We present a bio-inspired, hardware/software architecture to perform Markov Chain Monte Carlo sampling on probabilistic graphical models using energy aware hardware. We have developed algorithms and programming data flows for two recently developed multiprocessor architectures, the SpiNNaker and Parallella. We employ a neurally inspired sampling algorithm that abstracts the functionality of neurons in a biological network and exploits the neural dynamics to implement the sampling process. This algorithm maps nicely on the two hardware systems. Speedups as high as 1000 fold are achieved when performing inference using this approach, compared to algorithms running on traditional engineering workstations.