Parameter variations, noise susceptibility, and increasing energy dissipation of CMOS devices have been recognized as major challenges in circuit and micro-architecture design in the nanometer regime. Among these, parameter variations and noise susceptibility are increasingly causing CMOS devices to behave in an "unreliable" or "probabilistic" manner. To address these challenges, a shift in design paradigm, from current day deterministic designs to "statistical" or "probabilistic" designs is deemed inevitable. Motivated by these considerations, I introduce and define probabilistic Boolean logic, whose logical operators are by definition "correct" with a probability 1/2 <= p <= 1. While most of the laws of conventional Boolean logic can be naturally extended to be valid in the probabilistic case, there are a few significant departures. We also show that computations realized using implicitly probabilistic Boolean operators are more energy efficient than their counterparts which use explicit sources of randomness, in the context of probabilistic Boolean circuits as well as probabilistic models with state, Rabin automata. To demonstrate the utility of implicitly probabilistic elements, we study a family of probabilistic architectures: the probabilistic system-on-a-chip PSOC, based on CMOS devices rendered probabilistic due to noise, referred to as probabilistic CMOS or PCMOS devices. These architectures yield significant improvements, both in the energy consumed as well as in the performance in the context of probabilistic or randomized applications with broad utility. Finally, we extend the consideration of probability of correctness to arithmetic operations, through probabilistic arithmetic. We show that in the probabilistic context, substantial savings in energy over correct arithmetic operations may be achieved. This is the theoretical basis of the energy savings reported in the video decoding and radar processing applications that has been demonstrated in prior work.