WHAT IS A THEORY :

"A theory is the more impressive the greater the simplicity of its premises, the more varied the kinds of things that it relates and the more extended the area of its applicability. Therefore classical thermodynamics has made a deep impression on me. It is the only physical theory of universal content which I am convinced, within the areas of the applicability of its basic concepts, will never be overthrown." -- Einstein (1949)

ENTROPY : IN TERMS OF PROBABILITY

First we consider the probabilities for dice that every good backgammon players knows. Below, the left column indicates the sum of the two die; the next column lists all the possible combinations that give that sum; the third column counts the number of combinations for that sum. As you can see, there are a total of 36 different combinations for the two die, and each are equally probable to occur for "honest" dice. Thus the probability of getting a particular sum, as shown in the last column, is just the number of combinations divided by 36.

The most probable result, occurring one-sixth of the time, is to get seven.

Surprisingly, the first known successful calculation of the above probabilities didn't occur until the sixteenth century, by Girolamo Cardano. An exploration of why this should have occurred so late, plus a great deal more about probability, is the wonderful book by Peter L. Bernstein, Against the Gods: the remarkable story of risk (Wiley, 1996).

Surprisingly, the first known successful calculation of the above probabilities didn't occur until the sixteenth century, by Girolamo Cardano. An exploration of why this should have occurred so late, plus a great deal more about probability, is the wonderful book by Peter L. Bernstein, Against the Gods: the remarkable story of risk (Wiley, 1996), ISBN 0 471 29563 9.

The probabilities for dice lead us to our first definition of the entropy:

The entropy is a measure of the probability of a particular result.

Here,...