and all of this happens without any extra effort on your part. You see entropy in a more general sense in the observation that your room always seems to get cluttered, dishes are always waiting to be washed in the kitchen, the floor always needs sweeping, your desk is always a mess.

When you peddle that bike, a significant portion of the energy you use goes to overcoming friction, which dissipates that energy as heat, warming up and disordering your surroundings.1 Understanding entropy allows chemists and physicists to predict directionality and spontaneity in chemical reactions and other physical processes. For example, if you use a hand pump to fill your bike tire with air, some of the energy you expend is wasted in warming up the pump and the tire.

Physicists in the 1800s realized that any time energy is used to do work, some portion of that energy is unavoidably lost as heat to the surroundings. Entropy as a concept is simple enough to understand because you experience it every day. Today, I would like to speak about entropy: spiritual entropy. By the way, if any of you kids out there ever encounter a similar situation, and you end up wondering if you can just sweep all that sugar off the floor and back into the bin without mom noticing. Interestingly, what took them maybe five glorious minutes to create took me hours to clean up. The kitchen was a winter wonderland, and my sisters looked just like little sugared donuts. Soon, however, it became apparent even to me that the word they were calling out in such joyous chorus was, in fact, 'sugar.' I quickly investigated and found that my sisters had discovered mom's five-gallon bin of white sugar and were flinging armfuls into the air in delirious ecstasy. I was downstairs watching TV, like the conscientious babysitter that I was, when I heard my sisters in the kitchen upstairs excitedly calling out a word that sounded like 'stickers!' I shrugged and went back to watching my Transformers cartoons (this was the 80's). p 2) = I( p 1) + I( p 2): the information learned from independent events is the sum of the information learned from each event.I vividly remember babysitting my four little sisters one afternoon when I was about 12.I(1) = 0: events that always occur do not communicate information.I( p) is monotonically decreasing in p: an increase in the probability of an event decreases the information from an observed event, and vice versa.The amount of information acquired due to the observation of event i follows from Shannon's solution of the fundamental properties of information: To understand the meaning of −Σ p i log( p i), first define an information function I in terms of an event i with probability p i. This ratio is called metric entropy and is a measure of the randomness of the information. : 14–15Įntropy can be normalized by dividing it by information length. The entropy is zero: each toss of the coin delivers no new information as the outcome of each coin toss is always certain. The extreme case is that of a double-headed coin that never comes up tails, or a double-tailed coin that never results in a head.

Entropy, then, can only decrease from the value associated with uniform probability. Uniform probability yields maximum uncertainty and therefore maximum entropy. H ( X ) := − ∑ x ∈ X p ( x ) log ⁡ p ( x ) = E,

Entropy def