Mathematically,
information is defined as a reduction in uncertainty. Consider a scientist trapped in a
room. He has a coin and a telephone. He is told to flip the coin and then tell his
colleagues who are 500 miles away the results using the telephone. He is to repeat this
process until he is told otherwise.
Before the scientist flips the coin, he does not know whether it will land
heads or tails. There are two possible outcomes, and the scientist does not know which
will happen until he observes the results. Suppose that the first toss is heads. As soon
as the scientist observes this result, he has information. Two possibilities have been
reduced to one. Before observing the coin, the scientist was uncertain of the outcome.
After he observes the result, he is certain of the outcome. His colleagues do not have any
information until he tells them that the coin landed heads.
A unit of information is called a bit. Whenever two possible outcomes are
reduced to one, one bit of information is created. Thus, the scientist acquires one bit of
information each time he tosses the coin and observes the result.
Uncertainty depends on the number of possible outcomes. For example, if the
scientist is given a die to roll instead of a coin his uncertainty increases. With the
coin, there are only two possible outcomes, and with a die there are six. The reduction in
uncertainty for a die (six possible outcomes reduced to one outcome) is greater than it is
for the coin (two possible outcomes reduced to one outcome).
Thus, the scientist acquires more information when he observes the results of
tossing the die than when he observes the results of tossing the coin.
Next: Information Theory
Previous: Evolution and Information Theory |
