Consider a scientist
trapped in a room. He is given three coins, instructed to toss the coins one at a time,
and enter the results into the computer (figure 1.1). If the coin lands tails, he is
instructed to enter the letter T, and if it lands heads he is instructed to enter the
letter H. The combination for the door is H-T-H. The scientist has a 1 in 8 chance of
opening the door.
Figure 1.1: Trapped Scientist with Three Coins
When three coins are tossed, there are eight possible outcomes:
Coin 1 Coin 2 Coin 3
H H
H
H H
T
H T
H --------->opens the door
H T
T
T H
H
T H
T
T T
H
T T
T
The number of possible outcomes is easy to calculate. When the first coin is tossed, it
has two ways to land, heads or tails. The same rules apply for the second and third. So
the total number of possible outcomes is 2x2x2 = 8.
Whenever the scientist tosses a coin and observes the results, he acquires
information. When he tosses the first coin and observes the result, he acquires one bit of
information. After he observes the result of the second coin, he possesses 2 bits of
information. And after the third, he possesses 3 bits. Suppose on his first try to open
the door, all three coins land heads. After observing this event, the scientist will
possess 3 bits of information. He keeps trying, and after a few more tries, the first coin
lands head, the second lands tails and the third lands heads. When he enters this result
into the computer, the door opens. The scientist has acquired knowledge. The combination
for the door is H-T-H, and he now knows the combination.
Notice that every time the scientist tosses the coin he creates information,
but only one specific outcome creates useful information or knowledge.
One bit of information corresponds to each coin. In figure 1.1, all results
contain 3 bits of information. One result, H-T-H, contains 3 bits of knowledge.
Suppose that the combination is changed to
H-T-H-H-H-H-T-H-H-H-H-H-H-H-H-H-H-H-H-H. The scientist is given 20 coins, told to toss all
20, enter the results into the computer and observe the door. How much information is
generated every time the scientist tosses 20 coins and observes the result? Answer: 20
bits because there are 20 coins. While 20 bits of information is generated with each
attempt to open the door, only one possible outcome will open the door. This is the only
outcome that contains both information and knowledge.
With 20 coins, what is the probability that the scientist will find the
correct combination on the first try? Answer: multiply 2 by itself 20 times to determine
the total number of possible outcomes (1,048,576). Because only one of these outcomes will
open the door, the odds are 1 in 1,048,576 or approximately 1 in a million.
Exponents are a useful shorthand for representing a number multiplied by
itself many times. The phrase 2 multiplied by itself 20 times can be written as 220.
The number 10 multiplied by itself 86 times can be written as 1086. So the
number 5x104 = 5x10x10x10x10 = 50,000.
Next: More on Information Theory
Previous: Information Theory
|
