Measuring information | Journey into information theory | Computer Science | Khan Academy

Summary

In a whimsical narrative, Alice and Bob develop innovative communication methods to send messages between their treehouses, evolving from flames to electrical pulses. As they explore the efficiency of their system, they determine that the cost of transmission should correlate with the number of binary questions required to decode messages. They calculate the bits needed for various messages, such as coin flips, letters, and poker hands, and conclude that predictability in communication can enhance efficiency. Historical context is provided through Ralph Hartley's contributions to information theory, emphasizing the balance between randomness and predictability in messaging.

Takeaways

• 😀 Alice and Bob start with basic message transmission methods, evolving from flames and shutters to electrified wires and wireless communication.
• 😀 Alice charges customers based on the time taken to transmit messages, highlighting the need for a common unit of measurement.
• 😀 The concept of binary digits (bits) is introduced, with '1' representing a yes and '0' representing a no.
• 😀 For coin flips, each flip requires one question, leading to a total of 10 bits for transmitting 10 flips.
• 😀 To transmit a six-letter word, an average of 4.7 questions per letter is needed, resulting in approximately 28.2 bits.
• 😀 A poker hand, consisting of five cards, requires around 28.5 bits to transmit based on the average of 5.7 questions per card.
• 😀 Alice decides to charge one penny per bit, establishing a pricing model based on the calculated transmission lengths.
• 😀 Ralph Hartley's work in the 1920s laid the foundation for information theory, defining information in relation to the number of symbols and selections.
• 😀 Hartley's formula, H = N log S, quantifies information, where H is information, N is the number of symbols, and S is the number of possible selections.
• 😀 Real communication is influenced by predictability, allowing for more efficient transmissions and potentially reducing the number of required questions.

Q & A

What methods did Alice and Bob initially use to transmit messages?

Alice and Bob initially used flames at night and shutters during the day for message transmission.

How did Alice and Bob enhance their communication methods after using flames and shutters?

They enhanced their communication by using a wire, which they plucked in different ways, and eventually electrified it to send electrical pulses.

What are the three types of messages that Alice's customers wanted to send?

The three types of messages were a list of 10 coin flips, a six-letter word, and a poker hand.

How does Alice determine the price for transmitting messages?

Alice decides that the price of a message should depend on how long it takes to transmit it, measured in bits.

What is the significance of binary digits (bits) in the context of the script?

Binary digits (bits) represent the fundamental unit of information, where each bit can have a value of zero or one, allowing for the quantification of messages based on the number of questions needed for identification.

How many questions are needed to determine the outcome of 10 coin flips?

To determine the outcome of 10 coin flips, Alice would need to ask 10 questions, one for each flip.

What is the average number of questions needed to identify a six-letter word using an alphabet of 26 letters?

On average, approximately 4.7 questions are needed per letter, leading to around 28.2 questions for a six-letter word, which rounds up to 29 bits.

How is the number of questions for identifying a poker hand calculated?

For a poker hand, which consists of five cards from a deck of 52, the average number of questions needed is approximately 5.7 per card, totaling around 28.5 bits for the hand.

What does the term 'bit' refer to in the context of this communication model?

'Bit' is a shortened term for 'binary digit,' which is the unit used to measure the amount of information based on the minimum number of questions needed to define a message.

How does predictability affect the length of message transmission?

Predictability in communication can reduce the number of questions needed to identify a message, leading to shorter and more efficient transmissions compared to completely random selections.