Assumed audience
- Reading level: general adult.
- Background: no math required.
- Goal: understand information as a measure of uncertainty.
The core idea
Information is what reduces uncertainty. If you already know the outcome, you learn nothing new. If you are surprised, you gained information.
A simple example
- If a coin is fair, a flip gives you one bit of information.
- If a coin always lands heads, a flip gives you zero bits of information.
Why this matters
Information theory is a way to measure how much we learn from messages, data, or observations.