Part VI · Information & Entropy — 13

Quantum Entropy

Shannon’s formula counts the uncertainty in a list of probabilities. A quantum state isn’t a list — it’s a density matrix (chapter 6b). Von Neumann’s move was disarmingly simple: find that matrix’s hidden probabilities, and feed them to Shannon.

↩ before you start · keep these handy

·From Mixed States: a density matrix ρ is the bookkeeping for a partly-known qubit; arrow length r = 1 on the surface (pure), r = 0 dead-center (max mixed).

·From Ch. 12: Shannon entropy H = −Σ pᵢ log₂ pᵢ — average surprise of a probability list.

·log₂ reminder: log₂1 = 0 and log₂½ = −1, so −½ log₂½ = ½.

🔑 symbol decoder · every new mark, in plain words

S(ρ)the von Neumann entropy — bits of genuine ignorance in state ρ. λᵢ"lambda" — an eigenvalue of ρ: one of its hidden probabilities, all adding to 1. (1±r)/2for a qubit, the two eigenvalues — set entirely by the arrow length r. Trthe trace — add up a matrix's diagonal; here it just totals the eigenvalue terms. diagonala matrix rotated so only its diagonal is non-zero — then the diagonal is the probability list.

feel

How mixed, in one number?

Back on the Bloch ball, a pure state sits crisply on the surface; a mixed one sinks toward the murky center. We measured that with purity — but there’s a sharper currency. Von Neumann entropy answers a precise question: how many bits of genuine ignorance does this state hold? A state you know perfectly scores zero, no matter how exotic. A state that’s a coin-flip between two possibilities scores a full bit.

🃏 everyday picture

Imagine a single playing card on the table. If you watched it being placed face-up, you know it for certain — zero questions left to ask, zero entropy. If a friend slid it face-down from a freshly shuffled deck, you're missing real information. Von Neumann entropy measures only that face-down kind of ignorance. The clever part: a quantum state can look uncertain (a superposition) while still being fully face-up to you — and that case scores zero.

recapVon Neumann entropy is one number for "how many bits of genuine ignorance" a state carries.

play

Sink the arrow, raise the entropy

Drag the Bloch arrow inward. Its length r sets the two hidden probabilities — the matrix’s eigenvalues (1±r)/2. On the surface they are 1 and 0 (a sure thing, zero entropy); at the dead center they are ½ and ½ (a perfect coin, one full bit).

▸ von Neumann entropyS(ρ) = H(eigenvalues)

              
              1
              0
              center
              surface
              Bloch length r →
              
              mix
              
λ₊ (eigenvalue){{ lamPlus }}
λ₋ (eigenvalue){{ lamMinus }}
entropy S(ρ){{ entropy }} bits
{{ desc }}

recapSinking the arrow splits the eigenvalues toward ½,½ and drives entropy from 0 up to 1 bit.

math

Shannon, run on eigenvalues

Every density matrix can be rotated until it’s diagonal — and the diagonal entries, its eigenvalues λ_i, are exactly a probability list. Run those through Shannon’s formula and you have the von Neumann entropy:

S(ρ) = −Tr(ρ log₂ ρ) = − Σi λi log₂ λi
pure state:  eigenvalues 1, 0  →  S = −1·log₂1 − 0 = 0 bits
center of ball:  eigenvalues ½, ½  →  S = 1 bit  (max for one qubit)

For our qubit the two eigenvalues are (1±r)/2, so S depends only on the arrow’s length, never its direction — exactly why the curve above is drawn against r alone. It is the same ∩-shaped curve from chapter 12, just relabelled.

✎ worked example · an arrow shrunk to r = 0.6

eigenvalues from r: λ₊ = (1+0.6)/2 = 0.8, λ₋ = (1−0.6)/2 = 0.2. They sum to 1. ✓
surprise of each: −log₂0.8 ≈ 0.32 bits, −log₂0.2 ≈ 2.32 bits.
weight by eigenvalue: S = 0.8×0.32 + 0.2×2.32 = 0.26 + 0.46.
S(ρ) = 0.72 bits — partway between a pure state (0) and the maximally-mixed center (1). ✓

recapS(ρ) is just Shannon’s H run on ρ’s eigenvalues — for a qubit it depends only on r.

⚠ common misconception

“A superposition is uncertain, so it must have entropy.” No. The state |+⟩ is a wild superposition, yet it’s a pure state sitting on the surface — you know it exactly — so its entropy is zero. Superposition is definite knowledge written in a tilted basis; it is not ignorance.

Entropy only appears when you genuinely don’t know which pure state you hold — the kind of ignorance that arrives when a qubit entangles with an environment you can’t see. Which means the entropy you just dialled up is the same quantity that will set the true, unsqueezable size of a quantum message in the next chapter.

✓ you can now

✓find a qubit’s eigenvalues (1±r)/2 from its Bloch length and feed them to Shannon

✓compute S(ρ) and read it as bits of ignorance: 0 for pure, 1 for the maximally-mixed center

✓explain why a superposition has zero entropy — it’s known, not unknown

← 12 Counting Information next · 14 Data Compression →