AI Foundationspredict · compress · act

concepts → Objective

Objective

Information bottleneck

decade4 connections

keep what tells you the answer, throw away as much of the input as you can

the formal statement
maximise I(Z; Y) − β · I(X; Z)

Where it sits

The prism has six jobs across and eight layers down. Its primary cell is learn × L0. Hatched cells cannot exist — a GPU does not learn, an institution does not infer.

What must come first — and what it unlocks

Left to right is reading order, derived from the prerequisite_of edges. Nothing here is hand-ordered: the diagram is the graph.

Rate–distortion theoryInformation bottlenecknothing depends on it yet — a leaf in the reading order

Before it: Rate–distortion theory

What kind of thing it is — and what it is made of

It is made of: Information plane.

Where to read it

The chapter that introduces it, and any chapter that uses it again.

15The Bottleneckact IV · The Learning Floor

Where it comes from

paperThe Information Bottleneck MethodTishby, Pereira, Bialek · 1999
paperOn the Information Bottleneck Theory of Deep LearningSaxe, Bansal, Dapello, Advani, Kolchinsky, Tracey, Cox · 2018

Every connection

All 4 edges touching this node, grouped by relation family — the sections above are highlights from this list. Colours match the relation families inthe atlas.

Structure · 1
has as a partInformation planeFormat
Order · 1
Flow · 1
usesMutual informationQuantity
Lineage · 1
is attributed toNaftali TishbyPerson

This page is a projection of one node in src/data/concepts.ts. It has no prose file of its own — 521 declared edges produce all 340 of these pages. Edit an edge and both endpoints change.