AI Foundationspredict · compress · act

concepts → OpenQuestion

OpenQuestion

Data wall

timeless2 connectionsdraft

the stock of human-written text is finite, and scaling laws assume it is not

Where it sits

The prism has six jobs across and eight layers down. Its primary cell is no mode × L2. Hatched cells cannot exist — a GPU does not learn, an institution does not infer.

Overlays: Efficiency, cost & energy.

What must come first — and what it unlocks

Left to right is reading order, derived from the prerequisite_of edges. Nothing here is hand-ordered: the diagram is the graph.

nothing comes first — this is a starting pointData wallnothing depends on it yet — a leaf in the reading order

Where to read it

The chapter that introduces it, and any chapter that uses it again.

30The Big Readact VIII · The Attention Revolution

Where it comes from

paperWill We Run Out of Data? Limits of LLM Scaling Based on Human-Generated DataPablo Villalobos, Jaime Sevilla, Lennart Heim, Tamay Besiroglu, Marius Hobbhahn, Anson Ho · 2022

Every connection

All 2 edges touching this node, grouped by relation family — the sections above are highlights from this list. Colours match the relation families inthe atlas.

Flow · 1
usesPretrainingAlgorithm
Contrast · 1
trades off withScaling lawResult

This page is a projection of one node in src/data/concepts.ts. It has no prose file of its own — 521 declared edges produce all 340 of these pages. Edit an edge and both endpoints change.