Seeing Structure

Everything up to here was about models of language, images, and worlds. This act is about what happens when you aim one at a question that nature has been refusing to answer.

The best case to start with is also the most famous. It is worth being precise about why it worked, because the reason generalises — and the parts that did not work generalise too.

The problem, precisely

A protein is a chain of amino acids strung together in an order written down by DNA. That order is easy to read — sequencing is cheap and solved. But the chain does not stay straight: it folds into a specific three-dimensional shape, and the shape is what the protein does. Enzymes, antibodies and receptors are all the same chemistry in different geometries.

Protein folding is therefore the gap that mattered: sequence is known, function depends on shape, and shape was expensive. For fifty years getting a structure meant experiment — X-ray crystallography, later cryo-electron microscopy — a process of months to years per protein, sometimes with no answer at all. By 2020 the world had solved more than 175,000 structures.

Structure prediction is the shortcut: compute the shape from the sequence, without doing the experiment.

Why attention was the right tool

The breakthrough came from an architectural insight rather than from more compute, and it is worth understanding because it is not obvious.

A single sequence does not contain enough information to pin down a shape. What contains the information is many sequences: evolution has produced millions of variations of the same protein across species, and the ones that still fold correctly are the ones whose mutations are correlated. If position 40 and position 87 always change together, they are probably touching. That pattern of co-variation is a contact map. Reading it is almost exactly the operation attention performs — comparing every position against every other, weighted by relevance.

So AlphaFold 2 put attention in two places: over the amino-acid sequence, and over a multiple sequence alignment stacked on top of it. It replaced the previous approach of a pipeline of separately-trained stages with a single end-to-end network. In the blind community assessment CASP14 in 2020, it produced structures comparable to experimental accuracy for a majority of targets — and the Nature paper describing it appeared in 2021.

What happened after

Two things scaled up the result.

Coverage. The AlphaFold Protein Structure Database published predicted structures for over 214 million protein sequences by 2024 — effectively the known protein universe, not a hundred thousand examples. That is the difference between a demonstration and infrastructure: the expensive calculation was done once, for everything, and given away.

Scope. AlphaFold 3 (2024) changed the architecture again, using a diffusion-based model — the same family as the image and video models in this guide — and widened the question from a protein to a complex: proteins bound to nucleic acids, small molecules, ions. For interactions between proteins and other molecule types the paper reports at least a 50% improvement over existing methods.

That second part is the one with consequences, because most drugs work by binding to a protein, and predicting the binding is predicting the thing you are trying to design.

The pattern to carry forward is narrow and specific: AI did well where the answer was a structure and an instrument could check it. If that is the shape of success, then the obvious next question is whether it applies to systems bigger than a molecule — weather, whole materials.