AI Foundationspredict · compress · act
act III

The Instruments

Meaning becomes geometry. Similar things point the same way.

09

Everything Is a Vector

before this →Belief, Updated

A vector is a list of numbers — a point in space. In machine learning, that space is where meaning lives. A word, an image, a user, a document: each becomes a vector, and similarity becomes distance.

kingqueenrandom wordsmall angle = similaru · v = |u||v| cos θlarge when aligned
the dot product measures how much two vectors point the same way

The dot product

Multiply matching entries and add them up. That is the dot product, and when the vectors are normalised to length one, it is exactly the cosine of the angle between them. Aligned vectors score near 1; opposite ones near -1; unrelated ones near 0.

Matrix = many dot products at once

Stack a lot of vectors side by side and you get a matrix. Multiplying a vector by a matrix computes its dot product against every row simultaneously. This is why hardware vendors fight over matrix multiplication: it is where nearly all the arithmetic in a neural network happens. A whole network is, at bottom, a long chain of matrices with simple nonlinearities in between.

The geometry of meaning

why this turns out to be so powerful

Once words are points, meaning becomes arithmetic. Analogies fall out as vector offsets: king − man + woman lands near queen. You do not program that relationship. It emerges from how the vectors were learned.

the pictureAn embedding space is a map where nearby points mean similar things. Retrieval, clustering, and recommendation are all just navigation on this map.
OBJECTSwords, images, users, documents
VECTORSeach becomes a point in space
GEOMETRYdistance = similarity; arithmetic = meaning
introduces →vectordot productmatrixembedding space
← previousBelief, Updatednext →Rolling Downhill