AI Foundationspredict · compress · act
act V

The First Attempts

While AI chased symbols, control theory chased the same problem from the other side — and gave us reinforcement learning.

18

Feedback: The Other Tree

before this →The First Winter

There is a second lineage in the story, and it is easy to miss. Cybernetics and control theory asked: how does a system steer itself toward a goal in a changing world? Their answer was feedback.

controllersystemoutputfeedback — the error signal−
the control loop: measure, compare to the target, correct

The vocabulary of sequential decisions

Control theory gave us the formal furniture of decision-making. A Markov decision process (MDP) describes a world as states, actions, transitions, and rewards — with the Markov property: the future depends only on where you are now, not how you got there. That stripped-down model is the foundation of modern reinforcement learning.

The thermostat is the canonical example, and it is genuinely instructive: a sensor, a target, an error signal, and a correction. No symbols, no search, no knowledge base — just a loop that closes.

Two trees, one forest

symbols vs. feedback

Symbolic AI reasoned about the world from the outside. Control theory acted inside it, adapting moment to moment. Modern AI is where the two trees finally braid: a model that represents the world and a loop that acts in it.

the reunionAgentic AI is not a new idea. It is symbolic reasoning and feedback control, finally sharing the same substrate.
SENSEobserve the current state
COMPAREmeasure the gap to the goal
ACTcorrect, and repeat
introduces →feedbackcontrol theoryMarkov decision processcybernetics
← previousThe First Winternext →Learning from Blame