Belief, Updated
A probability is a degree of belief between 0 and 1. Bayes’ rule is the only correct way to update one of those beliefs when new evidence shows up.
In one line
Posterior = prior × likelihood, then renormalise.
The prior is what you believed. The likelihood is how well each hypothesis explains what you just saw. The posterior is what you believe now. Evidence that would be likely under many hypotheses barely moves you; evidence only one hypothesis predicts moves you a lot.
This is why a rare disease test can produce a positive result that is still probably wrong: if the disease is rare, the prior is tiny, and even a good test only lifts it so far. Bayes is a machine for not being fooled by base rates.
Learning is Bayesian at the core
Training a model on data is a form of Bayesian updating: start with a prior (the architecture, the initialization), see data (the likelihood), and end with a posterior over what the model should believe. When a model is “confidently wrong,” it has ignored its prior.