Courses / Beginner to intermediate
The maths behind AI
From functions and vectors to attention and image models, ending with a transformer you train yourself. 70 lessons in 14 modules, about 9 hours in all.
0 of 70 lessons done
Next up: Multiplication
Number basics
Multiplication from scratch, then bigger numbers, negatives, fractions, powers and the order of operations.
5 lessons, 32 minutes
- 1MultiplicationMultiplying is fast adding. Three rows of five is fifteen, and every number a model computes is built from steps this simple.5 min
- 2Bigger numbers and decimalsSplit each number into tens and ones, multiply the easy pieces, then add them up. Decimals follow the same rule with a point moved.6 min
- 3Negative numbersNumbers below zero, walked along a line. Adding a negative steps left, and multiplying by one flips you to the other side of zero.6 min
- 4Fractions, division and percentagesA fraction is a division waiting to happen, a decimal is its answer, and a percentage is that answer out of 100. Models report their confidence in all three.6 min
- 5Powers, roots and the order of operationsA power is repeated multiplication, a root undoes it, and three ways to work out square and cube roots by hand. Then the fixed order of operations.9 min
- Module 1 quiz: Number basics10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
Algebra: letters for numbers
Variables, solving equations, the slope of a line, and two equations at once: the doorway to linear algebra.
5 lessons, 31 minutes
- 1Letters for numbersA letter such as x is a box that can hold any number. Writing rules with letters lets one line describe millions of calculations.5 min
- 2Solving equationsAn equation says two things are equal. Do the same to both sides, and you can uncover the hidden number step by step.6 min
- 3Graphs, and what y = mx + c meansEvery point on a graph is a pair of numbers, x across and y up. A rule like y = mx + c turns each x into a y, and those points line up.7 min
- 4Straight lines and slopePlot y = mx + c and you get a straight line. The m is its slope, rise over run, and it is the first parameter a model ever learns.6 min
- 5Two equations at onceTwo unknowns need two equations. Each one is a line, the answer is where they cross, and writing them as a grid of numbers leads straight to matrices.7 min
- Module 2 quiz: Algebra: letters for numbers10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
Warm-up: the language of maths
Functions, powers, logarithms and the sigma sign. The few tools that everything else is written in.
3 lessons, 21 minutes
- 1FunctionsA function is a machine that takes a number in and gives a number out. Every model in AI, however large, is one of these.6 min
- 2Exponentials and logarithmsTwo functions that are each other's undo button. One grows by multiplying, the other counts the multiplications. AI uses both everywhere.8 min
- 3Sums, averages and the sigma signThe tall Greek letter in almost every AI formula just means "add these up". Learn to read it once and the formulas stop looking frightening.7 min
- Module 3 quiz: Warm-up: the language of maths10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
- Checkpoint 1: modules 1 to 325 questions with diagrams to drag, slide and click. Pass with 30 to open what comes next.50 marks
Linear algebra: vectors
How pictures, words and sounds become lists of numbers that a machine can compare.
5 lessons, 38 minutes
- 1Everything is a numberBefore a model can learn anything, pictures, words and sounds have to become lists of numbers. Every AI system starts here.6 min
- 2VectorsA vector is an ordered list of numbers. Drawn as an arrow it has a length and a direction, and both will matter.8 min
- 3The dot productOne number that says how much two vectors point the same way. It is the most used calculation in all of AI.8 min
- 4EmbeddingsGive every word a vector, place similar words close together, and directions in that space start to carry meaning.8 min
- 5Distance and nearest neighboursIf things are vectors, then "similar" means "close". Measuring closeness gives the simplest learning method there is, and the one behind search by meaning.8 min
- Module 4 quiz: Linear algebra: vectors10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
Linear algebra: matrices
Grids of numbers that stretch, turn and fold space. Every layer of a network is one.
5 lessons, 41 minutes
- 1Matrices move spaceA matrix is a grid of numbers and also a machine. Feed it a vector and it returns a new one. Watching what it does to a whole plane makes it easy to understand.9 min
- 2Matrix multiplicationMultiplying two matrices chains two transformations into one. The rule is rows times columns, and the shapes have to fit.8 min
- 3Layers and non-linearityOne layer of a network is a matrix multiplication plus a shift. Stack layers with a bend in between and the network can shape space in ways no single matrix can.9 min
- 4Transpose, identity and inverseThree small ideas that appear in almost every formula from here on. Flipping a matrix, the matrix that does nothing, and the matrix that undoes another.8 min
- 5Tensors and shapesA vector is a list and a matrix is a grid. Stack grids and you have a tensor. Reading shapes is the everyday working skill of anyone who builds models.7 min
- Module 5 quiz: Linear algebra: matrices10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
Derivatives
How a small nudge to an input changes the output, traced through a whole chain of steps.
5 lessons, 45 minutes
- 1Slope and the derivativeThe derivative answers one question. If I nudge the input a little, how much does the output change? All learning depends on it.9 min
- 2The chain ruleA handful of rules cover every function a network uses. The most important one handles functions inside functions.9 min
- 3GradientsA model's error depends on millions of parameters at once. The gradient collects one slope per parameter into a single vector that points uphill.9 min
- 4Backpropagation by handRun a tiny model forwards to get its error, then send the blame backwards with the chain rule. This is how every network finds its gradient.10 min
- 5Valleys, peaks and saddlesA zero gradient means the ground is level, but level ground comes in three kinds. Telling them apart explains a good deal about why training works at all.8 min
- Module 6 quiz: Derivatives10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
- Checkpoint 2: modules 4 to 625 questions with diagrams to drag, slide and click. Pass with 30 to open what comes next.50 marks
Gradient descent
The one-line rule that turns "how wrong am I?" into "which way should I move?".
5 lessons, 40 minutes
- 1Loss, a number for how wrongBefore a model can improve, "wrong" has to be turned into a single number. That number is the loss, and training is the effort to make it small.7 min
- 2Gradient descentCompute the gradient, take a small step the other way, repeat. One short update rule trains every neural network there is.9 min
- 3Fitting a linePut loss, gradient and update rule together and you have a complete learning algorithm. Watch it train a model with two parameters.8 min
- 4Momentum and AdamPlain gradient descent zig-zags in narrow valleys and crawls across flat ones. Two improvements fix that, and nearly every modern model is trained with them.9 min
- 5Batches, epochs and schedulesThe practical vocabulary of a training run. How much data goes into each step, how many times the data is used, and how the learning rate changes along the way.7 min
- Module 7 quiz: Gradient descent10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
Probability
A model never answers, it bets. This is the maths of making those bets and scoring them.
6 lessons, 50 minutes
- 1Probability and distributionsA language model does not know the next word. It assigns a probability to every word it could be. Here is the vocabulary for that kind of statement.8 min
- 2Conditional probability and the next tokenThe chance of the next letter depends on the letters before it. A probability that depends on context is exactly what a language model computes.9 min
- 3Softmax and temperatureA network outputs raw scores of any size and sign. Softmax converts them into a proper probability distribution, and temperature controls how bold it is.8 min
- 4Cross-entropyHow do you score a prediction that is a whole distribution? Look at the probability it gave to what actually happened, and take minus its logarithm.8 min
- 5Bayes' ruleHow to update a belief when new evidence arrives. It explains why a positive result is often less alarming than it sounds, and it is the logic behind learning itself.9 min
- 6Entropy and informationHow much uncertainty a distribution holds, measured in bits. It explains where the name cross-entropy comes from and what a loss value is really counting.8 min
- Module 8 quiz: Probability10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
Neural networks
Neurons, layers and backpropagation, assembled from the pieces you already have.
6 lessons, 53 minutes
- 1The neuronA neuron is a dot product, a bias and an activation. On its own it can do exactly one thing, which is to draw a straight boundary.8 min
- 2Layers and parametersPut neurons side by side to make a layer, and layers one after another to make a network. Then count what you have built.8 min
- 3Backpropagation through a networkThe same forward-and-backward procedure you did by hand, written with matrices so that it handles whole layers at once.10 min
- 4Training in practiceA real network, real data, a real training loop. Use it to see overfitting, capacity and the learning rate for yourself.10 min
- 5Classifying from start to finishOne complete classifier, from a drawing to a decision, using every piece of the last three modules. Then you train it yourself on your own handwriting.9 min
- 6RegularisationWays of stopping a network from memorising its training data. Each one makes training a little harder so that what is learned is more general.8 min
- Module 9 quiz: Neural networks10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
- Checkpoint 3: modules 7 to 925 questions with diagrams to drag, slide and click. Pass with 30 to open what comes next.50 marks
Images and convolution
How a network sees: small filters slid across a picture, stacked into layers, and run in reverse to draw.
4 lessons, 32 minutes
- 1Why images need their own kind of networkA picture is a tensor of brightness values. Feeding it to an ordinary network wastes parameters and throws away the one thing that matters most, which is where each pixel is.7 min
- 2ConvolutionSlide a small grid of weights across a picture and take a dot product at every position. That one operation finds edges, blurs, sharpens, and is the basis of machine vision.9 min
- 3From filters to a networkStack convolution layers, shrink the picture between them, and a network builds objects out of shapes and shapes out of edges.8 min
- 4How image generators workSystems that draw from a text prompt are trained on a strange task, which is removing noise. Run that skill in reverse from pure static and a picture appears.8 min
- Module 10 quiz: Images and convolution10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
Attention and transformers
How a model decides which earlier words matter for predicting the next one.
6 lessons, 53 minutes
- 1Tokens and positionsThe path from a sentence to the matrix a transformer works on. Split into tokens, look up embeddings, and add information about order.8 min
- 2AttentionAttention lets each token ask a question of every token before it and take a weighted mix of the answers. It is built from dot products and softmax.11 min
- 3The transformer blockAttention moves information between tokens. Add an MLP, two normalisations and two shortcut connections, and you have the unit that a GPT repeats again and again.10 min
- 4From logits to languageA trained model only ever outputs a distribution over the next token. Writing a paragraph means sampling from it over and over.8 min
- 5How tokenisers are builtReal models do not read letters or whole words. They read word pieces, found by an algorithm that merges the most frequent pairs again and again.8 min
- 6Heads, layers and what they learnOne attention pattern is not enough. A transformer runs several side by side and stacks them many layers deep, and each ends up with a job of its own.8 min
- Module 11 quiz: Attention and transformers10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
Using a language model
What happens between your prompt and the reply: context, fine-tuning, retrieval, and why models make things up.
5 lessons, 41 minutes
- 1What happens when you typeFollow a prompt through a language model, from the keys you press to the first word of the reply. Six steps, repeated once for every token.9 min
- 2Context, prompts and examplesA model's only working memory is the text in front of it. Understanding the context window explains what prompts do and why long conversations drift.8 min
- 3Pre-training and fine-tuningA language model is trained in stages. First on an enormous amount of general text, then on a much smaller amount chosen to shape how it behaves.8 min
- 4RetrievalA model's knowledge is frozen on the day its training ends. Retrieval fetches relevant text at the moment you ask and places it in the context, like an open-book exam.8 min
- 5Why models make things upA language model is built to continue text plausibly, not to check facts. Four pieces of maths from this course explain why a confident wrong answer is a natural result.8 min
- Module 12 quiz: Using a language model10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
- Checkpoint 4: modules 10 to 1225 questions with diagrams to drag, slide and click. Pass with 30 to open what comes next.50 marks
Other ways machines learn
Learning from reward, from ratings and from no labels at all, and how to tell whether any of it worked.
5 lessons, 39 minutes
- 1Three kinds of learningLearning from answers, learning from the data alone, and learning from reward. Nearly every system in AI is one of these or a mixture.7 min
- 2Reinforcement learningAn agent tries things, receives a reward now and then, and works out which of its earlier actions deserve the credit. One update rule is enough to learn a maze.9 min
- 3How recommendations workGive every person and every item a small vector, and learn the vectors from the ratings you have. The blanks in the table then fill themselves in.8 min
- 4ClusteringFinding groups in data when nobody has labelled anything. The k-means method needs two steps, repeated, and both are things you already know.7 min
- 5Measuring a classifierAccuracy can look excellent while a model is useless. Precision and recall say what kind of mistakes it makes, and a threshold lets you trade one for the other.8 min
- Module 13 quiz: Other ways machines learn10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
Train your own model
Build a language model by counting, then train a real transformer, watch it predict and ask it questions.
5 lessons, 45 minutes
- 1A language model from countingBuild a complete language model with nothing but a table of counts. It is weak, but it has every part of the real thing.8 min
- 2Train a tiny transformerEverything in this course runs here at once. Train a real transformer on your own device and learn to read what it shows you.12 min
- 3Watch it predictOpen your model and look at the one thing it actually does, which is to put a probability on every possible next character.8 min
- 4Ask your model a questionTrain a model on 44 questions and answers, then ask it things. It answers what it has seen and confidently makes up the rest, which is every chatbot in miniature.9 min
- 5What changes at scaleYour model and a frontier model share an architecture and a training algorithm. Here is what is actually different, and where to go from here.8 min
- Module 14 quiz: Train your own model10 questions with diagrams to drag, slide and click. Pass with 12 to open what comes next.20 marks
- Case study50 questions with diagrams to drag, slide and click. Pass with 30 to open what comes next.50 marks
- Final paper50 questions with diagrams to drag, slide and click. Pass with 60 to open what comes next.100 marks