How does AI work?

There is essentially no computer science in standard American K–12 education. That's a shame for many reasons, but especially now, because it leaves most people without a foundation to build intuition about AI and other advanced software.

Imagine if biology were missing from school, and then gene editing arrived. How would anyone make sense of CRISPR with no notion of cells or DNA? It would sound like magic, or like a threat, and there'd be no way to tell which.

So let's lay a foundation. We'll use nothing beyond high school stats, add one brick at a time, and end up with an approximate but valid understanding of how modern AI models work. Everything on this page is live: drag it, train it, break it.

1A line of best fit

The first brick is a familiar one. Here's some data on house prices: each dot is a house, with its size along the bottom (square feet) and its sale price up the side.

Even at a glance it matches intuition: in general, bigger house, bigger price. So we can draw a line through the cloud of dots and use it to predict the price of a house we haven't seen yet. Try it. Drag the sliders until the line looks right to you.

To predict a price, find the house's size on the bottom axis, go up to the line, and read off the height. In math terms, we plug x into the equation of the line we drew:

That's it. That's linear regression, and it's the seed of everything that follows.

2Loss: what makes a fit “best”

Some lines through the data are clearly better than others. But “clearly better” isn't something a computer can work with. We need a number.

That number is called the loss. To find it, measure the gap between each dot and the line (how much the line missed that house's real price), then add up all the gaps. Here's a bad line and a good line, side by side:

So the line of best fit is simply the line with the lowest loss. Throughout this page we'll report the average miss rather than the total, since “off by $40k per house” is easier to picture than a giant sum, but it's the same idea: lower is better.

In practice people usually square each miss before adding them up, which punishes big misses extra hard. The spirit is identical: one number that says how wrong the line is.

3Training: from a guess to the best fit

You found a good line by eye. How would a computer do it, with no eyes and no intuition?

Remember the line is y = mx + b. The x and y values come from the data, pairs like (1,200 sq ft, $290k) and (2,000 sq ft, $440k). Those are fixed. The only things we can change are m and b. So the simplest possible strategy is:

1. Pick a random m and b.
2. Calculate the loss for that line.
3. If it's the lowest loss so far, remember that m and b. Otherwise throw them out.
4. Repeat a bunch of times.

This loop is called training. Press the button and watch it happen. On the right is a map of every possible line: each spot is one choice of m and b, shaded by its loss. The yellow valley is where the good lines live.

Random guessing works, sort of. After 25 tries you usually have a decent line, and more tries can only help. But look at the map: most guesses land nowhere near the valley. Wasted effort.

A smarter move is to start somewhere and nudge. Try a slightly bigger and slightly smaller m, and the same for b. Keep whichever nudge lowers the loss, and when no nudge helps, take smaller steps. Hit “Nudge downhill” and watch the path walk into the valley.

Real AI training uses a refined version of nudging called gradient descent. Instead of test-driving each nudge, it uses calculus to work out the downhill direction for every parameter at once. Same idea, much faster.

4Parameters: the knobs we turn

Our example was as simple as it gets: one input (size), one output (price), and the equation for a line. The numbers training gets to change, m and b, are called parameters. x is the input and y is the output.

Nothing forces us to stop at two. If we wanted to use both house size and lot size, we'd have two inputs and three parameters:

And we could go wild, with any equation and as many parameters as we like:

Training works exactly the same way no matter how many there are: tweak the parameters, check the loss, keep what helps. And here's the punchline: parameter count is exactly how we measure the size of modern AI models. Look at the scale.

5Curves: more terms, more parameters

If parameter count is what matters, what equation do those billions of parameters live in? It isn't a line. A line can only ever be straight, and most things in the world aren't. Try fitting a line to a wave: no setting of m and b will ever do it.

The fix is to give the equation more terms. Keep the line, then add an x² term, an x³ term, and so on, each with its own parameter. This is a polynomial, the same kind you graphed in algebra class:

Each term is a simple shape on its own: x² is a bowl, x³ is an S, and higher powers get flatter in the middle and steeper at the edges. Scale each one by the right parameter, add them up, and they can bend into almost any smooth curve. The highest power, n, decides how much bending is available, and every extra power is one more parameter.

Pick a shape, press Train, and watch the curve bend itself to the data. Each press runs a fixed 3,000 training steps of the same loop as before: nudge every parameter downhill on the loss, 3,000 times over. Not happy with the fit? Press it again for 3,000 more. Click the plot to add your own dots.

Two inputs: a grid of parameters

With one input, the terms line up in a row: one parameter per power of x. Add a second input, z, and they form a grid. Every power of x gets multiplied by every power of z, and each combination gets its own parameter. Go up to the 9th power of each and that's a 10 × 10 grid, 100 parameters:

This is the picture to keep in your head: a model is an equation with a big grid of parameters. Real AI models don't use polynomials, which get unruly when they're huge. They use a different family of equations, designed to train well at enormous scale. But the parameters are still laid out in grids, and training is still the same loop: nudge every cell downhill on the loss.

Grids are also why the counts explode. A grid's size is its width times its height, so double both and you quadruple the parameters. One grid of 10,000 × 10,000 is already 100 million parameters. Stack a hundred of those and you're past 10 billion.

6Text: turning letters into numbers

So far everything has been numbers in, numbers out. How do we get words? Simple: number the letters.

Type anything to see it encoded:

The same kind of equation from our house example now translates letters into other letters. Not a useful translation yet, but notice nothing about the machinery changed. Numbers in, numbers out, the numbers just stand for letters now.

A better encoding: one-hot

Numbering has a hidden problem. It says b is “closer” to c than to z, and that m sits halfway between a and z. None of that means anything for letters. So real models use a one-hot encoding instead: every letter gets a row of 26 switches, one per letter of the alphabet, and exactly one switch is on. The rest are 0.

Now no letter is bigger, smaller, or closer than any other. They're just different switches. Type a word to see it one-hot encoded, one row per letter:

Functions that take letters

Our line was a function with one input: f(x) = mx + b. But a function can take as many inputs as we like, each with its own m:

A one-hot letter is just 26 inputs, so we can hand it straight to a function: f(one_hot). Each switch gets its own weight, and since only one switch is on, the output is simply the weight of whichever letter came in.

Here's one we can write by hand: return 1 if the letter is a vowel, 2 if it's a consonant. Give a, e, i, o, u a weight of 1, every other letter a weight of 2, and set b to 0:

Type a letter, or click one, and watch it go through:

We picked those 26 weights by hand because we already knew the rule. But they're parameters, the same as m and b in the house example. Which raises the obvious question: what if we don't know the rule, and let training find the weights from data instead?

Real models usually work in chunks of words called tokens instead of single letters, with one switch per token. The idea is exactly the same, just with a much longer row of switches.

7Autocomplete: predicting letters

This is the single biggest jump we're going to make, so let's line up the two pieces we already have.

1. We can predict numbers by fitting data. Show training some house sizes and prices, and it finds a line (or, with more terms, a curve) that predicts the price of a house it has never seen.
2. We can write text as numbers. A letter becomes a one-hot row of switches, and a function can take those switches as its inputs.

Put them together: can we fit a function to data so that it predicts letters?

Let's set our sights on autocomplete. Given the first four letters of a five-letter word, can we predict the fifth? Hand it w a t e and we want back r. We'll call the function autocomplete.

The input is four one-hot letters. The output is shaped like a one-hot too, one slot per letter, but instead of a single 1 it holds a probability for every letter: how likely the model thinks each one is to come next. Each row gets one extra slot, _, making 27. On the way in it means “blank” (fewer than four letters); on the way out it means “the word ends here.”

That picture also tells us how to train it. Our training data is just a list of words, and every word is a question with a known answer: the first four letters are the input, and the fifth is the right answer. The perfect output would be the one-hot of that right letter, 100% on it and 0% everywhere else. The loss is how far off we were from that.

For letters we measure “how far off” as loss = −log(chance on the right letter). It's 0 when the model puts 100% on the right letter and climbs quickly as that chance shrinks, so confident wrong guesses get punished hardest. Average it over every word and we have one number to push downhill, just like the average miss for house prices.

What's inside the box

Inside, autocomplete is an equation built from two grids of parameters, like the 10 × 10 grid in section 5, only bigger. Here's the whole journey from four letters to 27 probabilities, left to right in the diagram below:

1. Letters become switches. Each letter is one-hot, exactly like section 6, plus the extra blank switch: 27 switches per letter. Four letters make 4 × 27 = 108 switches, and only 4 of them are ever on at once.
2. A 108 × 64 grid mixes the switches into 64 middle numbers. Each middle number is a weighted sum over all 108 switches: multiply every switch (s) by its own weight from the grid and add them up, plus a bias. Since only 4 switches are on, that sum is really just four weights picked out by the four letters. Then the sum is squashed through an S-shaped curve that flattens out at both ends, so every middle number lands between −1 and 1. The squash plays the role the higher powers did in section 5: it's what lets the equation bend instead of being one giant straight-line sum. middle = squash(w₁·s₁ + w₂·s₂ + … + w₁₀₈·s₁₀₈ + c)
3. A 64 × 27 grid gives every possible answer a score. For each of the 27 answers there's one more weighted sum, this time over the 64 middle numbers. It's our old friend mx + b, with 64 m's. A high score means “the middle numbers that lit up point at this letter.” Below, r stands for any one of the 27 answers. score(r) = v₁·middle₁ + … + v₆₄·middle₆₄ + d
4. Scores become probabilities. Scores can be any number, even negative. To turn them into odds that are all positive and add up to 100%, raise e (≈ 2.718) to each score and divide by the total. This step is called softmax. The biggest score gets the biggest share, and a score just a little higher than the rest gets a lot more of the share. P(r) = e^score(r) ÷ (sum of e^score over all 27)

Count the knobs: a 108 × 64 grid of weights plus 64 biases for the middle numbers, then a 64 × 27 grid plus 27 biases for the scores. That's {{PARAMS}} parameters, and every one of them starts random.

Before training, its parameters are random, so it's useless. Ask it to finish w a t e _ and it might say b. We want water.

Training is the same loop as the house prices: for every word in the list, feed in the first four letters, measure the loss, and nudge all the parameters a little downhill. Then do it again, over and over. The only difference is we're turning {{PARAMS}} knobs instead of two. Press train and watch the one-hot picture above fill in too.

Try wate, hell, pian, zebr before and after training. Notice the model doesn't output one letter; it outputs a score for every letter, and we pick the likeliest. Those percentages will matter in the next step.

Why nudge instead of random guessing here? With two parameters, random guessing found a decent line in 25 tries. With {{PARAMS}} parameters the space of possibilities is so vast that random guessing would never stumble into a good spot, not in the lifetime of the universe. Nudging every parameter downhill at once gets there in seconds.

8From autocomplete to conversation

Our model takes four letters and predicts the next one. Now for the one move that turns autocomplete into a generator: feed the output back in as input.

There's no new model here. It's the exact same autocomplete function from section 7, just called in a loop. Start with just h, padded with blanks the same way as before. autocomplete predicts the next letter, say e. Tack it on and call autocomplete again: now the input is he, and it predicts l. Then hel → l. Then hell → o. Then hello → end of word.

Once the word is longer than four letters, autocomplete only sees the last four, since that's all it was built to take. Try it with the model you trained above. Each time, instead of always taking the top letter, we roll weighted dice using the model's percentages, so you get different words each run. The temperature slider flattens or sharpens those odds.

Low temperature sticks to the safest letter and mostly reproduces words it studied. High temperature takes risks and invents new, word-shaped nonsense. That's the same dial chatbots have.

Scale this up and you have the core of a chatbot. Instead of a couple hundred words, train on a large fraction of everything humans have written. Instead of four letters of context, give it hundreds of thousands of tokens. Instead of {{PARAMS}} parameters, use billions. Then train it further on examples of helpful conversations, so that the most likely continuation of a question is a good answer.

Each word a chatbot writes to you is one of these steps: predict the next token, add it, repeat. It isn't magic. It's math, done at enormous scale.

9Bigger is better, but not perfect

There's no secret ingredient separating our little model from the ones in the news. The recipe is the same: more parameters, more data, longer training runs.

Scale is what lets a model go beyond autocomplete. With enough parameters and enough text, it stops just memorizing which letters follow which and starts learning patterns across words: grammar, facts, and eventually something that looks a lot like reasoning.

Roughly, here's what shows up at each size:

1. 10KLearns tiny local patterns: which characters or short sequences tend to follow others. Produces word-like fragments, but not real language.
2. 10MStarts capturing basic language structure. Can imitate the style and shape of text, but coherence falls apart quickly.
3. 100MProduces fluent-looking sentences and learns a surprising amount of grammar and common associations, but has weak reasoning and limited world knowledge.
4. 1BBecomes a genuinely useful small language model. Can follow simple instructions, hold short conversations, summarize, classify, and answer basic questions, though it is still brittle.
5. 10BHas broad language competence and substantial world knowledge. Can explain ideas, write useful prose and code, and solve straightforward multi-step problems with reasonable consistency.
6. 100BA strong general-purpose model. Handles complex instructions, technical material, coding, and nontrivial reasoning much more reliably, though it still makes important mistakes.
7. 1TFrontier-scale territory. Potentially much stronger breadth, abstraction, and difficult problem-solving, but the parameter count alone no longer tells you much about capability because training quality and architecture matter enormously.

But no matter the size, a model is limited by the data it was trained on. Everything it knows comes from fitting that data, the same way our line could only learn from the houses we showed it.

That puts a fundamental ceiling on ability. Scaling up gets a model closer and closer to the best of what's in its training data, but there's no sudden jump to far-beyond-human intelligence waiting at the next size up. The LittleLearner paper shows this neatly: researchers trained a 5-billion-parameter model only on elementary-school material. It did well on elementary-school tasks, and collapsed on anything beyond them. Even extra training on advanced examples afterward couldn't lift it past what its original data covered.

10There is no magic

The AI models in the news are the same kind of thing we built here: an equation with grids of parameters, trained by nudging downhill on a loss. They're just much, much bigger.

It takes two things to describe a model: its equation and its parameters.

The equation is the recipe: which terms exist and how they're combined. Take the two-input polynomial from section 5. Its whole recipe fits on one line:

Then there are the actual values of the parameters: one number in every cell of the 10 × 10 grid. Hover or tap a number to see which term it belongs to.

Arriving at an equation and a set of parameter values is what training a model means. People pick the equation, and the loss-nudging loop from section 3 finds the values.

Once we have an equation and a set of parameters, we can use the model. That's called inference: feed in an input, get an output. Predicting a home's price with a line of best fit is inference. So is autocomplete guessing the next letter, and a chatbot generating text.

And the equation and the parameter values are shareable. If you know the equation and have the values, you can use a model without training it. That's great news, because training is the hard part, and using it is the useful part.

The 100 numbers above aren't random. They were trained ahead of time, before this page was ever loaded, to fit a landscape with one hill and one valley. Here they are saved in a file, just like one you'd download. Load it and the model works immediately, with zero training.

Some AI labs are secretive about their models. They don't share the equation or the parameter values. But lots of labs do. These are called open-weight models, and there are tons of them you can use. Here are the most-downloaded text generators on Hugging Face.

Their parameters can be downloaded and used just like our little example. In reality, the files are much bigger: roughly 1 GB per billion parameters. And the computer running them has to be powerful. Think MacBook Pro or stronger.

But the pattern is the same. These labs did the hard part: they came up with an equation, gathered the training data, and trained the parameters until the loss was low. Then they uploaded the equation and the parameter values to the internet, so anyone can download them and do the useful part.

What we built

1. A line of best fit turns an input into a prediction.
2. Loss is one number that says how wrong a prediction is.
3. Training adjusts the line to make the loss as low as possible.
4. The adjustable numbers are parameters. Model size is counted in them.
5. Add more terms and the equation can bend. With more inputs, the parameters form a grid.
6. Encode text as numbers and the same machinery works on language.
7. Autocomplete is an equation trained to predict the next letter.
8. Feed predictions back in and you get generation: the core of a chatbot.
9. Scale brings new abilities, but a model can't outgrow its training data.
10. There's no magic: a model is an equation plus parameters. Open-weight labs publish both, so anyone can run them.

That's an approximate but valid picture. The real systems add important machinery we skipped: tokens instead of letters, embeddings that give each token a rich numeric fingerprint, an architecture called the transformer that lets the model weigh every earlier word when predicting the next one, and extra training rounds with human feedback to make it helpful and careful. But every one of those pieces is built on the bricks above. Parameters, a loss, and a loop that nudges downhill.