This page needs no coding background.

Plain English · Page 1 of 4 · No coding background needed

How AI chat tools actually work

When you type into ChatGPT, Claude or Gemini, it can feel like someone is on the other side. There isn’t. There’s a very large math formula, called a language model, that does one thing, over and over: guess the next small piece of text.

This page shows how that works in four steps. Then the next pages show how the same trick powers AI “agents”, and what really gets sent each time.

The whole idea in four sentences

  1. Your words are chopped into small pieces, and each piece becomes a number.
  2. The model does math on those numbers and scores every possible next piece.
  3. One piece is picked and added to the end, and the math runs again, one piece at a time.
  4. It keeps no memory of you: each time, the app sends the whole conversation again.

This guide is four short pages

Read them in order. Each one opens as its own page, and a Next page button at the bottom of every page takes you on to the next.

  1. Page 1How AI worksHow a chat tool writes its reply, one small piece at a time.You’re here
  2. Page 2What agents areHow an ordinary program asks the model what to do next.Open
  3. Page 3Behind the scenesWhat really gets sent to the model each time, and why it adds up.Open
  4. Page 4Notes vs programsWhy a note of instructions isn’t the same as a program.Open

About 20 minutes for all four. Each page also makes sense on its own.

Start reading page 1

Step 1: Words become numbers

Words become numbers: what the AI model actually receives, in about a minute.
Read the transcript

Here’s a surprise… the AI model, the part that writes the reply, never sees your words. Not one. When you send a message, a program first chops it into small pieces called tokens. Some are whole words, some are bits of words. Even a comma gets its own. Then each piece is swapped for its number in one giant list, like a numbered phrasebook. Same piece, same number: “twice”, here, is ten thousand, nine hundred and seventeen. For everyone. Order forty-four seventy-one? That’s four pieces: four, four, seven, one. And both fours get the very same number. So by the time your message reaches the model, it’s a row of numbers. That’s all it ever gets. And numbers are something a computer is very good at… doing math with. Your words go in as numbers. The reply comes out as numbers… and is turned back into words.

A model is a math formula, and math needs numbers. So before a model sees your message, the text is chopped into small pieces called tokens. A token can be a whole word, part of a word, a space plus a word, a single digit or a punctuation mark.

Every token has its own number, called a token ID. The numbers come from a fixed list the model was built with, its vocabulary. The model we use for examples on this site is Qwen3, which anyone can download and inspect. Its list has about 150,000 pieces. Other models use their own lists, so the same word gets a different number there.

Here is a message we follow across this site. Priya, a customer, was charged twice for one order and writes to a support chat. This is her message, chopped by Qwen3’s real chopper. 13 words become 20 tokens.

“Hi, I got charged twice for order #4471. Can you sort it out?”

  1. Hi13048
  2. ,11
  3. space I358
  4. space got2684
  5. space charged11430
  6. space twice10917
  7. space for369
  8. space order1973
  9. space #671
  10. 419
  11. 419
  12. 722
  13. 116
  14. .13
  15. space Can2980
  16. space you498
  17. space sort3378
  18. space it432
  19. space out700
  20. ?30
13 words, 20 tokens. The sign marks a space that belongs to the piece.

More examples (8 more)

“I love tacos”

  1. I40
  2. space love2948
  3. space tacos84401
3 words, 3 tokens. Common words are usually one piece each.

“Unbelievable!”

  1. Un1806
  2. belie31798
  3. vable23760
  4. !0
1 word, 4 tokens. Yes, “!” is number 0. The numbers are just positions in a list; they don’t mean anything by themselves.

“strawberry”

  1. str495
  2. aw672
  3. berry15357
1 word, 3 tokens.

“Hello”, “space Hello” and “space hello”

  1. Hello9707
  2. space Hello21927
  3. space hello23811
The same word, 3 different numbers. A leading space or a capital makes it a different piece.

“charged”, “space charged” and “space Charged”

  1. charged36961
  2. space charged11430
  3. space Charg27617
  4. ed291
“space Charged” with a capital isn’t in the list at all, so it becomes two pieces.

“38.00”

  1. 318
  2. 823
  3. .13
  4. 015
  5. 015
5 tokens. In this model, numbers go digit by digit.

“Bonjour, ça va ?”

  1. Bonjour81581
  2. ,11
  3. space ça37943
  4. space va11164
  5. space ?937
4 words, 5 tokens. Other languages are in the same list.

“😀”

  1. 😀141334
1 emoji, 1 token.

These were worked out ahead of time with the real Qwen3 chopper. Typing your own isn’t possible here. Some AI companies offer a free token counter on their websites, and each one shows its own model’s pieces.

What to notice

  • Common words are usually one piece. “space charged” is 11430 and “space twice” is 10917.
  • The space usually travels with the next word, and capitals count. “charged” at the start of a line is 36961. “space charged” after a space is 11430. “space Charged” with a capital isn’t in the list at all, so it becomes two pieces, “space Charg” and “ed”.
  • Less common or long words get split. “Unbelievable” becomes “Un”, “belie”, “vable”.
  • In this model, numbers go digit by digit. “#4471” becomes “space #”, “4”, “4”, “7”, “1”. That matters later. (Some other models group up to three digits per piece, so “4471” might become “447” and “1”. Either way, a number is cut into pieces by its digits, not by what it means.)
  • A rough rule: in English, 100 tokens is about 75 words. Each model family has its own list, so the exact pieces differ from model to model.

See this with code

Step 2: Guess the next piece

One piece at a time: how a language model writes a reply, in about a minute.
Read the transcript

So what does the AI model do with that row of numbers? One thing: it guesses what comes next. Your phone does a tiny version. Type “see you”… and it offers “soon” or “later”. The model is that idea, grown enormous. Inside, it’s a huge math formula. Numbers go in. Out comes a score for every piece in its list. One piece is picked, usually a likely one, and added to the end. Then the whole row goes in again. Again. Again. That’s inference: running the formula to get the next piece. Those scores come from patterns learned from a mountain of text. So the guesses are often very good… and it can sound completely sure, while being wrong. It has no goals, and it doesn’t understand the way you do. It computes which piece is likely next. Nothing mysterious. Very good arithmetic… one piece at a time.

You’ve already used a tiny version of this.

Here’s the cycle

  1. The numbers for all the text so far go into the formula.
  2. Out comes a score for every token in the vocabulary, all 150,000 or so. A high score means “likely next”.
  3. One token is picked, usually a likely one.
  4. That token is added to the end, and the whole thing runs again on the slightly longer text.

Running the formula once to get those scores is called inference. So a 100-token reply takes about 100 runs, one per piece. That’s why answers appear on your screen a few words at a time.

The pick has a little chance mixed in, controlled by a setting. That’s why asking the same question twice can give you two different answers.

The model also has a special “end” token. When that one is picked, the reply stops.

Illustrative scores. We made these up to show the idea; the token pieces and IDs are real Qwen3 ones.

Demo 1: “Peanut butter and…”

Text so far

  1. P47
  2. ean5307
  3. ut332
  4. space butter14100
  5. space and323
Scores for the piece after “Peanut butter and”
Next pieceIDIllustrative score
space jelly picked51341 very likely
space jam or, by chance20072 possible
space banana43096 unlikely
space honey25744 unlikely
space chocolate17931 unlikely
everything else the rest, together

After “Pick one” (the likely piece)

  1. P47
  2. ean5307
  3. ut332
  4. space butter14100
  5. space and323
  6. space jelly51341

After “Pick a less likely one”

  1. P47
  2. ean5307
  3. ut332
  4. space butter14100
  5. space and323
  6. space jam20072

Same text in, a different pick out. Both are reasonable; neither was looked up.

Demo 2: Writing Priya’s reply

Text so far

  1. Hi13048
  2. space Pri27785
  3. ya7755
  4. ,11
  5. space sorry14589
  6. space about911
Pick 1 of 4: scores for the piece after “…sorry about”
Next pieceIDIllustrative score
space that picked429 very likely
space the279 possible
space this419 possible
space your697 unlikely
space any894 unlikely
everything else the rest, together

Text so far

  1. Hi13048
  2. space Pri27785
  3. ya7755
  4. ,11
  5. space sorry14589
  6. space about911
  7. space that429
Pick 2 of 4: scores for the piece after “…about that”
Next pieceIDIllustrative score
! picked0 very likely
.13 likely
,11 possible
everything else the rest, together

Text so far

  1. Hi13048
  2. space Pri27785
  3. ya7755
  4. ,11
  5. space sorry14589
  6. space about911
  7. space that429
  8. !0
Pick 3 of 4: scores for the piece after “…that!”
Next pieceIDIllustrative score
space You picked1446 very likely
space I358 likely
space We1205 possible
everything else the rest, together

Text so far

  1. Hi13048
  2. space Pri27785
  3. ya7755
  4. ,11
  5. space sorry14589
  6. space about911
  7. space that429
  8. !0
  9. space You1446
Pick 4 of 4: scores for the piece after “…that! You”
Next pieceIDIllustrative score
space were picked1033 very likely
space got2684 possible
space have614 possible
space was572 unlikely
everything else the rest, together

After 4 picks

  1. Hi13048
  2. space Pri27785
  3. ya7755
  4. ,11
  5. space sorry14589
  6. space about911
  7. space that429
  8. !0
  9. space You1446
  10. space were1033

The full reply carries on the same way, 34 pieces in all: “Hi Priya, sorry about that! You were charged twice for order #4471, so I’ve refunded the duplicate $38.00 charge.” A model writes every piece this way, even the “3” and “8” in $38.00.

See this with code (the “Generation: one token at a time” part)

Step 3: It’s math, not a mind

Inside, a model is a long chain of multiplications and additions. It uses billions of fixed numbers, called parameters.

The dials were set during training. Training means feeding the formula enormous amounts of text and nudging the dials, over and over, so that its guess for the next piece gets a little better each time. Afterwards it was tuned further on example conversations, so that the text it is most likely to write sounds like a helpful assistant.

So when it answers, it isn’t looking anything up, and it isn’t thinking the way a person does. It computes which piece is likely to come next, based on patterns in everything it was trained on. (Some apps add a web search. That’s the app around the model fetching pages and adding them to the text the model reads.)

Those patterns are rich enough that the results are often impressive: it can summarize, translate, explain and draft.

Three things follow from “it’s math”

  • It can sound sure and be wrong. A confident tone is another likely pattern. When a model makes something up, people call it a hallucination.
  • It writes numbers piece by piece; nothing in it works like a calculator. “38.00” comes out as five pieces: 3, 8, ., 0, 0, each picked because it scored high. The patterns are often good enough to get sums right, but nothing guarantees it. When a number must be exact, other software should check it.
  • It has no goals of its own. Between requests it isn’t waiting or wanting anything. The formula only runs when something sends it text.

See this with code (the “What to notice” notes on digits)

Step 4: It doesn’t remember you

Here’s the part that surprises most people. The model keeps nothing between replies. When you send your second message, the model has no record of your first one.

So how does a chat feel continuous? The app re-sends everything. Every time you hit send, the app puts together one long note: its own standing instructions (directions written by whoever built the app), the whole conversation so far, and your new message. The model reads all of it from the top, predicts a reply, and keeps none of it afterwards.

That whole note is what people mean by the prompt. Everything the model is shown on one request is also called its context.

  1. Message 1 sends: instructions + your message

    • Instructions
    • Your message 1
  2. Message 2 sends: instructions + message 1 + reply 1 + your new message

    • Instructions
    • Your message 1
    • Reply 1
    • Your message 2
  3. Message 3 sends: all of that + reply 2 + your newest message

    • Instructions
    • Your message 1
    • Reply 1
    • Your message 2
    • Reply 2
    • Your message 3
Each send carries everything before it. The dashed slip is the new message.

Three things follow

  • There’s a size limit. A model can only take in so many tokens at once. This is called its context window. For large models today it’s often hundreds of thousands of tokens. When a chat outgrows it, the app has to cut or summarize older parts, and details can get lost.
  • Long chats cost more. The companies that run models charge for every token the model reads and every token it writes. Re-sending a long conversation means paying to read it all again, every time. On a monthly plan, it’s why long chats use up your limits faster.
  • “Memory” features work the same way. When an app says it remembers you, it is saving short notes and adding them to the note it sends.

There’s a film about this Watch the note grow, message by message: Behind the scenes

See this with code (the messages list)

Putting it together

The whole engine: everything on this page in one machine, and the question that leads to agents, in about a minute.
Read the transcript

Let’s put it all together. You type a message. A program chops it into pieces… and every piece becomes a number. Those numbers go into the formula: billions of tiny dials, set long ago by training on mountains of text. It isn’t looking anything up. Out comes a score for every possible next piece. One piece is picked, and added to the end. Then the whole row goes round again. And again. Until the model picks the piece that means “the end”. The numbers turn back into words… and that’s your reply. Then you write back. The model kept nothing. So the app sends the whole thing again: its instructions, the chat so far, and your new message. Its memory is just a longer note. That’s the whole engine. Chat tools, writing helpers and AI agents all run on it. What changes is what gets built around it. Which raises a question. If the model only ever writes text… how does an agent actually refund Priya’s thirty-eight dollars? Turn the page.

  1. Your text
  2. Chopped into tokens
  3. Token IDs (numbers)
    1. The formula scores every possible next token
    2. One is picked and added to the end

    Round again. Each time round, the whole text so far goes in.

  4. The end token is picked
  5. The numbers are turned back into text you read
The whole cycle, from your message to the reply on your screen.

That’s the whole engine. Chat tools, writing assistants and AI agents all run on it. The difference is what’s built around it.

You can now explain:

  • What a token is, and why “charged” and “space Charged” get different numbers.
  • That the model only ever sees numbers.
  • That a reply is written one token at a time, each picked from scores for every possible token.
  • What “inference” means: one run of the formula.
  • Why the same question can get different answers.
  • Why a model can sound sure and still be wrong, and why it shouldn’t be trusted with exact sums.
  • Why the model doesn’t remember you, and what “memory” in an app really is.
  • Why long chats cost more and eventually hit a limit.

Next: What agents are If the model only writes text, how does an AI “agent” refund a customer?

Words used on this page

Model (language model)
A very large math formula that takes in text as numbers and gives back a score for every possible next piece of text.
Token
A small piece of text, such as a word, part of a word, a space plus a word, a single digit or a punctuation mark.
Token ID
The number that stands for a token in the model’s fixed list; the model only ever sees these numbers.
Vocabulary
The model’s fixed list of every token it can read or write, each with its own number (about 150,000 in the open model we show).
Tokenizing
Chopping text into tokens and swapping each one for its token ID.
Inference
Running the model’s formula once to get scores for the next token.
Prediction
The model’s scores for which token is likely to come next.
Picking (sampling)
Choosing one token from the scores, usually a likely one, with a little chance mixed in.
Parameters
The billions of numbers inside the formula, like dials, that were set during training.
Training
The process, finished before you ever use the model, of adjusting its numbers on enormous amounts of text until its predictions got good.
Hallucination
When a model’s text sounds confident but is wrong or made up.
Prompt
Everything sent to the model in one go: instructions, the conversation so far, and anything else added.
Standing instructions (system prompt)
Directions written by whoever built the app, sent at the top of every request.
Context
Everything the model is shown on one request: the whole note it reads before predicting. (In everyday talk, “prompt” and “context” often mean the same thing.)
Context window
The most tokens a model can take in at once; past that, older parts must be cut or summarized.
Memory (in AI apps)
Notes that the app saves and re-sends; the model itself keeps nothing between requests.

Sources