You type “Explain how a bicycle stays balanced for a curious twelve-year-old” and press Send. A few seconds later, a paragraph appears. The smoothness of that exchange hides several different jobs: arranging the input, converting text into numbers, running a trained model and deciding which output to show.
There is no single pipeline shared by every AI product. The journey below describes a common text assistant built around an autoregressive transformer, a model that generates a sequence using what came before it. Search, tools and image generation can add other processes around that core.
The app assembles more than your sentence
Your latest message is usually one part of the input. An application may also supply instructions about the assistant's behaviour and earlier conversation turns. Chat formats identify roles such as user and assistant, often using special control tokens to preserve that structure. 1
This material occupies the model's context window, its bounded working input for a response. Relevant document extracts or tool results may occupy space too. The generated output also uses context capacity. The training data is a separate thing; the model does not carry that entire dataset inside the active conversation. 2
As a conversation grows, an app may shorten or summarise earlier material to stay within its limits. Consequently, “we discussed this earlier” does not guarantee that every original detail remains available. For a task that depends on an exact quotation, providing the relevant passage again can remove that uncertainty. 2
Text becomes tokens and numerical vectors
A tokenizer breaks text into units called tokens and maps them to numerical identifiers. A token might represent a complete word, a word fragment, punctuation or a piece containing whitespace. Subword tokenizers let unfamiliar words be represented using familiar smaller pieces. 3
The boundaries depend on the tokenizer. A long word need not count as one token, and equivalent messages in different languages need not have similar token counts. This is why a word limit in your request and a model's token limit measure different things. Tokenization prepares the text for computation; it does not by itself interpret your intention. 3
Next, token identifiers map to learned vectors called embeddings. These ordered collections of numbers can be transformed mathematically. The model also incorporates token-position information, because word order changes meaning. How it represents position varies between architectures. 4
Attention connects the available context
Transformer layers repeatedly update those representations. Attention calculates weighted combinations of information from permitted positions. Other parts of each layer transform the results further. Several attention heads can capture different relationships simultaneously. 4
Compare “The bat flew out of the cave” with “She picked up the bat before batting practice.” When generating a response after either sentence, the model has different available evidence about the meaning of “bat”.
Attention is a mathematical operation, rather than a tiny reader choosing passages consciously. During causal processing, each token position can use earlier positions and itself. Its representation is therefore shaped by the sequence available up to that point. 4
The answer grows a token at a time
At the output stage, the model produces scores for possible next tokens. These can be converted into a probability distribution. A decoding procedure selects a continuation, adds it to the sequence and repeats the process. 45
If an answer begins “A bicycle stays,” its next step depends on both the original request and that beginning. The model continues using the expanding text. This repeated generation is what “autoregressive” means. It can support long explanations because the computation draws on learned patterns and the available context at every step. 5
Eventually, generation reaches a stopping condition, such as an end marker or an output-length limit. The token sequence is decoded back into readable text. An app may stream pieces as they become available, making the answer appear to be typed. That display behaviour does not imply a complete paragraph was waiting somewhere to be revealed. 5
Sampling influences which continuation appears
The highest-scoring next token is one possible choice. Greedy decoding takes that choice at each step. Sampling instead draws from a distribution of possible continuations, allowing different outputs from the same input. 6
Temperature changes how concentrated that distribution is. Lower positive values concentrate probability more strongly on high-scoring candidates; higher values spread it more widely. Top-p sampling limits candidates to a set accounting for a chosen cumulative probability. Different systems expose different controls, and some choose them automatically. 6
These controls influence variation, not whether a statement has been verified. A low-variation answer can still be wrong. Once one selected token differs, later predictions also receive a different sequence, so a small initial difference can grow into a substantially different paragraph.
Training and answering are different jobs
During training, a model's parameters, often called weights, are adjusted using examples and an objective. Language-model pretraining commonly includes learning to predict text continuations. At ordinary inference time, those learned parameters are used to process your input and generate output. They are usually not rewritten after each message. 7
Providing examples in a prompt can still change the answer. Show a few sentences rewritten in your preferred style and the model can use that pattern in the current context. This is in-context learning. The GPT-3 research demonstrated task adaptation through text examples without updating the model's parameters for those tasks. 7
Assistant behaviour also reflects later training. Instruction examples and feedback about preferred answers can shape how a model responds to requests. The InstructGPT research provides an influential example of supervised instruction training followed by reinforcement learning from human feedback. This improves behaviour without making every output correct. 8
An assistant may produce fluent text that adds an unsupported detail or accepts a mistaken premise. Helpful style and factual reliability are separate properties to assess. A plausible quotation still needs a source; a plausible calculation still needs checking. 8
Retrieval and tools add another loop
A model can generate from its learned parameters and supplied context without consulting a live information source. Access to current documents or external systems requires additional machinery.
Retrieval-augmented generation, often shortened to RAG, combines generation with information retrieved from an external collection. A search component identifies relevant material, and selected passages help condition the answer. The original RAG research combined a learned language model with a searchable Wikipedia index. The broader pattern can also support document collections. 9
Retrieval makes evidence available; it does not guarantee the best passage was found or correctly interpreted. When an answer cites a document, check that the cited passage actually supports the claim.
Tools add executable operations. A model might return a structured request for a calculation, database lookup or search. The surrounding application or service runs the operation and passes the result back. The model can then use that result in its next response. 10
For example, “What is tomorrow's weather?” may trigger a weather service call before any answer is composed. The returned forecast is new context. Typing the words “I checked the weather” would not itself perform that lookup. The capability comes from the connected system and its actual execution. 10
A useful prompt reduces missing decisions
Suppose you want a summary of notes about a school garden. “Make this better” leaves several decisions open. Better for whom? Shorter, clearer or more persuasive? Should the answer preserve every detail or choose the most important ones?
A more informative request might say, “Using the notes below, write a 120-word update for parents. Explain what the pupils planted, identify the next activity and keep proposals separate from completed work. If a date is missing, say so.”
This provides the task, audience, source material, length and an uncertainty rule. You can also give an example of the desired style. These additions change the context the model uses, while the underlying model remains the same. 17
Then evaluate the result against the job. Did it preserve the facts? Did it turn a proposed activity into a completed one? Does the word count fit? Precise instructions make the target clearer, but they are not an automatic enforcement mechanism. Revision remains useful when a constraint matters. 8
Image generation can follow a different process
A text prompt can also condition an image generator, but generating a picture need not mean choosing the next text token repeatedly.
In diffusion-based image synthesis, generation commonly starts from noise and applies learned denoising steps guided by conditions such as text. Latent diffusion performs this work in a compressed image representation, then decodes the result into pixels. The prompt influences what emerges during generation. 11
Flow matching is another framework. It trains a model to describe how samples should move along paths between distributions, such as from noise towards image data. Generation follows the learned vector field numerically. Diffusion and flow methods have mathematical connections, but “remove noise” is not a complete explanation of every flow formulation. 12
The important distinction is the generated representation and the process that updates it. A chat interface may pass your request to a specialised image model, so the same Send button can initiate a different kind of computation.
Questions readers often ask
Does the model remember everything in the conversation?
Only the material made available in its current context can directly inform that model call. Applications can preserve summaries or retrieve earlier information, but a long visible conversation is not proof that every original token remains in context. 2
Does correcting an answer immediately retrain the model?
Usually, the correction becomes context for the next response. That can change the answer without changing the model's weights. Actual model training is a separate operation. 7
Why can the same prompt produce different answers?
Sampling can select different continuations. The effective input may also differ if conversation history, retrieved material or app instructions change. Identical visible wording therefore need not mean identical model input or identical output. 16
Sources and further reading
Specifications, research papers and official documentation support the technical explanations. The examples and diagrams are original editorial material. Research checked on 23 October 2026.
- Hugging Face — Chat templates — Chat templates — Explains message roles, control tokens and the conversion of a chat into a token sequence.
- Anthropic — Context windows — Context windows — Documents the bounded context, accumulated messages, tool results and context-management behaviour.
- Hugging Face — Tokenization algorithms — Tokenization algorithms — Explains subword tokenization, vocabularies and tokenizer-dependent text segmentation.
- Vaswani and colleagues — Attention Is All You Need — Attention Is All You Need — Original transformer paper covering embeddings, positional information, attention and next-token probabilities.
- Hugging Face — Text generation — Text generation — Documents autoregressive generation, decoding token identifiers and stopping conditions.
- Hugging Face — Generation — Generation — Reference for greedy decoding, sampling, temperature and top-p controls.
- Brown and colleagues — Language Models are Few-Shot Learners — Language Models are Few-Shot Learners — Demonstrates in-context task examples without task-specific gradient updates or fine-tuning.
- Ouyang and colleagues — Training language models to follow instructions with human feedback — Training language models to follow instructions with human feedback — Research on instruction training, preference feedback and remaining factual or behavioural mistakes.
- Lewis and colleagues — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Foundational research combining language generation with an external searchable knowledge index.
- Anthropic — Tool use with Claude — Tool use with Claude — Documents structured tool calls, external execution and tool results returned to the model.
- Rombach and colleagues — High-Resolution Image Synthesis with Latent Diffusion Models — High-Resolution Image Synthesis with Latent Diffusion Models — Describes text-conditioned diffusion within a compressed image representation.
- Lipman and colleagues — Flow Matching for Generative Modeling — Flow Matching for Generative Modeling — Introduces flow matching and its relationship to paths connecting noise and data distributions.