How Does ChatGPT Work? From Training to Web Search and Citations

Knowing how ChatGPT works explains both its strengths and its errors, and tells you what makes a website likely to be cited in its answers. OpenAI launched ChatGPT in November 2022; its name comes from GPT, Generative Pre-trained Transformer.

What is the model behind ChatGPT?

ChatGPT runs on a large language model (LLM). An LLM is a statistical model with billions of parameters that has learned the relations between words, facts and styles from text. It holds patterns learned from those texts and uses them to produce plausible continuations of any text; individual pages are not kept as such.

The architecture is the transformer, introduced by Google researchers in the paper Attention Is All You Need in 2017. Its key mechanism, attention, lets the model weigh every word of the input against every other word, so it understands that "it" in a sentence refers to a noun three lines earlier.

How was ChatGPT trained?

1. Pre-training

The model reads a very large corpus of text (public web pages, books, code, licensed data) and learns one task: predict the next token. Repeated billions of times, this simple task forces it to learn grammar, facts, reasoning patterns and many languages. The result is a powerful text predictor that does not yet follow instructions well.

2. Supervised fine-tuning

People write examples of good answers to prompts, and the model is trained to imitate them. It learns the format of a helpful assistant: answer the question, structure the reply, ask for clarification when needed.

3. Reinforcement learning from human feedback

Reviewers compare several answers to the same prompt and rank them. A reward model learns those preferences, and the language model is then optimized to produce answers that score well. OpenAI described this method, called RLHF, in its InstructGPT research paper. It is what makes ChatGPT polite, structured and more careful about refusing harmful requests.

What happens when you send a message?

Tokens and the context window

A token is a chunk of text: in English, one token is on average about four characters, or three quarters of a word. The context window is the number of tokens the model can consider at once, including your conversation and any documents or web pages it reads. When a conversation becomes too long, the oldest parts fall out of view.

Generation word by word

At each step, the model computes a probability for every possible next token and picks one, with a controlled share of randomness. That is why the same question can get slightly different answers, and why ChatGPT writes fluently even when it is wrong: it optimizes for a likely continuation, without checking facts against a source.

How does ChatGPT search the web and cite sources?

A model only knows what was in its training data, up to a cutoff date. To answer about recent events, prices or products, ChatGPT can run a web search. OpenAI announced this feature as ChatGPT search in October 2024. The assistant rewrites your question into one or several queries, retrieves results from search providers and from OpenAI's own crawler, reads the pages, and writes an answer with links to its sources.

This method is called retrieval-augmented generation (RAG): the model generates text grounded in documents it has just retrieved. OpenAI documents its crawlers on its developer platform: OAI-SearchBot for search, GPTBot for training, and ChatGPT-User for pages opened on behalf of a user. OpenAI states that sites blocking OAI-SearchBot in robots.txt are not shown in ChatGPT search answers.

What are ChatGPT's limits?

• Hallucinations: it can state false facts, invented quotes or non-existent sources with confidence, because it predicts plausible text.
• Knowledge cutoff: without web search, it ignores everything after its training data.
• Context limits: very long documents or conversations exceed what it can consider at once.
• Sensitivity to wording: a slightly different prompt can produce a different answer, or a different list of recommended brands.
• Source selection: when it searches, it reads only a handful of pages, so being among them matters.

What does this mean for getting cited by ChatGPT?

Understanding the pipeline gives a clear checklist for any brand that wants to appear in ChatGPT answers:

1. Let the crawlers in: allow OAI-SearchBot in robots.txt, and serve content in HTML that works without heavy JavaScript.
2. Rank in classic search: ChatGPT search starts from search results, so pages that rank well on the underlying queries are more likely to be read.
3. Write extractable passages: a direct answer under each heading, short self-contained paragraphs, definitions, tables and sourced figures.
4. Be present where it reads: reviews, comparison articles, forums, media and Wikipedia shape what the model says about your brand, both in training data and in live search.
5. Stay consistent: the same name, description and facts across your site and third-party profiles help the model identify your brand as one entity.
6. Measure: ask the questions your customers ask and see whether you are cited. The free ChatGPT visibility checker gives a first answer in a minute.

ChatGPT predicts text from what it learned and, for fresh questions, from the few web pages it chooses to read: the brands it cites are those whose pages are easy to find, easy to extract and confirmed by other sources. To follow your presence in ChatGPT, Gemini and Claude over time and publish content built to be cited, Sorank tracks your prompts and writes optimized articles on autopilot, with a 3-day free trial.