Cyber SEO: 50% off
Preferences

Privacy is important to us, so you have the option of disabling certain types of storage that may not be necessary for the basic functioning of the website. Blocking categories may impact your experience on the website. More information (Cookie Policy)

Accept all cookies

Technical GEO: The 2026 Checklist to Get Your Site Read and Cited by AI

Technical GEO in 2026: AI crawler access, server rendering, llms.txt, structured data, speed and freshness, with a checklist to get cited by AI.

Published on
28/10/2026
Updated on
28/10/2026
Reading time
9 min
Cover of the article “Technical GEO”, with a screenshot of the Sorank software next to the title

Tools

On this page

In short

Technical GEO is the set of site settings that let AI assistants such as ChatGPT, Claude, Perplexity and Gemini fetch, read and quote your pages. It comes down to six checks: let the AI search crawlers in (robots.txt, firewall, CDN), serve your main content in the initial HTML because most AI crawlers do not execute JavaScript, keep pages fast and error-free, add structured data that matches the visible text, publish an llms.txt file as a low-cost extra, and signal freshness with accurate dates and sitemaps. None of this guarantees a citation, but any failure on these points removes your page from the candidates.

Want to go further? Ask

Quick quiz

How should you get mentioned by ChatGPT and other AI assistants?

Question 1 of 425%
What do you want to know or change?

Technical GEO is the infrastructure side of generative engine optimization: everything that happens before an AI model judges whether your content is worth citing. Content and authority decide who gets quoted, but a page that an AI crawler cannot reach, or reaches as an empty JavaScript shell, never enters the race. Google itself sums up the baseline for its AI features: a page must be indexed and eligible for a snippet, and there are no additional technical requirements for AI Overviews or AI Mode. The other AI assistants run their own crawlers, with their own limits, and that is where most sites lose visibility without knowing it. This guide covers each layer, from bot access to freshness, with a checklist you can run in an afternoon.

What is technical GEO, and how is it different from technical SEO?

Technical SEO makes a site crawlable and indexable for Googlebot. Technical GEO extends the same work to a dozen other bots that behave differently: they render less, crawl less often and follow fewer rules. The overlap is large (status codes, canonicals, sitemaps, speed), so a site with clean technical SEO already has most of the foundation. The differences sit in four places.

TopicTechnical SEO (Googlebot)Technical GEO (AI crawlers)
Bots to manageGooglebot, BingbotOAI-SearchBot, GPTBot, ChatGPT-User, Claude-SearchBot, ClaudeBot, Claude-User, PerplexityBot, Google-Extended and others
JavaScriptRendered, with a delayNot executed by most AI crawlers
Unit of valueThe page and its rankingThe passage that gets quoted in an answer
Opt-out logicBlock or noindex a pageSeparate choices for training, search and user-triggered visits
MeasurementSearch ConsoleServer logs and prompt tracking

The practical consequence: a site can rank well on Google and still be invisible to ChatGPT or Claude, because the content is built in JavaScript or because a firewall rule blocks unknown bots.

Which AI crawlers should you allow in robots.txt?

Each AI company now runs several bots, and each one has a different job. Blocking the wrong one can remove you from AI answers while you only meant to opt out of model training.

OpenAI

According to OpenAI's crawler documentation (checked October 2026), OAI-SearchBot decides whether your pages appear in ChatGPT search answers, GPTBot collects content that may be used to train its models, and ChatGPT-User fetches a page when a user asks for it. Disallowing GPTBot keeps your content out of training without removing you from ChatGPT search. Changes to robots.txt take about 24 hours to apply to search results, and robots.txt rules may not apply to ChatGPT-User because the visit is triggered by a person.

Anthropic

Anthropic follows the same three-way split, as its help center explains: ClaudeBot for training, Claude-SearchBot for search indexing and Claude-User for pages fetched during a conversation. Blocking Claude-SearchBot or Claude-User reduces your visibility in Claude's answers. Anthropic also warns that blocking its IP addresses is not a reliable opt-out, since the bot may then fail to read your robots.txt at all.

Google and the others

Google's AI Overviews and AI Mode use the regular Googlebot index. The Google-Extended token only controls the use of your content for Gemini training and grounding in some other Google systems, and Google states that it does not affect inclusion or ranking in Google Search. Perplexity uses PerplexityBot for its index. For a ready-made configuration per bot, see our guide to robots.txt rules for AI crawlers and the list of AI crawlers and their user agents.

GoalAllowOptional to block
Be cited in ChatGPTOAI-SearchBot, ChatGPT-UserGPTBot (training only)
Be cited in ClaudeClaude-SearchBot, Claude-UserClaudeBot (training only)
Be cited in PerplexityPerplexityBotNone
Appear in AI Overviews and AI ModeGooglebotGoogle-Extended (no effect on Search)

Watch out

Robots.txt is only half the check. Since July 2025, Cloudflare blocks AI crawlers by default on many configurations, and other firewalls and bot-protection tools do the same. Look at your CDN and security settings, then confirm in your server logs that the AI search bots receive a 200 status code.

Can AI crawlers read JavaScript content?

Mostly no, and this is the most common technical GEO failure. A Vercel analysis of its network (December 2024) found that none of the major AI crawlers rendered JavaScript: the bots of OpenAI, Anthropic, Meta, ByteDance and Perplexity download the raw HTML and stop there. ChatGPT and Claude did fetch JavaScript files but did not execute them. Gemini is the exception because it relies on Googlebot's rendering infrastructure. The same study measured the scale of this traffic: in one month, GPTBot made 569 million requests and Claude 370 million, against 4.5 billion for Googlebot.

If your product descriptions, prices, articles or FAQ are injected by a React, Vue or Angular app after page load, AI crawlers see an empty template. The fix is to serve the essential content in the initial HTML response:

  • Server-side rendering (SSR) or static generation for every page you want cited: main text, headings, tables, prices, author and dates.
  • Pre-rendering for older single-page apps that cannot move to SSR quickly.
  • Client-side rendering only for secondary elements: chat widgets, counters, personalization.

A quick test: open the page source (not the inspector) or fetch the URL with curl. If your main paragraph is missing from the raw HTML, AI crawlers cannot quote it. Our guide to JavaScript SEO covers the rendering options framework by framework, and the free AI crawlability audit checks what a bot receives.

How do speed, status codes and HTML structure affect AI citations?

AI assistants that browse in real time fetch several pages per question and work under tight time limits. A slow or broken page is skipped and another source is quoted. The Vercel study also found that ChatGPT and Claude crawlers hit 404 pages in about 34% of their fetches, against 8% for Googlebot, often by requesting outdated URLs. Three points matter most:

  1. Status codes: key pages return 200, redirects take a single hop, deleted pages that still receive bot requests are redirected to their closest equivalent.
  2. Response time: a fast server response and a light HTML document. Large pages filled with inline scripts push the useful text further down the file.
  3. Semantic HTML: real headings (h2, h3), paragraphs, lists and tables. Crawlers and models split pages into passages along these tags, so a clear structure produces clean, self-contained passages that can be quoted as they are.

Tables deserve a special mention: comparison data in an HTML table is easy to extract, while the same data in an image or a canvas chart is lost for every AI crawler.

Does structured data help with AI visibility?

Structured data helps machines identify what a page is about (an article, a product, an organization, a person) and connect it to known entities. Google is clear that no special schema is needed for its AI features, and that markup must match the visible text. Its value for technical GEO is clarity, not a ranking boost. The types worth implementing on most sites:

  • Organization with sameAs links to your official profiles, so models link your brand name to one entity.
  • Article or BlogPosting with author, datePublished and dateModified.
  • Product and Offer for e-commerce pages, with price and availability that match the page.
  • BreadcrumbList to describe the site structure.

Keep JSON-LD in the initial HTML, not injected by a tag manager, for the same JavaScript reason as above. The structured data audit for LLMs checks your markup against what a model can read, and our article on structured data for AI search details which types still matter.

Tip

Write the same key facts (company name, founding year, prices, locations) identically in your visible text, your schema and your official profiles. Consistent facts across sources make it easier for a model to describe your brand without mixing it up with another one.

Is llms.txt worth adding?

The llms.txt proposal, published by Jeremy Howard in September 2024, is a Markdown file at the root of a site that lists its most useful pages with a short description, so that an AI agent finds the right content without parsing navigation menus. It is a proposal and not a formal standard, it controls neither access nor indexing, and Google does not list it among the requirements for its AI features. Treat it as a low-cost extra: it takes less than an hour, it helps agents and coding assistants that look for it, and it forces you to decide which pages represent your site. You can draft one with the llms.txt generator, and offer clean Markdown versions of key pages if your stack allows it.

How do you signal freshness to AI assistants?

AI assistants that search the web favor recent and dated sources for any question where time matters (prices, tools, rules, statistics). Content freshness is partly an editorial matter, but the signals are technical:

  • A visible last updated date on the page, matching dateModified in the schema.
  • An XML sitemap with accurate lastmod values, changed only when the content really changes.
  • IndexNow pings for Bing and the other engines that support it, since several AI assistants rely on the Bing index; Google does not use IndexNow.
  • Stable URLs: updating an article at the same address keeps its links and history, while a new URL starts from zero and leaves a 404 behind.

The technical GEO checklist

CheckHow to verifyPriority
AI search bots allowed in robots.txtRead robots.txt for OAI-SearchBot, Claude-SearchBot, PerplexityBotHigh
No firewall or CDN blockServer logs show 200 responses for AI botsHigh
Main content in raw HTMLView source or curl the URLHigh
Key pages indexed by Google and BingSearch Console and Bing Webmaster ToolsHigh
200 status, single-hop redirectsCrawl with an SEO crawlerMedium
Semantic headings, lists and tablesRead the HTML outlineMedium
Organization and Article schemaSchema validatorMedium
Visible dates and accurate lastmodCompare page, schema and sitemapMedium
llms.txt at the rootOpen /llms.txtLow

Run it once on your 20 most important pages, then after every redesign, CMS change or security update. AI crawler logs are the only direct proof that the bots reach your pages; everything else is inference.

How do you know if technical GEO is working?

Technical fixes open the door; the result shows up in AI answers. Two measurements close the loop. First, server logs: visits from OAI-SearchBot, Claude-SearchBot and PerplexityBot should rise on the pages you fixed. Second, citations: track a fixed set of questions your customers ask and check, week after week, whether your brand is named and which sources are quoted instead. Patterns from large samples help to set expectations, as in our ChatGPT citation study.

Sorank, our software, handles that second part with AI mention tracking: 10 to 50 tracked prompts depending on the plan, on ChatGPT, Claude and Gemini, plus Grok from the Pro plan, with your visibility rate and the competitors cited in your place.

Sorank AI mention tracking dashboard showing the brand visibility rate and its share by ChatGPT, Gemini and Claude

Once the technical base is clean, the next limit is content: pages that answer questions clearly, with sources. The article generator writes and publishes optimized articles for Google and AI on autopilot, with internal links and sources.

Technical GEO decides whether AI assistants can read your site; content and authority decide whether they quote it. Fix access, rendering and structure first, then measure citations. Plans and the 3-day free trial are on the pricing page.

Frequently asked questions

Should I block GPTBot to protect my content?

Blocking GPTBot only tells OpenAI not to use your content for training its models. It does not remove your site from ChatGPT search answers, which depend on OAI-SearchBot. If you want to be cited in ChatGPT, keep OAI-SearchBot allowed whatever you decide for GPTBot.

Does Google use llms.txt for AI Overviews?

No. Google states that AI Overviews and AI Mode have no technical requirements beyond being indexed and eligible for a snippet in Search. llms.txt can help other AI agents find your key pages, but it has no effect on Google rankings or AI Overviews.

What is the difference between technical GEO and AEO?

AEO (answer engine optimization) and GEO describe the same goal: being quoted in answers generated by AI assistants. Technical GEO, sometimes called technical AEO, covers the infrastructure part of that work: bot access, rendering, speed, structured data and freshness. The content and authority parts come on top of it.

Free SEO & GEO audit

Get your content plan for 30 days for free

We analyse your competitors, your website and your AI visibility to create a tailored content plan for free.

Our free SEO tools

See all our free tools

Get a free audit of your site

THEY TRIED SORANK BEFORE YOU

Testimonials from ambitious founders

Organic growthE-commerce

0 to 10000 visitors in 5 months

Celio Fabianomonrobotlavevitre.fr
Organic growthSoftware

0 to 5000 clients in 8 months

Thibault BMsorank.com
Organic growthLeisure and aviation

0 to 1104 clicks in 4 months

Dominiqueflight-sensations-idf.com
Organic growthLead generation for installers

x98 clicks in 2 months

LeadBolt Agencyleadbolt.online
Organic growthMusic software

x55 clicks in 5 months

Valentin Cordiertiger-sounds.com