Technical GEO checklist: make your site readable for AI search engines

Your site is technically ready for AI search engines when crawlers such as OAI-SearchBot, Claude-SearchBot and PerplexityBot are allowed and able to fetch your pages, when your full text is in the HTML your server sends, and when your server responds quickly and reliably. Structured data, metadata and monitoring complete it. This checklist goes through every item, in the order in which you best work through them.

Some reassurance up front: Google says there are no extra technical requirements to appear in AI Overviews or AI Mode. Your page must be indexed and eligible to show with a snippet; good technical SEO is the foundation. The difference with classic SEO is mainly in the other AI search engines: they do not run JavaScript and have their own bots you need to let in separately.

Each part is explained in two deeper articles: how AI crawlers read your site (robots.txt, rendering, llms.txt) and the guide to JSON-LD and schema markup.

1. Access: are AI bots allowed to fetch your site?

Without access, the rest of the list does not count. First check that your robots.txt and firewall do not hold the AI bots back.

• Search bots are allowed: OAI-SearchBot (ChatGPT), Claude-SearchBot (Claude), PerplexityBot (Perplexity), Googlebot and Bingbot.
• User bots are not blocked: ChatGPT-User, Claude-User and Perplexity-User fetch a page when someone asks for it in a conversation.
• A deliberate choice about training: GPTBot, ClaudeBot, Google-Extended, Applebot-Extended and CCBot. Blocking is a choice about training, not about visibility in AI search. Note: Google-Extended also covers grounding in the Gemini apps.
• No separate Allow blocks that undo your exclusions. A bot only follows the block that best matches its name, not the * block as well.
• Firewall and CDN let AI bots through. Cloudflare has blocked AI crawlers by default on new domains since July 2025; check security plugins too.
• Sitemap is listed in robots.txt.
• Robots.txt tested in the robots.txt report in Search Console after every change.
• AI Overviews and AI Mode: if you do not want to be in them, use the generative AI control in Search Console, not robots.txt.

Explanation, the current bot names and two ready-made robots.txt variants are in setting up robots.txt per AI bot.

2. Rendering: is your content in the HTML?

AI crawlers only read the HTML your server returns. What only appears after JavaScript runs, GPTBot, ClaudeBot and PerplexityBot do not see.

• Important pages use SSR, SSG or ISR, not pure client-side rendering.
• Main text is in the source: “View page source” (Ctrl+U) or curl shows your text, not just an empty <div id="root">.
• Title, meta description, canonical and Open Graph are in the HTML and not set later by JavaScript.
• JSON-LD is in the HTML, not only injected by a script.
• No text behind “Loading…”, tabs or buttons that only fetch their content after a click.
• Images may lazy load, text may not.

# Is a sentence from your main text in the HTML a crawler receives?
curl -s -A "GPTBot/1.0" https://yourdomain.com/page | grep -i "a sentence from your text"

The comparison between CSR, SSR, SSG and ISR and what it means per platform (WordPress, Next.js, Nuxt, Astro) is in why AI crawlers do not see your JavaScript.

3. HTML structure: is your page easy to parse?

A crawler must be able to separate your main text from navigation and footer. Semantic HTML makes that easy.

• One <h1> per page, with <h2> and <h3> below it without skipping levels.
• Main content in <main> and <article>, navigation in <nav>, side content in <aside> and <footer>.
• Important content as text, not only in images, video or iframes.
• Every section starts with an answer sentence that makes sense without the rest. You do not need to chop your text into small pieces; Google calls that unnecessary. Clear sections do help visitors and systems that split your text.
• Tables and lists where you really compare or enumerate.

More on structure in content chunking for AI citations.

4. Structured data: are your facts right in machine-readable form?

Structured data makes your facts unambiguous and eligible for rich results in Google. There is no proven effect on AI citations: Ahrefs saw no clear positive effect across 1,885 pages. Do it right, but do not expect miracles.

• JSON-LD instead of Microdata or RDFa.
• Organization and WebSite site-wide, with a fixed @id.
• Article on blog posts, with author as an object, datePublished and dateModified.
• Product with Offer on product pages: price, currency, availability.
• BreadcrumbList where you have a hierarchy.
• FAQPage only with a visible FAQ. Google has not shown FAQ rich results since May 7, 2026; the markup may stay.
• Everything in the schema is also visible on the page.
• Validated with the Rich Results Test and the Schema Markup Validator, and checked in Search Console.

Examples per type, the status of FAQ rich results and a validation workflow are in the JSON-LD guide.

5. Speed and Core Web Vitals: does your server respond in time?

Speed does not get you cited more often, but a slow or faltering server causes missed crawls. No AI model favors fast sites; the link runs through two routes:

• Reachability. A crawler that gets no response, or a late one, does not fetch your page. What is not fetched cannot be cited. AI crawlers do not load images or scripts, so for them the server response and the size of the HTML matter most.
• Google rankings. Google uses Core Web Vitals in its ranking systems, and AI Overviews and AI Mode build on that same index. Bing, the foundation of Copilot, also weighs speed.

The thresholds for a good score, measured for 75% of your visitors (web.dev):

Metric	What it measures	Good	Poor
LCP (Largest Contentful Paint)	How fast the largest element is visible	2.5 s or less	More than 4 s
INP (Interaction to Next Paint)	How fast the page responds to a click or tap	200 ms or less	More than 500 ms
CLS (Cumulative Layout Shift)	How much the layout shifts while loading	0.1 or less	More than 0.25

INP replaced the old FID metric in March 2024. TTFB (Time to First Byte) is not a Core Web Vital, but for crawlers it is the most important: as a guideline, 0.8 seconds or less.

• LCP, INP and CLS in the green in PageSpeed Insights or the Core Web Vitals report in Search Console.
• TTFB 0.8 seconds or less on your most important pages.
• Caching on the server or through a CDN, so pages are not rebuilt for every request. Static generation (SSG) or ISR is faster here than SSR.
• Compression (gzip or Brotli) for HTML, CSS and JavaScript.
• Images in WebP or AVIF, with width and height (against CLS) and loading="lazy" only below the fold, not for your hero image.
• No unnecessarily heavy HTML from large inline scripts or styles.
• Server handles peaks: no 5xx errors or timeouts in your logs.

Measuring: PageSpeed Insights gives both lab data and field data from real Chrome users. Lighthouse in Chrome DevTools helps find causes. Whether crawlers suffer from your speed, you only see in your server logs (see part 8).

6. Sitemap, canonicals and redirects: does everything point to the right URL?

Duplicates and dead links cost crawls and confuse every system that has to choose which version to cite.

• XML sitemap with all indexable pages, at most 50,000 URLs per file.
• Real lastmod dates, only updated on a substantive change. Google ignores changefreq and priority.
• Sitemap submitted in Google Search Console and Bing Webmaster Tools.
• Canonical on every page, absolute and to itself or the preferred version.
• 301 redirects for moved or merged pages, no redirect chains.
• No internal links to 404s. In an analysis by Vercel and MERJ, 34.8% of ChatGPT’s requests ended on a 404 page, against 8.2% for Googlebot.
• No URL parameters that create duplicate pages.

7. Metadata and language: is every page unambiguous?

• Unique title per page, around 50 to 60 characters.
• Unique meta description, around 140 to 160 characters.
• Open Graph tags: og:title, og:description, og:image, og:type.
• <html lang="en"> on English pages.
• Hreflang on multilingual sites, with reciprocal references and an x-default.
• Publication and update dates visible on articles, matching your schema.

How to write the title and description per page is in meta description and SEO title. For images, read how to write good alt text.

8. Monitoring: do you know it keeps working?

Technical setups usually break during a deploy, not on their own. So check after every major change and keep an eye on a few signals.

• Server logs reviewed for OAI-SearchBot, ChatGPT-User, Claude-SearchBot, ClaudeBot, PerplexityBot and GPTBot.
• Status codes for bots: mostly 200; 403 points to your firewall, 5xx or 408 to a server that cannot keep up.
• Rendering tested after every framework or theme update.
• Structured data tested after every template change, preferably automatically in your build process.
• Robots.txt maintained: new bot names appear regularly.
• Citations followed: are your brand and pages mentioned in ChatGPT, Perplexity and AI Overviews?

# Which AI bots visited and which status codes did they get?
grep -E "OAI-SearchBot|ChatGPT-User|GPTBot|ClaudeBot|Claude-SearchBot|PerplexityBot" access.log \
  | awk '{print $9}' | sort | uniq -c

9. Optional: llms.txt

llms.txt is a Markdown file with an overview of your site for language models. It is cheap and does no harm, but Google does not use it for Search, and Ahrefs saw that 97% of files received not a single request in a month.

• If you use it: only existing pages, an accurate description per link, updated with every change.

More in llms.txt: what it is and what it does not do.

The checklist in short

Use this overview as a fixed check after every major change.

Access

• Search and user bots allowed in robots.txt
• Deliberate choice about training bots
• Firewall and CDN let AI bots through
• Sitemap in robots.txt

Rendering and HTML

• Main text, metadata and JSON-LD in the server HTML
• Semantic HTML with one H1 and logical headings
• Sections open with a standalone answer sentence

Structured data

• Organization, WebSite, Article and Product where needed
• Schema matches visible text and is validated

Speed

• LCP, INP and CLS in the green
• TTFB 0.8 seconds or less
• Caching, compression and light images

URLs and metadata

• Current sitemap, canonicals and 301 redirects
• No internal links to 404s
• Unique titles and descriptions, correct language

Monitoring

• AI bots in the server logs with status 200
• Check after every deploy
• Citations followed

Getting started

Start with parts 1 and 2: if AI bots are not allowed or able to read your page, nothing else matters. The technical scan runs through the most important items on your own domain.

Prefer not to do this by hand every time? GreatCompany checks your site continuously and also measures every day whether ChatGPT and other AI models mention your business. See pricing.

Read also

• How AI models choose their sources
• Tracking ChatGPT traffic in Google Analytics
• AI search, privacy and data protection