You blocked GPTBot in robots.txt to keep your content out of training runs — and then wondered why ChatGPT Search never cites your docs. Those are different systems. OpenAI’s own crawler docs say each bot is independent: allowing OAI-SearchBot keeps you eligible for ChatGPT search answers while disallowing GPTBot opts you out of foundation-model training.
This article is a Next.js App Router playbook for Generative Engine Optimization (GEO): how answer engines discover pages, which user-agents actually matter, how to configure robots.ts and sitemaps, how to keep HTML crawlable, and what llms.txt does — and does not — guarantee.
How LLMs find your site
Treat “appearing in an LLM” as three separate pipelines:
Pipeline






