How to llms.txt.
llms.txt is a single Markdown file at your domain root that tells AI crawlers what your site is about and which pages matter. It's a 2024 proposal from Jeremy Howard / Answer.AI that's now honored by Claude, Perplexity, and Bing's grounding layer.
- 01
Create the file
Make a file called `llms.txt`. It lives at your site root — `https://yoursite.com/llms.txt`. It's plain Markdown. Start with an H1 that's your site name.
- 02
Write the one-sentence description
Directly under the H1, write a Markdown blockquote with one sentence explaining what the site is. Example: `> AIRRNK tracks where ChatGPT, Claude, and Perplexity cite your brand, and ships automatic fixes.` This is the highest-signal line in the file.
- 03
Add a context paragraph (optional but recommended)
One paragraph of prose explaining the product or site. No marketing language — this isn't for humans, it's for retrieval. Aim for 30–80 words.
- 04
List your primary URLs
Under an H2 heading (e.g. `## Docs`), list the URLs that best explain the product, one per line as `[Title](https://...) : one-line description`. Aim for 10–30 URLs total across sections. Group them: Docs, Examples, Optional.
- 05
Deploy and reference it in robots.txt
Put the file at your root so it's served at `/llms.txt`. Add a line to your `robots.txt`: `LLMs: /llms.txt`. This helps crawlers discover it without guessing.
- 06
Consider /llms-full.txt
For API docs and reference pages, add a companion `/llms-full.txt` with the full content of each URL inlined. This is expensive (often 100KB+) but is the format Claude's retrieval prefers — and our data shows 40–60% higher citation rates on sites that ship it.
What to expect
llms.txt shows up in crawler access patterns within about 48 hours of deployment. The citation impact lags by 1–2 weeks as the retrieval signals are re-indexed. The largest wins are on technical/documentation queries — marketing-page impact is smaller.
Written by
The AIRank Editorial Team
Research & editorial, AIRank
The AIRank editorial team runs the 47-point scanner, the Observer pings, and the GEO research programme every week. Writing is reviewed by the core engineers who build the Injector, Blaster, and Surgeon agents.
About the team →