llms.txt Best Practices: The Most Common Mistakes and How to Avoid Them

The 5 most common mistakes

Most llms.txt files fail not because of technical issues – but because of content. These five mistakes come up most frequently:

Mistake #1

Too vague and non-specific

Descriptions like "We are an innovative company that develops solutions for our customers" provide no concrete context and could apply to thousands of websites.

→ Be specific: What exactly do you do, for whom, and with what result?

Mistake #2

Incorrect or outdated information

An llms.txt with outdated product names, old prices or discontinued services is worse than no llms.txt – if an AI system evaluates the file, incorrect information will be passed on.

→ Update llms.txt with every major change, and specify the date.

Mistake #3

Wrong encoding or broken special characters

Special characters not saved in UTF-8 cause encoding problems that impair the readability of the file.

→ Always save the file as UTF-8 without BOM and check with the validator.

Mistake #4

Sensitive information in llms.txt

Internal price lists, employee data, unpublished products or confidential business information do not belong in llms.txt – the file is public and accessible to everyone.

→ Only include publicly available information intended for users.

Mistake #5

Incorrect Content-Type output

If the web server does not deliver llms.txt as text/plain, the file may not be processed correctly. A common issue with CMS systems in particular.

→ Check after deployment: curl -I yourdomain.com/llms.txt

What belongs in – and what does not

Belongs in

A good llms.txt answers the basic questions about your website clearly and precisely:

Clear identity: Name of the website/organisation and what it does – in one sentence
Concrete offering: What specific products, services or content is available?
Target audience: Who is the website made for?
Key URLs: Links to the central pages – not all of them, just the most important
Contact: A reachable email address
Language(s): In what language is the content available?

✓ Good: "llmshub.de is a German-language platform for llms.txt and LLM visibility. We offer free tools for creating and validating llms.txt files as well as guides on GEO and AI crawlers."

✗ Bad: "Welcome to our website. We are experts in our field and help our customers to be successful."

Does not belong in

Advertising copy and marketing language: "Market leader", "unique", "revolutionary" – llms.txt is for factual information, not advertising messages
Complete page texts: That is what llms-full.txt is for – llms.txt is a structured overview, not a content dump
Technical implementation details: Which CMS is used or how the server is configured
Repetitions from robots.txt: Crawling rules belong in robots.txt, not in llms.txt
SEO keywords without context: A list of keywords is not explanatory text

Language and tone

llms.txt should be written in clearly structured, natural language. Not like a database schema, but not like an advertising text either.

Precision beats length

Two precise sentences about the actual offering are more valuable than a paragraph of vaguely formulated company philosophy.

✓ Precise: "AI-Ready Check prüft kostenlos ob eine Website technisch für KI-Suchmaschinen wie ChatGPT und Perplexity optimiert ist. Das Tool analysiert 20 Faktoren und gibt einen Score von 0–100."

✗ Vague: "Wir helfen Unternehmen dabei ihre digitale Präsenz zu verbessern und in der modernen KI-getriebenen Welt sichtbar zu bleiben."

Use the language of your audience

Write the llms.txt in the main language of your website and target audience. If your website is in English, write the llms.txt in English.

Tip: Write the llms.txt as if you were explaining to someone in two minutes what your website is and why it matters. No jargon, no filler – direct and concrete.

File size and performance

The llms.txt should stay lean. A rule of thumb: under 10 KB for llms.txt, under 100 KB for llms-full.txt.

What unnecessarily bloats the file size

Complete product descriptions or blog articles – these belong in llms-full.txt
Long lists of URLs – only link to the 5–10 most important pages
Redundant information that repeats itself
Binary characters or faulty encoding artefacts

Content-Type and encoding

The server must deliver the file correctly. This can easily be tested:

curl -I https://yourdomain.com/llms.txt

# Expected output – Content-Type must be text/plain:
HTTP/2 200
content-type: text/plain; charset=utf-8

Keeping it up to date

An outdated llms.txt is an underestimated risk: the file should reflect the actual current state of the website. Outdated information can – if evaluated by systems – lead to incorrect information being passed on.

When to update?

New products or services are launched
Existing offerings are discontinued or renamed
Contact details change
The target audience or positioning changes
Important new pages are created that should be linked

Specify the date

A ## Last Updated section helps assess how current the information is. Format: YYYY-MM or YYYY-MM-DD.

Warning: Do not use the date as decoration – if the update date is two years ago but the products have changed since then, the file no longer reflects reality.

llms-full.txt – when it makes sense

llms-full.txt is the optional extension to llms.txt: it contains the complete content of the most important pages. The difference lies in purpose:

llms.txt: Structured overview – who you are, what you offer, how to reach you
llms-full.txt: Complete content – all relevant texts in one file

When is llms-full.txt worth it?

llms-full.txt makes sense when your website has substantial content:

Documentation pages and knowledge bases
Extensive guides and tutorials
FAQ collections with many entries
Product databases or catalogues

For simple company websites or landing pages, llms-full.txt is a nice-to-have, not a necessity. An outdated llms-full.txt is worse than none at all.

Keeping size under control

llms-full.txt should stay under 500 KB. What goes in: the most important pages in full text, neatly structured with Markdown headings. What does not go in: navigation, footer, cookie banners, code snippets.

Best practices checklist

These points distinguish a good llms.txt from a very good one:

Description is concrete and specific – no generic marketing speak
File is accessible at yourdomain.com/llms.txt (HTTP 200)
Content-Type is text/plain; charset=utf-8
Encoding is UTF-8 without BOM – special characters are displayed correctly
File size under 10 KB
No sensitive or internal information included
Only the 5–10 most important pages linked – no complete URL list
Last update date is current and correct
Language matches the main language of the website
Content is updated when major website changes occur
Technically checked with the llms.txt Validator

Check your llms.txt now

The llmshub.de Validator shows in seconds whether your llms.txt is technically correct and where there is still room for improvement.

Validate llms.txt now →