Best Practices for Creating an llms.txt File

Search engines and AI assistants may feel similar to the end user, but the economics and responsibilities behind them are worlds apart.

When you search Google for “best running shoes for marathon runners”, the result is a ranked list of pages: reviews, blogs, product sites. Google isn't actually answering the question. It's showing you who else has talked about it. Whether the advice is fresh, contradictory, or flat wrong isn't Google's problem. Their job ends at the list.

That model is efficient. Google crawls sites occasionally - sometimes daily for big publishers, sometimes only a few times a year for smaller ones. It stores what it finds in its index, and when you search, it cheaply retrieves and re-ranks that cached data. Crawl occasionally, rank cheaply, display a list.

AI assistants can't operate that way.

When you ask an AI “What are the best running shoes for marathon runners?” you're not expecting a reading list. You're expecting a direct answer:

“The top marathon shoes right now are the Nike Vaporfly 3, ASICS Metaspeed Sky, and Saucony Endorphin Elite - each suited to different runner types.”

That makes the assistant responsible for the answer, not just the sourcing. And that responsibility carries a huge cost. Instead of pointing you to 20 sites, the model has to parse them, compare information, resolve contradictions, and generate a final statement. Do that 10,000 times a week, and the economics collapse.

This is the problem llms.txt is designed to solve.

Why llms.txt matters

llms.txt is not just schema or a sitemap under a new name. Those formats were built for the search model, where crawling is occasional and the output is a list.

llms.txt is built for the answer model, where AI assistants need a compact, authoritative, machine-friendly file they can trust - without re-parsing your site every time. It's how a business can say: here is the canonical truth about who we are, what we sell, and how to talk about us.

That makes it valuable for both sides:

• Businesses: ensure assistants stop hallucinating and start using your official words, facts, and product lists.
• AI systems: reduce redundant crawling, cut compute costs, and deliver faster, more accurate answers.
• Users: get trustworthy responses, not vague summaries stitched from the internet.

What an AI Website Profile should include

At Platinum.ai, we build what we call AI Website Profiles - structured llms.txt files that give assistants everything they need. A good profile includes:

• Human-readable summary - a plain-language bio of your company.
• Identity - name, domain, language, industry, official contacts.
• Explicitly missing info - stating what isn't published (e.g., “pricing not listed”) so AI doesn't invent.
• Official descriptions - taglines, product blurbs, and the wording you use on your site.
• Usage guidance - rules for how the file should be used (e.g., “only cite these facts and linked pages”).
• Provenance and maintenance - version number, last review date, update frequency.
• Context - services, product categories, unique differentiators, awards.
• People - key staff, leadership, or spokespeople (only public-facing).
• Customer actions - how users should engage (contact, purchase, book, etc.).
• Machine-friendly JSON - the entire profile encoded for fast parsing and caching.

Back to our running shoe example

Imagine a specialist running store publishes an AI Website Profile in llms.txt. Instead of AI scraping dozens of review blogs, it can pull directly:

• “We are a specialist retailer for distance running shoes, curated for marathon and ultramarathon training.”
• Top marathon shoes in stock, with attributes (weight, drop, cushioning, stability).
• Official contact info and policies.
• Usage guidance telling AI not to invent missing performance data.

So when users ask “What are the best running shoes for marathon runners?”, the assistant can give a clean, authoritative answer straight from the source:

“According to XYZ Running Store's AI Website Profile, their top marathon shoes are the Nike Vaporfly 3 (lightweight, carbon-plated for racing), the ASICS Metaspeed Sky (responsive cushioning for distance), and the Saucony Endorphin Elite (stability-focused).”

That's the difference. The assistant isn't guessing. It isn't improvising from outdated blogs. It's citing a verified profile.

The shift

If robots.txt was the infrastructure for the search era, llms.txt is the infrastructure for the AI era. The shift from list-maker to answer-maker changes the economics, the expectations, and the risks. Businesses that embrace it will be the ones whose voice gets carried accurately into the AI-driven web.