Key takeaways
- An agent can only repeat facts that exist as text. Prices in images and hours in pop-ups do not count.
- A robots.txt rule that blocks GPTBot or ClaudeBot keeps those agents from fetching the public pages.
- An /llms.txt file is the one-page brief: who you are, what you offer, what it costs, and how to start.
- A Site Scan shows the gaps. Asking ChatGPT about your own homepage shows what a customer would hear.
“Can they read it?” is the worry. “How do I get them to see it?” is the job. You do not need a new website. You need the facts an assistant can fetch, quote, and trust.
Can AI Agents Read Your Website walks through what an agent actually receives. The steps below are how you change that result.
Put the facts in real text
An agent keeps headings, paragraphs, lists, and links. It drops scripts, styles, and most images. If the price sits in a slider, the hours sit in a pop-up, or the menu is a PDF, those facts never arrive.
Write the offer in a sentence a stranger could repeat. “Emergency call-out, weekdays 7am to 7pm, from $149” survives the fetch. “Solutions that scale” does not tell anyone what you sell.
- Services, named, with who they are for
- Price, or an honest range, in HTML
- Hours and service area
- A phone number, email, or booking URL in the text
Let the answer agents in
Access comes before reading. If robots.txt disallows GPTBot, ClaudeBot, or Google’s crawlers, those agents never fetch the page. A public homepage that is blocked on purpose is invisible on purpose.
Allow the public pages. Keep admin, account, and draft paths closed. Best Practice robots.txt for the AI Age has the pattern, including a line that can point crawlers at your llms.txt file.
Publish one brief at /llms.txt
The pages can be correct and still be expensive to read. Menus, cookie copy, and repeated slogans burn tokens before the agent reaches the offer. Many of them stop and answer from an older source.
An AI Website Profile at /llms.txt is a plain-text brief written for that pass: who you are, what you offer, what it costs, where you are, and how to start. The site customers see stays as it is. The agent reads one file instead of reconstructing the business from banners.
Check what an agent actually gets
Open ChatGPT with browsing on and ask for your services, prices, hours, and service area. The blanks and the wrong details are the same ones a customer hears.
A Site Scan does the same check from the agent’s side: llms.txt, sitemap, structured data, crawler access, and how many tokens the homepage costs. Fix the first failure it names, then ask the assistant again.