What we observe
For the contact, booking and checkout pages we read, we look at what our request without a browser received there, when that page was in our sample of up to five: the page, a challenge or a refusal. A challenge on a page robots.txt asks AI crawlers to stay away from matches the owner’s own choice and is not counted.
Why it is here: the evidence
Documented A platform’s own published document or code says this is read: a standard the crawlers follow, or the documentation or source code of the company that runs the system.
Source: Cloudflare’s verified-bots documentation: its edge can challenge or turn away an automated visitor on any page. Source grade A: read at the source: the document, the code or the study itself.
Last re-checked ; due again within 30 days.
What we cannot see
What a named AI crawler, or a crawler a firewall can verify, receives there; we send only our own name. A page outside the sample has no reading.
How to fix it
If automated visitors are meant to reach these pages, set the edge or firewall to serve them there; most offer an allowance for verified bots.
- Layer
- Transact: Can an automated visitor reach contact, booking or checkout, and use the form there?
- Level
- One of the checks of Level 2: Is what it reads about your business the same everywhere, and can it find where to act?
- How common it is
- Not published yet. We publish a share only once at least 30 sites stand behind it.
- Check id
fact:path-challenge
Cite as: Invara, “How AI sees your site”, method version 1, check fact:path-challenge. https://clearsitedigital.com/how-ai-sees-sites/fact-path-challenge