Browse documentation
Daily workflowsUpdated April 24, 2026

How to audit your website for AI crawlability

Use the Website page to check whether PromptScout can reach, parse, and evaluate evidence-first signals that affect how AI systems understand your site.

PromptScout website audit preview showing site-level AI readiness checks

What the Website page checks

Whether your site is reachable, parse-able, and cite-able for AI systems. Think of it as an evidence-first crawlability check for AI search before you judge Monitoring or Sources.

What it tests

  • Domain reachable
  • Pages discovered
  • robots.txt, sitemap.xml, and llms.txt status
  • Metadata and machine-readable signals
  • Pass/fail checks per page
  • Prompt-aware coverage evidence where monitored prompts and owned pages line up

When to run a scan

  • After any meaningful site change — redesign, migration, CMS update.
  • When monitoring results look weak or inconsistent.
  • Before assuming a visibility problem is a content problem.

How reruns work

Website audits are deterministic: a rerun records what PromptScout can fetch and evaluate at that moment. It does not guess why a model cited or ignored a page.

  • Free teams keep the onboarding bootstrap audit as their Website state.
  • Paid teams can run dashboard full-site audits under the current daily limit.
  • Focused page audits have their own per-page daily limit and do not replace the latest full-site state.
  • Older runs stay available as evidence, so fixes can be compared against later scans.

How to use the results

  1. Fix foundational blockers first — reachability, wrong domain, missing files.
  2. Then check whether crawlable pages are also cited pages in Sources.
  3. Re-run Website after fixes, then re-run Monitoring to see if visibility moves.

This sequence stops you from optimizing pages AI still can't reach and helps the rest of your workflow stay grounded in real site conditions.

Keep these files healthy

  • robots.txt — don't accidentally block AI crawlers.
  • sitemap.xml — fresh and complete.
  • llms.txt — describe the pages you want AI systems to prioritize.

Next steps

  • Check citation coverage in Sources.
  • Re-run Monitoring after fixes.
  • Use Content when Website gaps and Insights point at the same page opportunity.

Related docs and next steps

Put this into practice

Use this workflow in PromptScout

Keep reading the docs, then run the workflow on your own brand data when you are ready.

Getting started guide

Move from setup to your first AI Visibility Check quickly.

PromptScout blog

Go deeper with practical AEO, GEO, and AI visibility playbooks.

Continue learning

  • Sources

    Documentation

    Use the Sources page to see which websites, documents, and communities AI systems cite when they recommend brands in your category.

  • Monitoring

    Documentation

    Use the Monitoring page to run monitors, inspect recent runs, and compare how your AI visibility changes across providers over time.

  • Brand

    Documentation

    Use the Brand page to control how PromptScout identifies your company across AI responses and to keep the monitored brand record accurate.

  • AI Citation & Source Tracking

    Guide

    See which sources AI assistants cite and where your authority can improve.

  • PromptScout Content Briefs Guide

    Guide

    Learn when PromptScout can turn monitor, Website, Insights, and source evidence into a saved content brief.

  • AI Brand Monitoring Platform

    Guide

    Track how ChatGPT, Gemini, Google AI Overviews, and Perplexity mention your brand over time.