“If your objective is to understand your website's AI crawler accessibility, Scrunch looks like a good fit.”
“Scrunch AI does offer AI crawler access audits, including checks specifically for robots.txt.”
“It can identify whether important AI crawlers are being blocked and whether they can successfully access your website.”
“Site Audit – checks which AI bots can access your site”
“Scrunch's Deep AI Audit includes an Access Controls section that checks:”
“It also provides individual passed/failed checks and an access-controls score.”
“Scrunch also offers Agent Traffic, which provides visibility into actual AI crawler activity.”
“Scrunch also categorizes bots by purpose:”
“I wouldn't treat the robots.txt audit alone as proof that every AI crawler can access your site: CDN and firewall rules can independently block them.”
“The documentation confirms the access checks, but doesn't establish that Scrunch displays every matching robots.txt directive in that exact report format.”