Guide-linked quiz · Medium Level
Robots.txt and AI Crawlers Quiz
Check how robots.txt groups crawler rules, how AI crawler tokens differ, and when site owners need search controls or real access protection.
Question 1 of 8
A publisher wants ChatGPT search discovery but excludes possible training crawls. Which distinction in OpenAI documentation matters?
Keep testing yourself
More quizzes picked for your level.