quiz bu lëkkale ak tegtal · Diggante Niveau

Robots.txt and AI Crawlers Quiz

Check how robots.txt groups crawler rules, how AI crawler tokens differ, and when site owners need search controls or real access protection.

Laaj 1 de 8

A publisher wants ChatGPT search discovery but excludes possible training crawls. Which distinction in OpenAI documentation matters?