Guide-linked quiz · Medium Level

Cloudflare AI Crawler Blocking and Pay Per Crawl Quiz

Understand AI Crawl Control, its block and charge actions, the Pay Per Crawl handshake, beta limits, and policy interactions.

Question 1 of 8

A site uses Cloudflare AI Crawl Control to block a listed crawler. Where is the decision enforced?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. A site uses Cloudflare AI Crawl Control to block a listed crawler. Where is the decision enforced?
    • At Cloudflare’s network/security layer for traffic routed through its service
    • Only inside the crawler’s voluntarily read robots.txt rules
    • In a browser cookie stored on the visitor’s device
    • In the search result title
  2. A crawler receives HTTP 402 and a crawler-price header. What does this response indicate?
    • The page was successfully delivered and charged
    • Payment is required for the requested content under the configured policy
    • The site has removed the page from search
    • The crawler has been authenticated as a human
  3. What confirms successful paid content delivery in the documented flow?
    • A robots.txt Allow line
    • HTTP 402 with no payment intent
    • HTTP 200 with a crawler-charged amount
    • A browser user-agent string alone
  4. Why can’t a site owner assume Pay Per Crawl will charge every bot?
    • HTTP 402 itself transfers payment
    • All bots pay automatically when a price exists
    • Robots.txt collects the fee
    • The crawler must support the payment exchange and be compatible/verified
  5. An owner sets a crawler to Charge but a WAF rule blocks it. What happens?
    • Both policies are combined into a double charge
    • The block overrides the charge action
    • The crawler is allowed without payment
    • The sitemap receives the charge instead
  6. How does Cloudflare currently describe Pay Per Crawl availability?
    • It is available only through robots.txt
    • It is mandatory on every website
    • It is a closed beta, so availability and terms may change
    • It guarantees revenue as soon as enabled
  7. A publisher considers putting a search crawler in the Charge group. Which risk does the guide identify?
    • Blocking or charging them may affect indexing and search visibility
    • Search bots cannot request public pages
    • Pricing automatically improves ranking
    • Charging disables all other crawler controls
  8. Before turning on a broad crawler policy, which first action follows the guide?
    • Assume user-agent text proves identity
    • Block every unfamiliar user-agent immediately
    • Forecast revenue from total page views
    • Review observed crawler identities and request patterns, then test a small scope

Keep testing yourself

More quizzes picked for your level.

View all quizzes