Ikibazo gihuza ibibazo · Hagati Urwego

Impamvu AI Yanze Gusaba Ikibazo

Menya kwangwa kubeshya, gusobanura imiterere yemewe, no kubungabunga imipaka yumutekano utagerageje kurenga kubirinda ibyifuzo bibi.

Inzira zijyanye no kuyoboraKurenga Kwangwa Mubiganiro bya AI
Ikibazo 1 Bya 8

How would you define over-refusal?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. How would you define over-refusal?
    • Answering a harmful request fully
    • Declining a benign request because it is incorrectly treated as unsafe
    • Asking a clarifying question
    • Refusing a request that would enable serious harm
  2. What was the scope of OR-Bench’s 2024 evaluation?
    • Every user conversation across all current chatbots
    • Only 25 harmless prompts
    • A test of image-generation quality
    • 80,000 seemingly toxic benign prompts, a hard subset, toxic controls, and 25 models across eight families
  3. Why should an OR-Bench refusal rate not be treated as universal?
    • It depends on benchmark categories, model versions, and how prompts were labeled
    • The benchmark contains no prompts
    • All models have identical safety behavior
    • Refusals cannot be counted
  4. What may help when a legitimate request is misunderstood?
    • Claiming the model must ignore all safeguards
    • Hiding the real purpose
    • Clarifying the benign goal, context, and safe boundaries
    • Adding instructions to bypass policy
  5. Is the goal of over-refusal mitigation to answer every request?
    • Yes, all refusals are errors
    • No; the system should reduce false refusals while still refusing harmful assistance
    • Yes, if a user says it is safe
    • No, because all benign requests should be refused
  6. Why test benign and harmful prompts that share vocabulary?
    • It checks whether the model uses context rather than only surface words
    • It guarantees zero false refusals
    • It tests response formatting only
    • It removes the need for safety tests
  7. What should a product team measure along with false refusals?
    • Only average answer length
    • Only positive user ratings
    • The number of words in the policy
    • Unsafe compliance on genuinely harmful requests
  8. If a request remains unsafe after clarification, how should a user proceed?
    • Use a jailbreak to force an answer
    • Repeat the request with coded wording
    • Accept the boundary and ask for a safe alternative
    • Assume every refusal is mistaken