Ikibazo gihuza ibibazo · Biragoye Urwego
Ibiciro bya Hallucination mubibazo byemewe bya AI
Gerageza uburyo ubushakashatsi bwa Stanford bwasobanuye kandi bupima imitekerereze ya AI yemewe n'amategeko, ibisubizo bidahwitse nibyo, n'impamvu ibisekuruza byongerewe imbaraga birananirana.
Ikibazo 1 Bya 8
Ubushakashatsi bwa Stanford "Hallucination-Free?" bwasobanuye iki igitekerezo cy'ibinyoma?
Reba buri kibazo8 ibibazo
Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.
- Ubushakashatsi bwa Stanford "Hallucination-Free?" bwasobanuye iki igitekerezo cy'ibinyoma?
- Any answer longer than the source document
- An answer that is either incorrect or misgrounded
- Only a citation to a case that does not exist
- Any answer the tool took more than a minute to produce
- Muri amagambo y'ubushakashatsi bwa Stanford, igisubizo 'kitari ku nkingi' ni iki?
- An answer that refuses to respond
- An answer written in the wrong language
- An answer with no citations at all
- An answer that cites a real source that does not support its claim
- Ni iki gikoresho cyagaragaje umusaruro mwiza mu igerageza rya Stanford rya 2024 ariko kigakomeza gukora ibitekerezo bitari ukuri ku bisubizo birenga 17% by'ibibazo?
- Lexis+ AI
- Westlaw AI-Assisted Research
- Ask Practical Law AI
- GPT-4 used as a comparison
- Ni uruhe rugero rw'ibitekerezo bitari ukuri "Large Legal Fictions" yasanze ku biganiro by'ubwenge bw'ikoranabuhanga rusange ku bibazo by'amategeko y'inkiko za leta bishobora kwemezwa?
- 1 to 3 percent
- 17 to 33 percent
- 69 to 88 percent
- 95 to 100 percent
- Kuki intambwe yo gushakisha ishobora kubona ububasha butari bwo n'ubwo ububiko bw'amakuru bukubiyemo ubwo bukwiye?
- Legal databases are encrypted and cannot be searched
- Retrieval only works for statutes, not cases
- Retrieval ranks documents alphabetically
- Retrieval matches similar wording or meaning, which is not the same as legal relevance
- What tendency did the research find when questions contained a false premise?
- Models tended to accept the premise and answer as if it were true
- Models always corrected the premise
- Models refused every such question
- Models replied only with the text of the relevant statute
- Which mitigation specifically helps flag a cited case that was later reversed or overruled?
- Raising the model's temperature setting
- Making answers shorter
- Translating the query into another language
- Connecting a citator such as KeyCite or Shepard's
- What marketing claim did the Stanford study's title directly question?
- That legal AI tools were cheaper than paralegals
- That some tools were 'hallucination-free'
- That legal AI could pass the bar exam
- That AI tools were approved by courts