Ururimi AI

Ibiciro bya Hallucination mubikoresho byemewe bya AI

Ibizamini byigenga byagaragaje ko ibikoresho by'ubushakashatsi bw'amategeko byifashisha AI byubatswe ku buryo bwa retrieval-augmented generation (RAG) bikiri gutanga ibisubizo bitari byo cyangwa bidafite ishingiro kenshi.

Umva iki gitaboAI-byavuzwe · ~3 min

Kuriyi page4 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Imbere y'igipimo cy'ibinyoma mu bikoresho bya AI by'amategeko
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Ubushakashatsi bwa Stanford bwa 2024 bwagaragaje hallucinations mu bisubizo birenga kimwe mu bitandatu n'ubwo igikoresho cy'ubucuruzi gikora neza cyane. Ibi bifite akamaro kuko abacuruzi batangaza ko ibi bikoresho bidakora hallucinations, kandi abavoka babishingiraho bakiri inshingano ku citation bose basaba.

Kwibira cyane

Two Stanford-affiliated studies shaped the debate. The first, "Large Legal Fictions" (2024), by Matthew Dahl, Varun Magesh, Mirac Suzgun and Daniel Ho, tested general-purpose chatbots on verifiable questions about federal court cases. It found hallucination rates between 69 and 88 percent on those questions. Models also tended to accept false premises built into the questions. Commercial legal vendors argued that their tools avoided this problem through retrieval-augmented generation, which grounds answers in real legal databases. Some marketing used phrases such as "hallucination-free." In 2024, researchers at Stanford's RegLab and Institute for Human-Centered AI tested that claim in a study titled "Hallucination-Free?" They ran more than 200 legal queries through Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI, with GPT-4 as a comparison. They counted an answer as a hallucination if it was either incorrect or misgrounded. Misgrounded means it cited a real source that does not support the claim. Lexis+ AI performed best, yet still hallucinated on more than 17 percent of queries. Westlaw's tool hallucinated on roughly a third. Ask Practical Law AI often gave incomplete answers. An early version of the paper was criticized because the researchers first lacked access to Westlaw's research tool; they later tested it and revised the paper. Retrieval reduces errors but does not remove them, for several reasons. The retriever may find documents that share words with the query but are not legally relevant. It may miss the controlling authority, or pull a case from the wrong jurisdiction or a superseded version of a rule. The model may then misread or ignore what was retrieved, blend sources together, or go along with a false premise. A common misconception is that a correct-looking citation means a correct answer. The misgrounded category shows that a real citation can support a false claim.

Gushyira mu bikorwa Isi

Umukozi asaba igikoresho cy'ubushakashatsi impamvu urukiko rwafashe icyemezo runaka mu rubanza rwari rwagombaga kugenda mu rundi ruhande. Igikoresho cyemera igitekerezo kibi kandi gisobanura rulling y'ibinyoma, kigaragaza ibitekerezo nyakuri.

Igikoresho gisubiza urubanza rw'inkiko rw'ukuri rufite isoko nyayo y'icyifuzo urwo rubanza rudakora. Ibi ni igisubizo 'misgrounded' kigaragara nk'icyizewe kugeza umuntu asomye ibitekerezo.

Igikoresho cy'ubushakashatsi kigaragaza icyemezo cyahinduwe nyuma, kuko cyakuyeho ibitekerezo bya kera kandi kitarebye amateka yacyo nyuma.

Ishami ry'amategeko rigerageza abacuruzi babiri ku bibazo byaryo 100 bifite ibisubizo bizwi. Ryandika niba buri citation ihari, niba ishyigikira ikirego, kandi niba igisubizo ari cyuzuye.

Imbere y'igipimo cy'ibinyoma mu bikoresho bya AI by'amategeko

Abacuruzi bavuguruye ibicuruzwa byabo kuva mu bushakashatsi bwa 2024, bityo ibipimo by'ikosa byihariye bishobora kutagaragaza verisiyo z'ubu. Ni yo mpamvu kugerageza kwigenga kenshi kurusha umubare umwe ari ngombwa kurushaho. Imirimo yo gupima ibipimo ikorerwa n'impande zindi n'amashuri y'amategeko ubwabo iragenda iba ihuriweho cyane. Gukoresha ibikoresho byiza byo gushakisha, imirimo isa n'iy'abakozi ibaza ku nkomoko zayo, no gushyira imizi ikomeye bishobora kugabanya amakosa. Nta kintu kugeza ubu kigaragaza ko uburyo bwo kubonera amakuru buzabikuraho burundu. Inkiko zikomeza kubona inyandiko zifite ibimenyetso by'ibinyoma cyangwa byatanzwe nabi, kandi zimwe zashyizeho ibihano. Mu gihe kizaza, isuzuma ry'umuntu kuri buri cyemezo cy'ibimenyetso biracyari inshingano y'umwuga, icyo umucuruzi yavuga cyose.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Umuvuduko n'igipimo. Ururimi rwakazi rushobora kugenda byihuse nta gutamba guhuzagurika.
  • Kugera no kugera. Yagura uburyo bwindimi nuburyo bwo gutumanaho.
  • Ibyemezo bisobanutse. Amakipe arashobora kumara umwanya munini murubanza mugihe automatike ikora gusubiramo.

Witondere

  • Ibintu bifatika bishobora kwinjiza bucece raporo, gushyigikira imigendekere, cyangwa ibisubizo byubushakashatsi.
  • Kwihuta byihuse birashobora gukora ibisubizo bidahuye mubisabwa bisa.
  • Ibyanditswe byumvikana birashobora kugaragara niba kugenzura kugenzura ari ntege.

Intambwe ikurikira

  1. Sobanura imiterere isohoka, amajwi, hamwe nubuziranenge mbere yo gutangira.
  2. Ibisubizo byibanze hamwe nisoko yizewe igihe cyose ukuri kwingirakamaro.
  3. Komeza kugenzura abantu kugenzura ibisubizo byinshi.
  4. Kurikirana uburyo bwo kunanirwa no kongera imyitozo cyangwa akazi gahoraho.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Hallucination Rates mu Bikoresho bya AI mu Mategeko ni iki?

Ibizamini byigenga byagaragaje ko ibikoresho by'ubushakashatsi bw'amategeko bya AI bikoresha retrieval-augmented generation (RAG) bikomeza gutanga ibisubizo bitari ukuri cyangwa bidafite ishingiro kenshi. Ubushakashatsi bwa Stanford bwakozwe mu 2024 bwagaragaje ibitekerezo by'ibinyoma mu bisubizo birenga kimwe kuri buri bisubizo bitandatu n'ubwo ari igikoresho cy'ubucuruzi gikora neza kurusha ibindi. Ibi bifite akamaro kuko abacuruzi bamamaza ibi bikoresho ko bitazazana ibitekerezo by'ibinyoma, kandi abavoka babikoresha bagomba kwitondera buri cyemezo bose batanga.

Ubushakashatsi bwa Stanford "Hallucination-Free?" bwasobanuye iki igitekerezo cy'ibinyoma?

Ubushakashatsi bwabariyemo ubwoko bubiri bw'ikosa: ibivugwa bidahwitse, n'ibivugwa bifite icyemezo kitagaragaza koko ko kibishyigikira.

Muri amagambo y'ubushakashatsi bwa Stanford, igisubizo 'kitari ku nkingi' ni iki?

Igisubizo kitari ku nkingi kigaragaza ububasha nyakuri, ariko ubwo bubasha ntibusaba ibyo igisubizo kivuga. Ibi bituma kigaragara nk'icyizewe kurusha uko kiri.

Ni iki gikoresho cyagaragaje umusaruro mwiza mu igerageza rya Stanford rya 2024 ariko kigakomeza gukora ibitekerezo bitari ukuri ku bisubizo birenga 17% by'ibibazo?

Lexis+ AI yagaragaje ibisubizo byiza, ariko ikomeza gukora ibitekerezo bitari ukuri ku bisubizo birenga 17% by'ibibazo. Igikoresho cya Westlaw cyakoze ibitekerezo bitari ukuri hafi kimwe cya gatatu.

Ni uruhe rugero rw'ibitekerezo bitari ukuri "Large Legal Fictions" yasanze ku biganiro by'ubwenge bw'ikoranabuhanga rusange ku bibazo by'amategeko y'inkiko za leta bishobora kwemezwa?

Ubushakashatsi bwagaragaje ko ibiganiro rusange by'ubwenge bw'ikoranabuhanga byakora ibitekerezo bitari ukuri hagati ya 69% na 88% by'igihe ku bibazo byihariye bishobora kwemezwa.

Kuki intambwe yo gushakisha ishobora kubona ububasha butari bwo n'ubwo ububiko bw'amakuru bukubiyemo ubwo bukwiye?

Abashakisha bapima ukugendana mu magambo cyangwa mu bisobanuro. Kunanirana, incamake y'argument y'ikindi ruhande, cyangwa urubanza ruvuye mu karere kitari ko, bishobora guhabwa amanota menshi batari ububasha bw'ingenzi.