Dzokera kuNhau
InnovationAI Understanding muchidimbu

Werewolf benchmark inowana maLLM anogona kuwandisa vapomeri vakavimbika

Mucherechedzo wemakumi mana akavhurika-huremu LLM masisitimu akawana kuti kupomera kunogona kushandura mamodheru anotenda kunyangwe mupomeri achienderana neanopikisa.

4 min readRead the primary source
Source-provided image accompanying Werewolf benchmark finds LLMs can overvalue trusted accusers
Primary-source documentKwakanyorwa
Muparidzi
arxiv.org
Source link
arxiv.orghttps://arxiv.org/abs/2609.12446
Source type
Gwaro rekutanga - chiziviso chepamutemo, bepa, faira, kana peji rebato rekutanga ratinoverenga zvakananga.
ContextNzwisisa izvi mumasekonzi makumi matanhatu

Tanga pano

Matemu akakosha

Benchmark
Muedzo wakamisikidzwa kana dhatabheti rinoshandiswa kuyera nekuenzanisa kuita kwemuenzaniso.
Mutauro Mukuru (LLM)
Mutauro wemodhi yakadzidziswa pane yakakura text corpora kugadzira nekuongorora zvinyorwa.
Annotation
Mazita akawedzerwa nevanhu kana metadata inoshandiswa kudzidzisa kana kuongorora mhando dzekudzidza dzemuchina.
Zviedze iwe pachakoAI Models Inotsanangurwa Mibvunzo

Chii chaitika

Vatsvagiri vakaunza Werewolf bhenji inoyera kuti kucherechedza maLLM anovandudza sei zvavanotenda mushure mekugamuchira fungidziro uye meseji yekupomera. Kupfuura 1,224 mameseji akatsanangurwa uye makumi mana akavhurika-huremu magadzirirwo emodhi, mamodheru akakura aiwanzo ratidza mapere echokwadi kubva munhoroondo yemutambo asi akaramba achikanganiswa zvakanyanya nekupomerwa uye kufungidzirwa kuvimbika kwemupomeri.

Bepa rinokurudzira kuongorora kuchinja kwekutenda pane kuvimba chete nemhedzisiro yemutambo wekubvisa magariro. Mukugadzirisa kwayo, modhi inocherekedza yemusha inogamuchira mameseji emitambo, kusanganisira kufungidzira uye kupomerwa, uye vaongorori vanoyera kuti zvavanotenda zvinoshanduka sei mushure memeseji yega yega.

Iyo abstract mishumo inobuda kubva ku40 yakavhurika-huremu LLM zvigadziriso uye 1,224 mameseji akatsanangurwa. Mienzaniso mikuru yakaita zvirinani pakusiyanisa makudo kubva kuvagari vemumusha vachishandisa nhoroondo yemutambo izere. Zvakadaro, kupomerana kwakaramba kuchiwedzera kufungirana kune ari kupomerwa uye kuderedza kunyumwira kune ari kupomera, kunyanya apo mupomeri akange atovimbwa.

Nzira yakataurwa yakaramba iripo kunyange apo mupomeri aivimbwa naye akanga akabatana nemapere. Mahombe modhi aikwanisa kuramba kupomerwa nevapomeri vavasina kuvimba navo kare, asi modhi dzinosvika mabhiriyoni zana nemakumi maviri dzichiri kunetseka kubatanidza zviri mupomero nekuvimbika kwekwakabva. Kwarinobva hakuna akadzama pa-modheni zvibodzwa, kusava nechokwadi kwenhamba, kana kudzokorora kwakazvimiririra mune zvinyorwa zvinopihwa.

Kwakabva mashoko: arxiv.org ↗

Nei zvichikosha

Chidzidzo chinoratidza hutera chaihwo mukutaurirana kwakarongeka: mhando dzinogona kusapatsanura zvakakwana kuvimbika kwemutauri kubva kune humbowo huri mukutaura kwemutauri. Izvo zvine basa kune vamiririri veLLM vanoshanda muzvirongwa umo mameseji anogona kusarudza, kunyengera, kana mhandu. Mhedzisiro iyi ibhenji uye kutsvaga kutsvaga, kwete humbowo hwekuti ese akaiswa AI masisitimu anoita nenzira iyi kana kuti ongororo yakawedzera kupfuura Werewolf.

Kune vaongorori vanoongorora vamiririri veLLM, mhedzisiro inoratidza kuti budiriro yemutambo wekupedzisira inogona kuvanza kusasimba kwakakosha mukufunga kwepakati nekuvandudza kutenda. Mutevedzeri unogona kusvika pasarudzo inonzwisisika uchiri kuwandisa uyo akapa chikumbiro pane kuongorora chikumbiro pamwe chete nehumbowo huripo.

Zvazvinoreva zvinogumira asi zvinobatsira: ongororo yevamiririri vanotaurirana kana kuita sarudzo kubva mumishumo yakawanda ingangoda kuyera shanduko yekutenda mushure memeseji yega yega, kusanganisira zviitiko apo mutauri anovimbika ari kurasisa. Chidzidzo ichi hachiratidze kuti maitiro akafanana anoitika mune zvigadzirwa zvakatumirwa kana mune zvisiri zvemitambo marongero.

Interactive Mechanism

Interactive Mechanism: Iyo Inonyatsoshanda

Ongorora ari pasi tekinoroji kuseri kwekusimudzira uku uchipindirana.

Thinking Budget (Test-Time Tokens):1,024 tokens
Complex Accuracy79%Math & Code Logic
Latency3.2sTime to first full output
Inference Cost$0.0092Per query estimated
Reasoning StyleStep VerificationInternal chain depth
Active Thinking Trace:
1Deconstruct user problem into formal constraints
2Propose candidate hypotheses & step-by-step calculation
3Self-correction: Backtrack and refute subtle edge cases
4Exhaustive consistency check & final output synthesis
Core takeaway: Test-time compute fundamentally changes AI economics. Instead of only scaling during pre-training, giving reasoning models more tokens at inference time allows them to systematically solve PhD-level STEM problems.
Interactive Concept Check+10 Points
AI Models Explained Quiz

Which component of an AI application is the machine-learning model itself?

Zvekutarisa zvinotevera

Iyo uye kodhi inowanikwa kuburikidza nesaiti yeprojekiti, asi sosi yacho haitaure marezinesi, ruzivo rwekutambira, kana kuti mhando dzakaongororwa dziripo kuti dzishandiswe neveruzhinji. Rimwe basa rinofanirwa kuyedza kana kutsvaga kuchiendeswa kune mamwe mabasa ekutaurirana, kana kudzidziswa kunovandudza tsime-zvirimo kufunga, uye kuti yakawanda sei mibairo inoenderana nekugadzirwa kwemutambo uye sarudzo dzezvirevo.

Vanyori vanoti bhenji uye kodhi inowanikwa pane yakabatana purojekiti peji. Nzvimbo yakapihwa hainyore zvinodikanwa zvekuwana, kupihwa rezenisi, mitengo, masisitimu anotsigirwa, kana kuti inosanganisira yemhando yezvinobuda kupfuura meseji yakanyorwa.

Zvinokosha zvisingazivikanwe zvinosanganisira kuti mameseji akagadzirwa sei uye akatsanangurwa, kuti kuvimba kwakamiswa sei, kana mhedzisiro yacho yakasimba pane imwe neimwe mhando, uye kana mamodheru akakura 'akavandudzwa emutambo-nhoroondo kufunga anoshandura mukuramba kuri nani kunyengedza.

Kudzokorodza kune mamwe mabasa ekutaurirana kwaizobatsira kuona kana iri Werewolf-chaiyo mhedzisiro kana mhedziso yakafara mukuti vamiririri veLLM vanosanganisa sei kutendeseka kwenzvimbo nezviri kupomerwa.

Related guides & Quizzes

AI Models InotsanangurwaAI AgentsTsika dzeAIEdza zvaunoziva - edza yemahara AI quizTarisa kumusoro izwi reAI mune yedu glossaryTevedza iyo AI modhi yekuburitsa tracker
Wakawana izvi zvinobatsira?