InoteveraGaidhi rinotevera
SWE-bhenji uye Coding Agent Benchmarks
Tekinoroji
Nhungamiro yehunyanzvi
Kuongorora maAI vamiririri kunoreva kuyera kuti yakawanda-nhanho system inosvika sei pazvinangwa mumamiriro echokwadi.
Teerera kugwaro iriAI-yakarondedzerwa · ~3 min
Izvo zvinobatanidzira kana yakapedza basa, kana nzira yayakatora yaive ine musoro, kana yakashandisa maturusi nemazvo, kuti yaidhura zvakadini, uye kana yakaramba yakachengeteka. MaAgents anoita pamusoro pematanho akawanda uye ane maside effects, saka kutarisa mhinduro imwe chete uchipesana nereferensi hakuna kukwana, uye ongororo dzakanaka dzinowanzo mhanyisa mumiriri mukati menzvimbo inodzorwa uye kuongorora mamiriro ekupedzisira.
Kuongororwa kweagent kunotarisa zvinhu zvakati. Kubudirira kwebasa kunobvunza kana chinangwa chakanyatsozadzikiswa, chakatariswa nekutarisa nharaunda mushure: bvunzo dzinopfuura, rekodhi chaiyo iripo, faira rine zviri mukati. Trajectory quality inobvunza kana nhanho dzacho dzaive nemusoro, pasina zvishwe, nhare dzisina musoro kana kufungidzira. Kurongeka kwekushandisa-turusi kunotarisa kana mumiririri akatora maturusi akakodzera ane nharo dzinoshanda uye akabata zvikanganiso. Mutengo uye latency track tokens, mafoni uye nguva yemadziro-wachi. Chengetedzo inotarisa kana mumiririri akadzivirira zviito zvinokuvadza kana zvisina mvumo uye kuramba jekiseni rekukurumidza. Mabhenji-akavakirwa kwezvakatipoteredza ave nzira inotungamira. SWE-bhenji, kubva kuPrinceton vaongorori, inokumbira vamiririri kuti vagadzirise chaiyo GitHub nyaya uye inotarisa mhedzisiro nemiedzo yeprojekiti, uye SWE-bhenji Verified idiki-inotariswa nevanhu. WebArena inopa mawebhusaiti ega ega ekubhurawuza mabasa. OSWorld inoedza vamiririri vanoshanda yakazara makomputa desktops. GAIA inobvunza mibvunzo inoda akawanda-nhanho tsvakiridzo uye maturusi. Iyo tau-bench suite inoteedzera vashandisi nemadomendi marongero, uye yakaunza pass^k metric inobvunza kana mumiririri akabudirira pane imwe neimwe k inodzokororwa kuedza. Kudhirowa kunoshandisa marudzi matatu makuru emacheki: kodhi-yakavakirwa cheki pane zvabuda, idzo dzakaringana uye dzakachipa; modhi-yakavakirwa magiredhi ayo anokora zvinyorwa zvichipesana nerubriki, izvo zvinochinjika asi zvinoda kutariswa zvinopesana nekutonga kwevanhu; uye kuongorora kwevanhu, iyo inonyanya kuvimbika uye inononoka. Chikanganiso chakajairika kuvimba bhodhi revatungamiriri veruzhinji chete. Mabhenji anogona kudonhedza mudhata rekudzidzisa, kuzara, kana kutadza kufanana nebasa rako pachako. Imwe mhosho kubata kumwe kumhanya sechokwadi, sezvo vamiririri vasingatarisire uye misiyano midiki muhuwandu hwebudiriro inogona kuve ruzha. Zvikwata zvinowana kukosha kwakanyanya kubva kune yakavanzika yekuongorora seti yakavakirwa kubva kune avo chaiwo mabasa uye kutadza, inomhanya ichidzokororwa uye inoteverwa nekufamba kwenguva.
Chikwata checoding-agent chinohwina kuyedza kwega kwega kana bvunzo dzakavanzika dzapfuura mushure memumiririri, sezvinoita SWE-bhenji, panzvimbo yekutonga kana iyo diff inotaridzika zvakanaka.
Mutengi-sevhisi mumiriri anomhanya achipesana nevashandisi vakaedzerwa uye dhatabhesi rekubhukirira. Magiredhi anotarisa kana iyo yekupedzisira dhatabhesi state inowirirana nemhedzisiro uye kana mumiriri akatevera refund policy.
Chikwata cheinjiniya chinoita basa rega rega kashanu uye chinoshuma kuti kangani mumiririri anobudirira pane ese mashanu, izvo zvinofumura hunhu husina kunaka hungahwanda kamwe chete.
Kambani inowedzera saiti yekuchengetedza umo mabasa anosanganisira kuraira kwakasimwa kudzima mafaera kana kuvuza data, uye inoverenga imwe nzira yakabudirira sekutadza kana mumiririri akateerera.
Sezvo vamiririri vachitora mabasa akareba, ongororo dziri kufamba dzichienda kumawere akareba, nharaunda dzechokwadi, uye zviyero zvekuvimbika uye mutengo pamwe nekugona kwepamusoro. Benchmark saturation uye kusvibiswa zvingangoramba zvichisundidzira vanogadzira kuti vagadzire mabasa matsva uye kuvimba zvakanyanya pane zvakavanzika, domain-chaiyo bvunzo seti. Chengetedzo uye chengetedzo ongororo, kusanganisira kuramba jekiseni nekukurumidza, iri kugamuchira zvakanyanya sezvo vamiririri vanowana mukana kune chaiwo masisitimu. Iko hakuna chiyero chakabvumiranwa chegiredhi yakavhurika-yakapera mumiriri maitiro, saka kubatanidza macheki emhedzisiro, akatemerwa modhi magiredhi uye wongororo yevanhu zvinogona kuramba zvichiitwa zvakajairika.
Tsamba yemahara
Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.
Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.
Zviedze iwe pachako
Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.
Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo
Kuongorora maAI vamiririri kunoreva kuyera kuti yakawanda-nhanho system inosvika sei pazvinangwa mumamiriro echokwadi. Izvo zvinobatanidzira kana yakapedza basa, kana nzira yayakatora yaive ine musoro, kana yakashandisa maturusi nemazvo, kuti yaidhura zvakadini, uye kana yakaramba yakachengeteka. MaAgents anoita pamusoro pematanho akawanda uye ane maside effects, saka kutarisa mhinduro imwe chete uchipesana nereferensi hakuna kukwana, uye ongororo dzakanaka dzinowanzo mhanyisa mumiriri mukati menzvimbo inodzorwa uye kuongorora mamiriro ekupedzisira.
Multi-step zviito zvinoshandura nharaunda, saka unofanirwa kutarisa kuti chii chaizvo chakaitika uye sei, kwete chinyorwa chekupedzisira.
Nzvimbo-yakavakirwa macheki, pano bvunzo dzepurojekiti, tongai kana kugadzirisa kunoshanda chaizvo.
Inoda budiriro pane ese k kuedza matanho kuenderana, izvo zvine basa kune vamiririri vakaiswa kune chaivo vashandisi.
Trajectory cheki inotonga nzira yakatorwa, kubata zvishwe uye marara izvo mhedzisiro inoongorora yega inopotsa.
Mamodeli magiredhi anochinjika asi anofanirwa kutariswa maererano nekutonga kwevanhu uye kutariswa kune yakarongeka.
Ramba uchidzidza
Mamwe madhairekitori akasarudzirwa nyaya iyi
InoteveraGaidhi rinotevera
SWE-bhenji uye Coding Agent Benchmarks
Tekinoroji