Nhungamiro yehunyanzvi

Spotlighting and Datamarking Defenses

Spotlighting mamaki kana kushandura mavara asina kuvimbwa saka modhi ine cue yekuti idata rekugadzirisa kwete rairo yekutevera.

Teerera kugwaro iriAI-yakarondedzerwa · ~3 min

Pa peji ino3 min verenga
  1. Pfupiso
  2. Kudzika Kwakadzika
  3. Kuitwa muNyika yeChokwadi
  4. Ramangwana reKutariswa uye Datamarking Dziviriro
  5. Kuzvishandisa
  6. Ramba Uchiongorora
  7. Mibvunzo inowanzo bvunzwa

Pfupiso

Tsvagiridzo inotsanangura kupatsanura, datamarking, uye encoding akasiyana, asi mhedzisiro inosungirirwa kune akaedzwa mhando uye kurwiswa; hapana kufomatidza trick inogadzira yakavimbiswa chengetedzo muganho.

Kudzika Kwakadzika

Spotlighting is a family of prompt transformations intended to make the provenance of untrusted content more visible in a model’s text context. A 2024 Microsoft-affiliated research paper describes three variants. Delimiting brackets a document with selected markers. Datamarking inserts a marker throughout the text, in the paper’s example replacing whitespace between words. Encoding transforms the document, for example using base64, and instructs the model to interpret the encoded block as data for the assigned task. Each variant also relies on a system instruction that explains how the marked text should be treated. The paper evaluated these methods on a synthetic indirect-injection corpus and older GPT-family model versions. It reported substantially lower attack success in its experiments, including a decrease from above 50% to below 2% for selected settings. Those numbers describe the paper’s particular tasks, attacks, and models; they are not expected production rates or guarantees for current systems. The paper also found encoding could impair underlying tasks for models less capable of decoding the text. This makes per-model utility evaluation important, not optional. Microsoft Foundry documents its Spotlighting preview as a document-attack control that uses base64 transformation and is supported only for models through the Chat Completions API. The docs say this increases document tokens and can cause long inputs to exceed limits, and note a possible user-visible encoding reference. That specific service behavior differs from a generic hand-built technique. In every case, marking helps signal provenance inside a shared text channel; it does not isolate data at a hardware or permission boundary. Test attacks that know the marker, preserve ordinary task quality, and restrict tool permissions so a successful injection has limited consequences.

Kuitwa muNyika yeChokwadi

Mupfupikisa anoisa gwaro rakadzoserwa pakati pemadelimita uye anoti mirairo iri mukati mendima inokwana kupfupikisa, kwete mirairo.

Pombi inoisa chicherechedzo pakati pemazwi mugwaro, ichitevera nzira yedatamarking, yozoongorora kana kupfupisa kwakajairika kuchiri kushanda.

Sisitimu inoshandura ndima ichishandisa encoding uye inoudza inokwanisa modhi nzira yekuidudzira sereferensi yemukati, yobva yayera huchokwadi hwebasa uye kurwisa kubudirira.

Chikwata chekuchengetedza chinobatanidza kumaka data nemidziyo yakaganhurirwa uye bvunzo dzekurwisa dzinokopa, dzinosiya, kana kutevedzera chimaka chakasarudzwa.

Ramangwana reKutariswa uye Datamarking Dziviriro

Provenance cues inogona kunyanya kuenderana nemodel interfaces, asi kukurumidza-level mamaki achiri kugovera yakafanana chiteshi chiteshi semavara avanonyora. Tsvagiridzo uye kushandiswa kwechigadzirwa kuchashanduka, uye mhedzisiro yavo ichasiyana nemhando yemhando uye dhizaini yekurwisa. Zvikwata zvinofanirwa kudzokorora mhando yebasa uye nyaya dzeanopikisa mushure mekuchinja kupi, uku vachichengeta yakazvimirira mvumo yekutarisa uye kutarisa. Dynamic mamaki anogona kuita kuti kumwe kupusa kuyedza kuome, asi ivo havagadziri yakaparadzana yakavimbika chiteshi. Ongorora maitiro matsva ekurwisa sezvo nzvimbo dzekudzosera dzichichinja.

Kuzvishandisa

Nei zvichikosha

  • Mutengo uye bhajeti. Zvisarudzo zvezvivakwa zvinotyaira kuita uye mutengo wekushandisa kwemakore.
  • Sarudzo dzakajeka. Dzidzo yehunyanzvi inobatsira zvikwata kusarudza murwi wakakodzera, kwete iwo mutsva chete.
  • Kudzora kwemhando yepamusoro. Sarudzo dzeinjiniya dziri nani dzinoderedza zviitiko zvekuvimbika mukugadzira.

Chenjera

  • Kugadzirisa imwe bhenji kunogona kuvanza yakafara system kushaya simba.
  • Infrastructure uye mari yekugadzirisa inowanzotarisirwa pasi.
  • Chengetedzo uye kucherechedzwa mapundu anogona kukura sezvo masisitimu anowedzera kuoma.

Matanho anotevera

  1. Tsanangura latency, mhando, uye mutengo zvinangwa usati waitwa.
  2. Benchmark pasi pechokwadi mutoro uye data mamiriro.
  3. Chishandiso chekutarisa zvikanganiso, kudonha, uye mushandisi maitiro.
  4. Gadzirira nzira dzekudzosera kumashure uye dzezviitiko usati wawedzera.

Ramba Uchiongorora

Tsamba yemahara

Tora iyo yemazuva ese AI pfupiso

Nhatu dzakasimbiswa dzeAI nyaya mangwanani ega ega pakati pevhiki, dzakanyorwa neChirungu chakajeka. Yemahara zvachose, hapana ads.

Imwe email zuva rega rega revhiki. Bvisa kunyoresa kamwe chete. Hatimbotengese kana kugovera kero yako.

Zviedze iwe pachako

Tora iyo Spotlighting uye Datamarking Dziviriro quiz

Mhinduro yepakarepo pamhinduro yega yega, uye chitupa chinogovaniswa chine ID inobatika kana wapedza kosi.

Tanga mibvunzo

Tsigira dzidzo yemahara yeAI. AI Understanding ndeye 501(c)(3) isingabatsiri - hapana ads, hapana paywall, zvachose. Ita chipo

Mibvunzo inowanzo bvunzwa

Chii chinonzi Spotlighting uye Datamarking Defenses?

Spotlighting mamaki kana kushandura mavara asina kuvimbwa saka modhi ine cue yekuti idata rekugadzirisa kwete rairo yekutevera. Tsvagiridzo inotsanangura kupatsanura, datamarking, uye encoding akasiyana, asi mhedzisiro inosungirirwa kune akaedzwa mhando uye kurwiswa; hapana kufomatidza trick inogadzira yakavimbiswa chengetedzo muganho.

Chii chinonzi spotlighting chinoedza kuratidza kune chimiro chemutauro?

Spotlighting inoshandisa shanduko uye mirairo kuti uone kuburitswa kwezvinhu zvisingavimbike.

Datamarking inoshanda sei mumuenzaniso wepepa?

Iyo bepa datamarking muenzaniso inosiya munhu akasarudzwa pamiganhu yezwi nekutsiva whitespace.

Chii chinofanira kutaurwa nevaverengi kubva mubepa rakashumwa kurwisa-kubudirira kudzikiswa?

Nhungamiro inodzika nhamba dzakataurwa kune mapepa akaedzwa modhi, kurwiswa, uye mabasa.

Ndeipi ganhuriro iyo bepa rakashuma pakukodha pamamodheru akaedzwa mashoma?

Iro bepa rinoshuma kuti mamwe mamodheru akaedzwa, anosanganisira GPT-3.5-Turbo, akanetsekana nemaencoded ekuisa uye mhando yebasa.

What does Microsoft Foundry’s documented Spotlighting preview do?

Microsoft zvinyorwa zvazvino zvinotsanangura base64-yakashandurwa gwaro zvinyorwa kune yayo Spotlighting preview.