Ubuyobozi bwa tekiniki

Spotlighting and Datamarking Defenses

Spotlighting ituma ikimenyetso cyangwa ihindura imyandikire y'ibinyoma kugira ngo mudeli ibone ikimenyetso ko ari amakuru agomba gukorwaho, aho kuba amabwiriza agomba gukurikizwa.

Umva iki gitaboAI-byavuzwe · ~3 min

Kuriyi page3 min soma
  1. Incamake
  2. Kwibira cyane
  3. Gushyira mu bikorwa Isi
  4. Kuzaza kwa Spotlighting n'Uburyo bwo Kurinda bwa Datamarking
  5. Kubishyira mubikorwa
  6. Komeza Ubushakashatsi
  7. Ibibazo bikunze kubazwa

Incamake

Ubushakashatsi busobanura uburyo bwo gutandukanya, gutanga ikimenyetso ku makuru, n'impinduka z'uburyo bwo gushyira mu buryo bwihariye, ariko ibisubizo bihuje na mudeli n'ibitero byagenzejwe; nta buryo bwo gutondeka bugira uruhare mu gukora urugero rwo kurinda rusubiramo.

Kwibira cyane

Spotlighting is a family of prompt transformations intended to make the provenance of untrusted content more visible in a model’s text context. A 2024 Microsoft-affiliated research paper describes three variants. Delimiting brackets a document with selected markers. Datamarking inserts a marker throughout the text, in the paper’s example replacing whitespace between words. Encoding transforms the document, for example using base64, and instructs the model to interpret the encoded block as data for the assigned task. Each variant also relies on a system instruction that explains how the marked text should be treated. The paper evaluated these methods on a synthetic indirect-injection corpus and older GPT-family model versions. It reported substantially lower attack success in its experiments, including a decrease from above 50% to below 2% for selected settings. Those numbers describe the paper’s particular tasks, attacks, and models; they are not expected production rates or guarantees for current systems. The paper also found encoding could impair underlying tasks for models less capable of decoding the text. This makes per-model utility evaluation important, not optional. Microsoft Foundry documents its Spotlighting preview as a document-attack control that uses base64 transformation and is supported only for models through the Chat Completions API. The docs say this increases document tokens and can cause long inputs to exceed limits, and note a possible user-visible encoding reference. That specific service behavior differs from a generic hand-built technique. In every case, marking helps signal provenance inside a shared text channel; it does not isolate data at a hardware or permission boundary. Test attacks that know the marker, preserve ordinary task quality, and restrict tool permissions so a successful injection has limited consequences.

Gushyira mu bikorwa Isi

Umuyoboro w'incamake ushyira inyandiko yafashwe hagati y'ibimenyetso by'urugero, maze usobanura ko amabwiriri ari mu ngingo ari ibikubiyemo bigomba gusobanurwa mu ncamake, aho kuba amabwiriza.

Umuyoboro ushyira ikimenyetso hagati y'amagambo mu nyandiko, ukurikije uburyo bwo gutanga ikimenyetso ku makuru, hanyuma ugerageza niba gusobanura mu ncamake nk'uko bisanzwe bikora neza.

Sisitemu ihindura ingingo ukoresheje uburyo bwo gushyira mu buryo bwihariye, maze isobanura mudeli ifite ubushobozi uko igomba gusoma nko umwanya w'ibikubiyemo, hanyuma igapima ubwiza bw'akazi n'ubusugire bw'igitero.

Itsinda ryo kurinda ryihuza gutanga ikimenyetso ku makuru n'ibikoresho bimeze mu buryo bwihariye, hanyuma rigapima ibitero byigisha, bigasiba, cyangwa bigahindura ikimenyetso cyahisemo.

Kuzaza kwa Spotlighting n'Uburyo bwo Kurinda bwa Datamarking

Ibimenyetso by'aho amakuru avuye ashobora kuba ari yo y'ibanze mu buryo bwo gukorana na mudeli, ariko ibimenyetso bya prompt bikomeje gukoresha umuyoboro umwe w'ibikubiyemo n'imyandikire yabyo. Ubushakashatsi n'uburyo bwo gukoresha bizahinduka, kandi ingaruka zabyo zizahinduka bitewe na versions za mudeli n'uburyo bwo gukora ibitero. Amatsinda agomba kongera kugenzura ubwiza bw'akazi n'ibibazo by'ibitero nyuma y'impinduka zose, aho gukomeza kugenzura uburenganzira bwihariye n'ubwihariye. Ibimenyetso bihindagurika bishobora gukora ko ibigeragezo byoroshye byo guhindura bigorana, ariko ntibikora umuyoboro wihariye w'ibyo kwemera. Reba imiterere mishya y'ibitero uko amasoko y'ibifashishwa ahinduka.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Fata ikizamini cya Spotlighting n'Uburyo bwo Kurinda bwa Datamarking

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Ni iki Spotlighting n'Uburyo bwo Kurinda bwa Datamarking?

Spotlighting ituma ikimenyetso cyangwa ihindura imyandikire y'ibinyoma kugira ngo mudeli ibone ikimenyetso ko ari amakuru agomba gukorwaho, aho kuba amabwiriza agomba gukurikizwa. Ubushakashatsi busobanura uburyo bwo gutandukanya, gutanga ikimenyetso ku makuru, n'impinduka z'uburyo bwo gushyira mu buryo bwihariye, ariko ibisubizo bihuje na mudeli n'ibitero byagenzejwe; nta buryo bwo gutondeka bugira uruhare mu gukora urugero rwo kurinda rusubiramo.

Spotlighting igamije kumenyesha iki mudeli y'ururimi?

Spotlighting ikoresha impinduka n'amabwiriza kugira ngo itange ikimenyetso cy'aho ibikubiyemo by'ibinyoma biherereye.

Datamarking ikora gute mu buryo bwavuzwe mu rupapuro?

Uburyo bwo gutanga ikimenyetso ku makuru bwavuzwe mu rupapuro bushyira inyuguti yihariye hagati y'amagambo aho gusimbura umwanya w'icyatsi.

Abasoma bagomba gusoma iki mu mibare y'ubusubiramo y'ibitero ivugwa mu rupapuro?

Iyi nshingano ihagarika imibare yavuzwe kuri mudeli, ibitero, n'ibikorwa byagenzejwe mu rupapuro.

Ni iki urupapuro rwavuze ko ari umwanzuro ku buryo bwo gushyira mu buryo bwihariye kuri mudeli zidafite ubushobozi bwinshi?

Urupapuro ruvuga ko mudeli zimwe na zimwe zagenzejwe, harimo GPT-3.5-Turbo, zari zifite ikibazo n'imyandikire yahinduwe n'ubwiza bw'akazi.

What does Microsoft Foundry’s documented Spotlighting preview do?

Amabwiriza ya Microsoft ya none asobanura ibikubiyemo by'inyandiko byahinduwe muri base64 ku isuzuma rya Spotlighting ryayo.