Ubuyobozi bwa tekiniki

Umutekano wihuse

Prompt security addresses attempts to make a language-model application treat untrusted content as instructions or disclose information it should protect.

Umva iki gitaboAI-byavuzwe · ~2 min

Kuriyi page2 min soma
  1. Incamake
  2. Ibyingenzi byingenzi
  3. Kwibira cyane
  4. Keep a retrieved instruction inside the document
  5. Gushyira mu bikorwa Isi
  6. Kubishyira mubikorwa
  7. Inkomoko no gusoma
  8. Komeza Ubushakashatsi
  9. Ibibazo bikunze kubazwa

Incamake

The problem can arise in user input, retrieved documents, webpages, images, or tool responses. Defenses must limit the consequences as well as detect suspicious text.

Ibyingenzi byingenzi

  1. Distinguish instructions from processed content.
  2. Limit permissions independently of the model.
  3. Test trust boundaries across input channels.

Kwibira cyane

Separate the user’s authorized task from the content being processed. A document may legitimately contain instructions as part of its subject matter. Those words should not automatically control the assistant’s tools, account access, or response policy. Keep privileges narrow. A summarization task generally does not require unrestricted file access or permission to send messages. Enforce those limits in the application so a model error cannot silently create a broader capability. Validate consequential outputs and actions against the original request. Check the destination, affected records, data being transmitted, and required approval. An external page claiming that the user approved something is not equivalent to an actual user instruction. Build regression cases that vary the location and format of untrusted instructions. Test direct input, retrieved passages, and tool results in a controlled environment. Review whether the application preserves useful task performance while resisting redirection. Avoid claims of a foolproof prompt-injection filter; layered controls and ongoing testing are more credible.

04Urugero rwakazi

Keep a retrieved instruction inside the document

  1. Construct a test page containing a normal policy paragraph and a sentence telling the assistant to upload unrelated files.

  2. Ask only for the policy summary. Verify that the summary uses relevant facts and that no upload tool is called.

  3. Repeat with the instruction in a quoted block and in a tool result to test the same trust boundary across formats.

Ibyo byerekana

This bounded defensive exercise verifies task adherence without using real private files.

Gushyira mu bikorwa Isi

Summarize a document that contains an instruction-like passage without executing it.

Check an outgoing tool action against the user’s original destination and purpose.

Kubishyira mubikorwa

Impamvu ari ngombwa

  • Igiciro na bije. Ibyemezo byubwubatsi bitwara imikorere nigiciro cyimikorere kumyaka.
  • Ibyemezo bisobanutse. Ubuhanga bwa tekinike bufasha amakipe guhitamo umurongo ukwiye, ntabwo ari shyashya gusa.
  • Kugenzura ubuziranenge. Guhitamo neza bya injeniyeri bigabanya ibintu byizewe mubikorwa.

Witondere

  • Gutezimbere igipimo kimwe gishobora guhisha intege nke za sisitemu.
  • Ibikorwa Remezo no kubungabunga akenshi usanga bidahabwa agaciro.
  • Icyuho cyumutekano no kwitegereza birashobora kwiyongera uko sisitemu igenda igorana.

Intambwe ikurikira

  1. Sobanura ubukererwe, ubuziranenge, nigiciro cyibiciro mbere yo kubishyira mubikorwa.
  2. Ibipimo byerekana umutwaro ufatika hamwe namakuru yimiterere.
  3. Gukurikirana ibikoresho kubikosa, drift, ningaruka zabakoresha.
  4. Tegura inzira yo gusubiza ibyabaye mbere yo gupima.

Inkomoko no gusoma

  1. OWASPPrompt injection risks and mitigations

Komeza Ubushakashatsi

Ibaruwa y'amakuru y'ubuntu

Ibuka amakuru ya buri munsi ya AI

Inkuru eshatu zizewe za AI buri gitondo cy'iminsi y'akazi, zanditse mu rurimi rworoshye. Ubuntu iteka ryose, nta kwamamaza.

Imeri imwe buri munsi y'akazi. Kwiyimura mu buryo bumwe. Ntitwigera tugurisha cyangwa ngo dusangire aderesi yawe.

Isuzume

Take the AI Prompt Security quiz

Ibisubizo ako kanya kuri buri kibazo, n'icyemezo gishobora gusangirwa gifite ID yagenzurwa umaze gutsinda isomo.

Tangira ikibazo

Fasha uburezi bwa AI bw'ubuntu. AI Understanding ni umuryango utegamiye kuri leta wa 501(c)(3) — nta kwamamaza, nta kwishyura, iteka ryose. Tanga impano

Ibibazo bikunze kubazwa

Can prompt injection be solved by banning one phrase?

No. The underlying issue is whether untrusted content can redirect behavior. Attacks can use many phrasings and formats.