Prompt Engineering

Prompt engineering is the craft of designing the input to a language model so its output is useful, accurate and in the right format, and then testing that input against real cases. A prompt is everything the model sees before it answers: a system message that sets its role and rules, the user's request, any documents or data supplied, examples of good answers and instructions about format. Because a language model generates text by predicting what is likely to come next given everything in front of it, changing that context changes which answers are likely. Prompt engineering is therefore less about magic words than about removing ambiguity and supplying what the model cannot guess.

A well-built prompt usually has five parts. First, a clear task: what to do and for whom. Second, context: the background a capable newcomer would need, including why the task matters, since Anthropic's guide notes that explaining the reason behind an instruction helps the model target its answer. Third, input data, clearly separated from instructions, for example by wrapping it in labeled tags. Fourth, examples of the desired output. Fifth, an explicit output format, such as a table, JSON fields or a word limit. Anthropic suggests a simple test: show the prompt to a colleague with no background on the task, and if they would be confused, the model probably will be too.

Several techniques have measured effects. Zero-shot prompting gives only an instruction. Few-shot prompting adds worked examples; the 2020 GPT-3 paper by Tom Brown and colleagues at OpenAI showed that a 175-billion-parameter model could perform new tasks from a handful of examples in the prompt, with no change to its parameters. Anthropic's current guide recommends three to five varied examples, each wrapped in tags so they are not mistaken for instructions. Chain-of-thought prompting, introduced by Jason Wei and colleagues at Google in January 2022, uses examples that show the reasoning, not just the answer. With eight such examples, PaLM 540B went from 17.9% to 56.9% on the GSM8K math word problems. Four months later, Takeshi Kojima and colleagues found that simply appending Let's think step by step raised an InstructGPT model from 10.4% to 40.7% on the same benchmark. Self-consistency, from Xuezhi Wang and colleagues, samples several reasoning paths and takes the most common final answer, adding 17.9 percentage points on GSM8K over a single chain of thought.

The history of the field is short and recent. Before 2020, using a model for a new task usually meant retraining it on labeled data. GPT-3 in May 2020 made in-context learning practical. The year 2022 brought chain of thought, zero-shot reasoning prompts and self-consistency. In 2023, two lines of research exposed limits. In February, Kai Greshake and colleagues demonstrated indirect prompt injection, in which instructions hidden in a web page or document hijack an AI application that reads it, including Bing's GPT-4-powered chat. In July, Nelson Liu and colleagues published Lost in the Middle, showing that models used information at the start or end of a long input much better than information buried in the middle. By 2025, OWASP listed prompt injection as the top risk for applications built on language models.

Prompt engineering is one of several ways to change a model's behavior, and it is usually the first to try. Fine-tuning retrains the model on many examples, which can lock in a style or format but costs time, data and money, and must be redone when the base model changes. Retrieval-augmented generation, often shortened to RAG, automatically fetches relevant documents and inserts them into the prompt, which is the usual fix when the model lacks facts rather than instructions. Switching models is sometimes the right answer, especially when the problem is speed or price rather than quality, since no rewording makes a large model faster or cheaper per token. Prompting cannot add knowledge the model never learned unless that knowledge is pasted in.

The disciplined version of the practice looks like software testing. It starts with three things before any tuning: a clear definition of success, a way to measure it, and a first draft. In practice, that means collecting 20 to 100 real inputs, writing down what a good answer looks like for each, running the prompt on all of them, and scoring the results. Every change to the prompt is then judged on the whole set, not on a single impressive example. Without this, it is easy to fix one case and quietly break five others.

The limits are real. Results are sensitive to small wording changes and to example order, so a prompt that works today may behave differently after a model update; vendors such as Anthropic now publish model-specific prompting advice and migration notes for exactly this reason. The chain-of-thought paper found the technique helped mainly in very large models and gave little benefit in small ones. Long inputs suffer from position effects, as Lost in the Middle showed. Most seriously, a prompt cannot reliably stop prompt injection, because the model reads instructions and data in the same stream of text. OWASP's recommended defenses therefore sit outside the prompt: least-privilege access for tools, deterministic checks on output format, filtering, clear marking of untrusted content and human approval for high-risk actions.

Costs are easy to estimate. Model providers charge per token, for both the prompt and the answer. Every few-shot example and every pasted document is paid for on every call, so a 3,000-token prompt sent 10,000 times a day consumes 30 million input tokens daily. Chain-of-thought prompting adds output tokens for the reasoning, and self-consistency multiplies the whole cost by the number of samples drawn. The trade-off is usually worth measuring: a slightly longer prompt that halves the error rate is often cheaper than the human time spent fixing mistakes.