Prompt Engineering — Frequently Asked Questions
What holds up in production — whether prompt engineering is a real skill, examples versus instructions, chain-of-thought, structured output, version control, and why your prompt broke without being changed.
The basics#
Is prompt engineering a real skill or a passing phase?#
The tricks are a passing phase; the discipline is not. Clever phrasing that squeezed better output from older models matters less each year as models improve.
What does not go away: stating the task precisely, giving examples of the output shape, handling untrusted content safely, defining refusal behaviour, and testing changes against a fixed set of cases. That is engineering, and it looks more like software practice every year.
What actually improves output most?#
In rough order: a clear, single task · examples of the desired output · explicit instruction on what to do when the answer is unknown · a specified output format · relevant context supplied rather than assumed.
Notice that none of these are tricks. Most disappointing output comes from an underspecified request, not from insufficient cleverness.
Do examples beat instructions?#
For anything with a specific shape, usually yes. Two well-chosen examples convey a format more reliably than a paragraph describing it, and they are less ambiguous.
Instructions are better for constraints and prohibitions — what to avoid, when to refuse, what must never appear.
Does "think step by step" still help?#
Less than it used to. Newer reasoning-capable models do this internally, and instructing it explicitly can be redundant or occasionally counterproductive.
It still helps on multi-step problems with weaker or smaller models. Test it on your own case rather than adopting it as a rule — that is the general answer to most prompt folklore.
Production concerns#
Where should prompts live?#
In version control, alongside the code, reviewed like code. Not in a database field, not in a vendor console, not pasted into a config at deploy time.
A prompt is program behaviour. Editing it directly in production is an untracked change to what your system does, and it is routinely the least controlled thing in an AI deployment.
How do we test a prompt?#
Fix a set of at least twenty real cases with known good outcomes, run them before and after any change, and record the pass rate along with the number of runs. Add every reported bad output to the set permanently.
Because output varies between runs, a single run is an anecdote. Report "18 of 20, over three runs" rather than a percentage from one pass.
Our prompt stopped working and we did not change it. How?#
The model changed underneath you. If you are pointing at a floating alias rather than a pinned version, the provider can update it and your behaviour shifts with no change on your side.
Pin the model version. Then upgrades become a deliberate change you test, rather than a mystery quality complaint nobody can reproduce.
How do we get reliably parseable output?#
Use the API's structured output or function-calling facility where available — it constrains generation rather than requesting cooperation. Where unavailable, specify the schema precisely, give an example, and write a parser that fails gracefully on malformed output rather than crashing.
Never assume the response is well-formed because it was well-formed in testing.
Long prompt or short prompt?#
As long as needed to specify the task, and no longer. Very long prompts accumulate contradictions — an instruction in paragraph two quietly conflicting with one in paragraph nine — and nobody notices because nobody reads the whole thing.
If a prompt has grown past a page, read it end to end looking specifically for contradictions. There is usually at least one.
Safety#
What is prompt injection, in practice?#
Content your system reads containing instructions aimed at the model — in a document, a web page, a support ticket, a tool response. The model cannot reliably distinguish instructions you wrote from instructions it encountered.
Delimit retrieved content clearly as data, state that it is not to be treated as instruction, and — crucially — do not let read content directly trigger anything consequential. The architectural control matters more than the wording.
Can we filter out injection attempts?#
Not reliably. The attacker also writes text, and any filter you build is something they can write around. Filtering raises the bar; it is not a boundary.
Design so that a successful injection has limited effect: narrow tool scope, confirmation on irreversible actions, permissions enforced outside the model.
Should the system prompt be secret?#
Assume it will leak, and design accordingly. Do not put credentials, internal identifiers or anything genuinely sensitive in it. Treating the system prompt as a security boundary is a common and misplaced assumption.
Working with the results#
Why does the same prompt give different answers?#
Because generation is probabilistic. Lowering temperature reduces variation and does not eliminate it, and some variation is often desirable.
The implication for testing is the important part: assert properties rather than exact strings, and report pass rates over multiple runs rather than a single comparison.
How do we handle prompts across multiple languages?#
Test in each language you support rather than assuming translation of the prompt is sufficient. Quality, refusal behaviour and formatting compliance all vary by language, usually degrading for languages less represented in training.
What is the most common mistake?#
Iterating on the prompt against three examples the author has in their head, shipping it, and discovering the failure modes in production. The fix is unglamorous: build the test set first, then iterate against it.