A Primer on Prompt Engineering

Prompt engineering is the practice of designing inputs so a large language model completes a task reliably. Every interaction has three roles:

• User — you
• Assistant — the model
• System — the broader context or worldview that shapes every reply

The system prompt is underrated. It sets tone, constraints, and what the model should refuse. For production use, treat it like an API contract: version it, test it, and review changes.

Key parameters

• Temperature — higher values increase randomness in token selection. Good for brainstorming; bad for structured output.
• Top_p (nucleus sampling) — limits the candidate token pool. Often tuned instead of temperature, not both at extremes.
• Max tokens — caps output length and cost. Always set explicitly in production.
• Stop sequences — hard boundaries that end generation. Useful for JSON blocks or sectioned answers.
• Frequency / presence penalty — reduces repetition in long outputs.

Rule of thumb: tune temperature or top_p, not both aggressively at once.

Prompt structure

Effective prompts usually include four parts:

1. Instruction — what to do
2. Input data — the material to work on
3. Output format — schema, bullets, JSON, markdown
4. Context — assumptions, audience, constraints

Example: "Summarize this support ticket in three bullets for an engineer" beats "summarize this."

Principles that survive model upgrades

Start simple, then iterate. A one-line prompt plus one failure case teaches more than a ten-page spec upfront.

Say what not to do. "Recommend a movie" vs "Recommend a movie; do not ask for personal information" — the second fails less often.

Separate reasoning from output when needed. Ask for steps in a scratchpad section, then a final answer. Many models now internalize this, but explicit structure still helps for audits.

Evaluate on real failures. Keep a small set of bad outputs from production. Regression-test prompts against them after every model or prompt change.

Common use cases

| Task | What works | |------|------------| | Summarization | Fixed length, audience, and "include / exclude" lists | | Extraction | Output schema with field names and examples | | Classification | 3–5 labeled examples (few-shot) | | Code generation | Repo conventions, test requirements, file paths | | Q&A over docs | Cite sources; refuse when context is missing |

Prompt engineering is not a substitute for product design. But for AI-native products, the prompt layer is part of the UX — price, latency, and trust all depend on it.