Skip to content

Prompt Templates for RAG Answers

How to structure the prompt that turns retrieved passages into a grounded, cited answer.

Editorial team 1 min read

The generation prompt is where retrieved context becomes an answer. Its structure affects faithfulness, citations and tone.

A Reliable Structure

  1. Role and audience: who the assistant serves.
  2. Grounding rules: answer only from the documents; say when the answer isn't there.
  3. The documents, clearly delimited, each with an ID, title, date and source.
  4. The question.
  5. Output requirements: length, format, citation style.

Example

You answer employees' questions about HR policy.
Use only the documents below. Cite document IDs in square brackets after each claim.
If the documents don't answer the question, say "I couldn't find that in the policies"
and suggest contacting HR.

<documents>
<doc id="1" title="Leave Policy" updated="2026-03-01">...</doc>
<doc id="2" title="Travel Policy" updated="2025-11-12">...</doc>
</documents>

Question: How many days of annual leave do part-time staff get?

Order Documents Sensibly

Put the most relevant documents first or last; models can pay less attention to material buried in the middle of long contexts.

Handle Conflicts

Tell the model what to do when documents disagree — prefer the most recent, and mention the conflict.

Keep It Versioned

Store templates in version control and re-run your evaluation set whenever they change.

Don't Overload

Including more documents isn't always better. Irrelevant context can reduce accuracy; re-rank and keep the most relevant.

More in RAG

All RAG guides →
RAG Guide · 1 min

RAG Architecture: The Components End to End

A map of a complete retrieval-augmented generation system, from ingestion to answer, and what each component is responsible for.

RAG 1 min read 6 Dec 2025

RAG Guide · 2 min

Document Parsing for RAG

Turning PDFs, slides, HTML and scans into clean, structured text — the unglamorous step that decides RAG quality.

RAG 2 min read 5 Dec 2025

RAG Guide · 2 min

Chunk Size and Overlap Tuning

How to choose chunk size and overlap for retrieval by testing against real questions rather than guessing.

RAG 2 min read 4 Dec 2025

RAG Guide · 2 min

Hybrid Search for RAG

Combining keyword and vector search so RAG finds both exact terms and paraphrased meaning.

RAG 2 min read 3 Dec 2025