# RAG System Specification

> Copy this file into your repository next to the code. A specification that lives in a wiki
> nobody opens is a document; one that sits beside the thing it describes is a contract.
> Every blank below is a decision someone has to make — leaving it blank is also a decision,
> just an unrecorded one.

**System name:** _______________
**Owner (person, not team):** _______________
**Date:** _______  **Version:** _______

## 1. The question this system answers

Write one sentence describing the question a user brings. If it takes a paragraph, the scope is
too wide for a first release.

> _We answer: _______________________________________________

**Who asks it today, and what do they do instead?** _______________

**What does a good answer save them?** (minutes, escalations, tickets) _______________

## 2. Corpus

| | |
|---|---|
| Document sets in scope | |
| Document sets explicitly OUT of scope | |
| **Named owner of each set** | |
| How often each set changes | |
| Where the truth lives if documents disagree | |

🔴 If any owner cell reads "nobody", stop. The most common failure of a retrieval system is a
stale document retrieved perfectly. Retrieval quality cannot fix content nobody maintains.

**Knowledge that is NOT written down anywhere:** _______________
(If the answer to the main question lives in people's heads, this is a documentation project
wearing a retrieval project's clothes.)

## 3. Chunking

| Decision | Value | Why |
|---|---|---|
| Split on | structure / fixed size | |
| Target chunk size | | |
| Overlap | | |
| Title + heading path prepended? | yes / no | |
| Metadata stored per chunk | owner, date, access level, type | |

## 4. Retrieval

| Decision | Value |
|---|---|
| Keyword search | yes / no |
| Semantic search | yes / no |
| **Hybrid** | yes / no |
| Embedding model + version (pinned) | |
| Chunks retrieved per query (k) | |
| Re-ranking | |
| Metadata filters available | |

> Semantic-only retrieval fails on exact identifiers — part numbers, error codes, policy
> references. Those are exactly what people search for. Record the decision either way.

## 5. Generation and refusal

| Decision | Value |
|---|---|
| Model + version (pinned) | |
| Answer only from supplied passages | yes / no |
| **Refusal instruction present** | yes / no |
| What the user sees when the answer is not in the corpus | |
| Output length limit | |
| Retrieved content delimited as data, not instructions | yes / no |

**Write the exact refusal text here:** _______________

## 6. Sources shown to the user

- Chunks used are displayed: yes / no
- Sources are clickable to the real document: yes / no
- Document date shown: yes / no

This is the mechanism by which users catch our mistakes. A system without it does not fail
loudly — it fails quietly, and people stop using it without telling you.

## 7. Permissions

| | |
|---|---|
| Retrieval filtered by the requesting user's entitlements | yes / no |
| Filtering happens at | query time / after generation |
| Privileged service account used | yes / no |
| Tested with users of different entitlements | yes / no |

🔴 A permissions failure here is a data breach, not a quality bug. Treat and report it as one.

## 8. Measurements — fill these BEFORE launch

| Metric | Target | Baseline measured | Date |
|---|---|---|---|
| Retrieval hit rate / recall@k | | | |
| Groundedness | | | |
| Refusal accuracy (unanswerable set) | | | |
| Over-refusal rate | | | |
| Cost per answer | | | |
| p95 latency | | | |

**Test set:** _____ questions, held where: _______________
Every reported bad answer is added to this set permanently. That is the only way the set stays
representative of what users actually ask rather than what we imagined they would ask.

## 9. Operations

| | |
|---|---|
| Re-index trigger | |
| Retrieval confidence logged | yes / no |
| Bad-answer reporting path | |
| Scheduled re-test cadence | |
| Who is paged, and for what | |

The corpus changes even when the code does not. A system with no scheduled re-test degrades
without a single deployment.

## 10. Sign-off

| | Name | Date |
|---|---|---|
| Built by | | |
| Content owner | | |
| Security review | | |
| Approved for users | | |