Release Test Report — Sample
A worked example of a test report that supports a go decision — what was tested, what was found, what was deliberately not tested, and the known defects being shipped on purpose.
Markdown. No sign-up, no email.
This is an illustrative example. The release, defects and figures are invented to show the shape of a report that a person can make a decision from. Copy the structure.
The report exists to answer one question: should this release ship? It answers it in section 1 and spends the rest showing the working, including what was not tested — which is the part that makes the recommendation honest.
Release test report — 4.7.0#
| Release | 4.7.0 — refund handling, bulk export, 14 defect fixes |
| Tested | 2026-07-27 to 2026-08-01 |
| Environment | Staging, production-shaped data, one production-scale load test |
| Prepared by | Test lead |
1. Recommendation#
Ship, with one feature behind a flag.
Refund handling and the 14 fixes are ready. Bulk export works correctly and is too slow at production data volumes; it ships disabled and is enabled per customer once the performance work lands.
Three known defects ship, listed in section 5. None is customer-visible in normal use, and each has an owner.
2. What was tested#
| Cases | Passed | Failed | |
|---|---|---|---|
| Automated regression | 2,340 | 2,338 | 2 |
| New feature — refunds | 84 | 81 | 3 |
| New feature — bulk export | 41 | 38 | 3 |
| Integration with the payment provider | 26 | 26 | — |
| Permissions across 6 roles | 96 | 95 | 1 |
| Exploratory, 3 sessions | — | — | 7 issues raised |
| Load, at 2× expected peak | 8 scenarios | 6 | 2 |
| Accessibility, refunds screen | — | — | 2 issues raised |
The two automated regression failures were both assertions that had gone stale against intended behaviour changes. Both were updated and reviewed as changes, not silently.
3. Refunds — the three failures#
Partial refund of a settled batch produced a rounding difference of one minor unit when the original payment split across three settlement lines. Found by an exploratory session, not by the scripted cases, which used single-line payments throughout. Fixed and re-tested.
Refund of a refunded payment was permitted, producing a negative balance. The requirement did not mention it. Fixed, and the requirement updated — this is a requirement defect, not an implementation one, and recording it that way is what stops the next one.
Refund notification used the payment date rather than the refund date. Fixed.
4. Bulk export — why it ships disabled#
Correct output on every case. The problem is time.
| Records | Duration | Memory |
|---|---|---|
| 1,000 | 4 s | 120 MB |
| 10,000 | 47 s | 900 MB |
| 100,000 | 9 min | 7.2 GB |
| 500,000 (largest real customer) | not completed | out of memory |
The implementation assembles the whole export in memory. The largest customer has 480,000 records and the feature was built and tested against a 5,000-record fixture.
Behind a flag rather than removed, because the code is correct and the fix is streaming rather than a redesign. Enabled per customer once it holds at 500,000.
5. Known defects shipping#
| Defect | Why it ships | Owner | |
|---|---|---|---|
| 1 | Export column order differs from 4.6 | Cosmetic; two customers notified | Product |
| 2 | Refund reason truncated at 200 characters in the audit log | Full reason retained in the payment record | Engineering |
| 3 | Role "finance viewer" can see refund totals it cannot act on | Read-only, no data exposure beyond the role's remit | Product |
Defect 3 was found by the permissions matrix and is the kind that gets waved through without one. It is a deliberate acceptance, recorded, with a name against it.
6. What was not tested#
Stated plainly. A report without this section reads as more complete than it is.
- Upgrade from 4.5. Only 4.6 → 4.7 was tested. Two customers are on 4.5 and their upgrade path is unverified.
- The Norwegian tax integration. The sandbox was unavailable all week. Unchanged in this release, so the risk is regression only.
- Sustained load. The load test ran 30 minutes. Behaviour over a full day is unknown.
- Browsers other than the two supported. Unchanged policy, stated for completeness.
- Data migration on a database over 2 TB. Largest tested was 400 GB. The migration is additive, so the risk is duration rather than correctness — but the duration is unknown.
The first and last are the two worth a decision before ship.
7. Exploratory sessions#
Three sessions, 90 minutes each, charter-based rather than scripted.
| Charter | Issues |
|---|---|
| Refund a payment in every state it can be in | 3, including the rounding defect |
| Use the export while data is changing underneath it | 2 |
| Attempt refunds as every role, including expired sessions | 2 |
Five of the seven issues, including the most serious, came from exploratory sessions. The scripted cases covered the paths somebody thought of in advance; the sessions covered the combinations. This ratio has been consistent across recent releases and is the argument for protecting the time.
8. Load#
Six of eight scenarios passed. The two failures were both bulk export, covered above. Refund handling held at twice expected peak with the 95th percentile at 1.9 seconds against a 3-second requirement.
Notes on using this format#
Lead with the recommendation. Most readers need only section 1. Everything after it is the working, for the people who need to check it.
Say what was not tested. Five items here, two of which need a decision. A report that omits them is read as covering everything.
List the defects you are shipping on purpose, with owners. Three known defects with names against them is a decision. The same three unmentioned is a discovery waiting to happen.
Report where issues came from. Five of seven from exploratory sessions is the strongest available argument for continuing to fund exploratory sessions.