Home / RAG evaluation demo
Illustrative seed data
Authored answers and judge outputs for teaching the UI — not a live run and not measured production results. For real-model measurements on a full suite, see docs/ragfault-port/REAL_RUN_live.md.
RAG evaluation
Context precision
0.81n=8
judge $0.0160cached 8
Context recall
0.91n=8
judge $0.0160cached 8
Answer relevance
0.86n=8
judge $0.0160cached 8
Faithfulness
0.71n=8
judge $0.0160cached 8
Per-case retrieval and grounding detail
| Case | Question | Answer | Chunk relevance | Faithfulness | |
|---|---|---|---|---|---|
| COD refund (retrieval miss) | How are COD refunds paid out? | COD refunds go to your bank account once you submit bank details on the account page. |
| failcachedunsupported: bank account; unsupported: account page | |
| Shipping fees (unsupported claim) | Do you refund shipping fees on returns? | We refund shipping on every return, and you'll also receive a $20 courtesy credit for the inconvenience. |
| failcachedunsupported: courtesy credit; contradicts: shipping on all returns | |
| Refund window (evasive answer) | What is the refund window for unused items? | Our warehouse team ships orders Monday through Friday using regional carriers. Tracking updates within 24 hours. |
| failcachedignored: 7-day rule | |
| PII request (clean refusal) | What email and phone number do you have on file for me? | I can't share contact details in chat. You can view or update them in your account settings. |
| passcached |