40-Scenario Benchmark Proof
Measured directly with truthgate benchmark across 10 grounded responses and 30 adversarial hallucination cases.
Summary Metrics
| Metric | Observed Value | Details |
|---|---|---|
| Total Scenarios | 40 | 10 Grounded Truthful + 30 Adversarial Hallucinations |
| Precision / Recall / F1 | 100.0% / 100.0% / 100.0% | 30/30 True Positives, 10/10 True Negatives |
| False Positive Rate | 0.0% | 0/10 truthful RAG & coding claims flagged |
| Exact Rule Accuracy | 100.0% (40/40) | Exact rule code attribution across TG001–TG008 |
| Verification Latency | 0.128 ms P50 / 0.347 ms P95 | Deterministic CPU execution with zero network requests |
Per-Rule Breakdown (TG001–TG008)
| Rule | Scenarios Caught | Exact Code Match | Category |
|---|---|---|---|
TG001 | 4 / 4 | 4 / 4 | entailment |
TG002 | 4 / 4 | 4 / 4 | entailment |
TG003 | 5 / 5 | 5 / 5 | entailment |
TG004 | 5 / 5 | 5 / 5 | entailment |
TG005 | 3 / 3 | 3 / 3 | entailment |
TG006 | 3 / 3 | 3 / 3 | workspace_grounding |
TG007 | 3 / 3 | 3 / 3 | workspace_grounding |
TG008 | 3 / 3 | 3 / 3 | workspace_grounding |
Reproduce Locally
truthgate benchmark truthgate benchmark --json