Reproducible Proof

Every metric below comes from running patchpilot benchmark, patchpilot demo, and pytest directly on v1.2.0.

Top-1 / Top-3100.0%
Token Reduction96.6%
Guard Recall / FPR100% / 0%
Guard p50 Latency0.101 ms

Subsystem Benchmark Suite (patchpilot benchmark)

Run the built-in harness locally with patchpilot benchmark --json to inspect machine-specific timings across all three subsystems.

Subsystem Benchmark Workload Accuracy / Recall Secondary Metric p50 Latency p95 Latency
AST Symbol & Call-Graph Indexer 50 files / 100 symbols 100.0% AST parse coverage Bidirectional call edges 4.19 ms 4.19 ms
Hybrid Fault Localizer 25 stack & call-graph fault scenarios 100.0% Top-1 / 100.0% Top-3 96.6% prompt token reduction 0.393 ms 0.789 ms
Anti-Reward-Hacking Guard (RH001–RH008) 20 adversarial + 20 legitimate patches 100.0% Recall / 100.0% Precision 0.0% False Positive Rate 0.101 ms 0.409 ms

End-to-End Repair Transaction Demo (patchpilot demo)

Observed execution output comparing an adversarial reward-hacking patch against a clean minimal bug fix inside TransactionalPatchWorkspace:

[1] HYBRID STACK-TRACE + AST CALL-GRAPH FAULT LOCALIZATION
  #1 billing/invoice.py::calculate_tiered_invoice (score=0.852, tokens=87) -> direct stack frame
  #2 billing/pricing.py::apply_volume_discount    (score=0.239, tokens=134) -> 1-hop AST neighbor
  #3 billing/tax.py::compute_regional_tax         (score=0.208, tokens=82) -> 1-hop AST neighbor
  Token Budget Used: 303/600 tokens across 3 suspect function slices.

[2] ADVERSARIAL REWARD-HACKING PATCH ATTEMPT (BLOCKED & ROLLED BACK)
  Status       : ROLLED_BACK (stage=integrity_guard)
  Risk Score   : 1.0000
  Violations   : [RH003] except...: pass, [RH001] @pytest.mark.skip injected, [RH002] assert True
  State Intact : True

[3] GENUINE MINIMAL PATCH TRANSACTION (COMMITTED)
  Status       : COMMITTED
  Verification : Verified calculate_tiered_invoice(1000, 10, 'DEFAULT') == 8925
  Verified     : True (SHA-256 RepairReceipt generated)

Architecture Decision Records