← WingFoil Benchmark — campaign 13fda3099afa, execution 1

D — Maintenance & Quality

Does the harness improve finding and fixing defects without regressions?

S2@1.0

M-Q1: hidden tests passed on the final snapshot, the defect tests among them (higher is better)

• baseline:
  95.8% n = 1 preliminary
• baseline-docs:
  95.8% n = 1 preliminary ±0.0 pp = same
• wingfoil wingfoil v0.3.0 · cc31d7a:
  95.8% n = 1 preliminary ±0.0 pp = same
  against baseline-docs ±0.0 pp = same

M-D3: regressions of tests that passed on the seed (lower is better)

• baseline:
  1 n = 1 preliminary
• baseline-docs:
  1 n = 1 preliminary ±0 = same
• wingfoil wingfoil v0.3.0 · cc31d7a:
  1 n = 1 preliminary ±0 = same
  against baseline-docs ±0 = same

Each suite on the final snapshot

• baseline: regression 16/16; reports-1 3/3; reports-2 2/2; false-report 0/1; reports-3 2/2
• baseline-docs: regression 16/16; reports-1 3/3; reports-2 2/2; false-report 0/1; reports-3 2/2
• wingfoil wingfoil v0.3.0 · cc31d7a: regression 16/16; reports-1 3/3; reports-2 2/2; false-report 0/1; reports-3 2/2

Hold-out

• baseline:
  hold-out 16/17 n = 1
• baseline-docs:
  hold-out 16/17 n = 1
• wingfoil wingfoil v0.3.0 · cc31d7a:
  hold-out 16/17 n = 1

Runs

• 13fda3099afa/1/runs/S2@1.0/baseline/gpt-5.6-terra/r1 — open with bench run show 13fda3099afa/1/runs/S2@1.0/baseline/gpt-5.6-terra/r1
• 13fda3099afa/1/runs/S2@1.0/baseline-docs/gpt-5.6-terra/r1 — open with bench run show 13fda3099afa/1/runs/S2@1.0/baseline-docs/gpt-5.6-terra/r1 — 2 attempts
• 13fda3099afa/1/runs/S2@1.0/wingfoil/gpt-5.6-terra/r1 — open with bench run show 13fda3099afa/1/runs/S2@1.0/wingfoil/gpt-5.6-terra/r1

Scenarios that also touch this category

• S1@1.0: primary category C