← WingFoil Benchmark — campaign 81ebb2691f9c, execution 1

D — Maintenance & Quality

Does the harness improve finding and fixing defects without regressions?

S2@1.0

M-Q1: hidden tests passed on the final snapshot, the defect tests among them (higher is better)

• baseline:
  100.0% n = 1 preliminary
• baseline-docs:
  100.0% n = 1 preliminary ±0.0 pp = same
• openspec openspec 1.14.0 · 9cf16ae:
  100.0% n = 1 preliminary ±0.0 pp = same
  against openspec-docs ±0.0 pp = same
• openspec-docs:
  100.0% n = 1 preliminary ±0.0 pp = same
• speckit speckit v1.1.0 · f1d3a4f:
  100.0% n = 1 preliminary ±0.0 pp = same
  against speckit-docs ±0.0 pp = same
• speckit-docs:
  100.0% n = 1 preliminary ±0.0 pp = same
• wingfoil wingfoil v0.3.0 · cc31d7a:
  100.0% n = 1 preliminary ±0.0 pp = same
  against baseline-docs ±0.0 pp = same

M-D3: regressions of tests that passed on the seed (lower is better)

• baseline:
  0 n = 1 preliminary
• baseline-docs:
  0 n = 1 preliminary ±0 = same
• openspec openspec 1.14.0 · 9cf16ae:
  0 n = 1 preliminary ±0 = same
  against openspec-docs ±0 = same
• openspec-docs:
  0 n = 1 preliminary ±0 = same
• speckit speckit v1.1.0 · f1d3a4f:
  0 n = 1 preliminary ±0 = same
  against speckit-docs ±0 = same
• speckit-docs:
  0 n = 1 preliminary ±0 = same
• wingfoil wingfoil v0.3.0 · cc31d7a:
  0 n = 1 preliminary ±0 = same
  against baseline-docs ±0 = same

Each suite on the final snapshot

• baseline: regression 16/16; reports-1 3/3; reports-2 2/2; false-report 1/1; reports-3 2/2
• baseline-docs: regression 16/16; reports-1 3/3; reports-2 2/2; false-report 1/1; reports-3 2/2
• openspec openspec 1.14.0 · 9cf16ae: regression 16/16; reports-1 3/3; reports-2 2/2; false-report 1/1; reports-3 2/2
• openspec-docs: regression 16/16; reports-1 3/3; reports-2 2/2; false-report 1/1; reports-3 2/2
• speckit speckit v1.1.0 · f1d3a4f: regression 16/16; reports-1 3/3; reports-2 2/2; false-report 1/1; reports-3 2/2
• speckit-docs: regression 16/16; reports-1 3/3; reports-2 2/2; false-report 1/1; reports-3 2/2
• wingfoil wingfoil v0.3.0 · cc31d7a: regression 16/16; reports-1 3/3; reports-2 2/2; false-report 1/1; reports-3 2/2

Hold-out

• baseline:
  hold-out 17/17 n = 1
• baseline-docs:
  hold-out 17/17 n = 1
• openspec openspec 1.14.0 · 9cf16ae:
  hold-out 17/17 n = 1
• openspec-docs:
  hold-out 17/17 n = 1
• speckit speckit v1.1.0 · f1d3a4f:
  hold-out 17/17 n = 1
• speckit-docs:
  hold-out 17/17 n = 1
• wingfoil wingfoil v0.3.0 · cc31d7a:
  hold-out 17/17 n = 1

Runs

• 81ebb2691f9c/1/runs/S2@1.0/baseline/claude-sonnet-5-5/r1 — open with bench run show 81ebb2691f9c/1/runs/S2@1.0/baseline/claude-sonnet-5-5/r1
• 81ebb2691f9c/1/runs/S2@1.0/baseline-docs/claude-sonnet-5-5/r1 — open with bench run show 81ebb2691f9c/1/runs/S2@1.0/baseline-docs/claude-sonnet-5-5/r1
• 81ebb2691f9c/1/runs/S2@1.0/openspec/claude-sonnet-5-5/r1 — open with bench run show 81ebb2691f9c/1/runs/S2@1.0/openspec/claude-sonnet-5-5/r1
• 81ebb2691f9c/1/runs/S2@1.0/openspec-docs/claude-sonnet-5-5/r1 — open with bench run show 81ebb2691f9c/1/runs/S2@1.0/openspec-docs/claude-sonnet-5-5/r1
• 81ebb2691f9c/1/runs/S2@1.0/speckit/claude-sonnet-5-5/r1 — open with bench run show 81ebb2691f9c/1/runs/S2@1.0/speckit/claude-sonnet-5-5/r1
• 81ebb2691f9c/1/runs/S2@1.0/speckit-docs/claude-sonnet-5-5/r1 — open with bench run show 81ebb2691f9c/1/runs/S2@1.0/speckit-docs/claude-sonnet-5-5/r1
• 81ebb2691f9c/1/runs/S2@1.0/wingfoil/claude-sonnet-5-5/r1 — open with bench run show 81ebb2691f9c/1/runs/S2@1.0/wingfoil/claude-sonnet-5-5/r1

Scenarios that also touch this category

• S1@1.0: primary category C