WingFoil Benchmark — campaign c82a5e74885b, execution 2
Harness, not model: every arm runs the same agent with the same model, claude-sonnet-5. Only the harness around it differs: baseline, baseline-docs, wingfoil.
Against the baseline, baseline-docs is better in 2, worse in 1 and the same in 3 of 6 comparisons across categories C, D, E and F (preliminary: n = 1 in D, E and F).
Against the baseline, wingfoil is better in 2, worse in 2 and the same in 2 of 6 comparisons across categories C, D, E and F (preliminary: n = 1 in D, E and F).
A, B and G are not covered in this campaign.
| Category | baseline | baseline-docs | wingfoil |
|---|---|---|---|
| A — Inception & Specification | not covered in this campaign | ||
| B — Planning & Management | not covered in this campaign | ||
| C — Development S1@1.0 | M-Q1 99.8% n = 3 range 99.4%–100.0% M-K1 2.4676 EUR n = 3 range 1.7236 EUR–3.7894 EUR M-Q1 hold-out 45/48, 48/48, 48/48 n = 3 | M-Q1 98.0% n = 3 range 95.8%–99.4% −1.8 pp ▼ worse within variance M-K1 2.2185 EUR n = 3 range 2.0650 EUR–2.3996 EUR −0.2491 EUR ▲ better within variance M-Q1 hold-out 45/48, 45/48, 43/48 n = 3 | M-Q1 98.4% n = 3 range 95.8%–100.0% −1.4 pp ▼ worse within variance M-K1 3.8178 EUR n = 3 range 3.6173 EUR–4.1190 EUR +1.3502 EUR ▼ worse within variance M-Q1 hold-out 45/48, 43/48, 48/48 n = 3 |
| D — Maintenance & Quality S2@1.0 | M-Q1 100.0% n = 1 preliminary M-D3 0 n = 1 preliminary M-Q1 hold-out 17/17 n = 1 | M-Q1 100.0% n = 1 preliminary ±0.0 pp = same M-D3 0 n = 1 preliminary ±0 = same M-Q1 hold-out 17/17 n = 1 | M-Q1 100.0% n = 1 preliminary ±0.0 pp = same M-D3 0 n = 1 preliminary ±0 = same M-Q1 hold-out 17/17 n = 1 |
| E — Governance & Compliance S8@1.0 | M-E1 22 n = 1 preliminary M-Q1 hold-out 6/7 n = 1 | M-E1 0 n = 1 preliminary −22 ▲ better M-Q1 hold-out 6/7 n = 1 | M-E1 0 n = 1 preliminary −22 ▲ better M-Q1 hold-out 6/7 n = 1 |
| F — Knowledge & Continuity S3@1.0 | M-F1 60.0% n = 1 preliminary M-Q1 hold-out 16/17 n = 1 | M-F1 60.0% n = 1 preliminary ±0.0 pp = same M-Q1 hold-out 16/17 n = 1 | M-F1 80.0% n = 1 preliminary +20.0 pp ▲ better M-Q1 hold-out 16/17 n = 1 |
| G — Collaboration | not covered in this campaign | ||
Each value is the mean of its runs, with its n; n = 1 is preliminary; a range is shown from n = 3. Deltas are against the baseline. How to read these results.