← WingFoil Benchmark — campaign 13fda3099afa, execution 1
Architect: A, E, F
Harness, not model: every arm runs the same agent with the same model, gpt-5.6-terra. Only the harness around it differs: baseline, baseline-docs, wingfoil.
Against the baseline, baseline-docs is better in 2, worse in 0 and the same in 0 of 2 comparisons across categories E and F (preliminary: n < 3 in E and F).
Against the baseline, wingfoil is better in 2, worse in 0 and the same in 0 of 2 comparisons across categories E and F (preliminary: n < 3 in E and F).
A is not covered in this campaign.
| Category | baseline | baseline-docs | wingfoil wingfoil v0.3.0 · cc31d7a |
|---|---|---|---|
| A — Inception & Specification | not covered in this campaign | ||
| E — Governance & Compliance S8@1.0 | M-E1 10 n = 1 preliminary M-Q1 hold-out 6/7 n = 1 | M-E1 0 n = 1 preliminary −10 ▲ better M-Q1 hold-out 6/7 n = 1 | M-E1 0 n = 1 preliminary −10 ▲ better M-Q1 hold-out 6/7 n = 1 |
| F — Knowledge & Continuity S3@1.0 | M-F1 60.0% n = 1 preliminary M-Q1 hold-out 16/17 n = 1 | M-F1 80.0% n = 1 preliminary +20.0 pp ▲ better M-Q1 hold-out 16/17 n = 1 | M-F1 100.0% n = 1 preliminary +40.0 pp ▲ better M-Q1 hold-out 16/17 n = 1 |
Each value is the mean of its runs, with its n; a single run is preliminary, and so is a comparison unless both sides have n ≥ 3; a range is shown from n = 3. Deltas are against the baseline. How to read these results; which tools are eligible, and why.