← How the benchmark works

How the wingfoil arm is set up, before its agent starts: the wingfoil harness at the campaign's pin, from the repository's arms/wingfoil/.

Setup script

#!/bin/bash
# The wingfoil arm's setup (REQ-RUN-03, adr-003 decisions 3, 6, 8, 9; REQ-RUN-17). The runner has
# already written the git identity (Benchmark Approver), copied this directory to ~/arm, the WingFoil
# under test to ~/harness.tgz and, if the scenario has one, its configuration to ~/scenario.
set -euo pipefail
# WORKSPACE and HOME are set otherwise only when the runner snapshots this configuration for the
# baseline-docs generator (task-015), in a one-off container with a build directory as its home.
WORKSPACE="${WORKSPACE:-/workspace}"
export PATH="$HOME/.local/bin:$PATH"
cd "$WORKSPACE"

# 1. The WingFoil under test, outside the workspace, called `wingfoil` (never `npx wingfoil`).
mkdir -p "$HOME/.local/bin"
tar -xzf "$HOME/harness.tgz" -C "$HOME"
printf '#!/bin/sh\nexec node %s/wingfoil/dist/cli.js "$@"\n' "$HOME" > "$HOME/.local/bin/wingfoil"
chmod +x "$HOME/.local/bin/wingfoil"
echo "WingFoil under test: commit $(cat "$HOME/wingfoil/.wingfoil-commit"), at $(command -v wingfoil)"

# 2. The project, as WingFoil's documentation sets it up: `init`'s Kanban scaffold, no workflow of the benchmark's own
#    (experiment design 1.5). `init` commits by itself.
wingfoil init --template Kanban > /dev/null

# 3. The scenario's configuration, if any (dl-005): copied over init's files, committed once.
if [ -d "$HOME/scenario" ]; then
  cp -R "$HOME/scenario/." "$WORKSPACE/"
  git add --all
  git commit --quiet --message "chore(wingfoil): apply the scenario configuration"
fi

# 4. The approver member, after the scenario's configuration, which may bring its own dna.yaml; added
#    only if no member has its e-mail yet. `dna add` commits by itself (since WingFoil v0.2.2; usage note N33
#    asked for this verb), and WingFoil reads approval authority from the committed dna.yaml (v0.2.2 and
#    v0.3.0, which decides every transition from HEAD). The team is
#    read whole first, as JSON: `grep -q` closing a pipe early would fail it under pipefail. A scenario's
#    dna.yaml naming another member "Benchmark Approver", or with no `approver` role, makes `dna add`
#    refuse, and the setup fail.
team="$(wingfoil --format json dna show team)"
if ! grep -qi '"approver@benchmark\.localhost"' <<< "$team"; then
  wingfoil dna add team.members --value "Benchmark Approver" --entry-email approver@benchmark.localhost \
    --entry-roles approver > /dev/null
fi
echo "setup done: $(git log --format=%s | head -n 3 | tr '\n' '|')"

Telemetry

The pinned wingfoil sends no telemetry: nothing is turned off.

Operating manual

The operating manual of the wingfoil arm, given to its agent.

Rules

The scenario's rules are wingfoil's own configuration, applied by the setup: no rules generator.

Docs control

baseline-docs: the same agent with the wingfoil arm's configuration rendered as documentation, without the harness. Kept from the configuration: .wingfoil/, docs/memory/.

ContentRenderedWhy
dna.yaml project, stacks, modulesyesthe project's description and parts
dna.yaml team and pathsnoWingFoil's process and navigation; the Benchmark Approver is the arm's approval mechanism
directives bound to developer, and the global onesyeswhat the wingfoil manual sends the agent to read
directives of other roles onlynothe manual does not send the agent to them
approved decision-logs and ADRsyesdecisions already taken: rules
Memory elements in any other state or typenoprocess state, not rules
workflows: name, description, phasesyesthe process the project follows
memory.yaml, Memory templates, workflows.yamlnothe tool's own mechanics

Git identity

Every arm commits as the same git identity, “Benchmark Approver” (adr-003, decision 7). In the wingfoil arm it is the approver member the setup adds to the project's team: WingFoil accepts the agent's approvals under it, once the neutral approver has replied (REQ-RUN-17). In a competitor arm it carries no approval authority.

Contest

Contest this setup: say which step is not as the tool's official documentation says, and how to correct it. An accepted correction runs as a new campaign; this one stays published.