← How the benchmark works

How the speckit arm is set up, before its agent starts: the speckit harness at the campaign's pin, from the repository's arms/speckit/.

Setup script

#!/usr/bin/env bash
# The speckit arm's setup (REQ-RUN-18), from task-065's spike: Spec Kit installed from its wheel bundle and
# initialized as its documentation says for Claude Code. The runner copies the harness artifact (the bundle, packed as
# a .tgz with the wheels flat at its root, REQ-FMT-12) to ~/harness.tgz, as it does for WingFoil; HOME and WORKSPACE
# are the runner's (the project rules' snapshot runs this with HOME=/build, WORKSPACE=/build/workspace).
set -euo pipefail
WORKSPACE="${WORKSPACE:-/workspace}"
export PATH="$HOME/.local/bin:$PATH"
cd "$WORKSPACE"
BUNDLE="$HOME/speckit-bundle"
mkdir -p "$BUNDLE" && tar -xzf "$HOME/harness.tgz" -C "$BUNDLE"
# Offline, from the bundle only: the wheel of the pinned tag and its dependencies, resolved when the bundle was built
# (for the run image's Python 3.11 on amd64: pyyaml's wheel is platform-specific).
uv tool install --offline --no-index --find-links "$BUNDLE" specify-cli
# `--integration claude` is required: without it a non-interactive init defaults to Copilot (docs/reference/core.md).
specify init --here --force --integration claude --script sh --ignore-agent-tools
# Telemetry: Spec Kit v1.1.0 has none (no telemetry or analytics code; install and init ran with --network none).

Telemetry

The pinned speckit sends no telemetry: nothing is turned off.

Operating manual

The operating manual of the speckit arm, given to its agent.

Rules

Written to .specify/memory/constitution.md: the scenario's rules — the directives its developer reads — rendered as the principles of Spec Kit's constitution, after the setup, in place of the template init leaves.

Docs control

speckit-docs: the same agent with the speckit arm's configuration rendered as documentation, without the harness. Kept from the configuration: .specify/memory/.

ContentRenderedWhy
the constitution's principlesyesthe scenario's rules, which the manual sends the agent to read
the constitution template's placeholdersnono project information
skills (.claude/skills/speckit-*)nothe tool's own mechanics: how its process is followed
templates, scripts, the workflow and its registry (.specify/)nothe tool's own mechanics
integration, option and manifest files (.specify/)nothe tool's own settings

Git identity

Every arm commits as the same git identity, “Benchmark Approver” (adr-003, decision 7). In the speckit arm, a competitor arm, it carries no approval authority: it is only a name.

Contest

Contest this setup: say which step is not as the tool's official documentation says, and how to correct it. An accepted correction runs as a new campaign; this one stays published.

Bundle

The Spec Kit bundle resolves its dependencies when it is built; their digests are in its SHA256SUMS, and the run installs only from the bundle.