REPRODUCE THE FROZEN WILDCLAWBENCH LOCAL COMPARISON

Use this repository and wildclawbench/runs/2026-10-12 as the authoritative recorded specification. First run python3 harness/validation/verify.py and read README.md, tasks.json, conditions.json, metrics-plan.json, provenance/campaign-snapshots/selection.json, runtime-provenance.json and wild-fairness-review.json. Check the public manifest, source manifest, frozen inventories, adaptation receipts and canonical input hashes. Public sanitization is documented; raw artifact bytes are not bundled, and historical commit labels alone do not verify source bytes. Hold scoring if a required input or runtime cannot be verified.

Create a separate empty workspace, optionally using python3 harness/prepare.py --workspace <new-workspace>. This stages code and selection only; it does not install, authenticate or validate a fresh host. Render host paths in runtime/mount templates explicitly. Rebuild pinned environments and install locked dependencies; use the documented native container aliases. Stage subscription export helpers in isolated agent sources, never an active developer checkout. Do not copy result files, complete markers, validity holds, acceptance mappings or historical run state into a fresh experiment.

Run exactly the 36 tasks in tasks.json / the final selection.json: six per domain, three independent attempts per task for Leon, Hermes and OpenClaw, totaling 324 candidate slots. Preserve task prompts, input bytes, native rubrics, published timeouts and recorded modifications. Do not use the stale 12-task pilot roster, replace tasks, update dependencies, silently alter budgets or select attempts by their scores.

Pin Leon commit 111ccc8de5719108eebd4906ed0485cba5de7a50, Hermes v0.21.6 commit 818c13be1dc4fd28987e1e881a9408224afd4535, OpenClaw v2026.10.1 commit e178c39e9aae3fa440735217f9aef9574f286677. Preserve native loops, prompts, tools and recorded presets, including Hermes persistent-memory/user-profile disabled, 90 main and 50 delegated iterations. Call these configured harness results, not factory-default products. Verify frozen byte inventories as well as declared version provenance.

All candidate/judge reasoning inference uses OpenAI directly through ChatGPT/Codex OAuth subscription, gpt-6.1-sol, high effort and service_tier priority requested. No OpenRouter, API billing keys or reasoning-model substitutions. Native GPT Image 2 subscription image generation/editing is permitted under the recorded auxiliary media policy and counted separately. Keep OAuth secrets local. Verify actual model, reasoning, endpoint and authentication; record returned service tier and never infer served Fast from priority requested.

Use Harbor dataset internlm/WildClawBench-Harbor revision 33d06066716dc2666721f08ad2784c96e9b44dc5. Restore omitted Social live inputs and the original Thread Consolidation server from internlm/WildClawBench revision 75f945578aa00cbdb8f46e4d42e4f4e98f704b4f, following source-review and recovery receipts. Keep the native 20% injected service-error policy. Apply identical documented immutable-initial-fixture repairs to both API-leakage Safety graders. Preserve native action scoring rules and weights, judge image fitting and repaired asynchronous media completion/public transcript capture. Verify task fixture and grader controls quickly before candidate inference. Retired original containers/images are intentional; recreate them from recorded recipes and dependencies, not latest tags without verification.

Keep one benchmark and one task cohort active at a time. Prepare immutable layers once per cohort, then run all three agents times three independent attempts concurrently: up to nine subject to recorded host/quota admission limits, each with two CPUs and 4,096 MB RAM. Use isolated mutable workspaces/profiles, retain published timeouts and never add a 30-minute candidate cutoff. Record hardware, network/software, actual admission/concurrency and provider allowance.

Export outputs, public transcripts, native trajectories, tool schemas/calls, request events, usage, native submetrics and integrity evidence before deleting each attempt's containers, volumes and networks. Reuse immutable images/dependencies until all nine consumers finish; retire task-owned resources immediately when no longer needed, suite layers only after their final consumer. Preserve evidence and pinned sources. Never globally prune unrelated Docker objects. Track cleanup receipts.

Use the recovered metrics contract: scores/criteria and completion; artifact/factual/safety correctness; tool-schema validity with coverage, structured execution errors and repeated signatures; task-specific semantic action correctness; response headers/first text/first tool/full response p50/p95 with sample counts; between-turn proxies; setup, waiting, execution and grading durations; input/output/cached/reasoning usage; inference starts, completed/failed/unclosed requests and observable retry/recovery counts; image/auxiliary and judge usage separately; output tokens/full-response effective throughput; per-task three scores, mean/median/range; matched successful-efficiency comparisons and selected-task uncertainty. New resource peak monitoring can be recorded separately, but it is unknown historically. Do not call schema validity semantic accuracy, response throughput decode speed or event-spacing pure turn latency. Missing usage remains unknown/lower-bound; cached/reasoning tokens are subsets. Do not add model-based diagnostics by default.

When a run fails, scores zero/very low, lacks required artifacts or shows concrete environment/judge errors, perform quick local evidence checks. Retain genuine task failures and deadlines. Repair only demonstrably broken wiring, inputs, lifecycle capture or grading; preserve originals and replace all affected slots consistently across agents. Artifact-only regrades do not rerun candidates or extend their task budget. Do not selectively rerun low scores. Obtain approval for selection/rubric changes beyond recorded repairs. Run a final contextual fairness audit before publishing.

Rank matched cohorts only, averaging three attempt scores per task and equally weighting six domains. Publish task counts, presets, modifications, returned service tiers, coverage and uncertainty. The same conditions reproduce methodology, not guaranteed numerical scores, because model/provider responses, live sources, native injected errors and judging vary.

If separately authorized to continue the full campaign, preserve frozen PinchBench 32 tasks and Terminal-Bench 4.0 21 tasks, in that order after WildClawBench, with one suite active at a time. Pinch native conversation reuse and file persistence across fresh sessions must first pass the recorded controls; prior three session controls failed and no scored Pinch runs are accepted. Public preparation does not resolve that gate. When resuming the owner's private campaign rather than creating a fresh reproduction, honor its pause/resume gate and never rerun valid accepted slots.