Harness profiles

A profile is the files and settings you layer onto a harness. It is part of the variant:

variants: [
  { harness: "opencode", model: "anthropic/claude-sonnet-4.6" },
  {
    harness: "opencode",
    model: "anthropic/claude-sonnet-4.6",
    profile: { name: "docs-writer", dir: "./profile" },
    sandbox: "daytona",
  },
]

The first variant is the stock harness. The second is your configured one. Each variant keeps its own runs, so you read a profile change against that profile’s history.

The directory

dir resolves against the eval file. Sphynx reads two directories and nothing else:

profile/
  home/
    .config/opencode/opencode.json
  workspace/
    AGENTS.md
    .claude/skills/review/SKILL.md
  system.md
  profile.json
  README.md

Path	Where it lands
`home/**`	The sandbox home directory, after the harness writes its own credentials and config, so a profile can override them
`workspace/**`	The workspace, after the source is cloned

Files outside those two directories are ignored unless the manifest names them, so a README.md can document the profile without shipping. node_modules and .git are skipped at any depth.

The manifest

profile.json holds what is not a file. It is optional.

profile.json

{
  "systemPrompt": "system.md",
  "env": { "DOCS_BASE_URL": "https://example.com" },
  "install": "pip install -r requirements.txt",
  "run": "python run.py"
}

Key	Meaning
`systemPrompt`	The agent’s system prompt. See System prompts
`env`	Environment variables for every process the trial starts. Never secrets
`install`	A command run once while the sandbox is prepared. Any harness
`run`	The command that starts your process. Command harness only, and required there

If systemPrompt, install or run names a file inside the profile directory, the file’s content is used. Otherwise the value is used as written, so "systemPrompt": "system.md" ships the file and "systemPrompt": "Be terse." ships the sentence. A value that names a file outside the directory is an error. env values are always literal. install runs after both file stages, so it can read what the profile shipped, and before the case’s prepare step. A profile that sets run on any harness other than command is rejected.

Never put a key in env. A profile is committed beside the eval file, stored unencrypted and readable by anyone who can read its runs. Provider keys belong in an env credential, which is sealed, injected at run time, and overrides any profile variable with the same name.

Limits

Limit	Value
Files	256
Characters per file	96,000
Characters in total	2,000,000
Path length	512 characters, under `home/` or `workspace/`, no `..` segment

Files are text. A file containing a NUL byte is skipped, so a checked-in binary does not fail the compile.

System prompts

The prompt is always written into the sandbox. Each harness receives it differently:

Harness	Delivery
`claude`	`--append-system-prompt-file`. A profile also adds `--add-dir` on the workspace, plus `--settings` and `--mcp-config` when it ships `.claude/settings.json` or `.mcp.json`. Claude runs with `--bare`, so a `CLAUDE.md` is not discovered and must be passed this way
`opencode`	`OPENCODE_CONFIG_CONTENT`, merging the profile’s own config with the prompt’s path in `instructions`
`codex`	`-c developer_instructions=…`, carrying the text, added to the built-in prompt rather than replacing it
`gemini`, `qwen`, `pi`, `fx`, `cursor`	Prepended to the instructions file the agent reads, `GEMINI.md` for Gemini and `AGENTS.md` for the rest
`command`	`SPHYNX_SYSTEM_PROMPT_FILE`, holding the path

For file-based delivery, check the trajectory to confirm the harness read the instructions.

Identity

The profile name is part of the variant. Two profiles on the same harness, model and sandbox are two variants with separate histories, the same as two harnesses. The profile version is not. It is a hash of the files, system prompt, environment, install and run commands. When you edit a profile, the next batch adds a run on the same variant, and each run records the version it used. Renaming a file changes the version. Renaming the profile creates a new variant.

const [latest, previous] = (
  await sphynx.evals.runs.list({ caseId: "writes-docs", variant: variantId })
).runs;

console.log(previous?.profileVersion, latest?.profileVersion);

Comparing fairly

Ship the same skills to every variant. Skills live in agent-specific directories such as .claude/skills and .agents/skills, so write them under workspace/ into every directory the compared harnesses read. Otherwise you measure the skills, not the harness. Hold the model constant. If a profile needs a provider another variant cannot use, or the model ids do not overlap, the variants are not comparable. Say so next to the results instead of reading the difference. Change one thing at a time. If the profile and the model both change between runs, you cannot tell which one moved the result.

Fundamentals

Workflows

Advanced evals

Tools

Harness profiles

The directory

The manifest

Limits

System prompts

Identity

Comparing fairly

​The directory

​The manifest

​Limits

​System prompts

​Identity

​Comparing fairly

The directory

The manifest

Limits

System prompts

Identity

Comparing fairly