> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sphynx.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Import a case file

> Turn existing JSON or YAML cases into a TypeScript suite

`sphynx eval import` converts an existing case file into a `*.eval.ts` suite.

```bash theme={null}
sphynx eval import --format evals-json ./evals.json --out ./release-notes.eval.ts
```

Without `--out`, the suite prints to stdout. A summary of converted and unconverted assertions goes to stderr.

| `--format` | Input |
| - | - |
| `evals-json` | One JSON file holding every case |
| `yaml` | One YAML file per case, or a directory of them |

The generated suite has a placeholder variant (`codex` on `gpt-5.6-sol`) and `trials: 3`, with each case's text passed through a `task` variable.

## evals-json

```json theme={null}
{
  "skill_name": "release notes",
  "evals": [
    {
      "id": 1,
      "name": "Lists the breaking change",
      "prompt": "Read CHANGELOG.md and write the release notes.",
      "expected_output": "A short note naming the breaking change.",
      "files": ["fixtures/changelog"],
      "assertions": [
        {
          "text": "names the removed flag",
          "kind": "content_contains_all",
          "needles": ["--legacy", "removed"]
        }
      ]
    }
  ]
}
```

`skill_name` and `evals` are required, and each case needs `id`, `prompt` and `assertions`. An assertion is a structured object or a prose string.

| `kind` | Generated check |
| - | - |
| `content_contains_any` | `containsAny(answer, needles)` |
| `content_contains_all` | `containsAll(answer, needles)` |
| `content_contains_none` | `containsNone(answer, needles)` |

Matching is case-insensitive. Prose assertions cannot be converted safely, so each becomes a placeholder that always fails:

```ts theme={null}
validate: async (context) => {
  const answer = await context.answer();

  return [
    /* Write this check, then delete the line under it: */
    /* The summary reads as though a maintainer wrote it. */
    unwritten("The summary reads as though a maintainer wrote it."),
  ].every(Boolean);
},
```

`expected_output` becomes a comment, not a check. `files` names directories but not their contents, so each case starts with `files({})` and a comment naming them.

## yaml

A directory imports every `*.yaml` and `*.yml` file in it, sorted by name.

```bash theme={null}
sphynx eval import --format yaml ./cases --out ./browsing.eval.ts
```

```yaml theme={null}
name: Checkout flow
task: Open the demo storefront and pay with the test card
judge_context:
  - Reach the basket page before paying
  - Use the test card, never a real one
max_steps: 12
```

`name` and `task` are required. The task becomes the prompt. Each `judge_context` entry needs human judgment, so it becomes a failing placeholder. A case without `judge_context` gets one placeholder asking what a good answer is.

`max_steps` becomes a comment, because Sphynx does not cap agent steps. YAML cases start with `files({})` because the format carries no source files.

## Finish the suite

1. Review the generated file.
2. Replace each `files({})` with fixture files or a [repository source](/evals/cases).
3. Set the harness, model and sandbox for each variant.
4. Replace every `unwritten(...)` placeholder with a real check.
5. Run it with `sphynx eval ./suite.eval.ts`.

An invalid file fails with its path, case and field. A directory is never partially imported.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.