> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sphynx.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Sphynx CLI

> Run eval suites and connect credentials from a terminal

## Install

<CodeGroup>
  ```bash npx theme={null}
  npx sphynx-sh --help
  ```

  ```bash global theme={null}
  npm install -g sphynx
  sphynx --help
  ```
</CodeGroup>

Set `SPHYNX_API_KEY` before commands that call Sphynx. `--help`, `--version`, `eval import` and `eval --local` work without it.

## Commands

| Command | Purpose |
| - | - |
| `sphynx eval [file] [options]` | Run suite files, or a stored case, as batches and gate on the results |
| `sphynx eval import --format <format> <path> [--out <file>]` | Turn a case file a team already wrote into a suite |
| `sphynx connectors <subcommand>` | Connect the credentials evals use |
| `sphynx whoami [--json]` | Show the organization and key `SPHYNX_API_KEY` acts for |
| `sphynx --completions <shell>` | Print completions for zsh, bash or fish |

## Check which organization a key uses

```bash theme={null}
sphynx whoami
```

```text theme={null}
organization       Acme
organization slug  acme
organization id    org_01J9...
key                ci
key starts with    anp_AbCd
permissions        evals:read, evals:write
```

Every run, suite and connector a key touches belongs to this organization. `--json` prints the same facts as JSON. The secret is never printed. `sphynx eval` also prints the organization once before its first batch.

## Run evals

```bash theme={null}
sphynx eval
sphynx eval ./evals/smoke.eval.ts
sphynx eval ./evals/smoke.eval.ts --case writes-hello --variant codex/gpt-5.6-sol
sphynx eval --case writes-hello
```

With no file, the CLI finds every `*.eval.ts` under the current directory (skipping `node_modules`, `dist`, `build` and hidden folders) and runs them one after another. Each suite file starts one batch: one run per case per variant, each with the suite's number of trials. The command waits for the batch and applies the gate. In an interactive terminal the grid updates as trials finish. Otherwise it prints the final grid once.

With `--case` and no file, the CLI runs a case Sphynx already stores again, on its newest version.

| Option | Purpose |
| - | - |
| `--case <id>` | Run one case. With a file, picks that case from it. Without one, runs the stored case again |
| `--variant <variant>` | Run one variant: `harness/model`, `harness/model@profile` or a profile name for a file, or a variant id for a stored case. Repeat it for more |
| `--local` | Run every case on every variant, with the suite's trials, on this machine instead of a cloud sandbox. See [local runs](/evals/local) |
| `--fail-on` | Gate, `failures` by default: fail when any run fails or has a trial that did not pass. `strict` also fails when a run or trial is missing or unfinished. `never` ignores trial results |
| `--timeout` | Seconds to wait per batch, default `1200`. A timeout fails the command but does not cancel the batch |
| `--output` | Write a JSON report with each file's batch ID, batch and problems |
| `--no-wait` | Start each batch and print its ID and runs, without waiting |
| `--json` | Print each finished batch as JSON instead of the grid |

Under every gate, a batch that fails to start, fails, or times out fails the command.

`sphynx eval import` reads an existing case file (or a directory of them) and prints a suite, or writes it with `--out`. It converts the mechanical assertions and leaves prose ones failing until someone writes the check. It reports how many of each on stderr. See [importing a case file](/evals/import).

## Connect credentials

`sphynx connectors add <integration>` connects a credential for the organization. It prompts for the secret, or reads it from stdin when not run in a terminal, so the secret never appears in your shell history. Pass `--default` to make it the one evals use, `--name` to label it and `--auth` to pick an auth method. Integrations that sign in through a browser are connected in the dashboard instead.

`sphynx connectors integrations` lists what can be connected, `connectors list` shows what the organization has, and `connectors remove <id>` deletes one.

## Exit codes

| Code | Meaning |
| - | - |
| `0` | Every batch passed the gate |
| `2` | The gate failed |
| `1` | An error, such as a batch that could not start or finish |
| `130` | Interrupted |

Failures print without a stack trace. Status messages go to stderr.

When Sphynx cannot be reached, the error says so and names the address it tried. The first request checks that the server answers and gives up within 5 seconds if it does not. After that, a read that gets no answer in 30 seconds gives up with an error that says the server was slow, not unreachable. A change is never cut off once the server has it, so it cannot be applied twice.

## Environment

| Variable | Purpose |
| - | - |
| `SPHYNX_API_KEY` | API key |
| `SPHYNX_BASE_URL` | API origin, default `https://api.sphynx.sh` |
| `SPHYNX_WEB_URL` | Origin for batch links, default `https://www.sphynx.sh` |
| `GITHUB_TOKEN` | Optional. With `GITHUB_REPOSITORY` set, `sphynx eval` posts a check named `sphynx` on the commit (the PR head on pull requests) |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.