Skip to main content

​
Install

Set SPHYNX_API_KEY before commands that call Sphynx. --help, --version, eval import and eval --local work without it.

​
Commands

​
Check which organization a key uses

Every run, suite and variable a key touches belongs to this organization. --json prints the same facts as JSON. The secret is never printed. sphynx eval also prints the organization once before its first batch.

​
Run evals

With no file, the CLI finds every *.eval.ts under the current directory (skipping node_modules, dist, build and hidden folders) and runs them one after another. Each suite file starts one batch: one run per case per variant, each with the suite’s number of trials. The command waits for the batch and applies the gate. In an interactive terminal the grid updates as trials finish. Otherwise it prints the final grid once. With --case and no file, the CLI runs a case Sphynx already stores again, on its newest version. Under every gate, a batch that fails to start, fails, or times out fails the command. sphynx eval import reads an existing case file (or a directory of them) and prints a suite, or writes it with --out. It converts the mechanical assertions and leaves prose ones failing until someone writes the check. It reports how many of each on stderr. See importing a case file.

​
Environment variables

sphynx env manages the keys and variables in Settings > Environment. It needs a key with credentials:read to list and credentials:write to change. set prompts for the value, or reads it from stdin when stdin is not a terminal (a pipe, a file or a heredoc), so the secret is never a command line argument. Pipe it from a variable or a file rather than typing it into a heredoc, which your shell may keep in its history:
Subscriptions, such as ChatGPT for Codex, are added in the dashboard.

​
Exit codes

Failures print without a stack trace. Status messages go to stderr. When Sphynx cannot be reached, the error says so and names the address it tried. The first request checks that the server answers and gives up within 5 seconds if it does not. After that, a read that gets no answer in 30 seconds gives up with an error that says the server was slow, not unreachable. A change is never cut off once the server has it, so it cannot be applied twice.

​
Environment