> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sphynx.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Mocks

> Give agents typed local MCP servers, CLIs and HTTP APIs

Mocks are TypeScript objects. Their schemas type your handlers and validate requests and responses at runtime. Each trial gets its own copy of every MCP server, CLI and HTTP API. Keep fixtures in shared files and assertions in validators.

```bash theme={null}
npm install sphynx-sh zod
```

## Share fixtures

Keep fixture data and schemas separate from each interface:

```ts fixtures/users.ts theme={null}
import { z } from "zod";

export const User = z.object({ id: z.string(), name: z.string() });
export const GetUserInput = z.object({ id: z.string() });

export const user = {
  id: "user_fixture",
  name: "Ada",
} satisfies z.infer<typeof User>;

export const getUser = (id: string) => {
  if (id !== user.id) throw new Error(`User not found: ${id}`);
  return user;
};
```

## MCP

```ts mcp/users.ts theme={null}
import { resource, server, tool } from "sphynx-sh/mcp";
import { GetUserInput, User, getUser, user } from "../fixtures/users";

export const usersMcp = server({
  name: "users",
  version: "1.0.0",
  tools: [
    tool({
      name: "users_get",
      inputSchema: GetUserInput,
      outputSchema: User,
      handler: ({ id }) => getUser(id),
    }),
  ],
  resources: [
    resource({
      name: "current_user",
      uri: "users://current",
      outputSchema: User,
      handler: () => user,
    }),
  ],
});
```

MCP mocks support tools and fixed-URI resources. Schemas must implement Standard Schema and expose JSON Schema. The examples use Zod 4.

## CLI

```ts cli/users.ts theme={null}
import { cli, command } from "sphynx-sh/cli";
import { GetUserInput, User, getUser } from "../fixtures/users";

export const usersCli = cli({
  path: "users",
  version: "1.0.0",
  commands: [
    command({
      path: ["get"],
      inputSchema: GetUserInput,
      outputSchema: User,
      options: { id: { description: "User ID", type: "string" } },
      handler: ({ id }) => getUser(id),
    }),
  ],
});
```

* `cli.path` is the executable name. `command.path` is a non-empty list of argv segments.
* Every input key needs an `options` entry. Its flag is the key in kebab case. Set `name` (with the leading `--`) to override it.
* String options accept `--id value` or `--id=value`. Boolean options are `true` when present, so make them optional or give them a schema default.
* Positional arguments are not supported.

The runtime generates help and version output, validates output, prints JSON to stdout and exits nonzero on failure.

`sphynx-sh` also exports a `command` for `validate`. If one file needs both, import one under another name.

## Attach and validate

Attach MCP and CLI mocks on the suite, then check the call logs in a validator.

```ts users.eval.ts theme={null}
import { suite, type Validator } from "sphynx-sh";
import { usersCli } from "./cli/users";
import { GetUserInput, user } from "./fixtures/users";
import { usersMcp } from "./mcp/users";

const requestedFixture = (input: unknown) => {
  const result = GetUserInput.safeParse(input);
  return result.success && result.data.id === user.id;
};

const validate: Validator = async ({ answer, cli, mcp }) => {
  const [text, cliCalls, mcpCalls] = await Promise.all([
    answer(),
    cli.calls("users"),
    mcp.calls("users"),
  ]);
  const called =
    cliCalls.some(({ input }) => requestedFixture(input)) ||
    mcpCalls.some(
      ({ input, kind, name }) =>
        kind === "tool" && name === "users_get" && requestedFixture(input)
    );
  return { passed: called && text.includes(user.name) };
};

export default suite({
  id: "users-interface",
  prompt: "Use an available interface to retrieve user_fixture and report its name.",
  mcp: [usersMcp],
  cli: [usersCli],
  cases: [{ id: "gets-user", validate }],
  variants: [{ harness: "codex", model: "model-id" }],
  trials: 1,
});
```

Check both the recorded call and the outcome. Call inputs and outputs are `unknown`, so parse them with the shared schema before asserting on values.

## HTTP APIs

Define endpoints with `api()` and `endpoint()`. Standard Schema types the handler input and checks each response against the schema for its status code. Handlers are plain TypeScript.

```ts mocks/catalog.ts theme={null}
import { api, endpoint } from "sphynx-sh/api";
import { z } from "zod";

const item = { id: "fixture", name: "Test item" };
const Item = z.object({ id: z.string(), name: z.string() });

export const catalog = api({
  name: "catalog",
  endpoints: [
    endpoint({
      method: "GET",
      path: "/items/:id",
      inputSchema: z.object({ params: z.object({ id: z.string() }) }),
      responses: {
        200: Item,
        404: z.object({ message: z.string() }),
      },
      handler: ({ params }) => params.id === item.id
        ? { status: 200, body: item }
        : { status: 404, body: { message: "Item not found" } },
    }),
  ],
});
```

Move `item` and `Item` into a shared fixture file to reuse them with MCP and CLI mocks. API names use lowercase letters, digits and hyphens.

### Attach an API

```ts catalog.eval.ts theme={null}
import { suite, type Validator } from "sphynx-sh";
import { catalog } from "./mocks/catalog";

const validate: Validator = async ({ api, answer }) => ({
  passed: (await api.calls("catalog")).some(
    ({ path, status }) => path === "/items/fixture" && status === 200
  ) && (await answer()).includes("Test item"),
});

export default suite({
  id: "catalog-api",
  api: [catalog],
  prompt: "Use the local catalog API to retrieve fixture and report its name.",
  cases: [{ id: "retrieve-item", validate }],
  variants: [{ harness: "codex", model: "model-id" }],
  trials: 1,
});
```

In each trial, Sphynx starts a loopback HTTP server on a free port and adds its base URL and endpoint list to the agent's input. Your prompt is unchanged. Curl, Python and SDK clients can call it directly.

Prepare functions and validators get the URL from `await api.url("catalog")`. No environment variables or global `fetch` are replaced.

### Requests and responses

* The input schema receives `{ params, query, headers, body }`. Headers are lowercase. Repeated query values are arrays. An empty body is `null`, and a nonempty body must be JSON. Use schema transforms for coercion.
* Handlers return `{ status, body, headers? }`. Use `null` for bodyless responses such as 204. The runtime sets content type and framing headers.
* An optional endpoint `description` is shown to the agent.
* Error responses you declare, such as 404 or 429, are normal mock behavior. Unmatched routes return 404 and are recorded with `matched: false`.
* A handler that throws, a response that fails its schema, or a body that cannot be serialized fails the mock. Nothing is forwarded upstream.

Handlers take a second argument, `{ signal, log }`. Call `log(value)` to attach evidence to the call, and use `signal` to cancel async work.

### Evidence

HTTP calls appear in the trial's Calls view and are available to judges. `api.calls(name?)` returns records with method, path, status, timing, whether a route matched, input, output, handler logs and errors. Calls to `api.url` and `api.calls` appear in the validator's evidence.

Input, output and log values are bounded snapshots with `text`, `format`, `state` and `truncated` fields. Check those before parsing captured JSON.

Authorization headers, cookies and common secret fields are redacted before recording. Add more field names (case-insensitive) with `api({ redact: ["accessToken"], ... })`. Before a call is stored in the trial, anything that looks like a key and every secret the trial knows, such as its harness and sandbox credentials, are also replaced with `[redacted]`, as described in [secrets in evidence](/evals/results#secrets-in-evidence). A secret with no known shape inside a string is still recorded, so keep credentials out of fixtures.

### Validator-owned servers

Use `withApi()` when a validator needs its own server with hidden test data. The server closes when the callback finishes or fails.

```ts theme={null}
import { withApi } from "sphynx-sh/api";

await withApi({
  api: catalog,
  run: async ({ url, calls }) => {
    const response = await fetch(`${url}/items/fixture`);
    console.info("HTTP evidence", await calls());
    return response.json();
  },
});
```

These calls are returned by `calls()`, not the trial's call log. Log them inside the validator to include them in its logs.

### Limits

* JSON HTTP endpoints only. No proxying, streaming, WebSockets or automatic recording.
* Request bodies up to 1 MiB, handlers up to 30 seconds, and 256 requests per mock runtime.
* `api.calls()` inside a validator returns values up to 64,000 characters. The trial stores each value at up to 16,000 characters, cut after redaction.

Mocks never call an upstream service, but they do not stop the agent from contacting other hosts. Use a sandbox egress policy for network isolation.

## Where mocks run

Mocks apply to every variant in the suite. Put variants in separate eval files when they need different mocks. CLI mocks are installed for every harness. MCP mocks are configured for every [built-in harness](/evals/variants), but not the [command harness](/evals/command-harness).

Handlers need no credentials for the service they simulate. Fixture data is bundled into the sandbox, so keep secrets out of it. Running the eval still needs `SPHYNX_API_KEY` and a model connection.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.