> ## Documentation Index
> Fetch the complete documentation index at: https://docs.veecle.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# test

> Check that your firmware does what it should on a virtual board.

**You**

Tell your agent what the firmware should do. It writes the suite, runs it on the virtual board, and returns a pass or fail for each check. The same checks can fail a pull request. Setup is on [Test in CI](/use-cases/ci-on-every-pr).

## When to use it

Use a test when each check needs a result. A firmware that stops printing fails the test. When you want the printed output, use [run](/tools/run).

## Ask your agent

```text theme={null}
This build is fixed at 35°C. Above 30°C it should log a blink every 3 seconds. Write a test that it boots, reaches that blink without a fault, and logs the blink within 3 seconds on the STM32F4 Discovery, and run it.
```

## What you get back

You get a pass or fail for each test. `succeeded` means the suite ran, not that every test passed. Check `result.summary`.

## What you can test

A suite advances virtual time and watches UART lines.

**Boot.** Expect a ready line within a few seconds.

**Init order.** Expect several lines in order. Each check only sees output printed after it starts.

**Timing.** A one-second tick is checked against chip-accurate interrupt timing. The call resolves to `matched` or `timed_out`. It does not report how long the interval actually was.

**Long-run liveness.** The firmware can run for minutes, in chunks of up to 60 seconds, then be checked for a line that shows it is still alive.

**HardFault.** The check passes when the line you expect shows up. If the firmware faults first, that line never arrives and the check fails. The suite does not report a HardFault on its own. A handler that prints the fault address makes the stop visible on the UART the suite already watches.

## Not supported yet

A suite observes firmware through UART only. GPIO input and button stimulus, pin-state or register and memory reads, and external sensors or I2C/SPI devices are not supported yet.

An LED blink counts when the firmware logs it. The suite cannot read the pin. A fixed temperature works the same way: build it into the firmware, then expect that line. The suite does not feed the value in.

**Agents**

## run\_test\_suite

`run_test_suite` takes:

**project\_id** `string` *required*

The project this test belongs to, from `create_project`. Ids look like `project_...`.

**firmware\_artifact\_id** `string` *required*

The firmware ELF, from `issue_upload_ticket`. Ids look like `artifact_...`.

**suite\_artifact\_id** `string` *required*

The TypeScript suite file, from a second `issue_upload_ticket`. Ids look like `artifact_...`.

**board** `object` *required*

The virtual board. `model` is a board model such as `stm32f4_discovery`. `uarts` is the list of UART names to capture, for example `["usart2"]`.

Your agent calls `issue_upload_ticket` twice, both with the same `project_id`. It PUTs the ELF to the first `upload_url` and the suite file to the second. `run_test_suite` returns `run_test_suite_id` as soon as the job starts. Pass that id to `wait_for_job` as `job_id`.

## How it works

The suite drives the simulation. The board is paused until the suite advances time. Virtual time moves only inside the awaited calls. Between those calls the board stays frozen.

`runUntilUartLine` only watches the UARTs named in `board.uarts`.

## Writing a suite

A suite is one TypeScript file:

```ts theme={null}
import { expect } from "jsr:@std/expect";

Deno.test("firmware greets", async () => {
  const outcome = await simulation.runUntilUartLine("Hello world!", 10000);
  expect(outcome.kind).toBe("matched");
});
```

Tests use `Deno.test`, or `describe` and `it` from `jsr:@std/testing/bdd`. The only imports available are `jsr:@std/testing/bdd`, `jsr:@std/expect`, and `zod`. No network, npm packages, or imports of other files. Chiplab type-checks the suite before it runs.

The server provides the `simulation` types itself. The suite must not reference `chiplab.d.ts`. Only the suite file is uploaded, and it is checked without network access, so a `/// <reference>` to the definitions, by local path or URL, fails to resolve and the suite does not compile.

For editor support, point the project's `deno.json` at the published definitions:

```json theme={null}
{ "compilerOptions": { "types": ["https://chiplab.veecle.ai/mcp/resources/schemas/chiplab.d.ts"] } }
```

A local `deno check` then needs `--allow-import` to fetch them.

## The simulation API

The suite gets a global `simulation`.

* `simulation.runForMillis(millis)` runs for `millis` of virtual time, then pauses.
* `simulation.runUntilUartLine(expected, maxMillis)` runs until a line containing `expected` appears, or `maxMillis` elapses. It resolves to `{ kind: "matched" }` or `{ kind: "timed_out" }`.

Both take 1 to 60000 ms.

Two matching rules:

* `runUntilUartLine` only matches output produced after the call starts.
* `expected` must not contain control characters such as a newline.

## Example checks

**Boot.** Expect `System ready` within 3 seconds.

```ts theme={null}
Deno.test("boots", async () => {
  const outcome = await simulation.runUntilUartLine("System ready", 3000);
  expect(outcome.kind).toBe("matched");
});
```

**Init order.** Expect several lines in order. Each call only matches output printed after it starts.

```ts theme={null}
Deno.test("init order", async () => {
  expect((await simulation.runUntilUartLine("clocks on", 3000)).kind).toBe("matched");
  expect((await simulation.runUntilUartLine("uart on", 3000)).kind).toBe("matched");
  expect((await simulation.runUntilUartLine("System ready", 3000)).kind).toBe("matched");
});
```

**Timing.** A 1 second tick should reach `tick 5` within 5.5 seconds. That relies on the chip-accurate interrupt timing. The call resolves to `matched` or `timed_out`. It does not report how long the interval actually was.

```ts theme={null}
Deno.test("one second tick", async () => {
  const outcome = await simulation.runUntilUartLine("tick 5", 5500);
  expect(outcome.kind).toBe("matched");
});
```

**Long-run liveness.** `runForMillis` takes at most 60000 ms, so a longer run is 60 second chunks. Then expect `alive`. The line has to be printed after those chunks, because the next call only sees output from that point on.

```ts theme={null}
Deno.test("stays alive", async () => {
  await simulation.runForMillis(60000);
  await simulation.runForMillis(60000);
  const outcome = await simulation.runUntilUartLine("alive", 3000);
  expect(outcome.kind).toBe("matched");
});
```

**Compile-time stub.** Build the firmware with a fixed raw value. The suite only expects the UART line. It does not feed that value in.

```ts theme={null}
Deno.test("stubbed temperature", async () => {
  const outcome = await simulation.runUntilUartLine("Temp: 23.5C", 3000);
  expect(outcome.kind).toBe("matched");
});
```

## Reading the result

`succeeded` means the suite ran, not that it passed. Check `result.summary`.

| What happened | `wait_for_job` |
| - | - |
| The suite ran | `status` is `succeeded`. `result.summary` has `planned`, `passed`, and `failed`. |
| The suite did not compile, or it threw before any test ran | `status` is `succeeded`. `result.summary` is `null`. Read `testsuite_stderr`. |
| The job failed | `status` is `failed`. The response has an error string and no artifacts. |

A suite that expects `Hello world!` on `stm32f4_discovery` / `usart2`, run against firmware that prints that line:

```json theme={null}
{
  "input": {
    "board": { "model": "stm32f4_discovery", "uarts": ["usart2"] },
    "firmware_artifact_id": "artifact_01m3ye77kfec79agpsjamg34nf",
    "suite_artifact_id": "artifact_01m3ye76e2fkrb99wrpwvae4xa"
  },
  "job_id": "run_test_suite_01m3ye78dkf5na6977req4trz6",
  "project_id": "project_01m3ydfkccerpswp3zh14rpeqh",
  "result": {
    "artifacts": ["testsuite_report", "testsuite_stdout", "firmware_uart"],
    "summary": { "failed": 0, "passed": 1, "planned": 1 }
  },
  "status": "succeeded",
  "timed_out": false
}
```

`testsuite_report` for that run:

```json theme={null}
{
  "planned": 1,
  "passed": 1,
  "failed": 0,
  "skipped": 0,
  "todo": 0,
  "bail_out": false,
  "tests": [{ "number": 1, "name": "firmware greets", "status": "passed" }],
  "diagnostics": ["# test.ts"]
}
```

The same suite against firmware that prints `before fault` and then hits a HardFault. The wait times out, the assertion fails, and the job still succeeds:

```json theme={null}
{
  "input": {
    "board": { "model": "stm32f4_discovery", "uarts": ["usart2"] },
    "firmware_artifact_id": "artifact_01m3ye7hgaeyya3vwmaye3ep9g",
    "suite_artifact_id": "artifact_01m3ye76e2fkrb99wrpwvae4xa"
  },
  "job_id": "run_test_suite_01m3ye7j9bfjhsv9spkmepxa5v",
  "project_id": "project_01m3ydfkccerpswp3zh14rpeqh",
  "result": {
    "artifacts": ["testsuite_report", "testsuite_stdout", "testsuite_stderr", "firmware_uart"],
    "summary": { "failed": 1, "passed": 0, "planned": 1 }
  },
  "status": "succeeded",
  "timed_out": false
}
```

`testsuite_report` for that run:

```json theme={null}
{
  "planned": 1,
  "passed": 0,
  "failed": 1,
  "skipped": 0,
  "todo": 0,
  "bail_out": false,
  "tests": [{ "number": 1, "name": "firmware greets", "status": "failed" }],
  "diagnostics": ["# test.ts"]
}
```

## Artifacts

`result.artifacts` lists what `issue_download_ticket` can download. Pass the `run_test_suite_id` as `job_id`.

* `testsuite_report`: JSON with the counts, plus `skipped`, `todo`, `bail_out`, per-test `tests` (`number`, `name`, `status`), and `diagnostics`.
* `testsuite_stdout` and `testsuite_stderr`: the suite's output, including `console.log`.
* `firmware_uart`: the firmware's UART output. A [run](/tools/run) calls this artifact `uart`.
* `simulator_stdout` and `simulator_stderr`: simulator logs.

Empty artifacts are omitted, so a passing suite may have no `testsuite_stderr`.

## Limits

The whole suite has 5 minutes of wall-clock time. Exceeding it fails the job with `internal server error` and produces no artifacts.

Each `runForMillis` and `runUntilUartLine` call takes 1 to 60000 ms.

Each `Deno.test` or top-level `describe` counts as one result. Steps and nested `it` calls are not reported individually.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.