test

You
Agents

Tell your agent what the firmware should do. It writes the suite, runs it on the virtual board, and returns a pass or fail for each check. The same checks can fail a pull request. Setup is on Test in CI.

When to use it

Use a test when each check needs a result. A firmware that stops printing fails the test. When you want the printed output, use run.

Ask your agent

This build is fixed at 35°C. Above 30°C it should log a blink every 3 seconds. Write a test that it boots, reaches that blink without a fault, and logs the blink within 3 seconds on the STM32F4 Discovery, and run it.

What you get back

You get a pass or fail for each test. succeeded means the suite ran, not that every test passed. Check result.summary.

What you can test

A suite advances virtual time and watches UART lines.Boot. Expect a ready line within a few seconds.Init order. Expect several lines in order. Each check only sees output printed after it starts.Timing. A one-second tick is checked against chip-accurate interrupt timing. The call resolves to matched or timed_out. It does not report how long the interval actually was.Long-run liveness. The firmware can run for minutes, in chunks of up to 60 seconds, then be checked for a line that shows it is still alive.HardFault. The check passes when the line you expect shows up. If the firmware faults first, that line never arrives and the check fails. The suite does not report a HardFault on its own. A handler that prints the fault address makes the stop visible on the UART the suite already watches.

Not supported yet

A suite observes firmware through UART only. GPIO input and button stimulus, pin-state or register and memory reads, and external sensors or I2C/SPI devices are not supported yet.An LED blink counts when the firmware logs it. The suite cannot read the pin. A fixed temperature works the same way: build it into the firmware, then expect that line. The suite does not feed the value in.

run_test_suite

run_test_suite takes:

string

required

The project this test belongs to, from create_project. Ids look like project_....

string

required

The firmware ELF, from issue_upload_ticket. Ids look like artifact_....

string

required

The TypeScript suite file, from a second issue_upload_ticket. Ids look like artifact_....

object

required

The virtual board. model is a board model such as stm32f4_discovery. uarts is the list of UART names to capture, for example ["usart2"].

Your agent calls issue_upload_ticket twice, both with the same project_id. It PUTs the ELF to the first upload_url and the suite file to the second. run_test_suite returns run_test_suite_id as soon as the job starts. Pass that id to wait_for_job as job_id.

How it works

The suite drives the simulation. The board is paused until the suite advances time. Virtual time moves only inside the awaited calls. Between those calls the board stays frozen.runUntilUartLine only watches the UARTs named in board.uarts.

Writing a suite

A suite is one TypeScript file:

import { expect } from "jsr:@std/expect";

Deno.test("firmware greets", async () => {
  const outcome = await simulation.runUntilUartLine("Hello world!", 10000);
  expect(outcome.kind).toBe("matched");
});

Tests use Deno.test, or describe and it from jsr:@std/testing/bdd. The only imports available are jsr:@std/testing/bdd, jsr:@std/expect, and zod. No network, npm packages, or imports of other files. Chiplab type-checks the suite before it runs.The server provides the simulation types itself. The suite must not reference chiplab.d.ts. Only the suite file is uploaded, and it is checked without network access, so a /// <reference> to the definitions, by local path or URL, fails to resolve and the suite does not compile.For editor support, point the project’s deno.json at the published definitions:

{ "compilerOptions": { "types": ["https://chiplab.veecle.ai/mcp/resources/schemas/chiplab.d.ts"] } }

A local deno check then needs --allow-import to fetch them.

The simulation API

The suite gets a global simulation.

simulation.runForMillis(millis) runs for millis of virtual time, then pauses.
simulation.runUntilUartLine(expected, maxMillis) runs until a line containing expected appears, or maxMillis elapses. It resolves to { kind: "matched" } or { kind: "timed_out" }.

Both take 1 to 60000 ms.Two matching rules:

runUntilUartLine only matches output produced after the call starts.
expected must not contain control characters such as a newline.

Example checks

Boot. Expect System ready within 3 seconds.

Deno.test("boots", async () => {
  const outcome = await simulation.runUntilUartLine("System ready", 3000);
  expect(outcome.kind).toBe("matched");
});

Init order. Expect several lines in order. Each call only matches output printed after it starts.

Deno.test("init order", async () => {
  expect((await simulation.runUntilUartLine("clocks on", 3000)).kind).toBe("matched");
  expect((await simulation.runUntilUartLine("uart on", 3000)).kind).toBe("matched");
  expect((await simulation.runUntilUartLine("System ready", 3000)).kind).toBe("matched");
});

Timing. A 1 second tick should reach tick 5 within 5.5 seconds. That relies on the chip-accurate interrupt timing. The call resolves to matched or timed_out. It does not report how long the interval actually was.

Deno.test("one second tick", async () => {
  const outcome = await simulation.runUntilUartLine("tick 5", 5500);
  expect(outcome.kind).toBe("matched");
});

Long-run liveness. runForMillis takes at most 60000 ms, so a longer run is 60 second chunks. Then expect alive. The line has to be printed after those chunks, because the next call only sees output from that point on.

Deno.test("stays alive", async () => {
  await simulation.runForMillis(60000);
  await simulation.runForMillis(60000);
  const outcome = await simulation.runUntilUartLine("alive", 3000);
  expect(outcome.kind).toBe("matched");
});

Compile-time stub. Build the firmware with a fixed raw value. The suite only expects the UART line. It does not feed that value in.

Deno.test("stubbed temperature", async () => {
  const outcome = await simulation.runUntilUartLine("Temp: 23.5C", 3000);
  expect(outcome.kind).toBe("matched");
});

Reading the result

succeeded means the suite ran, not that it passed. Check result.summary.

What happened	`wait_for_job`
The suite ran	`status` is `succeeded`. `result.summary` has `planned`, `passed`, and `failed`.
The suite did not compile, or it threw before any test ran	`status` is `succeeded`. `result.summary` is `null`. Read `testsuite_stderr`.
The job failed	`status` is `failed`. The response has an error string and no artifacts.

A suite that expects Hello world! on stm32f4_discovery / usart2, run against firmware that prints that line:

{
  "input": {
    "board": { "model": "stm32f4_discovery", "uarts": ["usart2"] },
    "firmware_artifact_id": "artifact_01m3ye77kfec79agpsjamg34nf",
    "suite_artifact_id": "artifact_01m3ye76e2fkrb99wrpwvae4xa"
  },
  "job_id": "run_test_suite_01m3ye78dkf5na6977req4trz6",
  "project_id": "project_01m3ydfkccerpswp3zh14rpeqh",
  "result": {
    "artifacts": ["testsuite_report", "testsuite_stdout", "firmware_uart"],
    "summary": { "failed": 0, "passed": 1, "planned": 1 }
  },
  "status": "succeeded",
  "timed_out": false
}

testsuite_report for that run:

{
  "planned": 1,
  "passed": 1,
  "failed": 0,
  "skipped": 0,
  "todo": 0,
  "bail_out": false,
  "tests": [{ "number": 1, "name": "firmware greets", "status": "passed" }],
  "diagnostics": ["# test.ts"]
}

The same suite against firmware that prints before fault and then hits a HardFault. The wait times out, the assertion fails, and the job still succeeds:

{
  "input": {
    "board": { "model": "stm32f4_discovery", "uarts": ["usart2"] },
    "firmware_artifact_id": "artifact_01m3ye7hgaeyya3vwmaye3ep9g",
    "suite_artifact_id": "artifact_01m3ye76e2fkrb99wrpwvae4xa"
  },
  "job_id": "run_test_suite_01m3ye7j9bfjhsv9spkmepxa5v",
  "project_id": "project_01m3ydfkccerpswp3zh14rpeqh",
  "result": {
    "artifacts": ["testsuite_report", "testsuite_stdout", "testsuite_stderr", "firmware_uart"],
    "summary": { "failed": 1, "passed": 0, "planned": 1 }
  },
  "status": "succeeded",
  "timed_out": false
}

testsuite_report for that run:

{
  "planned": 1,
  "passed": 0,
  "failed": 1,
  "skipped": 0,
  "todo": 0,
  "bail_out": false,
  "tests": [{ "number": 1, "name": "firmware greets", "status": "failed" }],
  "diagnostics": ["# test.ts"]
}

Artifacts

result.artifacts lists what issue_download_ticket can download. Pass the run_test_suite_id as job_id.

testsuite_report: JSON with the counts, plus skipped, todo, bail_out, per-test tests (number, name, status), and diagnostics.
testsuite_stdout and testsuite_stderr: the suite’s output, including console.log.
firmware_uart: the firmware’s UART output. A run calls this artifact uart.
simulator_stdout and simulator_stderr: simulator logs.

Empty artifacts are omitted, so a passing suite may have no testsuite_stderr.

Limits

The whole suite has 5 minutes of wall-clock time. Exceeding it fails the job with internal server error and produces no artifacts.Each runForMillis and runUntilUartLine call takes 1 to 60000 ms.Each Deno.test or top-level describe counts as one result. Steps and nested it calls are not reported individually.

Get started

Agents

Supported hardware

Use cases

Tools

Platform

When to use it

Ask your agent

What you get back

What you can test

Not supported yet

run_test_suite

How it works

Writing a suite

The simulation API

Example checks

Reading the result

Artifacts

Limits

​When to use it

​Ask your agent

​What you get back

​What you can test

​Not supported yet

​run_test_suite

​How it works

​Writing a suite

​The simulation API

​Example checks

​Reading the result

​Artifacts

​Limits

When to use it

Ask your agent

What you get back

What you can test

Not supported yet

run_test_suite

How it works

Writing a suite

The simulation API

Example checks

Reading the result

Artifacts

Limits