kevin/meshtastic_firmware
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

Extract mcp-server to its own repo (meshtastic/meshtastic-mcp) (#10861)

Browse Source 
The Python MCP server + hardware test harness that lived under mcp-server/
now has its own home at https://github.com/meshtastic/meshtastic-mcp
(published, versioned independently). Remove the in-tree copy and wire the
firmware repo to the standalone server externally.

- Delete mcp-server/ (96 files) and the 8 harness-coupled AI workflow files
  under .claude/commands/ and .github/prompts/ that drove ./mcp-server/
  run-tests.sh — those workflows now ship with meshtastic-mcp as skills.
- .mcp.json: register the server via
  `uvx --from git+https://github.com/meshtastic/meshtastic-mcp meshtastic-mcp`
  instead of a local ./mcp-server/.venv, keeping MESHTASTIC_FIRMWARE_ROOT="."
  so the MCP tools still work from this checkout with no local build.
- Repoint the remaining references (AGENTS.md, CLAUDE.md,
  .github/copilot-instructions.md, bin/regen-*.sh, docs, Screen.h,
  userPrefs.jsonc, test/fixtures/nodedb/README.md, .trunk/configs/.bandit)
  at the standalone repo. The MCP tool surface is unchanged — only the
  pytest harness moves out; run it from a meshtastic-mcp checkout with
  MESHTASTIC_FIRMWARE_ROOT pointed here.

No build/CI/platformio coupling existed, so nothing in the firmware build
changes.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Ben Meadors
2026-07-02 14:33:04 -05:00
committed by GitHub
co-authored by GitHub Claude Opus 4.8
parent 4673ed22fc
commit 510e9796f9
116 changed files with 93 additions and 18519 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/copilot-instructions.md
+30 -29
View File
@@ -2,12 +2,12 @@
> **TL;DR**
>
> | | |
> | -------------- | -------------------------------------------------------------------------------------------- |
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
> | Hardware tests | `./mcp-server/run-tests.sh` |
> | Format | `trunk fmt` |
> | Mirror docs | `AGENTS.md` (short pointer for agents that don't read this file) · `CLAUDE.md` (Claude Code) |
> | | |
> | -------------- | ---------------------------------------------------------------------------------------------------------------------- |
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
> | Hardware tests | [meshtastic/meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) (`MESHTASTIC_FIRMWARE_ROOT` → this checkout) |
> | Format | `trunk fmt` |
> | Mirror docs | `AGENTS.md` (short pointer for agents that don't read this file) · `CLAUDE.md` (Claude Code) |
>
> **Need this? It's here.**
>
@@ -737,15 +737,15 @@ Simulation testing: `bin/test-simulator.sh`
Quick entry point for new test modules: `test/README.md` (native unit-test authoring guide, skeleton, pitfalls, and setup checklist).
### Hardware-in-the-loop tests (`mcp-server/tests/`)
### Hardware-in-the-loop tests ([meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp))
Separate pytest suite that exercises real USB-connected Meshtastic devices. See the **MCP Server & Hardware Test Harness** section below for invocation, tier layout, and agent usage rules.
Separate pytest suite that exercises real USB-connected Meshtastic devices. It now lives in the standalone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) repo, run against a firmware checkout via `MESHTASTIC_FIRMWARE_ROOT`. See the **MCP Server & Hardware Test Harness** section below for invocation, tier layout, and agent usage rules.
## MCP Server & Hardware Test Harness
The `mcp-server/` directory houses a firmware-aware [MCP](https://modelcontextprotocol.io/) server plus a pytest-based integration suite. AI agents that speak MCP get a well-defined tool surface for flashing, configuring, and inspecting physical Meshtastic devices - use it instead of hand-rolling `pio` or `meshtastic --port` calls where possible. `mcp-server/README.md` is the operator-facing setup doc; this section is the agent-facing usage contract.
The firmware-aware [MCP](https://modelcontextprotocol.io/) server plus its pytest-based integration suite now live in the standalone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) repo. AI agents that speak MCP get a well-defined tool surface for flashing, configuring, and inspecting physical Meshtastic devices - use it instead of hand-rolling `pio` or `meshtastic --port` calls where possible. The meshtastic-mcp repo's README is the operator-facing setup doc; this section is the agent-facing usage contract.
The repo registers the server via `.mcp.json` at the repo root - Claude Code picks it up automatically once `mcp-server/.venv/` is built (`cd mcp-server && python3 -m venv .venv && .venv/bin/pip install -e '.[test]'`).
The repo registers the server via `.mcp.json` at the repo root - Claude Code / Copilot pick it up automatically and run it through `uvx --from git+https://github.com/meshtastic/meshtastic-mcp meshtastic-mcp`, so the MCP tools work with no local build. To run the pytest hardware harness instead, clone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) and point `MESHTASTIC_FIRMWARE_ROOT` at this firmware checkout.
### When to use which surface
@@ -757,7 +757,7 @@ The repo registers the server via `.mcp.json` at the repo root - Claude Code pic
| Flash firmware to a variant | `pio_flash` (any arch) or `erase_and_flash` (ESP32 factory install) |
| Stream serial logs while debugging | `serial_open` → `serial_read` loop → `serial_close` |
| Administer `userPrefs.jsonc` build-time constants | `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest` |
| Run the regression suite | `./mcp-server/run-tests.sh` (or `/test` slash command) |
| Run the regression suite | `./run-tests.sh` from a meshtastic-mcp checkout (or `/test` slash command) |
| Diagnose a specific device | `/diagnose [role]` slash command (read-only) |
| Triage a flaky test | `/repro <node-id> [count]` slash command |
@@ -765,7 +765,7 @@ The repo registers the server via `.mcp.json` at the repo root - Claude Code pic
### MCP tool surface (43 tools)
Grouped by purpose. Full argument shapes in `mcp-server/README.md`; a few high-value signatures are called out here.
Grouped by purpose. Full argument shapes in the meshtastic-mcp repo's README; a few high-value signatures are called out here.
- **Discovery & metadata**: `list_devices`, `list_boards`, `get_board`
- **Build & flash**: `build`, `clean`, `pio_flash`, `erase_and_flash` (ESP32 only), `update_flash` (ESP32 OTA), `touch_1200bps`
@@ -775,13 +775,13 @@ Grouped by purpose. Full argument shapes in `mcp-server/README.md`; a few high-v
- **userPrefs admin** (build-time constants, not runtime config): `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest`, `userprefs_testing_profile`
- **Vendor escape hatches**: `esptool_chip_info`, `esptool_erase_flash`, `esptool_raw`, `nrfutil_dfu`, `nrfutil_raw`, `picotool_info`, `picotool_load`, `picotool_raw`
- **USB power control** (via `uhubctl`, per-port PPPS toggle): `uhubctl_list` (read-only), `uhubctl_power(action='on'|'off', confirm=True)`, `uhubctl_cycle(delay_s, confirm=True)`. Target by raw `(location, port)` or by `role` (`"nrf52"`, `"esp32s3"`); role lookup checks `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` + `_PORT_<ROLE>` env vars first, falls back to VID auto-detection.
- **Observability** (UI tier + operator ad-hoc): `capture_screen(role, ocr=True)` - grabs a USB-webcam frame of the device OLED and optionally OCRs it. Requires `mcp-server[ui]` extras (`opencv-python-headless`, `easyocr`) and `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` env var; falls through to a 1×1 black PNG `NullBackend` when unconfigured.
- **Observability** (UI tier + operator ad-hoc): `capture_screen(role, ocr=True)` - grabs a USB-webcam frame of the device OLED and optionally OCRs it. Requires `meshtastic-mcp[ui]` extras (`opencv-python-headless`, `easyocr`) and `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` env var; falls through to a 1×1 black PNG `NullBackend` when unconfigured.
`confirm=True` is a tool-level gate on top of whatever permission prompt your MCP host shows. **Don't bypass it** by asking the host to auto-approve - it exists specifically because MCP hosts sometimes remember "always allow this tool" and that's dangerous for `factory_reset`, `erase_and_flash`, `uhubctl_power(action='off')`, and `uhubctl_cycle`.
**TCP / native-host nodes.** Setting `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` makes `list_devices` surface a `meshtasticd` daemon (e.g. the `native-macos` build) as a synthetic `tcp://host:port` entry, and `connect()` routes through `meshtastic.tcp_interface.TCPInterface` instead of `SerialInterface`. Every read/write/admin tool that flows through `connect()` works against the daemon transparently. USB-only tools (`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`, `serial_open`, `esptool_*`, `nrfutil_*`, `picotool_*`) raise a clear `ConnectionError` when handed a `tcp://` port; `pio_flash` against a `native*` env raises a `FlashError` (no upload step - use `build` and run the binary directly). The pytest harness still assumes USB-attached devices per role; TCP-aware fixtures are deferred. See `mcp-server/README.md` § "TCP / native-host nodes".
**TCP / native-host nodes.** Setting `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` makes `list_devices` surface a `meshtasticd` daemon (e.g. the `native-macos` build) as a synthetic `tcp://host:port` entry, and `connect()` routes through `meshtastic.tcp_interface.TCPInterface` instead of `SerialInterface`. Every read/write/admin tool that flows through `connect()` works against the daemon transparently. USB-only tools (`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`, `serial_open`, `esptool_*`, `nrfutil_*`, `picotool_*`) raise a clear `ConnectionError` when handed a `tcp://` port; `pio_flash` against a `native*` env raises a `FlashError` (no upload step - use `build` and run the binary directly). The pytest harness still assumes USB-attached devices per role; TCP-aware fixtures are deferred. See the meshtastic-mcp repo's README § "TCP / native-host nodes".
### Hardware test suite (`mcp-server/run-tests.sh`)
### Hardware test suite (`run-tests.sh`, from a meshtastic-mcp checkout)
The wrapper auto-detects connected devices (VID → role map: `0x239A` → `nrf52`, `0x303A`/`0x10C4` → `esp32s3`), maps each role to a PlatformIO env (`nrf52` → `rak4631`, `esp32s3` → `heltec-v3`, overridable via `MESHTASTIC_MCP_ENV_<ROLE>`), then invokes pytest. Zero pre-flight config needed from the operator.
@@ -801,22 +801,23 @@ Suite tiers (collected + run in this order via `pytest_collection_modifyitems`):
Invocation patterns:
```bash
./mcp-server/run-tests.sh # full suite (auto-bake-if-needed)
./mcp-server/run-tests.sh --force-bake # reflash before testing
./mcp-server/run-tests.sh --assume-baked # skip bake (caller vouches for device state)
./mcp-server/run-tests.sh tests/mesh # one tier
./mcp-server/run-tests.sh tests/mesh/test_direct_with_ack.py # one file
./mcp-server/run-tests.sh -k telemetry # name filter
# run from a meshtastic-mcp checkout, with MESHTASTIC_FIRMWARE_ROOT=/path/to/firmware
./run-tests.sh # full suite (auto-bake-if-needed)
./run-tests.sh --force-bake # reflash before testing
./run-tests.sh --assume-baked # skip bake (caller vouches for device state)
./run-tests.sh tests/mesh # one tier
./run-tests.sh tests/mesh/test_direct_with_ack.py # one file
./run-tests.sh -k telemetry # name filter
```
**No hardware detected?** The wrapper auto-narrows to `tests/unit/` only and prints `detected hub : (none)` in the pre-flight header. Agents interpreting the output should call this out explicitly - a 52-test green run without hardware is qualitatively different from a 12-unit-test green run.
**Artifacts every run produces:**
- `mcp-server/tests/report.html` - self-contained pytest-html. Each test gets a `Meshtastic debug` section with the tail of firmware log + device state dump. **Open this first** on failures; it's the canonical evidence source.
- `mcp-server/tests/junit.xml` - CI-parseable.
- `mcp-server/tests/reportlog.jsonl` - pytest-reportlog stream (`$report_type` keyed JSONL). Consumed by the live TUI.
- `mcp-server/tests/fwlog.jsonl` - firmware log mirror from the `meshtastic.log.line` pubsub topic. Populated by the `_firmware_log_stream` autouse session fixture.
- `tests/report.html` - self-contained pytest-html. Each test gets a `Meshtastic debug` section with the tail of firmware log + device state dump. **Open this first** on failures; it's the canonical evidence source.
- `tests/junit.xml` - CI-parseable.
- `tests/reportlog.jsonl` - pytest-reportlog stream (`$report_type` keyed JSONL). Consumed by the live TUI.
- `tests/fwlog.jsonl` - firmware log mirror from the `meshtastic.log.line` pubsub topic. Populated by the `_firmware_log_stream` autouse session fixture.
### Live TUI (`meshtastic-mcp-test-tui`)
@@ -836,7 +837,7 @@ A Textual-based live view that wraps `run-tests.sh`. Tails reportlog for per-tes
Launch:
```bash
cd mcp-server
# from a meshtastic-mcp checkout (MESHTASTIC_FIRMWARE_ROOT set)
.venv/bin/meshtastic-mcp-test-tui # full suite
.venv/bin/meshtastic-mcp-test-tui tests/mesh # args pass through to pytest
```
@@ -861,7 +862,7 @@ House rules for agents running these prompts:
### Key fixtures (test authors + agents debugging)
`mcp-server/tests/conftest.py` provides:
`tests/conftest.py` (in the meshtastic-mcp checkout) provides:
- **`_session_userprefs`** (autouse session) - snapshots `userPrefs.jsonc` at session start, merges the session test profile via `userprefs.merge_active(test_profile)`, restores at teardown. Four layers of safety: pytest teardown + `atexit` + sidecar file (`userPrefs.jsonc.mcp-session-bak`) + startup self-heal in `run-tests.sh`. **Do not edit `userPrefs.jsonc` from inside a test.**
- **`_firmware_log_stream`** (autouse session) - subscribes to `meshtastic.log.line` pubsub on every connected `SerialInterface` and mirrors lines to `tests/fwlog.jsonl`. Drives the TUI firmware-log pane.
@@ -877,13 +878,13 @@ Two firmware changes exist specifically so the test harness works reliably. **Ke
- **`src/mesh/StreamAPI.cpp` + `StreamAPI.h`** - `emitLogRecord` uses a dedicated `fromRadioScratchLog` + `txBufLog` pair and a `concurrency::Lock streamLock`. Before this fix, `debug_log_api_enabled=true` would tear `FromRadio` protobufs on the serial transport because `emitTxBuffer` and `emitLogRecord` shared a single scratch buffer. The conftest enables the log stream session-wide; without this fix the device would corrupt its own FromRadio replies mid-session.
- **`src/mesh/PhoneAPI.cpp`** - `ToRadio` `Heartbeat(nonce=1)` triggers `nodeInfoModule->sendOurNodeInfo(NODENUM_BROADCAST, true, 0, true)` for serial clients, mirroring the pre-existing behavior for TCP/UDP clients in `PacketAPI.cpp`. The mesh tests rely on this to force a NodeInfo broadcast right after connect so the peer discovers them before the test's first assertion.
If you're modifying `StreamAPI`, `PhoneAPI`, `NodeInfoModule`, or `userPrefs` flow, run `./mcp-server/run-tests.sh` at minimum before asking for review.
If you're modifying `StreamAPI`, `PhoneAPI`, `NodeInfoModule`, or `userPrefs` flow, run `./run-tests.sh` (from a meshtastic-mcp checkout, with `MESHTASTIC_FIRMWARE_ROOT` pointed here) at minimum before asking for review.
### Recovery playbooks
| Symptom | First check | Fix |
| --------------------------------------------------------------------------------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `userPrefs.jsonc` dirty after test run | `git status --porcelain userPrefs.jsonc` | If non-empty, re-run `./mcp-server/run-tests.sh` once - the pre-flight self-heal restores from sidecar. If still dirty, `git checkout userPrefs.jsonc`. |
| `userPrefs.jsonc` dirty after test run | `git status --porcelain userPrefs.jsonc` | If non-empty, re-run `./run-tests.sh` (from a meshtastic-mcp checkout) once - the pre-flight self-heal restores from sidecar. If still dirty, `git checkout userPrefs.jsonc`. |
| Port busy / wedged CP2102 on macOS | `lsof /dev/cu.usbserial-0001` | Kill the holder. USB replug if the kernel still reports busy. Often a stale `pio device monitor` or zombie `meshtastic_mcp` process. |
| nRF52 appears unresponsive | `list_devices` shows VID `0x239A` but `device_info` times out | `touch_1200bps(port=...)` drops it into the DFU bootloader → `pio_flash` re-installs. |
| Device fully wedged (Guru Meditation, frozen CDC, no DFU) | `list_devices` shows the VID but every admin call times out | `uhubctl_cycle(role="nrf52", confirm=True)` hard-power-cycles the port via USB hub PPPS. `baked_single`'s auto-recovery hook does this once automatically if uhubctl is installed. Falls back to physical replug if no PPPS hub. |
.github/prompts/mcp-diagnose.prompt.md
-64
View File
@@ -1,64 +0,0 @@
---
mode: agent
description: Device health report via the meshtastic MCP tools (Copilot equivalent of the Claude Code /diagnose slash command)
---
# `/mcp-diagnose` - device health report
Equivalent of `.claude/commands/diagnose.md`. Use when the operator asks to "check the devices", "what's the mesh looking like", "is nrf52 alive", etc.
This prompt assumes the meshtastic MCP server is registered with your VS Code Copilot agent. If it isn't, fall back to running `./mcp-server/run-tests.sh tests/unit` plus a short `device_info` script via the terminal.
## What to do
1. **Enumerate hardware** via the `list_devices` MCP tool (with `include_unknown=True`). For each entry where `likely_meshtastic=True`, capture `port`, `vid`, `pid`, `description`.
2. **Apply the operator's filter** (if any):
- No filter → every likely-meshtastic device.
- `nrf52` → `vid == 0x239a`
- `esp32s3` → `vid == 0x303a` or `vid == 0x10c4`
- A `/dev/cu.*` path → only that port.
- Anything else → substring match on port.
3. **For each selected device, in sequence (don't parallelize - SerialInterface holds an exclusive port lock):**
- `device_info(port=<p>)` → `my_node_num`, `long_name`, `short_name`, `firmware_version`, `hw_model`, `region`, `num_nodes`, `primary_channel`
- `list_nodes(port=<p>)` → peer count, which peers have `publicKey`, SNR/RSSI distribution
- `get_config(section="lora", port=<p>)` → region, preset, channel_num, tx_power, hop_limit
- If anything looks off (can't connect, `num_nodes` wrong, missing `firmware_version`), open a short firmware-log window: `serial_open(port=<p>, env=<inferred>)`, wait 3 seconds, `serial_read(session_id, max_lines=100)`, `serial_close(session_id)`. Infer env from VID (0x239a → `rak4631`, 0x303a/0x10c4 → `heltec-v3`) unless an `MESHTASTIC_MCP_ENV_<ROLE>` env var overrides it.
4. **Hub health** (call once, not per-device): `uhubctl_list()` - enumerates every USB hub the host sees. Cross-reference each Meshtastic device's VID to find which hub + port it's on. Flag in the report if:
- No hub advertises `ppps=true` → `tests/recovery/` can't run; hard-recovery via `uhubctl_cycle` isn't available.
- A Meshtastic device is on a non-PPPS hub → note it; moving to a PPPS hub unlocks auto-recovery.
- `uhubctl_list` raises `ConfigError: uhubctl not found` → report as "uhubctl not installed"; don't treat as a device fault.
5. **Render per-device report** as a compact block:
```text
[nrf52 @ /dev/cu.usbmodem1101] fw=2.7.23.bce2825, hw=RAK4631
owner : Meshtastic 40eb / 40eb
region/band : US, channel 88, LONG_FAST
tx_power : 30 dBm, hop_limit=3
peers : 1 (esp32s3 0x433c2428, pubkey ✓, SNR 6.0 / RSSI -24 dBm)
primary ch : McpTest
hub : 1-1.3 port 2 (PPPS, uhubctl-controllable)
firmware : no panics in last 3s
```
Flag abnormalities inline with `⚠︎ <short reason>` - missing pubkey on a known peer, region UNSET, mismatched channel name, device on non-PPPS hub, etc.
6. **Cross-device correlation** (when >1 device selected):
- Do both see each other in `nodesByNum`?
- Do `region`, `channel_num`, `modem_preset` match across devices?
- Do the primary channel names match? (Different name → different PSK → no decode.)
7. **Suggest next steps only for recognizable failure modes**, never speculatively:
- Stale PKI one-way → "`/mcp-test tests/mesh/test_direct_with_ack.py` - the test's retry+nodeinfo-ping heals this."
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
- Device unreachable, DFU reachable → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds and the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
- CP2102-wedged-driver on macOS → see `run-tests.sh` notes.
## Hard constraints
- **Read-only.** No `set_config`, no `reboot`, no `factory_reset`, no `flash`. If the operator wants mutation, they'll escalate explicitly.
- **Open/query/close per device.** Never hold multiple SerialInterfaces to the same port. The port lock is exclusive.
- **Don't infer env beyond the VID map** - if the operator has an unusual board, ask them which env to use rather than guessing.
.github/prompts/mcp-repro.prompt.md
-68
View File
@@ -1,68 +0,0 @@
---
mode: agent
description: Re-run a specific test N times to triage flakes; diff firmware logs between passes and failures (Copilot equivalent of the Claude Code /repro slash command)
---
# `/mcp-repro` - flakiness triage for one test
Equivalent of `.claude/commands/repro.md`. Use when the operator says "that one test is flaky - dig in", "repro the direct_with_ack failure", "why does X sometimes fail?".
## What to do
1. **Parse the operator's input** into two pieces:
- **Test identifier** - either a pytest node id (has `::` or starts with `tests/`) or a `-k`-style filter (plain substring like `direct_with_ack`).
- **Count** - integer, default `5`, cap at `20`. If the operator asks for 50, negotiate down and explain (airtime + USB wear).
2. **Sanity-check the hub** via the `list_devices` MCP tool. If the test name references `nrf52` or `esp32s3` and the matching VID isn't present, stop and report - re-running won't help.
3. **Loop** N times. Each iteration:
```bash
./mcp-server/run-tests.sh <test-id> --tb=short -p no:cacheprovider
```
`-p no:cacheprovider` keeps pytest from caching anything between iterations. Capture: exit code, duration, and (on failure) the `Meshtastic debug` firmware-log section from `mcp-server/tests/report.html`.
4. **Tally** results as you go:
```text
attempt 1: PASS (42s)
attempt 2: FAIL (128s) ← fw log captured
attempt 3: PASS (39s)
attempt 4: FAIL (121s)
attempt 5: PASS (41s)
--------------------------------------------------
pass rate: 3/5 (60%) | mean duration: 74s
```
5. **On mixed outcomes, diff the firmware logs** between one representative pass and one representative fail. Focus on:
- Error-level lines present only in failures (`PKI_UNKNOWN_PUBKEY`, `Alloc an err=`, `Skip send`, `No suitable channel`, `NAK`)
- Timing around the assertion point (broadcast sent? ACK received? retry fired?)
- Device-state fields that changed between attempts
Surface the top 3 differences as a compact "passes when / fails when" table with uptime timestamps. Don't dump full logs.
6. **Classify** the flake into one of:
- **LoRa airtime collision** - pass rate improves with fewer concurrent transmitters. Suggest a `time.sleep` gap or retry bump in the test body.
- **PKI key staleness** - first attempt fails, subsequent ones pass; existing retry-loop pattern in `test_direct_with_ack.py` is the fix.
- **NodeInfo cooldown** - `Skip send NodeInfo since we sent it <600s ago` in fail-only logs; needs a `broadcast_nodeinfo_ping()` warmup.
- **Hardware-specific** - one direction consistently fails, firmware versions differ, CP2102 driver wedged, etc. For a device wedged past `touch_1200bps`, recommend `uhubctl_cycle(role=..., confirm=True)` to hard-power-cycle its hub port (requires `uhubctl` installed).
- **Device went dark mid-run** - fails from some iteration onward and never recovers; firmware log stops arriving. Almost always a Guru crash with frozen CDC. Recommend `uhubctl_cycle` before the next iteration; escalate to replug if that also fails.
- **Unknown** - say so. Don't invent a root cause.
7. **Report back** with:
- Pass rate + mean duration.
- Classification + the specific log evidence for it.
- A concrete next step (tighter assertion, more retries, open `/mcp-diagnose`, file a bug, nothing).
## Examples
- `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip[esp32s3->nrf52] 10` - 10 runs of that parametrized case.
- `broadcast_delivers` - no `::`, no `tests/`; treat as `-k broadcast_delivers`; runs every match 5 times.
- `tests/telemetry/test_device_telemetry_broadcast.py 3` - shorter count for a slow test.
## Notes
- If the FIRST attempt fails and the rest pass, that's a state-leak signature - suggest starting from `--force-bake` or a clean device state rather than chasing the first-failure firmware logs.
- If ALL N fail, this isn't a flake - it's a regression. Say so, stop iterating, escalate to `/mcp-test` for full-suite context.
- Don't rebuild firmware during triage. Flakes that only reproduce under different firmware belong in a separate session with a plan.
.github/prompts/mcp-test.prompt.md
-57
View File
@@ -1,57 +0,0 @@
---
mode: agent
description: Run the mcp-server test suite and interpret results (Copilot equivalent of the Claude Code /test slash command)
---
# `/mcp-test` - mcp-server test runner with interpretation
Equivalent of the Claude Code `/test` slash command in `.claude/commands/test.md`. Use this when the operator asks you to "run the tests", "check the mcp test suite", "run the mesh tests", etc.
## What to do
1. **Invoke the wrapper** from the firmware repo root:
```bash
./mcp-server/run-tests.sh [pytest-args]
```
If the operator specified a subset (e.g. "just the mesh tests"), pass it through as `tests/mesh` or a pytest `-k filter`. If they said nothing, use the wrapper's defaults (full suite with pytest-html report).
The wrapper auto-detects connected Meshtastic devices, maps each to its PlatformIO env, exports the required env vars, and invokes pytest. Zero pre-flight config needed from the operator.
2. **Read the pre-flight header** (first few lines of wrapper output). The `detected hub :` line lists role → port → env mappings. If it reads `(none)`, the wrapper narrowed to `tests/unit` only - call that out explicitly so the operator knows hardware tiers were skipped.
3. **On pass**: one-line summary like `N passed, M skipped in <duration>`. Don't enumerate test names. DO mention any non-placeholder SKIPs and name the cause:
- `"role not present on hub"` → device unplugged; operator should reconnect.
- `"firmware not baked with USERPREFS_UI_TEST_LOG"` → tests/ui skipped; the UI-log compile macro isn't in the baked firmware. Suggest `--force-bake`.
- `"uhubctl not installed"` → tests/recovery + `test_peer_offline_recovery` skipped. Suggest `brew install uhubctl` / `apt install uhubctl`.
- `"no PPPS-capable hubs detected"` → tests/recovery skipped because the attached hub doesn't support per-port power switching; won't run on that setup.
- `"opencv-python-headless is not installed"` → tests/ui auto-deselected by `run-tests.sh`. Suggest `pip install -e 'mcp-server/.[ui]'`.
4. **On failure**: open `mcp-server/tests/report.html` (pytest-html output, self-contained) and extract the `Meshtastic debug` section for each failed test. That section includes a firmware log stream (last 200 lines) and device state dump. For each failure, summarise:
- test name
- one-line assertion message
- the specific firmware log lines that explain why (look for `PKI_UNKNOWN_PUBKEY`, `Skip send NodeInfo`, `Error=`, `Guru Meditation`, `assertion failed`, `No suitable channel`)
- for UI-tier failures also check `mcp-server/tests/ui_captures/<session>/<test>/transcript.md` (per-step frame + OCR)
5. **Classify each failure** as one of:
- **Transient flake** - LoRa collision, first-attempt NAK with self-heal pattern, timing-sensitive assertion. Suggest `/mcp-repro <test-id>` to confirm.
- **Environmental** - device unreachable, port busy, CP2102 driver wedged on macOS. Suggest recovery in escalation order: (a) replug USB, (b) `touch_1200bps` + `pio_flash` for nRF52 DFU, (c) `uhubctl_cycle(role=..., confirm=True)` for a device wedged past DFU (needs `uhubctl` installed; `baked_single` does this once automatically when available). Also check `git status userPrefs.jsonc`.
- **Regression** - same assertion fails repeatedly on re-runs, firmware log shows novel errors. Identify the firmware module likely responsible.
6. **Do NOT run destructive recovery automatically**. If a failure looks like it needs a reflash, factory*reset, `uhubctl_cycle`, or replug - \_describe the steps* and let the operator decide. Never burn airtime or flash cycles without approval.
## Arguments convention
Operators generally invoke this prompt either with no arguments (full suite) or with a specific subset. Examples:
- `tests/mesh` - one tier
- `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip` - one test
- `--force-bake` - reflash devices first
- `-k telemetry` - name-filter
## Side-effects to confirm in your summary
- `userPrefs.jsonc` should be clean after a successful run. The session fixture in `mcp-server/tests/conftest.py` (`_session_userprefs`) snapshots and restores. Check `git status --porcelain userPrefs.jsonc` and report if it's non-empty.
- `mcp-server/tests/report.html` and `junit.xml` regenerate on every run.
- The wrapper prints a warning if a `.mcp-session-bak` sidecar was left over from a crashed prior session and auto-restores from it - mention that if it happened.