Extract mcp-server to its own repo (meshtastic/meshtastic-mcp) (#10861)
The Python MCP server + hardware test harness that lived under mcp-server/ now has its own home at https://github.com/meshtastic/meshtastic-mcp (published, versioned independently). Remove the in-tree copy and wire the firmware repo to the standalone server externally. - Delete mcp-server/ (96 files) and the 8 harness-coupled AI workflow files under .claude/commands/ and .github/prompts/ that drove ./mcp-server/ run-tests.sh — those workflows now ship with meshtastic-mcp as skills. - .mcp.json: register the server via `uvx --from git+https://github.com/meshtastic/meshtastic-mcp meshtastic-mcp` instead of a local ./mcp-server/.venv, keeping MESHTASTIC_FIRMWARE_ROOT="." so the MCP tools still work from this checkout with no local build. - Repoint the remaining references (AGENTS.md, CLAUDE.md, .github/copilot-instructions.md, bin/regen-*.sh, docs, Screen.h, userPrefs.jsonc, test/fixtures/nodedb/README.md, .trunk/configs/.bandit) at the standalone repo. The MCP tool surface is unchanged — only the pytest harness moves out; run it from a meshtastic-mcp checkout with MESHTASTIC_FIRMWARE_ROOT pointed here. No build/CI/platformio coupling existed, so nothing in the firmware build changes. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
GitHub
Claude Opus 4.8
parent
4673ed22fc
commit
510e9796f9
@@ -1,49 +0,0 @@
|
|||||||
# Claude Code slash commands for the mcp-server test suite
|
|
||||||
|
|
||||||
Three AI-assisted workflows wrapping `mcp-server/run-tests.sh` and the meshtastic MCP tools. Each one has a twin in `.github/prompts/` for Copilot users.
|
|
||||||
|
|
||||||
| Slash command | What it does | Copilot equivalent |
|
|
||||||
| --------------------- | ------------------------------------------------------------------------- | ---------------------------------------- |
|
|
||||||
| `/test [args]` | Runs the test suite (auto-detects hardware) and interprets failures | `.github/prompts/mcp-test.prompt.md` |
|
|
||||||
| `/diagnose [role]` | Read-only device health report via the meshtastic MCP tools | `.github/prompts/mcp-diagnose.prompt.md` |
|
|
||||||
| `/repro <test> [n=5]` | Re-runs one test N times, diffs firmware logs between passes and failures | `.github/prompts/mcp-repro.prompt.md` |
|
|
||||||
|
|
||||||
## Why two surfaces
|
|
||||||
|
|
||||||
The Claude Code commands and Copilot prompts cover the same three workflows but each speaks its host's idiom:
|
|
||||||
|
|
||||||
- **Claude Code** (`/test`) uses `$ARGUMENTS` for pass-through, has direct access to Bash + all MCP tools registered in the user's settings, and runs in the terminal context.
|
|
||||||
- **Copilot** (`/mcp-test`) runs in VS Code's agent mode; it has terminal + MCP access too but typically asks the operator to confirm inputs interactively.
|
|
||||||
|
|
||||||
A contributor using either IDE gets equivalent assistance. Keep the two in sync when behavior changes - the diff of intent should be minimal.
|
|
||||||
|
|
||||||
## House rules
|
|
||||||
|
|
||||||
- **No destructive writes without explicit operator approval.** Skills that could reflash, factory-reset, or reboot a device must describe the action and stop - the operator authorizes.
|
|
||||||
- **Interpret failures, don't just echo them.** The skill body should pull firmware log lines from `mcp-server/tests/report.html` (the `Meshtastic debug` section, attached by `tests/conftest.py::pytest_runtest_makereport`) and classify the failure.
|
|
||||||
- **Keep MCP tool calls sequential per port.** SerialInterface holds an exclusive port lock; two parallel tool calls on the same port deadlock.
|
|
||||||
- **Never speculate about root cause.** If the evidence doesn't support a classification, say "unknown" and list what you'd need to disambiguate.
|
|
||||||
|
|
||||||
## Adding a new command
|
|
||||||
|
|
||||||
1. Write the Claude Code version at `.claude/commands/<name>.md` with YAML frontmatter:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
description: one-line purpose (used for auto-invocation by the model)
|
|
||||||
argument-hint: [optional-hint]
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Write the Copilot equivalent at `.github/prompts/mcp-<name>.prompt.md` with:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
mode: agent
|
|
||||||
description: ...
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Add the row to the table above. Cross-link in both bodies.
|
|
||||||
|
|
||||||
4. Smoke-test on Claude Code first (`/<name>` should appear in autocomplete), then in VS Code Copilot (`/mcp-<name>` in Chat).
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
---
|
|
||||||
description: Produce a device health report using the meshtastic MCP tools (device_info, list_nodes, get_config, short serial log capture)
|
|
||||||
argument-hint: [role=all|nrf52|esp32s3|<port>]
|
|
||||||
---
|
|
||||||
|
|
||||||
# `/diagnose` - device health report
|
|
||||||
|
|
||||||
Call the meshtastic MCP tool bundle and format a structured health report for one or all detected devices. Zero guesswork for the operator.
|
|
||||||
|
|
||||||
## What to do
|
|
||||||
|
|
||||||
1. **Enumerate hardware.** Call `mcp__meshtastic__list_devices(include_unknown=True)`. For each entry where `likely_meshtastic=True`, capture `port`, `vid`, `pid`, `description`.
|
|
||||||
|
|
||||||
2. **Filter by `$ARGUMENTS`**:
|
|
||||||
- No args, `all` → every likely-meshtastic device.
|
|
||||||
- `nrf52` → only devices with `vid == 0x239a`.
|
|
||||||
- `esp32s3` → only devices with `vid == 0x303a` or `vid == 0x10c4`.
|
|
||||||
- A `/dev/cu.*` path → only that one port.
|
|
||||||
- Anything else → treat as a substring match against the `port` string.
|
|
||||||
|
|
||||||
3. **For each selected device, in sequence (NOT parallel - SerialInterface holds an exclusive port lock):**
|
|
||||||
- `mcp__meshtastic__device_info(port=<p>)` - captures `my_node_num`, `long_name`, `short_name`, `firmware_version`, `hw_model`, `region`, `num_nodes`, `primary_channel`.
|
|
||||||
- `mcp__meshtastic__list_nodes(port=<p>)` - count of peers, which ones have `publicKey` set, SNR/RSSI distribution.
|
|
||||||
- `mcp__meshtastic__get_config(section="lora", port=<p>)` - region, preset, channel_num, tx_power, hop_limit.
|
|
||||||
- Optionally, if the device seems unhappy (fails to connect, `num_nodes==1` when ≥2 are plugged in, missing firmware*version), open a short firmware log window: `mcp__meshtastic__serial_open(port=<p>, env=<inferred-env>)`, wait 3s, `serial_read(session_id=<s>, max_lines=100)`, `serial_close(session_id=<s>)`. The env should be inferred from the VID map in `mcp-server/run-tests.sh` (nrf52 → rak4631, esp32s3 → heltec-v3) unless `MESHTASTIC_MCP_ENV*<ROLE>` is set.
|
|
||||||
|
|
||||||
4. **Hub health** (call once, not per-device): `mcp__meshtastic__uhubctl_list()` - enumerates every USB hub the host can see. Note which hubs advertise `ppps=true` and which hub hosts each Meshtastic device (cross-reference by VID). Flag it in the report if:
|
|
||||||
- No hub advertises PPPS → `tests/recovery/` can't run on this setup; hard-recovery via `uhubctl_cycle` isn't available.
|
|
||||||
- A Meshtastic device is on a non-PPPS hub → note it; operator may want to move the device to a PPPS hub to unlock auto-recovery.
|
|
||||||
- `uhubctl_list` raises `ConfigError: uhubctl not found` → just say `uhubctl not installed` in the report; don't treat as a fault.
|
|
||||||
|
|
||||||
5. **Render per-device report** as:
|
|
||||||
|
|
||||||
```text
|
|
||||||
[nrf52 @ /dev/cu.usbmodem1101] fw=2.7.23.bce2825, hw=RAK4631
|
|
||||||
owner : Meshtastic 40eb / 40eb
|
|
||||||
region/band : US, channel 88, LONG_FAST
|
|
||||||
tx_power : 30 dBm, hop_limit=3
|
|
||||||
peers : 1 (esp32s3 0x433c2428, pubkey ✓, SNR 6.0 / RSSI -24 dBm)
|
|
||||||
primary ch : McpTest
|
|
||||||
hub : 1-1.3 port 2 (PPPS, uhubctl-controllable)
|
|
||||||
firmware : no panics in last 3s; NodeInfoModule emitted 2 broadcasts
|
|
||||||
```
|
|
||||||
|
|
||||||
Keep it scannable. If a field is missing or abnormal (no pubkey for a known peer, region=UNSET, num_nodes inconsistent with the hub, device on non-PPPS hub), flag it inline with a short `⚠︎ <one-line reason>`.
|
|
||||||
|
|
||||||
6. **Cross-device correlation** (only when >1 device is inspected):
|
|
||||||
- Do both sides see each other in `nodesByNum`? If one does and the other doesn't, that's asymmetric NodeInfo - flag it.
|
|
||||||
- Do the LoRa configs match? (region, channel_num, modem_preset should all agree; mismatch = no mesh)
|
|
||||||
- Do the primary channel NAMES match? Mismatch = different PSK = no decode.
|
|
||||||
|
|
||||||
7. **Recorder slice (cheap, always available).** The mcp-server runs an autouse log recorder that's been collecting from every connected device. Pull two short slices to surface anything weird that's already happened:
|
|
||||||
- `mcp__meshtastic__logs_window(start="-2m", level="WARN|ERROR|CRIT", max_lines=20)` - recent firmware errors. If empty, say "no recent errors"; don't manufacture concern.
|
|
||||||
- `mcp__meshtastic__telemetry_timeline(window="1h", field="free_heap", max_points=60)` - heap trend. If `slope_per_min < -50`, flag it and recommend `/leakhunt window=6h` for a deeper read; otherwise just note the current free heap.
|
|
||||||
- If `recorder_status` shows `running:false` or `files.telemetry.last_ts` is null, note "recorder has no telemetry yet - enable `set_debug_log_api(True)` to populate" and skip this step gracefully.
|
|
||||||
|
|
||||||
8. **Suggest next actions only for specific, recognisable failure modes**:
|
|
||||||
- Stale PKI pubkey one-way → "run `/test tests/mesh/test_direct_with_ack.py` - the retry + nodeinfo-ping heals this in the test path."
|
|
||||||
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
|
|
||||||
- Device unreachable, reachable via DFU → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds AND the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
|
|
||||||
- CP2102-wedged-driver on macOS → see the note in `run-tests.sh`.
|
|
||||||
- Heap slope strongly negative → "run `/leakhunt window=6h` for a full timeline + classification."
|
|
||||||
|
|
||||||
## What NOT to do
|
|
||||||
|
|
||||||
- No writes. No `set_config`, no `reboot`, no `factory_reset`. This is a read-only diagnostic skill - if the operator wants to change state, they'll ask explicitly.
|
|
||||||
- No `flash` / `erase_and_flash`. Those are separate escalations.
|
|
||||||
- No holding SerialInterface across tool calls - open, query, close; next device. The port lock is exclusive.
|
|
||||||
@@ -1,103 +0,0 @@
|
|||||||
---
|
|
||||||
description: Hunt for memory leaks (and other slow degradations) by reading the persistent recorder's heap timeline + log slice over a window
|
|
||||||
argument-hint: [window=1h] [field=free_heap] [variant=local]
|
|
||||||
---
|
|
||||||
|
|
||||||
<!-- markdownlint-disable MD029 -->
|
|
||||||
|
|
||||||
# `/leakhunt` - read the recorder, classify a memory leak
|
|
||||||
|
|
||||||
Use the always-on recorder (`mcp-server/.mtlog/`) to read a heap timeline plus the matching log slice and produce a one-page verdict: **steady / slow leak / fragmentation / OOM-imminent**. No firmware changes, no special build flags - the LocalStats telemetry packet that the firmware already broadcasts every ~60 s carries `heap_free_bytes` and `heap_total_bytes`.
|
|
||||||
|
|
||||||
## Two signal paths - pick the right one
|
|
||||||
|
|
||||||
| Path | Build flag | Cadence | Per-thread attribution | Cost |
|
|
||||||
| --------------------- | ---------------- | -------------- | ---------------------- | ------------------------- |
|
|
||||||
| LocalStats packet | (default) | ~60 s | No | Free - always on |
|
|
||||||
| `[heap N]` log prefix | `-DDEBUG_HEAP=1` | every log line | Yes (Thread X leaked) | Bigger flash + log volume |
|
|
||||||
|
|
||||||
Both feed the same `telemetry_timeline(field="free_heap")` query - when DEBUG_HEAP is on, the recorder synthesizes telemetry rows from log prefixes (tagged `source: debug_heap`), so a single timeline call gets whichever signal is available. **For a slow leak diagnosis, the default path is plenty** (60 s cadence over 6 h = 360 points; linear regression over that nails sub-100-byte/min slopes). **DEBUG_HEAP is for attribution** - when the slope is real and you need to know which thread is leaking.
|
|
||||||
|
|
||||||
## What to do
|
|
||||||
|
|
||||||
1. **Parse `$ARGUMENTS`**: optional `window` (default `1h`, accepts `30m`/`6h`/`-3d`/etc.), optional `field` (default `free_heap`; alternates: `total_heap`, `battery_level`, anything in the LocalStats variant), optional `variant` (default `local`; alternates: `device`, `environment`, `power`, `airQuality`, `health`).
|
|
||||||
|
|
||||||
2. **Verify the recorder is alive** - call `mcp__meshtastic__recorder_status`. Check:
|
|
||||||
- `running == True`
|
|
||||||
- `files.telemetry.lines > 0` (at least one telemetry packet recorded - if zero, the device hasn't broadcast LocalStats yet OR `set_debug_log_api` has never been on; tell the operator to run `mcp__meshtastic__set_debug_log_api(enabled=True)` and wait one device-update interval)
|
|
||||||
- `files.telemetry.last_ts` within the last 5 minutes (if older, the device is silent - log that, not "leak detected")
|
|
||||||
|
|
||||||
3. **Detect whether DEBUG_HEAP is active** - `mcp__meshtastic__logs_window(start="-2m", grep=r"\\[heap \\d+\\]", max_lines=3)`. If any line matches, the firmware has the prefix → DEBUG_HEAP is on, expect higher-cadence data and `heap_event` rows. If zero matches over the last 2 minutes, you're on the LocalStats-only path.
|
|
||||||
|
|
||||||
4. **Pull the timeline** - `mcp__meshtastic__telemetry_timeline(window=$window, variant=$variant, field=$field, max_points=200)`. Read:
|
|
||||||
- `samples` - how many raw points contributed
|
|
||||||
- `min`, `max` - total swing
|
|
||||||
- `slope_per_min` - units per minute (linear regression over the whole window)
|
|
||||||
|
|
||||||
5. **Pull the log context for the same window** - `mcp__meshtastic__logs_window(start="-${window}", grep="Heap status|leaked heap|freed heap|out of memory|Alloc an err|panic|abort", max_lines=200)`. These are the strings the firmware emits when something memory-related happens (`DEBUG_HEAP` builds emit `"Heap status:"` and `"leaked heap"` lines; production builds emit `"Alloc an err"` on failure and `"out of memory"` on OOM).
|
|
||||||
|
|
||||||
6. **Pull marker events** so we know if the operator labeled phases - `mcp__meshtastic__events_window(start="-${window}", kind="mark|connection_lost|connection_established")`. If a `connection_lost` overlaps a sharp drop, that's not a leak; that's a reboot.
|
|
||||||
|
|
||||||
6a. **(DEBUG_HEAP only) Per-thread attribution** - `mcp__meshtastic__logs_window(start="-${window}", grep="leaked heap", max_lines=200)`. Each row has a structured `heap_event` field with `{kind, thread, before, after, delta}`. Aggregate by thread: sum the `delta` over the window per thread name. The thread with the largest cumulative negative delta is your suspect. Note the count too - a thread with 50× small leaks is different from 1× big leak.
|
|
||||||
|
|
||||||
7. **Classify** based on what the data says, NOT on what you wish it said. Use these rules in order:
|
|
||||||
- **Insufficient data** (< 5 samples): say so. Suggest a longer window or longer wait. Stop.
|
|
||||||
- **Reboot mid-window**: if any `connection_lost` event is present AND `free_heap` jumped UP at that timestamp, the device rebooted. Note it; pre-reboot trend may be a leak but you only have part of the curve.
|
|
||||||
- **OOM-imminent**: any `Alloc an err=` or `out of memory` line in the log slice. This trumps everything; flag urgently.
|
|
||||||
- **Slow leak**: `slope_per_min < -50` AND `max - min > 1000` AND no reboot. The heap is monotonically (or near-monotonically) declining. Estimate time-to-zero: `min / -slope_per_min` minutes. Surface it.
|
|
||||||
- **Fragmentation suspect**: `slope_per_min` close to zero (|x| < 50) BUT min trends down across the window AND the log slice shows `Alloc an err` warnings WITHOUT total OOM. Means free total is OK but largest contiguous block is shrinking. Recommend a `DEBUG_HEAP` build to confirm.
|
|
||||||
- **Steady**: |slope_per_min| < 50, no error lines. Heap is fine.
|
|
||||||
- **Recovery curve**: slope is POSITIVE - heap recovered. Either a workload completed or GC fired. Note it; not a leak.
|
|
||||||
|
|
||||||
8. **Report**:
|
|
||||||
|
|
||||||
```text
|
|
||||||
/leakhunt window=6h field=free_heap variant=local
|
|
||||||
────────────────────────────────────────────────────
|
|
||||||
recorder : running, telem last_ts 8s ago
|
|
||||||
build : DEBUG_HEAP=ON (per-line prefix detected)
|
|
||||||
samples : 14,200 over 6h (cadence ~1.5s, log-line synth)
|
|
||||||
free_heap : min 92,344 / max 124,008 / range 31,664
|
|
||||||
slope : -82 bytes/min (negative - heap declining)
|
|
||||||
reboots : none in window
|
|
||||||
OOM events : none
|
|
||||||
error lines : 3× "Alloc an err=ESP_ERR_NO_MEM" at +4h12m, +5h08m, +5h44m
|
|
||||||
thread leaks : (DEBUG_HEAP) MeshPacket -3,124 B over 18 events
|
|
||||||
Router -1,408 B over 4 events
|
|
||||||
others -240 B
|
|
||||||
verdict : SLOW LEAK - primary suspect MeshPacket thread
|
|
||||||
est. time-to-OOM: ~1,127 min (~18.8 h) at current slope
|
|
||||||
evidence : (3 log line citations with uptimes)
|
|
||||||
```
|
|
||||||
|
|
||||||
Then: **what to do next.**
|
|
||||||
- SLOW LEAK, **DEBUG_HEAP off** → recommend rebuilding with the flag and re-running this skill. Concrete one-liner the operator can copy:
|
|
||||||
```text
|
|
||||||
mcp__meshtastic__build(env="<env>", build_flags={"DEBUG_HEAP": 1})
|
|
||||||
mcp__meshtastic__pio_flash(env="<env>", port="<port>", confirm=True)
|
|
||||||
```
|
|
||||||
After flash, set debug_log_api back on and wait one window; re-run `/leakhunt`.
|
|
||||||
- SLOW LEAK, **DEBUG_HEAP on** → cite the top-leaking thread name from step 6a. Point at the corresponding source file (`grep -rn "ThreadName(\"<name>\")" src/`); the operator decides what to fix.
|
|
||||||
- FRAGMENTATION SUSPECT → propose pre-allocating any per-packet buffers; or rebuilding with `CONFIG_HEAP_TASK_TRACKING=y` on ESP32 to see who's holding the largest blocks.
|
|
||||||
- OOM-IMMINENT → flag for immediate attention; don't wait for the next telemetry interval.
|
|
||||||
- STEADY → say so; stop. Don't invent problems.
|
|
||||||
|
|
||||||
## What NOT to do
|
|
||||||
|
|
||||||
- Don't assume a leak from a single dip. LocalStats fires every ~60 s and the firmware naturally allocates+frees on each broadcast cycle; one packet sees the trough. Look at the slope, not the deltas.
|
|
||||||
- Don't recommend code changes. This skill diagnoses; the operator decides what to fix.
|
|
||||||
- Don't enable `set_debug_log_api` automatically - if it's off, telemetry isn't reaching pubsub anyway, and the recorder will be empty. Tell the operator to flip it on and wait, then re-run.
|
|
||||||
- Don't run heavy workloads to "trigger the leak." The recorder is passive; we read what's there.
|
|
||||||
|
|
||||||
## Companion: `mark_event` for stress runs
|
|
||||||
|
|
||||||
If the operator wants to test under stimulus (e.g. blast 50 broadcasts and see what the heap does), they can frame the experiment with markers:
|
|
||||||
|
|
||||||
```text
|
|
||||||
mark_event("burst-start")
|
|
||||||
… run the workload …
|
|
||||||
mark_event("burst-end")
|
|
||||||
/leakhunt window=15m
|
|
||||||
```
|
|
||||||
|
|
||||||
The markers land in both `events.jsonl` and `logs.jsonl`, so the report can show "free_heap dipped 8 KB during the burst window, recovered to baseline within 2 LocalStats cycles" → not a leak.
|
|
||||||
@@ -1,70 +0,0 @@
|
|||||||
---
|
|
||||||
description: Re-run a specific test N times in isolation to triage flakes, diff firmware logs between passes and failures
|
|
||||||
argument-hint: <test-node-id> [count=5]
|
|
||||||
---
|
|
||||||
|
|
||||||
<!-- markdownlint-disable MD029 -->
|
|
||||||
|
|
||||||
# `/repro` - flakiness triage for one test
|
|
||||||
|
|
||||||
Re-run a single pytest node ID N times in isolation, track pass rate, and surface what's _different_ in the firmware logs between the passing attempts and the failing ones. Turns "it's flaky, I guess" into "it fails when X, passes when Y."
|
|
||||||
|
|
||||||
## What to do
|
|
||||||
|
|
||||||
1. **Parse `$ARGUMENTS`**: first token is the pytest node id (e.g. `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip[nrf52->esp32s3]`); second token is an integer count (default `5`, cap at `20`). If the first token doesn't look like a test path (no `::` and no `tests/` prefix), treat the whole `$ARGUMENTS` as a `-k` filter instead.
|
|
||||||
|
|
||||||
2. **Sanity-check the hub first** (so we're not measuring "nothing plugged in" N times): call `mcp__meshtastic__list_devices`. If the test name contains `nrf52` or `esp32s3` and the matching VID isn't present, stop and report - re-running won't help.
|
|
||||||
|
|
||||||
3. **Loop N times**. For each iteration:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./mcp-server/run-tests.sh <test-id> --tb=short -p no:cacheprovider
|
|
||||||
```
|
|
||||||
|
|
||||||
Capture: exit code, duration, and (on failure) the `Meshtastic debug` firmware log section from `mcp-server/tests/report.html`. `-p no:cacheprovider` suppresses pytest's `.pytest_cache` writes so iterations don't influence each other.
|
|
||||||
|
|
||||||
4. **Track a small structured tally**:
|
|
||||||
|
|
||||||
```text
|
|
||||||
attempt 1: PASS (42s)
|
|
||||||
attempt 2: FAIL (128s) ← firmware log 200-line tail captured
|
|
||||||
attempt 3: PASS (39s)
|
|
||||||
attempt 4: FAIL (121s)
|
|
||||||
attempt 5: PASS (41s)
|
|
||||||
--------------------------------------
|
|
||||||
pass rate: 3/5 (60%) | mean duration: 74s
|
|
||||||
```
|
|
||||||
|
|
||||||
5. **On mixed outcomes**: diff the firmware log tails between a representative passing attempt and a representative failing attempt. Focus on:
|
|
||||||
- Error-level lines only present in failures (`PKI_UNKNOWN_PUBKEY`, `Alloc an err=`, `Skip send`, `No suitable channel`)
|
|
||||||
- Timing around the assertion event - did a broadcast go out, was there an ACK, did NAK fire?
|
|
||||||
- Device state fields that changed (nodesByNum entries, region/preset, channel_num)
|
|
||||||
|
|
||||||
Surface the top 3 differences as a "passes when / fails when" table. Don't dump full logs - pull specific lines with uptime timestamps.
|
|
||||||
|
|
||||||
5a. **Archive recorder slices per attempt** (no extra device interaction; the recorder runs autouse). Right after each attempt finishes, capture its `(start_ts, end_ts)` and call `mcp__meshtastic__recorder_export(start=<start>, end=<end>, dest_dir="mcp-server/tests/repro_artifacts/<safe-test-id>/attempt_<n>/")`. This drops a `logs.jsonl`, `telemetry.jsonl`, `packets.jsonl`, and `events.jsonl` snapshot scoped to the attempt window. Use these for cross-attempt diffs in step 5: `jq '.line' logs.jsonl` is faster than re-running the test, and the telemetry slice lets you compare heap behavior across attempts.
|
|
||||||
|
|
||||||
6. **Classify the flake** into one of:
|
|
||||||
- **LoRa airtime collision** → pass rate improves with fewer concurrent transmitters; propose a `time.sleep` gap or retry bump in the test body.
|
|
||||||
- **PKI key staleness** → fails on first attempt, passes after self-heal; existing retry loop in `test_direct_with_ack.py` handles this.
|
|
||||||
- **NodeInfo cooldown** → `Skip send NodeInfo since we sent it <600s ago` in fail-only logs; needs `broadcast_nodeinfo_ping()` warmup.
|
|
||||||
- **Hardware-specific** (one direction fails, other passes; one device's firmware is older; driver wedged) → specific recovery pointer. For a device that's wedged past `touch_1200bps`, the next escalation is `uhubctl_cycle(role=..., confirm=True)` to hard-power-cycle its hub port (requires `uhubctl` installed).
|
|
||||||
- **Device went dark mid-run** → fails from some attempt onward, never recovers, firmware log stops arriving. Almost always hardware: a Guru crash + frozen CDC. Hard-power-cycle via `uhubctl_cycle(role=..., confirm=True)` before the next iteration; if that also fails, escalate to replug.
|
|
||||||
- **Genuinely unknown** → say so; don't invent a root cause.
|
|
||||||
|
|
||||||
7. **Report back** with:
|
|
||||||
- Pass rate and mean duration.
|
|
||||||
- Classification + evidence (the specific log lines that support it).
|
|
||||||
- A suggested next step (re-run with specific args, open `/diagnose`, edit a specific test file, nothing).
|
|
||||||
|
|
||||||
## Examples
|
|
||||||
|
|
||||||
- `/repro tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip[esp32s3->nrf52] 10` - runs 10 times, diffs firmware logs.
|
|
||||||
- `/repro broadcast_delivers` - no `::`, no `tests/`, so interpreted as `-k broadcast_delivers`; runs every matching test the default 5 times.
|
|
||||||
- `/repro tests/telemetry/test_device_telemetry_broadcast.py 3` - shorter run for a slow test.
|
|
||||||
|
|
||||||
## Constraints
|
|
||||||
|
|
||||||
- Don't exceed `count=20` per invocation - airtime and USB wear add up. If the user asks for 50, negotiate down.
|
|
||||||
- Don't rebuild firmware as part of triage; flakes that only reproduce under different firmware belong in a separate session.
|
|
||||||
- If the FIRST attempt fails AND the rest all pass, that's a classic "state leak from a prior test" → say so and suggest running with `--force-bake` or starting from a clean state rather than chasing the first failure.
|
|
||||||
@@ -1,47 +0,0 @@
|
|||||||
---
|
|
||||||
description: Run the mcp-server test suite (auto-detects devices) and interpret the results
|
|
||||||
argument-hint: [pytest-args]
|
|
||||||
---
|
|
||||||
|
|
||||||
# `/test` - mcp-server test runner with interpretation
|
|
||||||
|
|
||||||
Run `mcp-server/run-tests.sh` and make sense of the output so the operator doesn't have to.
|
|
||||||
|
|
||||||
## What to do
|
|
||||||
|
|
||||||
1. **Invoke the wrapper.** From the firmware repo root, run:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./mcp-server/run-tests.sh $ARGUMENTS
|
|
||||||
```
|
|
||||||
|
|
||||||
The wrapper auto-detects connected Meshtastic devices, maps each to its PlatformIO env, exports the required `MESHTASTIC_MCP_ENV_*` env vars, and invokes pytest. If the user passed no arguments, the wrapper supplies a sensible default set (`tests/ --html=tests/report.html --self-contained-html --junitxml=tests/junit.xml -v --tb=short`). A `--report-log=tests/reportlog.jsonl` arg is always appended (unless the operator passed their own). `--assume-baked` is deliberately NOT in the defaults - `test_00_bake.py` has its own skip-if-already-baked check and runs the ~8 s verification by default. Operators can opt into the fast path with `--assume-baked`, or force a reflash with `--force-bake`.
|
|
||||||
|
|
||||||
2. **Read the pre-flight header.** First ~6 lines print the detected hub (role → port → env). If that line reads `detected hub : (none)`, the wrapper will narrow to `tests/unit` only - say so explicitly in your summary so the operator knows hardware tiers were skipped.
|
|
||||||
|
|
||||||
3. **On pass**: one-line summary of the form `N passed, M skipped in <duration>`. Don't enumerate the test names - the user can read those. Do mention any SKIPPED tests and name the cause:
|
|
||||||
- `"role not present on hub"` → device unplugged; operator knows to reconnect.
|
|
||||||
- `"firmware not baked with USERPREFS_UI_TEST_LOG"` → tests/ui skipped because the macro isn't in firmware yet; suggest `--force-bake`.
|
|
||||||
- `"uhubctl not installed"` → tests/recovery + peer-offline skipped; suggest `brew install uhubctl` / `apt install uhubctl`.
|
|
||||||
- `"no PPPS-capable hubs detected"` → tests/recovery skipped because the hub doesn't support per-port power; the tier will never run on that setup.
|
|
||||||
- `"opencv-python-headless is not installed"` → tests/ui auto-deselected by run-tests.sh; suggest `pip install -e 'mcp-server/.[ui]'`.
|
|
||||||
|
|
||||||
4. **On failure**: for every FAILED test, open `mcp-server/tests/report.html` and extract the `Meshtastic debug` section for that test. pytest-html embeds the firmware log stream + device state dump there; the 200-line firmware log tail is usually enough to explain the failure. Summarise: which test, one-line assertion message, the firmware log lines that matter (things like `PKI_UNKNOWN_PUBKEY`, `Skip send NodeInfo`, `Error=`, `Guru Meditation`, `assertion failed`). For UI-tier failures also glance at `mcp-server/tests/ui_captures/<session>/<test>/transcript.md` - it records each step's frame + OCR.
|
|
||||||
|
|
||||||
5. **Classify the failure** as one of:
|
|
||||||
- **Transient/flake**: LoRa collision, timing-sensitive assertion, first-attempt NAK + successful retry pattern. Propose `/repro <test_node_id>` to confirm.
|
|
||||||
- **Environmental**: device unreachable, port busy, CP2102 driver wedged. Suggest the specific recovery in escalation order: (a) replug USB, (b) `touch_1200bps(port=...)` + `pio_flash` for nRF52 DFU, (c) `uhubctl_cycle(role="nrf52", confirm=True)` when a device is fully wedged past DFU (needs `uhubctl` installed - `baked_single`'s auto-recovery hook does this once automatically). Also check `git status userPrefs.jsonc`.
|
|
||||||
- **Regression**: same assertion fails repeatedly, firmware log shows a new/unusual error. Surface the diff between expected and observed, identify the module likely responsible.
|
|
||||||
|
|
||||||
6. **Never run destructive recovery automatically.** If a failure looks like it needs a reflash, factory*reset, `uhubctl_cycle`, or USB replug, \_describe what to do* - don't execute. The operator decides.
|
|
||||||
|
|
||||||
## Arguments handling
|
|
||||||
|
|
||||||
- No args → wrapper's defaults (full suite).
|
|
||||||
- `$ARGUMENTS` passed verbatim to the wrapper, which passes them to pytest.
|
|
||||||
- Common operator invocations: `/test tests/mesh`, `/test tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip`, `/test --force-bake`, `/test -k telemetry`.
|
|
||||||
|
|
||||||
## Side-effects to mention in summary
|
|
||||||
|
|
||||||
- The session fixture snapshots `userPrefs.jsonc` at session start and restores at teardown (plus on `atexit`). After a clean run, `git status userPrefs.jsonc` should be empty. If the wrapper's pre-flight printed a warning about a stale sidecar, call that out - means a prior session crashed.
|
|
||||||
- `mcp-server/tests/report.html` and `junit.xml` are regenerated on every run; the HTML is self-contained (shareable).
|
|
||||||
@@ -2,12 +2,12 @@
|
|||||||
|
|
||||||
> **TL;DR**
|
> **TL;DR**
|
||||||
>
|
>
|
||||||
> | | |
|
> | | |
|
||||||
> | -------------- | -------------------------------------------------------------------------------------------- |
|
> | -------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||||
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
|
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
|
||||||
> | Hardware tests | `./mcp-server/run-tests.sh` |
|
> | Hardware tests | [meshtastic/meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) (`MESHTASTIC_FIRMWARE_ROOT` → this checkout) |
|
||||||
> | Format | `trunk fmt` |
|
> | Format | `trunk fmt` |
|
||||||
> | Mirror docs | `AGENTS.md` (short pointer for agents that don't read this file) · `CLAUDE.md` (Claude Code) |
|
> | Mirror docs | `AGENTS.md` (short pointer for agents that don't read this file) · `CLAUDE.md` (Claude Code) |
|
||||||
>
|
>
|
||||||
> **Need this? It's here.**
|
> **Need this? It's here.**
|
||||||
>
|
>
|
||||||
@@ -737,15 +737,15 @@ Simulation testing: `bin/test-simulator.sh`
|
|||||||
|
|
||||||
Quick entry point for new test modules: `test/README.md` (native unit-test authoring guide, skeleton, pitfalls, and setup checklist).
|
Quick entry point for new test modules: `test/README.md` (native unit-test authoring guide, skeleton, pitfalls, and setup checklist).
|
||||||
|
|
||||||
### Hardware-in-the-loop tests (`mcp-server/tests/`)
|
### Hardware-in-the-loop tests ([meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp))
|
||||||
|
|
||||||
Separate pytest suite that exercises real USB-connected Meshtastic devices. See the **MCP Server & Hardware Test Harness** section below for invocation, tier layout, and agent usage rules.
|
Separate pytest suite that exercises real USB-connected Meshtastic devices. It now lives in the standalone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) repo, run against a firmware checkout via `MESHTASTIC_FIRMWARE_ROOT`. See the **MCP Server & Hardware Test Harness** section below for invocation, tier layout, and agent usage rules.
|
||||||
|
|
||||||
## MCP Server & Hardware Test Harness
|
## MCP Server & Hardware Test Harness
|
||||||
|
|
||||||
The `mcp-server/` directory houses a firmware-aware [MCP](https://modelcontextprotocol.io/) server plus a pytest-based integration suite. AI agents that speak MCP get a well-defined tool surface for flashing, configuring, and inspecting physical Meshtastic devices - use it instead of hand-rolling `pio` or `meshtastic --port` calls where possible. `mcp-server/README.md` is the operator-facing setup doc; this section is the agent-facing usage contract.
|
The firmware-aware [MCP](https://modelcontextprotocol.io/) server plus its pytest-based integration suite now live in the standalone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) repo. AI agents that speak MCP get a well-defined tool surface for flashing, configuring, and inspecting physical Meshtastic devices - use it instead of hand-rolling `pio` or `meshtastic --port` calls where possible. The meshtastic-mcp repo's README is the operator-facing setup doc; this section is the agent-facing usage contract.
|
||||||
|
|
||||||
The repo registers the server via `.mcp.json` at the repo root - Claude Code picks it up automatically once `mcp-server/.venv/` is built (`cd mcp-server && python3 -m venv .venv && .venv/bin/pip install -e '.[test]'`).
|
The repo registers the server via `.mcp.json` at the repo root - Claude Code / Copilot pick it up automatically and run it through `uvx --from git+https://github.com/meshtastic/meshtastic-mcp meshtastic-mcp`, so the MCP tools work with no local build. To run the pytest hardware harness instead, clone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) and point `MESHTASTIC_FIRMWARE_ROOT` at this firmware checkout.
|
||||||
|
|
||||||
### When to use which surface
|
### When to use which surface
|
||||||
|
|
||||||
@@ -757,7 +757,7 @@ The repo registers the server via `.mcp.json` at the repo root - Claude Code pic
|
|||||||
| Flash firmware to a variant | `pio_flash` (any arch) or `erase_and_flash` (ESP32 factory install) |
|
| Flash firmware to a variant | `pio_flash` (any arch) or `erase_and_flash` (ESP32 factory install) |
|
||||||
| Stream serial logs while debugging | `serial_open` → `serial_read` loop → `serial_close` |
|
| Stream serial logs while debugging | `serial_open` → `serial_read` loop → `serial_close` |
|
||||||
| Administer `userPrefs.jsonc` build-time constants | `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest` |
|
| Administer `userPrefs.jsonc` build-time constants | `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest` |
|
||||||
| Run the regression suite | `./mcp-server/run-tests.sh` (or `/test` slash command) |
|
| Run the regression suite | `./run-tests.sh` from a meshtastic-mcp checkout (or `/test` slash command) |
|
||||||
| Diagnose a specific device | `/diagnose [role]` slash command (read-only) |
|
| Diagnose a specific device | `/diagnose [role]` slash command (read-only) |
|
||||||
| Triage a flaky test | `/repro <node-id> [count]` slash command |
|
| Triage a flaky test | `/repro <node-id> [count]` slash command |
|
||||||
|
|
||||||
@@ -765,7 +765,7 @@ The repo registers the server via `.mcp.json` at the repo root - Claude Code pic
|
|||||||
|
|
||||||
### MCP tool surface (43 tools)
|
### MCP tool surface (43 tools)
|
||||||
|
|
||||||
Grouped by purpose. Full argument shapes in `mcp-server/README.md`; a few high-value signatures are called out here.
|
Grouped by purpose. Full argument shapes in the meshtastic-mcp repo's README; a few high-value signatures are called out here.
|
||||||
|
|
||||||
- **Discovery & metadata**: `list_devices`, `list_boards`, `get_board`
|
- **Discovery & metadata**: `list_devices`, `list_boards`, `get_board`
|
||||||
- **Build & flash**: `build`, `clean`, `pio_flash`, `erase_and_flash` (ESP32 only), `update_flash` (ESP32 OTA), `touch_1200bps`
|
- **Build & flash**: `build`, `clean`, `pio_flash`, `erase_and_flash` (ESP32 only), `update_flash` (ESP32 OTA), `touch_1200bps`
|
||||||
@@ -775,13 +775,13 @@ Grouped by purpose. Full argument shapes in `mcp-server/README.md`; a few high-v
|
|||||||
- **userPrefs admin** (build-time constants, not runtime config): `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest`, `userprefs_testing_profile`
|
- **userPrefs admin** (build-time constants, not runtime config): `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest`, `userprefs_testing_profile`
|
||||||
- **Vendor escape hatches**: `esptool_chip_info`, `esptool_erase_flash`, `esptool_raw`, `nrfutil_dfu`, `nrfutil_raw`, `picotool_info`, `picotool_load`, `picotool_raw`
|
- **Vendor escape hatches**: `esptool_chip_info`, `esptool_erase_flash`, `esptool_raw`, `nrfutil_dfu`, `nrfutil_raw`, `picotool_info`, `picotool_load`, `picotool_raw`
|
||||||
- **USB power control** (via `uhubctl`, per-port PPPS toggle): `uhubctl_list` (read-only), `uhubctl_power(action='on'|'off', confirm=True)`, `uhubctl_cycle(delay_s, confirm=True)`. Target by raw `(location, port)` or by `role` (`"nrf52"`, `"esp32s3"`); role lookup checks `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` + `_PORT_<ROLE>` env vars first, falls back to VID auto-detection.
|
- **USB power control** (via `uhubctl`, per-port PPPS toggle): `uhubctl_list` (read-only), `uhubctl_power(action='on'|'off', confirm=True)`, `uhubctl_cycle(delay_s, confirm=True)`. Target by raw `(location, port)` or by `role` (`"nrf52"`, `"esp32s3"`); role lookup checks `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` + `_PORT_<ROLE>` env vars first, falls back to VID auto-detection.
|
||||||
- **Observability** (UI tier + operator ad-hoc): `capture_screen(role, ocr=True)` - grabs a USB-webcam frame of the device OLED and optionally OCRs it. Requires `mcp-server[ui]` extras (`opencv-python-headless`, `easyocr`) and `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` env var; falls through to a 1×1 black PNG `NullBackend` when unconfigured.
|
- **Observability** (UI tier + operator ad-hoc): `capture_screen(role, ocr=True)` - grabs a USB-webcam frame of the device OLED and optionally OCRs it. Requires `meshtastic-mcp[ui]` extras (`opencv-python-headless`, `easyocr`) and `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` env var; falls through to a 1×1 black PNG `NullBackend` when unconfigured.
|
||||||
|
|
||||||
`confirm=True` is a tool-level gate on top of whatever permission prompt your MCP host shows. **Don't bypass it** by asking the host to auto-approve - it exists specifically because MCP hosts sometimes remember "always allow this tool" and that's dangerous for `factory_reset`, `erase_and_flash`, `uhubctl_power(action='off')`, and `uhubctl_cycle`.
|
`confirm=True` is a tool-level gate on top of whatever permission prompt your MCP host shows. **Don't bypass it** by asking the host to auto-approve - it exists specifically because MCP hosts sometimes remember "always allow this tool" and that's dangerous for `factory_reset`, `erase_and_flash`, `uhubctl_power(action='off')`, and `uhubctl_cycle`.
|
||||||
|
|
||||||
**TCP / native-host nodes.** Setting `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` makes `list_devices` surface a `meshtasticd` daemon (e.g. the `native-macos` build) as a synthetic `tcp://host:port` entry, and `connect()` routes through `meshtastic.tcp_interface.TCPInterface` instead of `SerialInterface`. Every read/write/admin tool that flows through `connect()` works against the daemon transparently. USB-only tools (`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`, `serial_open`, `esptool_*`, `nrfutil_*`, `picotool_*`) raise a clear `ConnectionError` when handed a `tcp://` port; `pio_flash` against a `native*` env raises a `FlashError` (no upload step - use `build` and run the binary directly). The pytest harness still assumes USB-attached devices per role; TCP-aware fixtures are deferred. See `mcp-server/README.md` § "TCP / native-host nodes".
|
**TCP / native-host nodes.** Setting `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` makes `list_devices` surface a `meshtasticd` daemon (e.g. the `native-macos` build) as a synthetic `tcp://host:port` entry, and `connect()` routes through `meshtastic.tcp_interface.TCPInterface` instead of `SerialInterface`. Every read/write/admin tool that flows through `connect()` works against the daemon transparently. USB-only tools (`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`, `serial_open`, `esptool_*`, `nrfutil_*`, `picotool_*`) raise a clear `ConnectionError` when handed a `tcp://` port; `pio_flash` against a `native*` env raises a `FlashError` (no upload step - use `build` and run the binary directly). The pytest harness still assumes USB-attached devices per role; TCP-aware fixtures are deferred. See the meshtastic-mcp repo's README § "TCP / native-host nodes".
|
||||||
|
|
||||||
### Hardware test suite (`mcp-server/run-tests.sh`)
|
### Hardware test suite (`run-tests.sh`, from a meshtastic-mcp checkout)
|
||||||
|
|
||||||
The wrapper auto-detects connected devices (VID → role map: `0x239A` → `nrf52`, `0x303A`/`0x10C4` → `esp32s3`), maps each role to a PlatformIO env (`nrf52` → `rak4631`, `esp32s3` → `heltec-v3`, overridable via `MESHTASTIC_MCP_ENV_<ROLE>`), then invokes pytest. Zero pre-flight config needed from the operator.
|
The wrapper auto-detects connected devices (VID → role map: `0x239A` → `nrf52`, `0x303A`/`0x10C4` → `esp32s3`), maps each role to a PlatformIO env (`nrf52` → `rak4631`, `esp32s3` → `heltec-v3`, overridable via `MESHTASTIC_MCP_ENV_<ROLE>`), then invokes pytest. Zero pre-flight config needed from the operator.
|
||||||
|
|
||||||
@@ -801,22 +801,23 @@ Suite tiers (collected + run in this order via `pytest_collection_modifyitems`):
|
|||||||
Invocation patterns:
|
Invocation patterns:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./mcp-server/run-tests.sh # full suite (auto-bake-if-needed)
|
# run from a meshtastic-mcp checkout, with MESHTASTIC_FIRMWARE_ROOT=/path/to/firmware
|
||||||
./mcp-server/run-tests.sh --force-bake # reflash before testing
|
./run-tests.sh # full suite (auto-bake-if-needed)
|
||||||
./mcp-server/run-tests.sh --assume-baked # skip bake (caller vouches for device state)
|
./run-tests.sh --force-bake # reflash before testing
|
||||||
./mcp-server/run-tests.sh tests/mesh # one tier
|
./run-tests.sh --assume-baked # skip bake (caller vouches for device state)
|
||||||
./mcp-server/run-tests.sh tests/mesh/test_direct_with_ack.py # one file
|
./run-tests.sh tests/mesh # one tier
|
||||||
./mcp-server/run-tests.sh -k telemetry # name filter
|
./run-tests.sh tests/mesh/test_direct_with_ack.py # one file
|
||||||
|
./run-tests.sh -k telemetry # name filter
|
||||||
```
|
```
|
||||||
|
|
||||||
**No hardware detected?** The wrapper auto-narrows to `tests/unit/` only and prints `detected hub : (none)` in the pre-flight header. Agents interpreting the output should call this out explicitly - a 52-test green run without hardware is qualitatively different from a 12-unit-test green run.
|
**No hardware detected?** The wrapper auto-narrows to `tests/unit/` only and prints `detected hub : (none)` in the pre-flight header. Agents interpreting the output should call this out explicitly - a 52-test green run without hardware is qualitatively different from a 12-unit-test green run.
|
||||||
|
|
||||||
**Artifacts every run produces:**
|
**Artifacts every run produces:**
|
||||||
|
|
||||||
- `mcp-server/tests/report.html` - self-contained pytest-html. Each test gets a `Meshtastic debug` section with the tail of firmware log + device state dump. **Open this first** on failures; it's the canonical evidence source.
|
- `tests/report.html` - self-contained pytest-html. Each test gets a `Meshtastic debug` section with the tail of firmware log + device state dump. **Open this first** on failures; it's the canonical evidence source.
|
||||||
- `mcp-server/tests/junit.xml` - CI-parseable.
|
- `tests/junit.xml` - CI-parseable.
|
||||||
- `mcp-server/tests/reportlog.jsonl` - pytest-reportlog stream (`$report_type` keyed JSONL). Consumed by the live TUI.
|
- `tests/reportlog.jsonl` - pytest-reportlog stream (`$report_type` keyed JSONL). Consumed by the live TUI.
|
||||||
- `mcp-server/tests/fwlog.jsonl` - firmware log mirror from the `meshtastic.log.line` pubsub topic. Populated by the `_firmware_log_stream` autouse session fixture.
|
- `tests/fwlog.jsonl` - firmware log mirror from the `meshtastic.log.line` pubsub topic. Populated by the `_firmware_log_stream` autouse session fixture.
|
||||||
|
|
||||||
### Live TUI (`meshtastic-mcp-test-tui`)
|
### Live TUI (`meshtastic-mcp-test-tui`)
|
||||||
|
|
||||||
@@ -836,7 +837,7 @@ A Textual-based live view that wraps `run-tests.sh`. Tails reportlog for per-tes
|
|||||||
Launch:
|
Launch:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd mcp-server
|
# from a meshtastic-mcp checkout (MESHTASTIC_FIRMWARE_ROOT set)
|
||||||
.venv/bin/meshtastic-mcp-test-tui # full suite
|
.venv/bin/meshtastic-mcp-test-tui # full suite
|
||||||
.venv/bin/meshtastic-mcp-test-tui tests/mesh # args pass through to pytest
|
.venv/bin/meshtastic-mcp-test-tui tests/mesh # args pass through to pytest
|
||||||
```
|
```
|
||||||
@@ -861,7 +862,7 @@ House rules for agents running these prompts:
|
|||||||
|
|
||||||
### Key fixtures (test authors + agents debugging)
|
### Key fixtures (test authors + agents debugging)
|
||||||
|
|
||||||
`mcp-server/tests/conftest.py` provides:
|
`tests/conftest.py` (in the meshtastic-mcp checkout) provides:
|
||||||
|
|
||||||
- **`_session_userprefs`** (autouse session) - snapshots `userPrefs.jsonc` at session start, merges the session test profile via `userprefs.merge_active(test_profile)`, restores at teardown. Four layers of safety: pytest teardown + `atexit` + sidecar file (`userPrefs.jsonc.mcp-session-bak`) + startup self-heal in `run-tests.sh`. **Do not edit `userPrefs.jsonc` from inside a test.**
|
- **`_session_userprefs`** (autouse session) - snapshots `userPrefs.jsonc` at session start, merges the session test profile via `userprefs.merge_active(test_profile)`, restores at teardown. Four layers of safety: pytest teardown + `atexit` + sidecar file (`userPrefs.jsonc.mcp-session-bak`) + startup self-heal in `run-tests.sh`. **Do not edit `userPrefs.jsonc` from inside a test.**
|
||||||
- **`_firmware_log_stream`** (autouse session) - subscribes to `meshtastic.log.line` pubsub on every connected `SerialInterface` and mirrors lines to `tests/fwlog.jsonl`. Drives the TUI firmware-log pane.
|
- **`_firmware_log_stream`** (autouse session) - subscribes to `meshtastic.log.line` pubsub on every connected `SerialInterface` and mirrors lines to `tests/fwlog.jsonl`. Drives the TUI firmware-log pane.
|
||||||
@@ -877,13 +878,13 @@ Two firmware changes exist specifically so the test harness works reliably. **Ke
|
|||||||
- **`src/mesh/StreamAPI.cpp` + `StreamAPI.h`** - `emitLogRecord` uses a dedicated `fromRadioScratchLog` + `txBufLog` pair and a `concurrency::Lock streamLock`. Before this fix, `debug_log_api_enabled=true` would tear `FromRadio` protobufs on the serial transport because `emitTxBuffer` and `emitLogRecord` shared a single scratch buffer. The conftest enables the log stream session-wide; without this fix the device would corrupt its own FromRadio replies mid-session.
|
- **`src/mesh/StreamAPI.cpp` + `StreamAPI.h`** - `emitLogRecord` uses a dedicated `fromRadioScratchLog` + `txBufLog` pair and a `concurrency::Lock streamLock`. Before this fix, `debug_log_api_enabled=true` would tear `FromRadio` protobufs on the serial transport because `emitTxBuffer` and `emitLogRecord` shared a single scratch buffer. The conftest enables the log stream session-wide; without this fix the device would corrupt its own FromRadio replies mid-session.
|
||||||
- **`src/mesh/PhoneAPI.cpp`** - `ToRadio` `Heartbeat(nonce=1)` triggers `nodeInfoModule->sendOurNodeInfo(NODENUM_BROADCAST, true, 0, true)` for serial clients, mirroring the pre-existing behavior for TCP/UDP clients in `PacketAPI.cpp`. The mesh tests rely on this to force a NodeInfo broadcast right after connect so the peer discovers them before the test's first assertion.
|
- **`src/mesh/PhoneAPI.cpp`** - `ToRadio` `Heartbeat(nonce=1)` triggers `nodeInfoModule->sendOurNodeInfo(NODENUM_BROADCAST, true, 0, true)` for serial clients, mirroring the pre-existing behavior for TCP/UDP clients in `PacketAPI.cpp`. The mesh tests rely on this to force a NodeInfo broadcast right after connect so the peer discovers them before the test's first assertion.
|
||||||
|
|
||||||
If you're modifying `StreamAPI`, `PhoneAPI`, `NodeInfoModule`, or `userPrefs` flow, run `./mcp-server/run-tests.sh` at minimum before asking for review.
|
If you're modifying `StreamAPI`, `PhoneAPI`, `NodeInfoModule`, or `userPrefs` flow, run `./run-tests.sh` (from a meshtastic-mcp checkout, with `MESHTASTIC_FIRMWARE_ROOT` pointed here) at minimum before asking for review.
|
||||||
|
|
||||||
### Recovery playbooks
|
### Recovery playbooks
|
||||||
|
|
||||||
| Symptom | First check | Fix |
|
| Symptom | First check | Fix |
|
||||||
| --------------------------------------------------------------------------------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| --------------------------------------------------------------------------------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `userPrefs.jsonc` dirty after test run | `git status --porcelain userPrefs.jsonc` | If non-empty, re-run `./mcp-server/run-tests.sh` once - the pre-flight self-heal restores from sidecar. If still dirty, `git checkout userPrefs.jsonc`. |
|
| `userPrefs.jsonc` dirty after test run | `git status --porcelain userPrefs.jsonc` | If non-empty, re-run `./run-tests.sh` (from a meshtastic-mcp checkout) once - the pre-flight self-heal restores from sidecar. If still dirty, `git checkout userPrefs.jsonc`. |
|
||||||
| Port busy / wedged CP2102 on macOS | `lsof /dev/cu.usbserial-0001` | Kill the holder. USB replug if the kernel still reports busy. Often a stale `pio device monitor` or zombie `meshtastic_mcp` process. |
|
| Port busy / wedged CP2102 on macOS | `lsof /dev/cu.usbserial-0001` | Kill the holder. USB replug if the kernel still reports busy. Often a stale `pio device monitor` or zombie `meshtastic_mcp` process. |
|
||||||
| nRF52 appears unresponsive | `list_devices` shows VID `0x239A` but `device_info` times out | `touch_1200bps(port=...)` drops it into the DFU bootloader → `pio_flash` re-installs. |
|
| nRF52 appears unresponsive | `list_devices` shows VID `0x239A` but `device_info` times out | `touch_1200bps(port=...)` drops it into the DFU bootloader → `pio_flash` re-installs. |
|
||||||
| Device fully wedged (Guru Meditation, frozen CDC, no DFU) | `list_devices` shows the VID but every admin call times out | `uhubctl_cycle(role="nrf52", confirm=True)` hard-power-cycles the port via USB hub PPPS. `baked_single`'s auto-recovery hook does this once automatically if uhubctl is installed. Falls back to physical replug if no PPPS hub. |
|
| Device fully wedged (Guru Meditation, frozen CDC, no DFU) | `list_devices` shows the VID but every admin call times out | `uhubctl_cycle(role="nrf52", confirm=True)` hard-power-cycles the port via USB hub PPPS. `baked_single`'s auto-recovery hook does this once automatically if uhubctl is installed. Falls back to physical replug if no PPPS hub. |
|
||||||
|
|||||||
@@ -1,64 +0,0 @@
|
|||||||
---
|
|
||||||
mode: agent
|
|
||||||
description: Device health report via the meshtastic MCP tools (Copilot equivalent of the Claude Code /diagnose slash command)
|
|
||||||
---
|
|
||||||
|
|
||||||
# `/mcp-diagnose` - device health report
|
|
||||||
|
|
||||||
Equivalent of `.claude/commands/diagnose.md`. Use when the operator asks to "check the devices", "what's the mesh looking like", "is nrf52 alive", etc.
|
|
||||||
|
|
||||||
This prompt assumes the meshtastic MCP server is registered with your VS Code Copilot agent. If it isn't, fall back to running `./mcp-server/run-tests.sh tests/unit` plus a short `device_info` script via the terminal.
|
|
||||||
|
|
||||||
## What to do
|
|
||||||
|
|
||||||
1. **Enumerate hardware** via the `list_devices` MCP tool (with `include_unknown=True`). For each entry where `likely_meshtastic=True`, capture `port`, `vid`, `pid`, `description`.
|
|
||||||
|
|
||||||
2. **Apply the operator's filter** (if any):
|
|
||||||
- No filter → every likely-meshtastic device.
|
|
||||||
- `nrf52` → `vid == 0x239a`
|
|
||||||
- `esp32s3` → `vid == 0x303a` or `vid == 0x10c4`
|
|
||||||
- A `/dev/cu.*` path → only that port.
|
|
||||||
- Anything else → substring match on port.
|
|
||||||
|
|
||||||
3. **For each selected device, in sequence (don't parallelize - SerialInterface holds an exclusive port lock):**
|
|
||||||
- `device_info(port=<p>)` → `my_node_num`, `long_name`, `short_name`, `firmware_version`, `hw_model`, `region`, `num_nodes`, `primary_channel`
|
|
||||||
- `list_nodes(port=<p>)` → peer count, which peers have `publicKey`, SNR/RSSI distribution
|
|
||||||
- `get_config(section="lora", port=<p>)` → region, preset, channel_num, tx_power, hop_limit
|
|
||||||
- If anything looks off (can't connect, `num_nodes` wrong, missing `firmware_version`), open a short firmware-log window: `serial_open(port=<p>, env=<inferred>)`, wait 3 seconds, `serial_read(session_id, max_lines=100)`, `serial_close(session_id)`. Infer env from VID (0x239a → `rak4631`, 0x303a/0x10c4 → `heltec-v3`) unless an `MESHTASTIC_MCP_ENV_<ROLE>` env var overrides it.
|
|
||||||
|
|
||||||
4. **Hub health** (call once, not per-device): `uhubctl_list()` - enumerates every USB hub the host sees. Cross-reference each Meshtastic device's VID to find which hub + port it's on. Flag in the report if:
|
|
||||||
- No hub advertises `ppps=true` → `tests/recovery/` can't run; hard-recovery via `uhubctl_cycle` isn't available.
|
|
||||||
- A Meshtastic device is on a non-PPPS hub → note it; moving to a PPPS hub unlocks auto-recovery.
|
|
||||||
- `uhubctl_list` raises `ConfigError: uhubctl not found` → report as "uhubctl not installed"; don't treat as a device fault.
|
|
||||||
|
|
||||||
5. **Render per-device report** as a compact block:
|
|
||||||
|
|
||||||
```text
|
|
||||||
[nrf52 @ /dev/cu.usbmodem1101] fw=2.7.23.bce2825, hw=RAK4631
|
|
||||||
owner : Meshtastic 40eb / 40eb
|
|
||||||
region/band : US, channel 88, LONG_FAST
|
|
||||||
tx_power : 30 dBm, hop_limit=3
|
|
||||||
peers : 1 (esp32s3 0x433c2428, pubkey ✓, SNR 6.0 / RSSI -24 dBm)
|
|
||||||
primary ch : McpTest
|
|
||||||
hub : 1-1.3 port 2 (PPPS, uhubctl-controllable)
|
|
||||||
firmware : no panics in last 3s
|
|
||||||
```
|
|
||||||
|
|
||||||
Flag abnormalities inline with `⚠︎ <short reason>` - missing pubkey on a known peer, region UNSET, mismatched channel name, device on non-PPPS hub, etc.
|
|
||||||
|
|
||||||
6. **Cross-device correlation** (when >1 device selected):
|
|
||||||
- Do both see each other in `nodesByNum`?
|
|
||||||
- Do `region`, `channel_num`, `modem_preset` match across devices?
|
|
||||||
- Do the primary channel names match? (Different name → different PSK → no decode.)
|
|
||||||
|
|
||||||
7. **Suggest next steps only for recognizable failure modes**, never speculatively:
|
|
||||||
- Stale PKI one-way → "`/mcp-test tests/mesh/test_direct_with_ack.py` - the test's retry+nodeinfo-ping heals this."
|
|
||||||
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
|
|
||||||
- Device unreachable, DFU reachable → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds and the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
|
|
||||||
- CP2102-wedged-driver on macOS → see `run-tests.sh` notes.
|
|
||||||
|
|
||||||
## Hard constraints
|
|
||||||
|
|
||||||
- **Read-only.** No `set_config`, no `reboot`, no `factory_reset`, no `flash`. If the operator wants mutation, they'll escalate explicitly.
|
|
||||||
- **Open/query/close per device.** Never hold multiple SerialInterfaces to the same port. The port lock is exclusive.
|
|
||||||
- **Don't infer env beyond the VID map** - if the operator has an unusual board, ask them which env to use rather than guessing.
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
---
|
|
||||||
mode: agent
|
|
||||||
description: Re-run a specific test N times to triage flakes; diff firmware logs between passes and failures (Copilot equivalent of the Claude Code /repro slash command)
|
|
||||||
---
|
|
||||||
|
|
||||||
# `/mcp-repro` - flakiness triage for one test
|
|
||||||
|
|
||||||
Equivalent of `.claude/commands/repro.md`. Use when the operator says "that one test is flaky - dig in", "repro the direct_with_ack failure", "why does X sometimes fail?".
|
|
||||||
|
|
||||||
## What to do
|
|
||||||
|
|
||||||
1. **Parse the operator's input** into two pieces:
|
|
||||||
- **Test identifier** - either a pytest node id (has `::` or starts with `tests/`) or a `-k`-style filter (plain substring like `direct_with_ack`).
|
|
||||||
- **Count** - integer, default `5`, cap at `20`. If the operator asks for 50, negotiate down and explain (airtime + USB wear).
|
|
||||||
|
|
||||||
2. **Sanity-check the hub** via the `list_devices` MCP tool. If the test name references `nrf52` or `esp32s3` and the matching VID isn't present, stop and report - re-running won't help.
|
|
||||||
|
|
||||||
3. **Loop** N times. Each iteration:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./mcp-server/run-tests.sh <test-id> --tb=short -p no:cacheprovider
|
|
||||||
```
|
|
||||||
|
|
||||||
`-p no:cacheprovider` keeps pytest from caching anything between iterations. Capture: exit code, duration, and (on failure) the `Meshtastic debug` firmware-log section from `mcp-server/tests/report.html`.
|
|
||||||
|
|
||||||
4. **Tally** results as you go:
|
|
||||||
|
|
||||||
```text
|
|
||||||
attempt 1: PASS (42s)
|
|
||||||
attempt 2: FAIL (128s) ← fw log captured
|
|
||||||
attempt 3: PASS (39s)
|
|
||||||
attempt 4: FAIL (121s)
|
|
||||||
attempt 5: PASS (41s)
|
|
||||||
--------------------------------------------------
|
|
||||||
pass rate: 3/5 (60%) | mean duration: 74s
|
|
||||||
```
|
|
||||||
|
|
||||||
5. **On mixed outcomes, diff the firmware logs** between one representative pass and one representative fail. Focus on:
|
|
||||||
- Error-level lines present only in failures (`PKI_UNKNOWN_PUBKEY`, `Alloc an err=`, `Skip send`, `No suitable channel`, `NAK`)
|
|
||||||
- Timing around the assertion point (broadcast sent? ACK received? retry fired?)
|
|
||||||
- Device-state fields that changed between attempts
|
|
||||||
|
|
||||||
Surface the top 3 differences as a compact "passes when / fails when" table with uptime timestamps. Don't dump full logs.
|
|
||||||
|
|
||||||
6. **Classify** the flake into one of:
|
|
||||||
- **LoRa airtime collision** - pass rate improves with fewer concurrent transmitters. Suggest a `time.sleep` gap or retry bump in the test body.
|
|
||||||
- **PKI key staleness** - first attempt fails, subsequent ones pass; existing retry-loop pattern in `test_direct_with_ack.py` is the fix.
|
|
||||||
- **NodeInfo cooldown** - `Skip send NodeInfo since we sent it <600s ago` in fail-only logs; needs a `broadcast_nodeinfo_ping()` warmup.
|
|
||||||
- **Hardware-specific** - one direction consistently fails, firmware versions differ, CP2102 driver wedged, etc. For a device wedged past `touch_1200bps`, recommend `uhubctl_cycle(role=..., confirm=True)` to hard-power-cycle its hub port (requires `uhubctl` installed).
|
|
||||||
- **Device went dark mid-run** - fails from some iteration onward and never recovers; firmware log stops arriving. Almost always a Guru crash with frozen CDC. Recommend `uhubctl_cycle` before the next iteration; escalate to replug if that also fails.
|
|
||||||
- **Unknown** - say so. Don't invent a root cause.
|
|
||||||
|
|
||||||
7. **Report back** with:
|
|
||||||
- Pass rate + mean duration.
|
|
||||||
- Classification + the specific log evidence for it.
|
|
||||||
- A concrete next step (tighter assertion, more retries, open `/mcp-diagnose`, file a bug, nothing).
|
|
||||||
|
|
||||||
## Examples
|
|
||||||
|
|
||||||
- `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip[esp32s3->nrf52] 10` - 10 runs of that parametrized case.
|
|
||||||
- `broadcast_delivers` - no `::`, no `tests/`; treat as `-k broadcast_delivers`; runs every match 5 times.
|
|
||||||
- `tests/telemetry/test_device_telemetry_broadcast.py 3` - shorter count for a slow test.
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- If the FIRST attempt fails and the rest pass, that's a state-leak signature - suggest starting from `--force-bake` or a clean device state rather than chasing the first-failure firmware logs.
|
|
||||||
- If ALL N fail, this isn't a flake - it's a regression. Say so, stop iterating, escalate to `/mcp-test` for full-suite context.
|
|
||||||
- Don't rebuild firmware during triage. Flakes that only reproduce under different firmware belong in a separate session with a plan.
|
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
---
|
|
||||||
mode: agent
|
|
||||||
description: Run the mcp-server test suite and interpret results (Copilot equivalent of the Claude Code /test slash command)
|
|
||||||
---
|
|
||||||
|
|
||||||
# `/mcp-test` - mcp-server test runner with interpretation
|
|
||||||
|
|
||||||
Equivalent of the Claude Code `/test` slash command in `.claude/commands/test.md`. Use this when the operator asks you to "run the tests", "check the mcp test suite", "run the mesh tests", etc.
|
|
||||||
|
|
||||||
## What to do
|
|
||||||
|
|
||||||
1. **Invoke the wrapper** from the firmware repo root:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./mcp-server/run-tests.sh [pytest-args]
|
|
||||||
```
|
|
||||||
|
|
||||||
If the operator specified a subset (e.g. "just the mesh tests"), pass it through as `tests/mesh` or a pytest `-k filter`. If they said nothing, use the wrapper's defaults (full suite with pytest-html report).
|
|
||||||
|
|
||||||
The wrapper auto-detects connected Meshtastic devices, maps each to its PlatformIO env, exports the required env vars, and invokes pytest. Zero pre-flight config needed from the operator.
|
|
||||||
|
|
||||||
2. **Read the pre-flight header** (first few lines of wrapper output). The `detected hub :` line lists role → port → env mappings. If it reads `(none)`, the wrapper narrowed to `tests/unit` only - call that out explicitly so the operator knows hardware tiers were skipped.
|
|
||||||
|
|
||||||
3. **On pass**: one-line summary like `N passed, M skipped in <duration>`. Don't enumerate test names. DO mention any non-placeholder SKIPs and name the cause:
|
|
||||||
- `"role not present on hub"` → device unplugged; operator should reconnect.
|
|
||||||
- `"firmware not baked with USERPREFS_UI_TEST_LOG"` → tests/ui skipped; the UI-log compile macro isn't in the baked firmware. Suggest `--force-bake`.
|
|
||||||
- `"uhubctl not installed"` → tests/recovery + `test_peer_offline_recovery` skipped. Suggest `brew install uhubctl` / `apt install uhubctl`.
|
|
||||||
- `"no PPPS-capable hubs detected"` → tests/recovery skipped because the attached hub doesn't support per-port power switching; won't run on that setup.
|
|
||||||
- `"opencv-python-headless is not installed"` → tests/ui auto-deselected by `run-tests.sh`. Suggest `pip install -e 'mcp-server/.[ui]'`.
|
|
||||||
|
|
||||||
4. **On failure**: open `mcp-server/tests/report.html` (pytest-html output, self-contained) and extract the `Meshtastic debug` section for each failed test. That section includes a firmware log stream (last 200 lines) and device state dump. For each failure, summarise:
|
|
||||||
- test name
|
|
||||||
- one-line assertion message
|
|
||||||
- the specific firmware log lines that explain why (look for `PKI_UNKNOWN_PUBKEY`, `Skip send NodeInfo`, `Error=`, `Guru Meditation`, `assertion failed`, `No suitable channel`)
|
|
||||||
- for UI-tier failures also check `mcp-server/tests/ui_captures/<session>/<test>/transcript.md` (per-step frame + OCR)
|
|
||||||
|
|
||||||
5. **Classify each failure** as one of:
|
|
||||||
- **Transient flake** - LoRa collision, first-attempt NAK with self-heal pattern, timing-sensitive assertion. Suggest `/mcp-repro <test-id>` to confirm.
|
|
||||||
- **Environmental** - device unreachable, port busy, CP2102 driver wedged on macOS. Suggest recovery in escalation order: (a) replug USB, (b) `touch_1200bps` + `pio_flash` for nRF52 DFU, (c) `uhubctl_cycle(role=..., confirm=True)` for a device wedged past DFU (needs `uhubctl` installed; `baked_single` does this once automatically when available). Also check `git status userPrefs.jsonc`.
|
|
||||||
- **Regression** - same assertion fails repeatedly on re-runs, firmware log shows novel errors. Identify the firmware module likely responsible.
|
|
||||||
|
|
||||||
6. **Do NOT run destructive recovery automatically**. If a failure looks like it needs a reflash, factory*reset, `uhubctl_cycle`, or replug - \_describe the steps* and let the operator decide. Never burn airtime or flash cycles without approval.
|
|
||||||
|
|
||||||
## Arguments convention
|
|
||||||
|
|
||||||
Operators generally invoke this prompt either with no arguments (full suite) or with a specific subset. Examples:
|
|
||||||
|
|
||||||
- `tests/mesh` - one tier
|
|
||||||
- `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip` - one test
|
|
||||||
- `--force-bake` - reflash devices first
|
|
||||||
- `-k telemetry` - name-filter
|
|
||||||
|
|
||||||
## Side-effects to confirm in your summary
|
|
||||||
|
|
||||||
- `userPrefs.jsonc` should be clean after a successful run. The session fixture in `mcp-server/tests/conftest.py` (`_session_userprefs`) snapshots and restores. Check `git status --porcelain userPrefs.jsonc` and report if it's non-empty.
|
|
||||||
- `mcp-server/tests/report.html` and `junit.xml` regenerate on every run.
|
|
||||||
- The wrapper prints a warning if a `.mcp-session-bak` sidecar was left over from a crashed prior session and auto-restores from it - mention that if it happened.
|
|
||||||
@@ -1,8 +1,12 @@
|
|||||||
{
|
{
|
||||||
"mcpServers": {
|
"mcpServers": {
|
||||||
"meshtastic": {
|
"meshtastic": {
|
||||||
"command": "./mcp-server/.venv/bin/python",
|
"command": "uvx",
|
||||||
"args": ["-m", "meshtastic_mcp"],
|
"args": [
|
||||||
|
"--from",
|
||||||
|
"git+https://github.com/meshtastic/meshtastic-mcp",
|
||||||
|
"meshtastic-mcp"
|
||||||
|
],
|
||||||
"env": {
|
"env": {
|
||||||
"MESHTASTIC_FIRMWARE_ROOT": "."
|
"MESHTASTIC_FIRMWARE_ROOT": "."
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,10 +12,10 @@
|
|||||||
# defensive loops over flaky sources (pubsub handlers, device
|
# defensive loops over flaky sources (pubsub handlers, device
|
||||||
# re-enumeration polls). One failed iteration shouldn't abort the loop.
|
# re-enumeration polls). One failed iteration shouldn't abort the loop.
|
||||||
# B404 import_subprocess
|
# B404 import_subprocess
|
||||||
# mcp-server wraps PlatformIO, esptool, nrfutil, picotool, and the
|
# the bin/ proto + fixture tooling (regen-py-protos, gen-fake-nodedb-seed,
|
||||||
# pytest test-runner — subprocess is a load-bearing import here, not
|
# seed-json-to-proto) shells out to protoc and helper scripts — subprocess
|
||||||
# a smell. The "consider possible security implications" advisory is
|
# is a load-bearing import here, not a smell. The "consider possible security
|
||||||
# redundant given the file-level review already applied.
|
# implications" advisory is redundant given the file-level review already applied.
|
||||||
# B603 subprocess_without_shell_equals_true
|
# B603 subprocess_without_shell_equals_true
|
||||||
# all subprocess calls use a static argv list; `shell=False` is the
|
# all subprocess calls use a static argv list; `shell=False` is the
|
||||||
# default and we never string-interpolate user input into the command.
|
# default and we never string-interpolate user input into the command.
|
||||||
|
|||||||
@@ -2,12 +2,12 @@
|
|||||||
|
|
||||||
> **TL;DR**
|
> **TL;DR**
|
||||||
>
|
>
|
||||||
> | | |
|
> | | |
|
||||||
> | -------------- | ------------------------------------------------------------------------- |
|
> | -------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||||
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
|
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
|
||||||
> | Hardware tests | `./mcp-server/run-tests.sh` |
|
> | Hardware tests | [meshtastic/meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) (`MESHTASTIC_FIRMWARE_ROOT` → this checkout) |
|
||||||
> | Format | `trunk fmt` |
|
> | Format | `trunk fmt` |
|
||||||
> | Mirror docs | `.github/copilot-instructions.md` (canonical) · `CLAUDE.md` (Claude Code) |
|
> | Mirror docs | `.github/copilot-instructions.md` (canonical) · `CLAUDE.md` (Claude Code) |
|
||||||
>
|
>
|
||||||
> **Need this? It's here.**
|
> **Need this? It's here.**
|
||||||
>
|
>
|
||||||
@@ -18,7 +18,7 @@
|
|||||||
> | New module skeleton | inherit `ProtobufModule<T>` in `src/mesh/ProtobufModule.h` |
|
> | New module skeleton | inherit `ProtobufModule<T>` in `src/mesh/ProtobufModule.h` |
|
||||||
> | Observer / event wiring | `src/Observer.h` |
|
> | Observer / event wiring | `src/Observer.h` |
|
||||||
|
|
||||||
This repository is the [Meshtastic](https://meshtastic.org) firmware - a C++17 embedded codebase targeting ESP32 / nRF52 / RP2040 / STM32WL / Linux-Portduino LoRa mesh radios - plus a Python MCP server in `mcp-server/` that AI agents use to flash, configure, and test connected devices.
|
This repository is the [Meshtastic](https://meshtastic.org) firmware - a C++17 embedded codebase targeting ESP32 / nRF52 / RP2040 / STM32WL / Linux-Portduino LoRa mesh radios. The Python MCP server that AI agents use to flash, configure, and test connected devices now lives in its own repo, [meshtastic/meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp); this repo registers it via `.mcp.json` (run through `uvx`) so its tools are available automatically.
|
||||||
|
|
||||||
## Primary instruction file
|
## Primary instruction file
|
||||||
|
|
||||||
@@ -35,15 +35,15 @@ This file (`AGENTS.md`) is a short pointer + quick reference for agents that don
|
|||||||
| Clean + rebuild | `pio run -e <env> -t clean && pio run -e <env>` |
|
| Clean + rebuild | `pio run -e <env> -t clean && pio run -e <env>` |
|
||||||
| Flash a device | `pio run -e <env> -t upload --upload-port <port>` (or use the `pio_flash` MCP tool) |
|
| Flash a device | `pio run -e <env> -t upload --upload-port <port>` (or use the `pio_flash` MCP tool) |
|
||||||
| Run firmware unit tests (native) | `./bin/run-tests.sh` (preferred - ASan/LSan + RED/AMBER/GREEN verdict); or raw: `~/.platformio/penv/bin/python -m platformio test -e native > /tmp/test_out.txt 2>&1` |
|
| Run firmware unit tests (native) | `./bin/run-tests.sh` (preferred - ASan/LSan + RED/AMBER/GREEN verdict); or raw: `~/.platformio/penv/bin/python -m platformio test -e native > /tmp/test_out.txt 2>&1` |
|
||||||
| Run MCP hardware tests | `./mcp-server/run-tests.sh` |
|
| Run MCP hardware tests | From a [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) checkout: `MESHTASTIC_FIRMWARE_ROOT=/path/to/firmware ./run-tests.sh` |
|
||||||
| Live TUI test runner | `mcp-server/.venv/bin/meshtastic-mcp-test-tui` |
|
| Live TUI test runner | `uvx --from git+https://github.com/meshtastic/meshtastic-mcp meshtastic-mcp-test-tui` |
|
||||||
| Format before commit | `trunk fmt` |
|
| Format before commit | `trunk fmt` |
|
||||||
| Regenerate protobuf bindings | `bin/regen-protos.sh` |
|
| Regenerate protobuf bindings | `bin/regen-protos.sh` |
|
||||||
| Generate CI matrix | `./bin/generate_ci_matrix.py all [--level pr]` |
|
| Generate CI matrix | `./bin/generate_ci_matrix.py all [--level pr]` |
|
||||||
|
|
||||||
## MCP server (device + test automation)
|
## MCP server (device + test automation)
|
||||||
|
|
||||||
The `mcp-server/` package exposes ~32 MCP tools for device discovery, building, flashing, serial monitoring, and live-node administration. Tools are grouped as:
|
The [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) server exposes ~32 MCP tools for device discovery, building, flashing, serial monitoring, and live-node administration. Tools are grouped as:
|
||||||
|
|
||||||
- **Discovery**: `list_devices`, `list_boards`, `get_board`
|
- **Discovery**: `list_devices`, `list_boards`, `get_board`
|
||||||
- **Build & flash**: `build`, `clean`, `pio_flash`, `erase_and_flash` (ESP32 factory), `update_flash` (ESP32 OTA), `touch_1200bps`
|
- **Build & flash**: `build`, `clean`, `pio_flash`, `erase_and_flash` (ESP32 factory), `update_flash` (ESP32 OTA), `touch_1200bps`
|
||||||
@@ -53,19 +53,13 @@ The `mcp-server/` package exposes ~32 MCP tools for device discovery, building,
|
|||||||
- **userPrefs admin**: `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest`, `userprefs_testing_profile`
|
- **userPrefs admin**: `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest`, `userprefs_testing_profile`
|
||||||
- **Vendor escape hatches**: `esptool_*`, `nrfutil_*`, `picotool_*`
|
- **Vendor escape hatches**: `esptool_*`, `nrfutil_*`, `picotool_*`
|
||||||
|
|
||||||
Setup: `cd mcp-server && python3 -m venv .venv && .venv/bin/pip install -e '.[test]'`. The repo registers the server via `.mcp.json` - Claude Code picks it up automatically.
|
Setup: nothing to build - `.mcp.json` runs the server via `uvx --from git+https://github.com/meshtastic/meshtastic-mcp meshtastic-mcp`, so Claude Code picks it up automatically. To run the pytest hardware harness, clone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) and set `MESHTASTIC_FIRMWARE_ROOT` to this firmware checkout.
|
||||||
|
|
||||||
See `mcp-server/README.md` for argument shapes and the **MCP Server & Hardware Test Harness** section of `.github/copilot-instructions.md` for agent usage rules (tool surface, fixture contract, firmware integration points, recovery playbooks).
|
See the meshtastic-mcp repo's README for argument shapes and the **MCP Server & Hardware Test Harness** section of `.github/copilot-instructions.md` for agent usage rules (tool surface, fixture contract, firmware integration points, recovery playbooks).
|
||||||
|
|
||||||
## Slash commands (AI-assisted workflows)
|
## Slash commands (AI-assisted workflows)
|
||||||
|
|
||||||
Three test-and-diagnose workflows exist as slash commands:
|
The test-and-diagnose workflows (`/test`, `/diagnose`, `/repro`, `/leakhunt`) now ship with the [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) repo as skills - run them from a checkout of that repo (with `MESHTASTIC_FIRMWARE_ROOT` pointed here). The MCP tools they build on are still available in this repo via `.mcp.json`.
|
||||||
|
|
||||||
- **`/test` (Claude Code) / `/mcp-test` (Copilot)** - run the hardware test suite and interpret failures
|
|
||||||
- **`/diagnose` / `/mcp-diagnose`** - read-only device health report
|
|
||||||
- **`/repro` / `/mcp-repro`** - flakiness triage: re-run one test N times, diff firmware logs between passes and failures
|
|
||||||
|
|
||||||
Bodies live in `.claude/commands/` and `.github/prompts/` respectively. `.claude/commands/README.md` is the index.
|
|
||||||
|
|
||||||
## Encryption at a glance
|
## Encryption at a glance
|
||||||
|
|
||||||
@@ -108,8 +102,8 @@ Sequence these; don't parallelize on the same port.
|
|||||||
|
|
||||||
1. Build locally: `pio run -e <env>`
|
1. Build locally: `pio run -e <env>`
|
||||||
2. Flash the test device: `pio_flash(env=..., port=..., confirm=True)`
|
2. Flash the test device: `pio_flash(env=..., port=..., confirm=True)`
|
||||||
3. Run the suite: `./mcp-server/run-tests.sh tests/<tier>` or `/test tests/<tier>`
|
3. Run the suite from a meshtastic-mcp checkout: `MESHTASTIC_FIRMWARE_ROOT=/path/to/firmware ./run-tests.sh tests/<tier>` (or the `/test` skill)
|
||||||
4. On failure, open `mcp-server/tests/report.html` → `Meshtastic debug` section for the firmware log tail + device state dump
|
4. On failure, open the run's `tests/report.html` → `Meshtastic debug` section for the firmware log tail + device state dump
|
||||||
5. Iterate
|
5. Iterate
|
||||||
|
|
||||||
### Debugging a flaky test
|
### Debugging a flaky test
|
||||||
@@ -120,25 +114,23 @@ Sequence these; don't parallelize on the same port.
|
|||||||
|
|
||||||
## Where to look
|
## Where to look
|
||||||
|
|
||||||
| Path | What's there |
|
| Path | What's there |
|
||||||
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
| -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `src/` | Firmware C++ source (`mesh/`, `modules/`, `platform/`, `graphics/`, `gps/`, `motion/`, `mqtt/`, …) |
|
| `src/` | Firmware C++ source (`mesh/`, `modules/`, `platform/`, `graphics/`, `gps/`, `motion/`, `mqtt/`, …) |
|
||||||
| `src/mesh/` | Core: NodeDB, Router, Channels, CryptoEngine, radio interfaces, StreamAPI, PhoneAPI |
|
| `src/mesh/` | Core: NodeDB, Router, Channels, CryptoEngine, radio interfaces, StreamAPI, PhoneAPI |
|
||||||
| `src/modules/` | Feature modules; `Telemetry/Sensor/` has 50+ I2C sensor drivers |
|
| `src/modules/` | Feature modules; `Telemetry/Sensor/` has 50+ I2C sensor drivers |
|
||||||
| `variants/` | 200+ hardware variant definitions (`variant.h` + `platformio.ini` per board) |
|
| `variants/` | 200+ hardware variant definitions (`variant.h` + `platformio.ini` per board) |
|
||||||
| `protobufs/` | `.proto` definitions; regenerate with `bin/regen-protos.sh` |
|
| `protobufs/` | `.proto` definitions; regenerate with `bin/regen-protos.sh` |
|
||||||
| `test/` | Firmware unit tests (19 suites; `./bin/run-tests.sh` preferred, falls back to `pio test -e native`) |
|
| `test/` | Firmware unit tests (19 suites; `./bin/run-tests.sh` preferred, falls back to `pio test -e native`) |
|
||||||
| `mcp-server/` | Python MCP server + pytest hardware integration tests |
|
| [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) | Standalone MCP server + tiered pytest hardware harness (`unit/`, `mesh/`, `telemetry/`, `monitor/`, `recovery/`, `ui/`, `fleet/`, `admin/`, `provisioning/`) - registered here via `.mcp.json` |
|
||||||
| `mcp-server/tests/` | Tiered pytest suite: `unit/`, `mesh/`, `telemetry/`, `monitor/`, `recovery/`, `ui/`, `fleet/`, `admin/`, `provisioning/` |
|
| `.github/prompts/` | Copilot prompt bodies (firmware scaffolding: new module / sensor / variant) |
|
||||||
| `.claude/commands/` | Claude Code slash command bodies |
|
| `.github/copilot-instructions.md` | **Primary agent instructions - read this** |
|
||||||
| `.github/prompts/` | Copilot prompt bodies (mirrors of the Claude Code ones) |
|
| `.github/workflows/` | CI pipelines |
|
||||||
| `.github/copilot-instructions.md` | **Primary agent instructions - read this** |
|
| `.mcp.json` | MCP server registration for Claude Code |
|
||||||
| `.github/workflows/` | CI pipelines |
|
|
||||||
| `.mcp.json` | MCP server registration for Claude Code |
|
|
||||||
|
|
||||||
## Recovery one-liners
|
## Recovery one-liners
|
||||||
|
|
||||||
- **`userPrefs.jsonc` dirty after a test run?** Re-run `./mcp-server/run-tests.sh` once (pre-flight self-heals from the sidecar). If still dirty: `git checkout userPrefs.jsonc`.
|
- **`userPrefs.jsonc` dirty after a test run?** Re-run the meshtastic-mcp harness once (pre-flight self-heals from the sidecar). If still dirty: `git checkout userPrefs.jsonc`.
|
||||||
- **nRF52 not responding?** `mcp__meshtastic__touch_1200bps(port=...)` drops it into the DFU bootloader, then `pio_flash` re-installs.
|
- **nRF52 not responding?** `mcp__meshtastic__touch_1200bps(port=...)` drops it into the DFU bootloader, then `pio_flash` re-installs.
|
||||||
- **Device fully wedged (no DFU)?** `mcp__meshtastic__uhubctl_cycle(role="nrf52", confirm=True)` hard-power-cycles it via USB hub PPPS. Needs `uhubctl` installed (`brew install uhubctl` / `apt install uhubctl`); on Linux without udev rules, permission errors fail fast, so use `sudo uhubctl` yourself or configure udev access.
|
- **Device fully wedged (no DFU)?** `mcp__meshtastic__uhubctl_cycle(role="nrf52", confirm=True)` hard-power-cycles it via USB hub PPPS. Needs `uhubctl` installed (`brew install uhubctl` / `apt install uhubctl`); on Linux without udev rules, permission errors fail fast, so use `sudo uhubctl` yourself or configure udev access.
|
||||||
- **Port busy?** `lsof <port>` to find the holder. Usually a stale `pio device monitor` or zombie `meshtastic_mcp` process. Kill it.
|
- **Port busy?** `lsof <port>` to find the holder. Usually a stale `pio device monitor` or zombie `meshtastic_mcp` process. Kill it.
|
||||||
|
|||||||
@@ -2,12 +2,12 @@
|
|||||||
|
|
||||||
> **TL;DR**
|
> **TL;DR**
|
||||||
>
|
>
|
||||||
> | | |
|
> | | |
|
||||||
> | -------------- | ------------------------------------------------------------------ |
|
> | -------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||||
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
|
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
|
||||||
> | Hardware tests | `./mcp-server/run-tests.sh` |
|
> | Hardware tests | [meshtastic/meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) (`MESHTASTIC_FIRMWARE_ROOT` → this checkout) |
|
||||||
> | Format | `trunk fmt` |
|
> | Format | `trunk fmt` |
|
||||||
> | Mirror docs | `.github/copilot-instructions.md` (canonical) · `AGENTS.md` |
|
> | Mirror docs | `.github/copilot-instructions.md` (canonical) · `AGENTS.md` |
|
||||||
>
|
>
|
||||||
> **Need this? It's here.**
|
> **Need this? It's here.**
|
||||||
>
|
>
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ attribute access) to use the new namespace (e.g., `meshtastic_v25`).
|
|||||||
|
|
||||||
Why: the .proto files declare `package meshtastic;`, so protoc emits
|
Why: the .proto files declare `package meshtastic;`, so protoc emits
|
||||||
`from meshtastic import mesh_pb2 as ...` lines. That would shadow the PyPI
|
`from meshtastic import mesh_pb2 as ...` lines. That would shadow the PyPI
|
||||||
`meshtastic` package which other parts of the mcp-server depend on. Renaming
|
`meshtastic` package which the meshtastic-mcp tooling depends on. Renaming
|
||||||
to a local namespace keeps both available.
|
to a local namespace keeps both available.
|
||||||
|
|
||||||
Usage:
|
Usage:
|
||||||
|
|||||||
@@ -22,9 +22,10 @@ if [[ ! -d bin/_generated/meshtastic ]]; then
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
# 2) Pick a Python interpreter that has the meshtastic deps installed.
|
# 2) Pick a Python interpreter that has the meshtastic deps installed.
|
||||||
# Prefer the mcp-server venv (most likely to be set up by the operator).
|
# Prefer a local venv; otherwise fall back to system python3 (install the
|
||||||
|
# `meshtastic` package there, or run this under `uv run --with meshtastic`).
|
||||||
PY="python3"
|
PY="python3"
|
||||||
for cand in mcp-server/.venv/bin/python3 .venv/bin/python3; do
|
for cand in .venv/bin/python3; do
|
||||||
if [[ -x "$cand" ]]; then
|
if [[ -x "$cand" ]]; then
|
||||||
PY="$cand"
|
PY="$cand"
|
||||||
break
|
break
|
||||||
@@ -70,4 +71,5 @@ echo "Done. To load on Portduino native:"
|
|||||||
echo " cp build/fixtures/nodedb/nodes_v25_1000.proto ~/.portduino/default/prefs/nodes.proto"
|
echo " cp build/fixtures/nodedb/nodes_v25_1000.proto ~/.portduino/default/prefs/nodes.proto"
|
||||||
echo ""
|
echo ""
|
||||||
echo "To push to a hardware device:"
|
echo "To push to a hardware device:"
|
||||||
echo " Use the mcp-server tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"
|
echo " Use the meshtastic-mcp tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"
|
||||||
|
echo " (MCP server: https://github.com/meshtastic/meshtastic-mcp)"
|
||||||
|
|||||||
@@ -8,7 +8,7 @@
|
|||||||
# Namespace rewrite:
|
# Namespace rewrite:
|
||||||
# The .proto files declare `package meshtastic;`, which makes protoc emit
|
# The .proto files declare `package meshtastic;`, which makes protoc emit
|
||||||
# imports like `from meshtastic import mesh_pb2`. That conflicts with the
|
# imports like `from meshtastic import mesh_pb2`. That conflicts with the
|
||||||
# PyPI `meshtastic` package (which the mcp-server relies on for its
|
# PyPI `meshtastic` package (which the meshtastic-mcp tooling relies on for its
|
||||||
# SerialInterface/BLEInterface transport). We post-process the generated
|
# SerialInterface/BLEInterface transport). We post-process the generated
|
||||||
# files to live under `meshtastic_v25` instead — both the directory layout
|
# files to live under `meshtastic_v25` instead — both the directory layout
|
||||||
# and all internal imports — so they coexist cleanly with the PyPI package.
|
# and all internal imports — so they coexist cleanly with the PyPI package.
|
||||||
|
|||||||
@@ -374,15 +374,15 @@ via the `LOG_INFO "Route to … stale"` / "Resetting next hop" lines). Use this
|
|||||||
|
|
||||||
### 3. Hardware via meshtastic MCP (auto-detect; 3+ devices for a real hop)
|
### 3. Hardware via meshtastic MCP (auto-detect; 3+ devices for a real hop)
|
||||||
|
|
||||||
- `mcp-server/tests/mesh/test_nexthop_multihop_recovery.py` - **the multi-hop validator
|
- `meshtastic-mcp/tests/mesh/test_nexthop_multihop_recovery.py` - **the multi-hop validator
|
||||||
for this work** (added on this branch). Self-discovers an A - relay - C line, asserts a
|
for this work** (added on this branch). Self-discovers an A - relay - C line, asserts a
|
||||||
directed DM is delivered across the relay (next_hop + M1/M2/M3 engaged), and asserts
|
directed DM is delivered across the relay (next_hop + M1/M2/M3 engaged), and asserts
|
||||||
delivery recovers after the relay is power-cycled (M3). Skips unless the bench is a true
|
delivery recovers after the relay is power-cycled (M3). Skips unless the bench is a true
|
||||||
multi-hop line (≥3 roles via `--hub-profile`, endpoints out of direct RF range).
|
multi-hop line (≥3 roles via `--hub-profile`, endpoints out of direct RF range).
|
||||||
- `mcp-server/tests/mesh/test_direct_with_ack.py` - happy-path regression: a fresh/unique
|
- `meshtastic-mcp/tests/mesh/test_direct_with_ack.py` - happy-path regression: a fresh/unique
|
||||||
route still delivers a want_ack DM on the first/second try (M4's gate must keep this
|
route still delivers a want_ack DM on the first/second try (M4's gate must keep this
|
||||||
green).
|
green).
|
||||||
- `mcp-server/tests/mesh/test_peer_offline_recovery.py` - 2-device recovery validator: peer
|
- `meshtastic-mcp/tests/mesh/test_peer_offline_recovery.py` - 2-device recovery validator: peer
|
||||||
off mid-conversation then back. Must stay green and ideally recover in fewer attempts.
|
off mid-conversation then back. Must stay green and ideally recover in fewer attempts.
|
||||||
|
|
||||||
### 4. Build / format sanity
|
### 4. Build / format sanity
|
||||||
@@ -411,7 +411,7 @@ production; sanity-check RAM headroom on the smallest nRF52 build for the ~384 B
|
|||||||
without it. The intermediate-node failure-count path and the M4 A/B therefore have unit
|
without it. The intermediate-node failure-count path and the M4 A/B therefore have unit
|
||||||
coverage of their logic but no end-to-end multi-node run yet.
|
coverage of their logic but no end-to-end multi-node run yet.
|
||||||
- **Hardware / multi-hop tier** - a committable bench test now exists:
|
- **Hardware / multi-hop tier** - a committable bench test now exists:
|
||||||
`mcp-server/tests/mesh/test_nexthop_multihop_recovery.py`. It self-discovers a real
|
`meshtastic-mcp/tests/mesh/test_nexthop_multihop_recovery.py`. It self-discovers a real
|
||||||
multi-hop pair (A - relay - C), asserts a directed DM is delivered across the relay, and
|
multi-hop pair (A - relay - C), asserts a directed DM is delivered across the relay, and
|
||||||
asserts delivery recovers after the relay is power-cycled (the M3 path). It
|
asserts delivery recovers after the relay is power-cycled (the M3 path). It
|
||||||
`pytest.skip`s cleanly unless the bench is a true line with endpoints out of direct RF
|
`pytest.skip`s cleanly unless the bench is a true line with endpoints out of direct RF
|
||||||
|
|||||||
@@ -1,35 +0,0 @@
|
|||||||
.venv/
|
|
||||||
__pycache__/
|
|
||||||
*.py[cod]
|
|
||||||
*.egg-info/
|
|
||||||
.pytest_cache/
|
|
||||||
.mypy_cache/
|
|
||||||
dist/
|
|
||||||
build/
|
|
||||||
|
|
||||||
# Persistent device-log capture (recorder + Datadog cursor).
|
|
||||||
# Cross-session JSONL streams written by the autouse Recorder singleton
|
|
||||||
# (see src/meshtastic_mcp/recorder/). Lives outside tests/ so the pytest
|
|
||||||
# fixture truncate doesn't touch it.
|
|
||||||
.mtlog/
|
|
||||||
|
|
||||||
# Test harness artifacts
|
|
||||||
tests/report.html
|
|
||||||
tests/junit.xml
|
|
||||||
tests/reportlog.jsonl
|
|
||||||
tests/fwlog.jsonl
|
|
||||||
# Subprocess-output tee from pio/esptool/nrfutil/picotool (live flash
|
|
||||||
# progress for the TUI; also a post-run diagnostic for plain CLI runs).
|
|
||||||
tests/flash.log
|
|
||||||
tests/tool_coverage.json
|
|
||||||
tests/.coverage
|
|
||||||
htmlcov/
|
|
||||||
# Persistent run counter for meshtastic-mcp-test-tui header.
|
|
||||||
tests/.tui-runs
|
|
||||||
# Cross-run history (TUI duration sparkline).
|
|
||||||
tests/.history/
|
|
||||||
# Reproducer bundles (TUI `x` export on failed tests).
|
|
||||||
tests/reproducers/
|
|
||||||
# UI-tier camera captures + per-test transcripts. Regenerated every run;
|
|
||||||
# left on disk for human review between runs.
|
|
||||||
tests/ui_captures/
|
|
||||||
@@ -1,413 +0,0 @@
|
|||||||
# Meshtastic MCP Server
|
|
||||||
|
|
||||||
An [MCP](https://modelcontextprotocol.io) server for working with the Meshtastic firmware repo and connected devices. Lets Claude Code / Claude Desktop:
|
|
||||||
|
|
||||||
- Discover USB-connected Meshtastic devices
|
|
||||||
- Enumerate PlatformIO board variants (166+) with Meshtastic metadata
|
|
||||||
- Build, clean, flash, erase-and-flash (factory), and OTA-update firmware
|
|
||||||
- Read serial logs via `pio device monitor` (with board-specific exception decoders)
|
|
||||||
- Trigger 1200bps touch-reset for bootloader entry (nRF52, ESP32-S3, RP2040)
|
|
||||||
- Query and administer a running node via the [`meshtastic` Python API](https://github.com/meshtastic/python): owner name, config (LocalConfig + ModuleConfig), channels, messaging, reboot/shutdown/factory-reset
|
|
||||||
- Call `esptool`, `nrfutil`, `picotool` directly when PlatformIO doesn't cover the operation
|
|
||||||
|
|
||||||
## Design principle
|
|
||||||
|
|
||||||
**PlatformIO first.** Its `pio run -t upload` knows the correct protocol, offsets, and post-build chain for every variant in `variants/`. Direct vendor-tool wrappers (`esptool_*`, `nrfutil_*`, `picotool_*`) exist as escape hatches for operations pio doesn't cover (blank-chip erase, DFU `.zip` packages, BOOTSEL-mode inspection).
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- Python ≥ 3.11
|
|
||||||
- [PlatformIO Core](https://platformio.org/install/cli) - `pio` on `$PATH` or at `~/.platformio/penv/bin/pio`
|
|
||||||
- The Meshtastic firmware repo checked out somewhere (set via `MESHTASTIC_FIRMWARE_ROOT`)
|
|
||||||
- Optional: `esptool`, `nrfutil`, `picotool` on `$PATH` (or under the firmware venv at `.venv/bin/`) if you want to use the direct-tool wrappers
|
|
||||||
|
|
||||||
## Install
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd <firmware-repo>/mcp-server
|
|
||||||
python3 -m venv .venv
|
|
||||||
.venv/bin/pip install -e .
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
MESHTASTIC_FIRMWARE_ROOT=<firmware-repo> .venv/bin/python -m meshtastic_mcp
|
|
||||||
```
|
|
||||||
|
|
||||||
The server blocks on stdin (that's correct - it speaks MCP over stdio). Ctrl-C to exit.
|
|
||||||
|
|
||||||
## Register with Claude Code
|
|
||||||
|
|
||||||
Edit `~/.claude/settings.json` (global) or `<firmware-repo>/.claude/settings.local.json` (project-only):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"mcpServers": {
|
|
||||||
"meshtastic": {
|
|
||||||
"command": "<firmware-repo>/mcp-server/.venv/bin/python",
|
|
||||||
"args": ["-m", "meshtastic_mcp"],
|
|
||||||
"env": {
|
|
||||||
"MESHTASTIC_FIRMWARE_ROOT": "<firmware-repo>"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Replace `<firmware-repo>` with the absolute path, e.g. `/Users/you/GitHub/firmware`. Restart Claude Code after editing.
|
|
||||||
|
|
||||||
## Register with Claude Desktop
|
|
||||||
|
|
||||||
Same `mcpServers` block, but in `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows).
|
|
||||||
|
|
||||||
## Tools (43)
|
|
||||||
|
|
||||||
### Discovery & metadata
|
|
||||||
|
|
||||||
| Tool | What it does |
|
|
||||||
| -------------- | ------------------------------------------------------------------------------------------ |
|
|
||||||
| `list_devices` | USB/serial port listing, flags likely-Meshtastic candidates |
|
|
||||||
| `list_boards` | PlatformIO envs with `custom_meshtastic_*` metadata; filters by arch/supported/query/level |
|
|
||||||
| `get_board` | Full env dict incl. raw pio config |
|
|
||||||
|
|
||||||
### Build & flash
|
|
||||||
|
|
||||||
| Tool | What it does |
|
|
||||||
| ----------------- | -------------------------------------------------------------------- |
|
|
||||||
| `build` | `pio run -e <env>` (+ mtjson target) |
|
|
||||||
| `clean` | `pio run -e <env> -t clean` |
|
|
||||||
| `pio_flash` | `pio run -e <env> -t upload --upload-port <port>` - any architecture |
|
|
||||||
| `erase_and_flash` | ESP32 full factory flash via `bin/device-install.sh` |
|
|
||||||
| `update_flash` | ESP32 OTA app-partition update via `bin/device-update.sh` |
|
|
||||||
| `touch_1200bps` | 1200-baud open/close to trigger USB CDC bootloader entry |
|
|
||||||
|
|
||||||
### Serial log sessions
|
|
||||||
|
|
||||||
Backed by long-running `pio device monitor` subprocesses with a 10k-line ring buffer per session and board-specific filters (`esp32_exception_decoder` auto-selected when you pass `env=`).
|
|
||||||
|
|
||||||
| Tool | What it does |
|
|
||||||
| -------------- | ------------------------------------------------------------------ |
|
|
||||||
| `serial_open` | Start a monitor session; returns `session_id` |
|
|
||||||
| `serial_read` | Cursor-based pull; reports `dropped` if lines aged out of the ring |
|
|
||||||
| `serial_list` | All active sessions |
|
|
||||||
| `serial_close` | Terminate a session |
|
|
||||||
|
|
||||||
### Device reads
|
|
||||||
|
|
||||||
| Tool | What it does |
|
|
||||||
| ------------- | --------------------------------------------------------------------------- |
|
|
||||||
| `device_info` | my_node_num, long/short name, firmware version, region, channel, node count |
|
|
||||||
| `list_nodes` | Full node database with position, SNR, RSSI, last_heard, battery |
|
|
||||||
|
|
||||||
_The tool tables below document 38 currently registered MCP server tools._
|
|
||||||
|
|
||||||
### Device writes
|
|
||||||
|
|
||||||
| Tool | What it does |
|
|
||||||
| ------------------- | -------------------------------------------------------------------------- |
|
|
||||||
| `set_owner` | Long name + optional short name (≤4 chars) |
|
|
||||||
| `get_config` | One section or all (LocalConfig + ModuleConfig) |
|
|
||||||
| `set_config` | Dot-path field write: `lora.region`=`"US"`, `device.role`=`"ROUTER"`, etc. |
|
|
||||||
| `get_channel_url` | Primary-only or include_all=admin URL |
|
|
||||||
| `set_channel_url` | Import channels from a Meshtastic URL |
|
|
||||||
| `set_debug_log_api` | Enable or disable debug logging for the Meshtastic Python API client |
|
|
||||||
| `send_text` | Broadcast or direct text message |
|
|
||||||
| `reboot` | `localNode.reboot(secs)` - requires `confirm=True` |
|
|
||||||
| `shutdown` | `localNode.shutdown(secs)` - requires `confirm=True` |
|
|
||||||
| `factory_reset` | `localNode.factoryReset(full?)` - requires `confirm=True` |
|
|
||||||
|
|
||||||
### Direct hardware tools (escape hatches)
|
|
||||||
|
|
||||||
| Tool | What it does |
|
|
||||||
| --------------------- | --------------------------------------------------------- |
|
|
||||||
| `esptool_chip_info` | Read chip, MAC, crystal, flash size |
|
|
||||||
| `esptool_erase_flash` | Full-chip erase (destructive) |
|
|
||||||
| `esptool_raw` | Pass-through; confirm=True required for write/erase/merge |
|
|
||||||
| `nrfutil_dfu` | DFU-flash a `.zip` package |
|
|
||||||
| `nrfutil_raw` | Pass-through |
|
|
||||||
| `picotool_info` | Read Pico BOOTSEL-mode info |
|
|
||||||
| `picotool_load` | Load a UF2 |
|
|
||||||
| `picotool_raw` | Pass-through |
|
|
||||||
|
|
||||||
### USB power control (uhubctl)
|
|
||||||
|
|
||||||
| Tool | What it does |
|
|
||||||
| --------------- | ----------------------------------------------------------- |
|
|
||||||
| `uhubctl_list` | Enumerate USB hubs + attached-device VID/PID (read-only) |
|
|
||||||
| `uhubctl_power` | Drive a hub port `on` or `off`; `off` requires confirm=True |
|
|
||||||
| `uhubctl_cycle` | Off → wait `delay_s` → on; confirm=True required |
|
|
||||||
|
|
||||||
Target a port by explicit `(location, port)` (raw uhubctl syntax like
|
|
||||||
`location="1-1.3", port=2`) or by `role` (`"nrf52"`, `"esp32s3"`). Role
|
|
||||||
lookup checks `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` +
|
|
||||||
`MESHTASTIC_UHUBCTL_PORT_<ROLE>` env vars first, then auto-detects via VID
|
|
||||||
against `uhubctl`'s output.
|
|
||||||
|
|
||||||
Requires [`uhubctl`](https://github.com/mvp/uhubctl) on PATH:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
brew install uhubctl # macOS
|
|
||||||
apt install uhubctl # Debian/Ubuntu
|
|
||||||
```
|
|
||||||
|
|
||||||
Modern macOS + PPPS-capable hubs generally work without root. On Linux
|
|
||||||
without udev rules, or on old macOS with driver quirks, you may need
|
|
||||||
`sudo`. If uhubctl returns a permission error the MCP tool raises a
|
|
||||||
clear `UhubctlError` pointing at the
|
|
||||||
[udev-rules / sudo fallback](https://github.com/mvp/uhubctl#linux-usb-permissions)
|
|
||||||
rather than auto-`sudo`'ing mid-run.
|
|
||||||
|
|
||||||
## Safety
|
|
||||||
|
|
||||||
- **All destructive flash/admin tools require `confirm=True`** as a tool-level gate, on top of any permission prompt from Claude.
|
|
||||||
- **Serial port is exclusive.** If a `serial_*` session is active on a port, `device_info`/admin tools on the same port will fail fast with a pointer at the active `session_id`. Close the session first.
|
|
||||||
- **Flash confirmation by architecture**: `erase_and_flash` / `update_flash` error if the env's architecture isn't ESP32 - use `pio_flash` for nRF52/RP2040/STM32.
|
|
||||||
|
|
||||||
## Environment variables
|
|
||||||
|
|
||||||
| Var | Default | Purpose |
|
|
||||||
| -------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `MESHTASTIC_FIRMWARE_ROOT` | walks up from cwd for `platformio.ini` | Pin the firmware repo |
|
|
||||||
| `MESHTASTIC_PIO_BIN` | `~/.platformio/penv/bin/pio` → `$PATH` `pio` → `platformio` | Override `pio` location |
|
|
||||||
| `MESHTASTIC_ESPTOOL_BIN` | `<firmware>/.venv/bin/esptool` → `$PATH` | Override esptool |
|
|
||||||
| `MESHTASTIC_NRFUTIL_BIN` | `$PATH` | Override nrfutil |
|
|
||||||
| `MESHTASTIC_PICOTOOL_BIN` | `$PATH` | Override picotool |
|
|
||||||
| `MESHTASTIC_MCP_SEED` | `mcp-<user>-<host>` | PSK seed for test-harness session (CI override) |
|
|
||||||
| `MESHTASTIC_MCP_FLASH_LOG` | `<mcp-server>/tests/flash.log` | Tee target for pio/esptool/nrfutil subprocess output (TUI tails it) |
|
|
||||||
| `MESHTASTIC_MCP_TCP_HOST` | unset | `host` or `host:port` of a `meshtasticd` daemon to surface as a TCP device (see "TCP / native-host nodes" below) |
|
|
||||||
|
|
||||||
## TCP / native-host nodes
|
|
||||||
|
|
||||||
The `native-macos` and `native` PlatformIO envs build a headless `meshtasticd`
|
|
||||||
binary that runs on the host (Apple Silicon / Intel macOS, or Linux Portduino).
|
|
||||||
The daemon exposes the meshtastic TCP API on port `4403` rather than a USB
|
|
||||||
serial endpoint - point the MCP server at it via `MESHTASTIC_MCP_TCP_HOST`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Build + run a daemon on this host (see variants/native/portduino/platformio.ini
|
|
||||||
# for full Homebrew prereqs and CH341 LoRa-adapter setup).
|
|
||||||
pio run -e native-macos
|
|
||||||
~/.meshtasticd/meshtasticd
|
|
||||||
|
|
||||||
# 2. Point the MCP server at it.
|
|
||||||
export MESHTASTIC_MCP_TCP_HOST=localhost # or host:port, default port 4403
|
|
||||||
```
|
|
||||||
|
|
||||||
**First-run gotcha - MAC address.** `meshtasticd` derives its MAC from the
|
|
||||||
USB adapter's serial-number / product strings. Many cheap CH341 dongles
|
|
||||||
(MeshStick included - VID 0x1A86 / PID 0x5512) ship with `iSerialNumber=0`
|
|
||||||
and `iProduct=0`, so the daemon aborts on boot with `*** Blank MAC Address
|
|
||||||
not allowed!`. Set the MAC explicitly in `config.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# Under General:
|
|
||||||
MACAddress: 02:CA:FE:BA:BE:01
|
|
||||||
```
|
|
||||||
|
|
||||||
Use a locally-administered address (first byte's second-LSB set, e.g.
|
|
||||||
`02:*` / `06:*` / `0A:*` / `0E:*`) to avoid colliding with a real OUI.
|
|
||||||
|
|
||||||
There is also a `--hwid AA:BB:CC:DD:EE:FF` CLI flag visible in
|
|
||||||
`meshtasticd --help`, but it is **currently broken** in
|
|
||||||
`MAC_from_string()` (`src/platform/portduino/PortduinoGlue.cpp`): the
|
|
||||||
function strips colons from its parameter but then reads bytes from the
|
|
||||||
global `portduino_config.mac_address`, so `--hwid` is silently overridden
|
|
||||||
when `MACAddress:` is also set, and crashes the daemon (uncaught
|
|
||||||
`std::invalid_argument: stoi: no conversion`) when it isn't. Use the YAML
|
|
||||||
form until that's fixed upstream.
|
|
||||||
|
|
||||||
`list_devices` will surface the daemon as `tcp://localhost:4403` with
|
|
||||||
`likely_meshtastic=True`, so `device_info`, `list_nodes`, `get_config`,
|
|
||||||
`set_config`, `set_owner`, `send_text`, `userprefs_*`, and the admin RPCs
|
|
||||||
auto-select it when no `port` is passed. Pass `port="tcp://other-host:9999"`
|
|
||||||
explicitly to target a different daemon.
|
|
||||||
|
|
||||||
**Tools that don't apply to a TCP/native node** (no USB hardware to operate
|
|
||||||
on) raise a clear `ConnectionError` rather than failing mysteriously:
|
|
||||||
`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`,
|
|
||||||
`serial_open` (use info/admin tools directly), and the vendor escape hatches
|
|
||||||
`esptool_*`, `nrfutil_*`, `picotool_*`. `pio_flash` against a `native*` env
|
|
||||||
similarly raises - there's no upload step; use `build` and run the binary
|
|
||||||
directly.
|
|
||||||
|
|
||||||
The pytest harness in `tests/` still assumes USB-attached devices per role -
|
|
||||||
TCP-aware fixtures are not part of this surface yet.
|
|
||||||
|
|
||||||
## Hardware Test Suite
|
|
||||||
|
|
||||||
`mcp-server/tests/` holds a pytest-based integration suite that exercises
|
|
||||||
real USB-connected Meshtastic devices against the MCP server surface. Separate
|
|
||||||
from the native C++ unit tests in the firmware repo's top-level `test/`
|
|
||||||
directory - this one validates the device-facing behavior end-to-end.
|
|
||||||
|
|
||||||
### Invocation
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./mcp-server/run-tests.sh # full suite (auto-detect + auto-bake-if-needed)
|
|
||||||
./mcp-server/run-tests.sh --force-bake # reflash devices before testing
|
|
||||||
./mcp-server/run-tests.sh --assume-baked # skip the bake step (caller vouches for state)
|
|
||||||
./mcp-server/run-tests.sh tests/mesh # one tier
|
|
||||||
./mcp-server/run-tests.sh tests/mesh/test_traceroute.py # one file
|
|
||||||
./mcp-server/run-tests.sh -k telemetry # pytest name filter
|
|
||||||
```
|
|
||||||
|
|
||||||
The wrapper auto-detects connected devices (VID `0x239A` → `nrf52` → env
|
|
||||||
`rak4631`; `0x303A` or `0x10C4` → `esp32s3` → env `heltec-v3`), exports
|
|
||||||
`MESHTASTIC_MCP_ENV_<ROLE>` env vars, and invokes pytest. Overrides via
|
|
||||||
per-role env vars: `MESHTASTIC_MCP_ENV_NRF52=heltec-mesh-node-t114 ./run-tests.sh`.
|
|
||||||
|
|
||||||
No hardware connected? The wrapper narrows to `tests/unit/` only and says so
|
|
||||||
in the pre-flight header.
|
|
||||||
|
|
||||||
### Tiers (run in this order)
|
|
||||||
|
|
||||||
- **`bake`** (`tests/test_00_bake.py`) - flashes both hub roles with the
|
|
||||||
session's test profile. Has a skip-if-already-baked check (region + channel
|
|
||||||
match); `--force-bake` overrides.
|
|
||||||
- **`unit`** - pure Python, no hardware. boards / PIO wrapper /
|
|
||||||
userPrefs-parse / testing-profile fixtures.
|
|
||||||
- **`mesh`** - 2-device mesh: formation, broadcast delivery, direct+ACK,
|
|
||||||
traceroute, bidirectional. Parametrized over both directions. Includes
|
|
||||||
`test_peer_offline_recovery` which uses uhubctl to power-cycle one peer
|
|
||||||
mid-conversation and verifies the mesh recovers (skips without uhubctl).
|
|
||||||
- **`telemetry`** - periodic telemetry broadcast + on-demand request/reply
|
|
||||||
(`TELEMETRY_APP` with `wantResponse=True`).
|
|
||||||
- **`monitor`** - boot log has no panic markers within 60 s of reboot.
|
|
||||||
- **`recovery`** - `uhubctl` power-cycle round-trip: verifies the hub port
|
|
||||||
can be toggled off/on, the device re-enumerates with the same
|
|
||||||
`my_node_num`, and NVS-resident config (region, channel, modem preset)
|
|
||||||
survives a hard reset. Requires `uhubctl` on PATH; skips cleanly otherwise.
|
|
||||||
- **`ui`** - input-broker-driven screen navigation (`AdminMessage.send_input_event`
|
|
||||||
injection → `Screen::handleInputEvent` → frame transition). Parametrized
|
|
||||||
on the screen-bearing role (heltec-v3 OLED). Captures images via USB
|
|
||||||
webcam + OCRs them for HTML-report evidence. Requires `pip install -e '.[ui]'`
|
|
||||||
and `MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=<index>`; tier is auto-deselected
|
|
||||||
if `cv2` isn't importable.
|
|
||||||
- **`fleet`** - PSK-seed isolation: two labs with different seeds never
|
|
||||||
overlap.
|
|
||||||
- **`admin`** - owner persistence across reboot, channel URL round-trip,
|
|
||||||
`lora.hop_limit` persistence.
|
|
||||||
- **`provisioning`** - region/channel baking, userPrefs survive
|
|
||||||
`factory_reset(full=False)`.
|
|
||||||
|
|
||||||
#### UI tier setup
|
|
||||||
|
|
||||||
The `tests/ui/` tier drives the on-device OLED via the firmware's existing
|
|
||||||
`AdminMessage.send_input_event` RPC (no firmware changes required) and
|
|
||||||
verifies transitions via a macro-gated log line + camera + OCR. Summary:
|
|
||||||
|
|
||||||
1. Install extras: `pip install -e 'mcp-server/.[ui]'` - pulls in
|
|
||||||
`opencv-python-headless`, `numpy`, `easyocr`, `Pillow`. First easyocr
|
|
||||||
run downloads ~100 MB of models to `~/.EasyOCR/`; an autouse session
|
|
||||||
fixture pre-warms the reader so per-test OCR is <100 ms after that.
|
|
||||||
2. Point a USB webcam at the heltec-v3 OLED. Discover its index:
|
|
||||||
```bash
|
|
||||||
.venv/bin/python -c "import cv2; [print(i, cv2.VideoCapture(i).read()[0]) for i in range(5)]"
|
|
||||||
```
|
|
||||||
3. Export the per-role device env var:
|
|
||||||
```bash
|
|
||||||
export MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0
|
|
||||||
```
|
|
||||||
4. Run:
|
|
||||||
```bash
|
|
||||||
./run-tests.sh tests/ui -v
|
|
||||||
```
|
|
||||||
Captures land under `tests/ui_captures/<session_seed>/<test_id>/`, one
|
|
||||||
PNG + `.ocr.txt` per `frame_capture()` call, with a per-test
|
|
||||||
`transcript.md` stepping through event → frame → OCR. The HTML report
|
|
||||||
embeds the full image strip inline (pass or fail).
|
|
||||||
|
|
||||||
On macOS, `cv2.VideoCapture(0)` triggers the TCC Camera permission prompt
|
|
||||||
on first use. Pre-grant Terminal (or your IDE's terminal) before running.
|
|
||||||
The `OpenCVBackend` fails fast on 10 consecutive black frames so a silent
|
|
||||||
permission denial surfaces as a clear error, not an empty PNG strip.
|
|
||||||
|
|
||||||
No camera? Set `MESHTASTIC_UI_CAMERA_BACKEND=null` (or leave the device var
|
|
||||||
unset). Tests still exercise the event-injection path and log assertions;
|
|
||||||
captures just become 1×1 black PNGs.
|
|
||||||
|
|
||||||
### Artifacts (regenerated every run, under `tests/`)
|
|
||||||
|
|
||||||
- `report.html` - self-contained pytest-html report. Each test gets a
|
|
||||||
**Meshtastic debug** section attached on failure with a 200-line firmware
|
|
||||||
log tail + device-state dump. Open this first on failures.
|
|
||||||
- `junit.xml` - CI-parseable.
|
|
||||||
- `reportlog.jsonl` - `pytest-reportlog` event stream; consumed by the TUI.
|
|
||||||
- `fwlog.jsonl` - firmware log mirror (`meshtastic.log.line` pubsub → JSONL).
|
|
||||||
- `flash.log` - tee of all pio / esptool / nrfutil / picotool subprocess
|
|
||||||
output during the run (driven by `MESHTASTIC_MCP_FLASH_LOG`).
|
|
||||||
|
|
||||||
### Live TUI
|
|
||||||
|
|
||||||
```bash
|
|
||||||
.venv/bin/meshtastic-mcp-test-tui
|
|
||||||
.venv/bin/meshtastic-mcp-test-tui tests/mesh # pytest args pass through
|
|
||||||
```
|
|
||||||
|
|
||||||
Textual-based wrapper over `run-tests.sh` with a live test tree, tier
|
|
||||||
counters, pytest output pane, firmware-log pane, and a device-status strip.
|
|
||||||
Key bindings: `r` re-run focused, `f` filter, `d` failure detail, `g` open
|
|
||||||
`report.html`, `x` export reproducer bundle, `l` cycle fw-log filter, `q`
|
|
||||||
quit (SIGINT → SIGTERM → SIGKILL escalation).
|
|
||||||
|
|
||||||
Set `MESHTASTIC_UI_TUI_CAMERA=1` to mount a bottom-of-screen **UI camera**
|
|
||||||
panel. Left side: the latest capture PNG rendered as Unicode half-blocks
|
|
||||||
(via `rich-pixels`, works in any terminal - no kitty/sixel required).
|
|
||||||
Right side: live transcript tail ("step 3 - frame 4/8 name=nodelist_nodes
|
|
||||||
|
|
||||||
- OCR: Nodes 2/2") so you can see every event-injection and its result
|
|
||||||
as each UI test runs. Requires the `[ui]` extras for image rendering; the
|
|
||||||
transcript alone works without them.
|
|
||||||
|
|
||||||
### Slash commands
|
|
||||||
|
|
||||||
Three AI-assisted workflows are wired up for Claude Code operators
|
|
||||||
(`.claude/commands/`) and Copilot operators (`.github/prompts/`):
|
|
||||||
`/test` (run + interpret), `/diagnose` (read-only health report), `/repro`
|
|
||||||
(flake triage, N-times re-run with log diff).
|
|
||||||
|
|
||||||
### House rules (for human + agent contributors)
|
|
||||||
|
|
||||||
- Session-scoped fixtures in `tests/conftest.py` snapshot + restore
|
|
||||||
`userPrefs.jsonc`; **never edit `userPrefs.jsonc` from inside a test**.
|
|
||||||
Use the `test_profile` / `no_region_profile` fixtures for ephemeral
|
|
||||||
overrides.
|
|
||||||
- `SerialInterface` holds an **exclusive port lock**; sequence calls
|
|
||||||
open → mutate → close, then next device. No parallel calls to the
|
|
||||||
same port.
|
|
||||||
- Directed PKI-encrypted sends need **bilateral NodeInfo warmup** -
|
|
||||||
both sides must hold the other's current pubkey. See
|
|
||||||
`tests/mesh/_receive.py::nudge_nodeinfo_port` and the three directed-
|
|
||||||
send tests (`test_direct_with_ack`, `test_traceroute`,
|
|
||||||
`test_telemetry_request_reply`) for the canonical pattern.
|
|
||||||
|
|
||||||
## Layout
|
|
||||||
|
|
||||||
```text
|
|
||||||
mcp-server/
|
|
||||||
├── pyproject.toml
|
|
||||||
├── README.md
|
|
||||||
└── src/meshtastic_mcp/
|
|
||||||
├── __main__.py # entry: python -m meshtastic_mcp
|
|
||||||
├── server.py # FastMCP app + @app.tool() registrations (thin)
|
|
||||||
├── config.py # firmware_root, pio_bin, esptool_bin, etc.
|
|
||||||
├── pio.py # subprocess wrapper (timeouts, JSON, tail_lines)
|
|
||||||
├── devices.py # list_devices (findPorts + comports)
|
|
||||||
├── boards.py # list_boards / get_board (pio project config parse + cache)
|
|
||||||
├── flash.py # build, clean, flash, erase_and_flash, update_flash, touch_1200bps
|
|
||||||
├── serial_session.py # SerialSession + reader thread + ring buffer
|
|
||||||
├── registry.py # session registry + per-port locks
|
|
||||||
├── connection.py # connect(port) ctx mgr - SerialInterface + port lock
|
|
||||||
├── info.py # device_info, list_nodes
|
|
||||||
├── admin.py # set_owner, get/set_config, channels, send_text, reboot/shutdown/factory_reset
|
|
||||||
└── hw_tools.py # esptool / nrfutil / picotool wrappers
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
- **"Could not locate Meshtastic firmware root"** - set `MESHTASTIC_FIRMWARE_ROOT`.
|
|
||||||
- **"Could not find `pio`"** - install PlatformIO or set `MESHTASTIC_PIO_BIN`.
|
|
||||||
- **"Port is held by serial session ..."** - call `serial_close(session_id)` or `serial_list` to find it.
|
|
||||||
- **`factory.bin` not found after build** - the env may not be ESP32; only ESP32 envs produce a `.factory.bin`.
|
|
||||||
- **`touch_1200bps` reported `new_port: null`** - the device may not have 1200bps-reset stdio, or the bootloader re-uses the same port name. Check `list_devices` manually.
|
|
||||||
@@ -1,54 +0,0 @@
|
|||||||
[project]
|
|
||||||
name = "meshtastic-mcp"
|
|
||||||
version = "0.1.0"
|
|
||||||
description = "MCP server for Meshtastic firmware development: device discovery, PlatformIO tooling, flashing, serial monitoring, and device administration via the meshtastic Python API."
|
|
||||||
readme = "README.md"
|
|
||||||
requires-python = ">=3.11"
|
|
||||||
license = { text = "GPL-3.0-only" }
|
|
||||||
authors = [{ name = "thebentern" }]
|
|
||||||
dependencies = ["mcp>=1.2", "pyserial>=3.5", "meshtastic>=2.7.8"]
|
|
||||||
|
|
||||||
[project.optional-dependencies]
|
|
||||||
dev = ["pytest>=7"]
|
|
||||||
test = [
|
|
||||||
"pytest>=8",
|
|
||||||
"pytest-html>=4",
|
|
||||||
"pytest-reportlog>=0.4",
|
|
||||||
"pytest-timeout>=2.3",
|
|
||||||
"coverage[toml]>=7",
|
|
||||||
"pyyaml>=6",
|
|
||||||
# textual is required by the `meshtastic-mcp-test-tui` script (see
|
|
||||||
# `src/meshtastic_mcp/cli/test_tui.py`). Bundled into `test` rather than a
|
|
||||||
# separate `[tui]` extra because v1 expects test operators are the only
|
|
||||||
# consumers; revisit if install cost pushes back.
|
|
||||||
"textual>=0.50",
|
|
||||||
]
|
|
||||||
# UI test tier + `capture_screen` MCP tool. Optional because the ML OCR
|
|
||||||
# model alone is ~100 MB and camera hardware is user-supplied.
|
|
||||||
# pip install -e '.[ui]' - full (OpenCV + easyocr)
|
|
||||||
# pip install -e '.[ui-min]' - image capture only, no OCR
|
|
||||||
ui = [
|
|
||||||
"opencv-python-headless>=4.9",
|
|
||||||
"numpy>=1.26",
|
|
||||||
"easyocr>=1.7",
|
|
||||||
"Pillow>=10.0",
|
|
||||||
# Renders the latest camera capture as Unicode half-blocks in the TUI
|
|
||||||
# (MESHTASTIC_UI_TUI_CAMERA=1). Terminal-agnostic - no kitty / sixel
|
|
||||||
# dependency. Pure Python, tiny.
|
|
||||||
"rich-pixels>=3.0",
|
|
||||||
]
|
|
||||||
ui-min = ["opencv-python-headless>=4.9", "numpy>=1.26"]
|
|
||||||
|
|
||||||
[project.scripts]
|
|
||||||
meshtastic-mcp = "meshtastic_mcp.__main__:main"
|
|
||||||
# Live TUI wrapping run-tests.sh - shells out to the same script the plain
|
|
||||||
# CLI uses, tails pytest-reportlog for per-test state, and polls the device
|
|
||||||
# list at startup + post-run (port lock forces it to stay idle during the run).
|
|
||||||
meshtastic-mcp-test-tui = "meshtastic_mcp.cli.test_tui:main"
|
|
||||||
|
|
||||||
[build-system]
|
|
||||||
requires = ["hatchling"]
|
|
||||||
build-backend = "hatchling.build"
|
|
||||||
|
|
||||||
[tool.hatch.build.targets.wheel]
|
|
||||||
packages = ["src/meshtastic_mcp"]
|
|
||||||
@@ -1,274 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# mcp-server hardware test runner.
|
|
||||||
#
|
|
||||||
# Auto-detects connected Meshtastic devices, maps each to its PlatformIO env
|
|
||||||
# via the same role table the pytest fixtures use, exports the right
|
|
||||||
# MESHTASTIC_MCP_ENV_* env vars, and invokes pytest.
|
|
||||||
#
|
|
||||||
# Usage:
|
|
||||||
# ./run-tests.sh # full suite, default pytest args
|
|
||||||
# ./run-tests.sh tests/mesh # subset (any pytest args pass through)
|
|
||||||
# ./run-tests.sh --force-bake # override one default with another
|
|
||||||
# MESHTASTIC_MCP_ENV_NRF52=foo ./run-tests.sh # override env per role
|
|
||||||
# MESHTASTIC_MCP_SEED=ci-run-42 ./run-tests.sh # override PSK seed
|
|
||||||
#
|
|
||||||
# If zero supported devices are detected, only the unit tier runs.
|
|
||||||
#
|
|
||||||
# Also restores `userPrefs.jsonc` from the session-backup sidecar if a prior
|
|
||||||
# run exited abnormally (belt to conftest.py's atexit suspenders).
|
|
||||||
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
# cd to the script's directory so relative paths resolve consistently no
|
|
||||||
# matter where the user invoked from.
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
||||||
cd "$SCRIPT_DIR"
|
|
||||||
|
|
||||||
VENV_PY="$SCRIPT_DIR/.venv/bin/python"
|
|
||||||
if [[ ! -x $VENV_PY ]]; then
|
|
||||||
echo "error: $VENV_PY not found or not executable." >&2
|
|
||||||
echo " Bootstrap the venv first:" >&2
|
|
||||||
echo " cd $SCRIPT_DIR && python3 -m venv .venv && .venv/bin/pip install -e '.[test]'" >&2
|
|
||||||
exit 2
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Resolve firmware root the same way conftest.py does (this script sits in
|
|
||||||
# mcp-server/, firmware repo root is one level up).
|
|
||||||
FIRMWARE_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
|
||||||
USERPREFS_PATH="$FIRMWARE_ROOT/userPrefs.jsonc"
|
|
||||||
USERPREFS_SIDECAR="$USERPREFS_PATH.mcp-session-bak"
|
|
||||||
|
|
||||||
# ---------- Pre-flight: recover stale userPrefs.jsonc from prior crash ----
|
|
||||||
# If conftest.py's atexit hook didn't fire (SIGKILL, kernel panic, OS
|
|
||||||
# restart), the sidecar is the ground truth. Self-heal before running so we
|
|
||||||
# don't bake the previous run's dirty state into this run's firmware.
|
|
||||||
if [[ -f $USERPREFS_SIDECAR ]]; then
|
|
||||||
echo "[pre-flight] found $USERPREFS_SIDECAR from a prior abnormal exit;" >&2
|
|
||||||
echo " restoring userPrefs.jsonc before starting." >&2
|
|
||||||
cp "$USERPREFS_SIDECAR" "$USERPREFS_PATH"
|
|
||||||
rm -f "$USERPREFS_SIDECAR"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# If userPrefs.jsonc has uncommitted changes BEFORE the run starts, that's
|
|
||||||
# worth warning about - tests will snapshot this dirty state and restore to
|
|
||||||
# it at the end, which may not be what the operator wants.
|
|
||||||
if command -v git >/dev/null 2>&1; then
|
|
||||||
cd "$FIRMWARE_ROOT"
|
|
||||||
# Capture the git status into a local first - SC2312 flags command
|
|
||||||
# substitution inside `[[ -n ... ]]` because the exit code of `git
|
|
||||||
# status` is masked. A two-step assignment makes the failure path
|
|
||||||
# explicit (non-git, missing file) and keeps the bracket test clean.
|
|
||||||
_git_status_porcelain="$(git status --porcelain userPrefs.jsonc 2>/dev/null || true)"
|
|
||||||
if [[ -n $_git_status_porcelain ]]; then
|
|
||||||
echo "[pre-flight] warning: userPrefs.jsonc has uncommitted changes." >&2
|
|
||||||
echo " Tests will snapshot THIS state and restore to it" >&2
|
|
||||||
echo " at teardown. If that's not intended, run:" >&2
|
|
||||||
echo " git checkout userPrefs.jsonc" >&2
|
|
||||||
echo " and re-invoke." >&2
|
|
||||||
fi
|
|
||||||
cd "$SCRIPT_DIR"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# ---------- Seed default --------------------------------------------------
|
|
||||||
# Per-machine default so repeated runs from the same operator land on the
|
|
||||||
# same PSK (makes --assume-baked valid across invocations). Operator can
|
|
||||||
# override with an explicit env var if they want isolation (e.g. CI).
|
|
||||||
if [[ -z ${MESHTASTIC_MCP_SEED-} ]]; then
|
|
||||||
WHO="$(whoami 2>/dev/null || echo anon)"
|
|
||||||
HOST="$(hostname -s 2>/dev/null || echo host)"
|
|
||||||
export MESHTASTIC_MCP_SEED="mcp-${WHO}-${HOST}"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# ---------- Flash progress log --------------------------------------------
|
|
||||||
# pio.py / hw_tools.py tee subprocess output (pio run -t upload, esptool,
|
|
||||||
# nrfutil, picotool) to this file line-by-line as it arrives when this env
|
|
||||||
# var is set. The TUI tails it so the operator sees live flash progress
|
|
||||||
# instead of 3 minutes of silence during `test_00_bake.py`. Plain CLI users
|
|
||||||
# also benefit - the log is a post-run diagnostic even without the TUI.
|
|
||||||
# Truncate at session start so each run gets a clean log.
|
|
||||||
export MESHTASTIC_MCP_FLASH_LOG="$SCRIPT_DIR/tests/flash.log"
|
|
||||||
: >"$MESHTASTIC_MCP_FLASH_LOG"
|
|
||||||
|
|
||||||
# ---------- Detect connected hardware -------------------------------------
|
|
||||||
# In-process call to the same Python API the test fixtures use, so the
|
|
||||||
# script never drifts from what pytest sees. Returns a JSON object
|
|
||||||
# {role: port, ...}.
|
|
||||||
ROLES_JSON="$(
|
|
||||||
"$VENV_PY" - <<'PY'
|
|
||||||
import json
|
|
||||||
import sys
|
|
||||||
|
|
||||||
sys.path.insert(0, "src")
|
|
||||||
from meshtastic_mcp import devices
|
|
||||||
|
|
||||||
# Role → canonical VID map. Kept in sync with
|
|
||||||
# `tests/conftest.py::hub_profile` defaults; if that changes, this must too.
|
|
||||||
ROLE_BY_VID = {
|
|
||||||
0x239A: "nrf52", # Adafruit / RAK nRF52 native USB (app + DFU)
|
|
||||||
0x303A: "esp32s3", # Espressif native USB (ESP32-S3)
|
|
||||||
0x10C4: "esp32s3", # CP2102 USB-UART (common on Heltec/LilyGO ESP32 boards)
|
|
||||||
}
|
|
||||||
|
|
||||||
out: dict[str, str] = {}
|
|
||||||
for dev in devices.list_devices(include_unknown=True):
|
|
||||||
vid_raw = dev.get("vid") or ""
|
|
||||||
try:
|
|
||||||
if isinstance(vid_raw, str) and vid_raw.startswith("0x"):
|
|
||||||
vid = int(vid_raw, 16)
|
|
||||||
else:
|
|
||||||
vid = int(vid_raw)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
continue
|
|
||||||
role = ROLE_BY_VID.get(vid)
|
|
||||||
# First port wins per role - matches hub_devices fixture semantics.
|
|
||||||
if role and role not in out:
|
|
||||||
out[role] = dev["port"]
|
|
||||||
|
|
||||||
json.dump(out, sys.stdout)
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
|
|
||||||
# ---------- Map role → pio env --------------------------------------------
|
|
||||||
# Honor MESHTASTIC_MCP_ENV_<ROLE> operator overrides; fall back to the
|
|
||||||
# same defaults hardcoded in tests/conftest.py::_DEFAULT_ROLE_ENVS.
|
|
||||||
resolve_env() {
|
|
||||||
local role="$1"
|
|
||||||
local default="$2"
|
|
||||||
local upper
|
|
||||||
upper="$(echo "$role" | tr '[:lower:]' '[:upper:]')"
|
|
||||||
local var="MESHTASTIC_MCP_ENV_${upper}"
|
|
||||||
eval "local override=\${$var:-}"
|
|
||||||
if [[ -n $override ]]; then
|
|
||||||
echo "$override"
|
|
||||||
else
|
|
||||||
echo "$default"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
NRF52_PORT="$(echo "$ROLES_JSON" | "$VENV_PY" -c 'import json,sys; print(json.loads(sys.stdin.read()).get("nrf52", ""))')"
|
|
||||||
ESP32S3_PORT="$(echo "$ROLES_JSON" | "$VENV_PY" -c 'import json,sys; print(json.loads(sys.stdin.read()).get("esp32s3", ""))')"
|
|
||||||
|
|
||||||
DETECTED=""
|
|
||||||
if [[ -n $NRF52_PORT ]]; then
|
|
||||||
NRF52_ENV="$(resolve_env nrf52 rak4631)"
|
|
||||||
export MESHTASTIC_MCP_ENV_NRF52="$NRF52_ENV"
|
|
||||||
DETECTED="${DETECTED} nrf52 @ ${NRF52_PORT} -> env=${NRF52_ENV}\n"
|
|
||||||
fi
|
|
||||||
if [[ -n $ESP32S3_PORT ]]; then
|
|
||||||
ESP32S3_ENV="$(resolve_env esp32s3 heltec-v3)"
|
|
||||||
export MESHTASTIC_MCP_ENV_ESP32S3="$ESP32S3_ENV"
|
|
||||||
DETECTED="${DETECTED} esp32s3 @ ${ESP32S3_PORT} -> env=${ESP32S3_ENV}\n"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# ---------- Pre-flight summary --------------------------------------------
|
|
||||||
# Surface what pytest is about to do with respect to the bake phase: the
|
|
||||||
# operator should see "will verify + bake if needed" by default, so a
|
|
||||||
# 3-minute flash appearing mid-run isn't a surprise. Detection of the
|
|
||||||
# explicit overrides is best-effort - we just scan $@ for the known flags.
|
|
||||||
_bake_mode="auto (verify + bake if needed)"
|
|
||||||
for _arg in "$@"; do
|
|
||||||
case "$_arg" in
|
|
||||||
--assume-baked) _bake_mode="skip (--assume-baked)" ;;
|
|
||||||
--force-bake) _bake_mode="force (--force-bake)" ;;
|
|
||||||
*) ;; # any other arg: pass-through; bake mode unchanged
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
echo "mcp-server test runner"
|
|
||||||
echo " firmware root : $FIRMWARE_ROOT"
|
|
||||||
echo " seed : $MESHTASTIC_MCP_SEED"
|
|
||||||
echo " bake : $_bake_mode"
|
|
||||||
if [[ -n $DETECTED ]]; then
|
|
||||||
echo " detected hub :"
|
|
||||||
printf "%b" "$DETECTED"
|
|
||||||
else
|
|
||||||
echo " detected hub : (none)"
|
|
||||||
fi
|
|
||||||
echo
|
|
||||||
|
|
||||||
# ---------- Invoke pytest -------------------------------------------------
|
|
||||||
# If no devices detected, only the unit tier would produce meaningful
|
|
||||||
# PASS/FAIL - every hardware test would SKIP with "role not present". We
|
|
||||||
# narrow to tests/unit explicitly so the summary reads as "no hardware,
|
|
||||||
# unit suite only" instead of "big skip count looks suspicious".
|
|
||||||
# Keep terminal output condensed (`-q -r fE`) so skip-heavy runs do not print
|
|
||||||
# each skipped test in full; skip counts still appear in pytest's summary.
|
|
||||||
if [[ -z $DETECTED && $# -eq 0 ]]; then
|
|
||||||
echo "[pre-flight] no supported devices detected; running unit tier only."
|
|
||||||
echo
|
|
||||||
exec "$VENV_PY" -m pytest tests/unit -q -r fE --report-log=tests/reportlog.jsonl
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Default pytest args when the user passed none. Power users can invoke
|
|
||||||
# `./run-tests.sh tests/mesh -v --tb=long` and skip all of these defaults.
|
|
||||||
#
|
|
||||||
# NOTE: `--assume-baked` is DELIBERATELY omitted here. `tests/test_00_bake.py`
|
|
||||||
# has an internal skip-if-already-baked check (`_bake_role`: query device_info,
|
|
||||||
# compare region + primary_channel to the session profile, skip on match).
|
|
||||||
# So the fast path is ~8-10 s of verification overhead when the devices are
|
|
||||||
# already baked - negligible next to the 2-6 min suite runtime. Letting
|
|
||||||
# test_00_bake.py run means a fresh device, a re-seeded session, or a post-
|
|
||||||
# factory-reset device gets flashed automatically instead of silently
|
|
||||||
# skipping half the hardware tests with "not baked with session profile"
|
|
||||||
# errors. Power users who know their hardware is current and want to shave
|
|
||||||
# those seconds can pass `--assume-baked` explicitly.
|
|
||||||
# Defaults also use condensed reporting (`-q -r fE`) to avoid listing every
|
|
||||||
# skipped test verbatim while still surfacing failures/errors and summary data.
|
|
||||||
if [[ $# -eq 0 ]]; then
|
|
||||||
set -- tests/ \
|
|
||||||
--html=tests/report.html --self-contained-html \
|
|
||||||
--junitxml=tests/junit.xml \
|
|
||||||
-q -r fE --tb=short
|
|
||||||
fi
|
|
||||||
|
|
||||||
# UI tier requires opencv-python-headless (and ideally easyocr). If it's
|
|
||||||
# not installed, auto-deselect tests/ui so operators without the [ui]
|
|
||||||
# extra still get a green run. Printed in yellow; silent when cv2 is
|
|
||||||
# present.
|
|
||||||
_cv2_ok=0
|
|
||||||
if "$VENV_PY" -c "import cv2" >/dev/null 2>&1; then
|
|
||||||
_cv2_ok=1
|
|
||||||
fi
|
|
||||||
_running_ui=0
|
|
||||||
for _arg in "$@"; do
|
|
||||||
case "$_arg" in
|
|
||||||
*tests/ui* | tests/) _running_ui=1 ;;
|
|
||||||
*) ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
if [[ $_running_ui -eq 1 && $_cv2_ok -eq 0 ]]; then
|
|
||||||
printf '\033[33m[pre-flight] tests/ui tier detected, but opencv-python-headless is not installed - deselecting.\033[0m\n'
|
|
||||||
printf ' install with: .venv/bin/pip install -e "mcp-server/.[ui]"\n'
|
|
||||||
echo
|
|
||||||
set -- "$@" --ignore=tests/ui
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Recovery tier needs `uhubctl` on PATH - it power-cycles devices via USB
|
|
||||||
# hub PPPS. The tier's conftest already skips cleanly, so this is just a
|
|
||||||
# friendly heads-up before the skip happens. `baked_single`'s auto-
|
|
||||||
# recovery hook also benefits from having uhubctl available across the
|
|
||||||
# whole suite.
|
|
||||||
if ! command -v uhubctl >/dev/null 2>&1; then
|
|
||||||
printf "\033[33m[pre-flight] uhubctl not found on PATH - recovery tier will skip, and\n"
|
|
||||||
printf " wedged-device auto-recovery is disabled.\033[0m\n"
|
|
||||||
printf " install with: brew install uhubctl (macOS) or apt install uhubctl (Debian/Ubuntu).\n"
|
|
||||||
echo
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Always emit `tests/reportlog.jsonl` (unless the operator explicitly passed
|
|
||||||
# their own `--report-log=...`). Consumers - notably the
|
|
||||||
# `meshtastic-mcp-test-tui` TUI - tail the reportlog for live per-test state.
|
|
||||||
# Appending here means power-user invocations like `./run-tests.sh tests/mesh`
|
|
||||||
# also produce it, not just the all-defaults invocation.
|
|
||||||
_has_report_log=0
|
|
||||||
for _arg in "$@"; do
|
|
||||||
case "$_arg" in
|
|
||||||
--report-log | --report-log=*) _has_report_log=1 ;;
|
|
||||||
*) ;; # any other arg: no-op; loop continues
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
if [[ $_has_report_log -eq 0 ]]; then
|
|
||||||
set -- "$@" --report-log=tests/reportlog.jsonl
|
|
||||||
fi
|
|
||||||
|
|
||||||
exec "$VENV_PY" -m pytest "$@"
|
|
||||||
@@ -1,217 +0,0 @@
|
|||||||
{
|
|
||||||
"title": "Meshtastic Firmware - Recorder Stream",
|
|
||||||
"description": "Live view of `.mtlog/` streams shipped by `mtlog_to_datadog.py`. Heap, packet volume, log levels, errors. One row per port.",
|
|
||||||
"widgets": [
|
|
||||||
{
|
|
||||||
"definition": {
|
|
||||||
"title": "Free heap (bytes)",
|
|
||||||
"type": "timeseries",
|
|
||||||
"show_legend": true,
|
|
||||||
"requests": [
|
|
||||||
{
|
|
||||||
"queries": [
|
|
||||||
{
|
|
||||||
"name": "free_heap",
|
|
||||||
"data_source": "metrics",
|
|
||||||
"query": "avg:mesh.local.heap_free_bytes{service:meshtastic-firmware} by {port}"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"response_format": "timeseries",
|
|
||||||
"display_type": "line"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"yaxis": { "label": "bytes" }
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"definition": {
|
|
||||||
"title": "Heap slope (bytes/min) - last 1h",
|
|
||||||
"type": "query_value",
|
|
||||||
"precision": 0,
|
|
||||||
"requests": [
|
|
||||||
{
|
|
||||||
"queries": [
|
|
||||||
{
|
|
||||||
"name": "slope",
|
|
||||||
"data_source": "metrics",
|
|
||||||
"query": "derivative(avg:mesh.local.heap_free_bytes{service:meshtastic-firmware})",
|
|
||||||
"aggregator": "avg"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"response_format": "scalar"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"conditional_formats": [
|
|
||||||
{ "comparator": "<", "value": -100, "palette": "white_on_red" },
|
|
||||||
{ "comparator": "<", "value": 0, "palette": "white_on_yellow" },
|
|
||||||
{ "comparator": ">=", "value": 0, "palette": "white_on_green" }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"definition": {
|
|
||||||
"title": "Total heap (bytes)",
|
|
||||||
"type": "timeseries",
|
|
||||||
"requests": [
|
|
||||||
{
|
|
||||||
"queries": [
|
|
||||||
{
|
|
||||||
"name": "total_heap",
|
|
||||||
"data_source": "metrics",
|
|
||||||
"query": "avg:mesh.local.heap_total_bytes{service:meshtastic-firmware} by {port}"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"response_format": "timeseries",
|
|
||||||
"display_type": "line"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"definition": {
|
|
||||||
"title": "Battery level (%)",
|
|
||||||
"type": "timeseries",
|
|
||||||
"requests": [
|
|
||||||
{
|
|
||||||
"queries": [
|
|
||||||
{
|
|
||||||
"name": "battery",
|
|
||||||
"data_source": "metrics",
|
|
||||||
"query": "avg:mesh.device.battery_level{service:meshtastic-firmware} by {port}"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"response_format": "timeseries",
|
|
||||||
"display_type": "line"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"yaxis": { "min": "0", "max": "105" }
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"definition": {
|
|
||||||
"title": "Air utilization (TX %)",
|
|
||||||
"type": "timeseries",
|
|
||||||
"requests": [
|
|
||||||
{
|
|
||||||
"queries": [
|
|
||||||
{
|
|
||||||
"name": "airutil",
|
|
||||||
"data_source": "metrics",
|
|
||||||
"query": "avg:mesh.device.air_util_tx{service:meshtastic-firmware} by {port}"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"response_format": "timeseries",
|
|
||||||
"display_type": "line"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"definition": {
|
|
||||||
"title": "Channel utilization (%)",
|
|
||||||
"type": "timeseries",
|
|
||||||
"requests": [
|
|
||||||
{
|
|
||||||
"queries": [
|
|
||||||
{
|
|
||||||
"name": "chutil",
|
|
||||||
"data_source": "metrics",
|
|
||||||
"query": "avg:mesh.device.channel_utilization{service:meshtastic-firmware} by {port}"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"response_format": "timeseries",
|
|
||||||
"display_type": "line"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"definition": {
|
|
||||||
"title": "Log volume by level",
|
|
||||||
"type": "timeseries",
|
|
||||||
"show_legend": true,
|
|
||||||
"requests": [
|
|
||||||
{
|
|
||||||
"response_format": "timeseries",
|
|
||||||
"display_type": "bars",
|
|
||||||
"queries": [
|
|
||||||
{
|
|
||||||
"name": "log_count",
|
|
||||||
"data_source": "logs",
|
|
||||||
"indexes": ["*"],
|
|
||||||
"compute": { "aggregation": "count" },
|
|
||||||
"search": { "query": "service:meshtastic-firmware" },
|
|
||||||
"group_by": [
|
|
||||||
{
|
|
||||||
"facet": "@level",
|
|
||||||
"limit": 10,
|
|
||||||
"sort": { "order": "desc", "aggregation": "count" }
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"definition": {
|
|
||||||
"title": "Recent ERROR / CRIT firmware logs",
|
|
||||||
"type": "list_stream",
|
|
||||||
"requests": [
|
|
||||||
{
|
|
||||||
"response_format": "event_list",
|
|
||||||
"query": {
|
|
||||||
"data_source": "logs_stream",
|
|
||||||
"query_string": "service:meshtastic-firmware (status:error OR @level:ERROR OR @level:CRIT)",
|
|
||||||
"indexes": [],
|
|
||||||
"sort": { "column": "timestamp", "order": "desc" }
|
|
||||||
},
|
|
||||||
"columns": [
|
|
||||||
{ "field": "timestamp", "width": "auto" },
|
|
||||||
{ "field": "host", "width": "auto" },
|
|
||||||
{ "field": "@port", "width": "auto" },
|
|
||||||
{ "field": "@level", "width": "auto" },
|
|
||||||
{ "field": "@thread", "width": "auto" },
|
|
||||||
{ "field": "message", "width": "stretch" }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"definition": {
|
|
||||||
"title": "Recorder marker events",
|
|
||||||
"type": "list_stream",
|
|
||||||
"requests": [
|
|
||||||
{
|
|
||||||
"response_format": "event_list",
|
|
||||||
"query": {
|
|
||||||
"data_source": "logs_stream",
|
|
||||||
"query_string": "service:meshtastic-firmware @level:MARK",
|
|
||||||
"indexes": [],
|
|
||||||
"sort": { "column": "timestamp", "order": "desc" }
|
|
||||||
},
|
|
||||||
"columns": [
|
|
||||||
{ "field": "timestamp", "width": "auto" },
|
|
||||||
{ "field": "host", "width": "auto" },
|
|
||||||
{ "field": "message", "width": "stretch" }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"template_variables": [
|
|
||||||
{
|
|
||||||
"name": "port",
|
|
||||||
"prefix": "port",
|
|
||||||
"available_values": [],
|
|
||||||
"default": "*"
|
|
||||||
},
|
|
||||||
{ "name": "host", "prefix": "host", "available_values": [], "default": "*" }
|
|
||||||
],
|
|
||||||
"layout_type": "ordered",
|
|
||||||
"notify_list": [],
|
|
||||||
"reflow_type": "auto"
|
|
||||||
}
|
|
||||||
@@ -1,389 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""Forward selected recorder JSONL streams to Datadog.
|
|
||||||
|
|
||||||
Reads `.mtlog/logs.jsonl` and `.mtlog/telemetry.jsonl`, ships logs to the
|
|
||||||
Logs Intake API and telemetry numerics to the Metrics v2 series API.
|
|
||||||
Resumes from `.mtlog/.dd-cursor.json` so a daemon restart doesn't
|
|
||||||
duplicate rows already shipped from the current live files.
|
|
||||||
|
|
||||||
This forwarder does not currently backfill rotated `.jsonl.gz` archives.
|
|
||||||
If the recorder rotates before this process drains the live file, or the
|
|
||||||
forwarder is down across a rotation, those older rows are skipped.
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
DD_API_KEY=... ./scripts/mtlog_to_datadog.py --tail
|
|
||||||
./scripts/mtlog_to_datadog.py --once # catch up + exit
|
|
||||||
./scripts/mtlog_to_datadog.py --since 3600 # backfill last hour from start
|
|
||||||
|
|
||||||
Default `DD_SITE` is `us5.datadoghq.com` - the team's Datadog instance.
|
|
||||||
Override via `DD_SITE=...` env var or `--site` flag for one-offs.
|
|
||||||
|
|
||||||
The forwarder is a separate process by design - a Datadog outage or
|
|
||||||
auth failure must not backpressure the recorder. We exit non-zero on
|
|
||||||
fatal config errors (missing API key) and keep retrying on transient
|
|
||||||
network/HTTP errors.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import socket
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Iterator
|
|
||||||
|
|
||||||
try:
|
|
||||||
import requests
|
|
||||||
except ImportError:
|
|
||||||
print(
|
|
||||||
"requests is required. Install it in the mcp-server venv: "
|
|
||||||
"uv pip install requests",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
sys.exit(2)
|
|
||||||
|
|
||||||
|
|
||||||
_DEFAULT_LOG_DIR = Path(__file__).resolve().parents[1] / ".mtlog"
|
|
||||||
_LOG_INTAKE_TPL = "https://http-intake.logs.{site}/api/v2/logs"
|
|
||||||
_METRICS_TPL = "https://api.{site}/api/v2/series"
|
|
||||||
_LOG_BATCH = 50
|
|
||||||
_METRICS_BATCH = 100
|
|
||||||
_MAX_RETRIES = 5
|
|
||||||
_RETRY_BASE_S = 1.5
|
|
||||||
|
|
||||||
|
|
||||||
# --- streaming JSONL with byte-position cursor -------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class _StreamReader:
|
|
||||||
"""Reads a single rotating JSONL with cursor-based resume.
|
|
||||||
|
|
||||||
This tails only the live `.jsonl` file. The recorder rotates files
|
|
||||||
(live `.jsonl` → `.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`), which means
|
|
||||||
the live file shrinks abruptly. We detect that via inode change OR live
|
|
||||||
size < cursor position, and reset the live-file cursor to 0.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, path: Path, cursor: dict[str, Any]):
|
|
||||||
self.path = path
|
|
||||||
self.cursor = cursor
|
|
||||||
|
|
||||||
def _state(self) -> tuple[int, int]:
|
|
||||||
"""Return (inode, size) for the live file. (0, 0) if missing."""
|
|
||||||
try:
|
|
||||||
st = self.path.stat()
|
|
||||||
return (st.st_ino, st.st_size)
|
|
||||||
except FileNotFoundError:
|
|
||||||
return (0, 0)
|
|
||||||
|
|
||||||
def iter_new_records(self) -> Iterator[dict[str, Any]]:
|
|
||||||
ino, size = self._state()
|
|
||||||
last_ino = self.cursor.get("ino")
|
|
||||||
last_pos = int(self.cursor.get("pos") or 0)
|
|
||||||
if ino == 0:
|
|
||||||
return
|
|
||||||
if last_ino is not None and last_ino != ino:
|
|
||||||
# Rotation happened. Start over.
|
|
||||||
last_pos = 0
|
|
||||||
if last_pos > size:
|
|
||||||
# Live file truncated/shrunk under us - recorder rotated.
|
|
||||||
last_pos = 0
|
|
||||||
try:
|
|
||||||
with self.path.open("r", encoding="utf-8") as fh:
|
|
||||||
fh.seek(last_pos)
|
|
||||||
for line in fh:
|
|
||||||
line = line.rstrip("\n")
|
|
||||||
if not line:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
yield json.loads(line)
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
continue
|
|
||||||
last_pos = fh.tell()
|
|
||||||
except FileNotFoundError:
|
|
||||||
return
|
|
||||||
self.cursor["ino"] = ino
|
|
||||||
self.cursor["pos"] = last_pos
|
|
||||||
|
|
||||||
|
|
||||||
def _load_cursor(path: Path) -> dict[str, Any]:
|
|
||||||
if not path.exists():
|
|
||||||
return {}
|
|
||||||
try:
|
|
||||||
return json.loads(path.read_text())
|
|
||||||
except (OSError, json.JSONDecodeError):
|
|
||||||
return {}
|
|
||||||
|
|
||||||
|
|
||||||
def _save_cursor(path: Path, data: dict[str, Any]) -> None:
|
|
||||||
tmp = path.with_suffix(".json.tmp")
|
|
||||||
tmp.write_text(json.dumps(data, separators=(",", ":")))
|
|
||||||
tmp.replace(path)
|
|
||||||
|
|
||||||
|
|
||||||
# --- Datadog clients ---------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class _DDSession:
|
|
||||||
"""Pool one HTTPS session, share retry logic."""
|
|
||||||
|
|
||||||
def __init__(self, api_key: str, site: str, hostname: str) -> None:
|
|
||||||
self.api_key = api_key
|
|
||||||
self.site = site
|
|
||||||
self.hostname = hostname
|
|
||||||
self.session = requests.Session()
|
|
||||||
self.session.headers.update(
|
|
||||||
{
|
|
||||||
"DD-API-KEY": api_key,
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
def _post(self, url: str, payload: Any) -> bool:
|
|
||||||
for attempt in range(_MAX_RETRIES):
|
|
||||||
try:
|
|
||||||
resp = self.session.post(url, json=payload, timeout=30)
|
|
||||||
except requests.RequestException as e:
|
|
||||||
_wait_retry(attempt, f"network error: {e}")
|
|
||||||
continue
|
|
||||||
if 200 <= resp.status_code < 300:
|
|
||||||
return True
|
|
||||||
if resp.status_code in (408, 429, 500, 502, 503, 504):
|
|
||||||
_wait_retry(
|
|
||||||
attempt,
|
|
||||||
f"HTTP {resp.status_code} retrying",
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
print(
|
|
||||||
f"datadog refused: {resp.status_code} {resp.text[:200]}",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
return False
|
|
||||||
return False
|
|
||||||
|
|
||||||
def send_logs(self, records: list[dict[str, Any]]) -> int:
|
|
||||||
if not records:
|
|
||||||
return 0
|
|
||||||
url = _LOG_INTAKE_TPL.format(site=self.site)
|
|
||||||
sent = 0
|
|
||||||
for i in range(0, len(records), _LOG_BATCH):
|
|
||||||
batch = records[i : i + _LOG_BATCH]
|
|
||||||
if self._post(url, batch):
|
|
||||||
sent += len(batch)
|
|
||||||
return sent
|
|
||||||
|
|
||||||
def send_metrics(self, series: list[dict[str, Any]]) -> int:
|
|
||||||
if not series:
|
|
||||||
return 0
|
|
||||||
url = _METRICS_TPL.format(site=self.site)
|
|
||||||
sent = 0
|
|
||||||
for i in range(0, len(series), _METRICS_BATCH):
|
|
||||||
batch = series[i : i + _METRICS_BATCH]
|
|
||||||
if self._post(url, {"series": batch}):
|
|
||||||
sent += len(batch)
|
|
||||||
return sent
|
|
||||||
|
|
||||||
|
|
||||||
def _wait_retry(attempt: int, reason: str) -> None:
|
|
||||||
wait = _RETRY_BASE_S * (2**attempt)
|
|
||||||
print(
|
|
||||||
f" retry {attempt + 1}/{_MAX_RETRIES} in {wait:.1f}s ({reason})",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
time.sleep(wait)
|
|
||||||
|
|
||||||
|
|
||||||
# --- record → datadog payload ------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _log_record_to_dd(rec: dict[str, Any], host: str) -> dict[str, Any]:
|
|
||||||
line = rec.get("line") or ""
|
|
||||||
tags = [
|
|
||||||
f"role:{rec.get('role')}",
|
|
||||||
f"port:{rec.get('port')}",
|
|
||||||
]
|
|
||||||
level = rec.get("level")
|
|
||||||
if level:
|
|
||||||
tags.append(f"level:{level}")
|
|
||||||
tag = rec.get("tag")
|
|
||||||
if tag:
|
|
||||||
tags.append(f"thread:{tag}")
|
|
||||||
return {
|
|
||||||
"ddsource": "meshtastic-firmware",
|
|
||||||
"service": "meshtastic-firmware",
|
|
||||||
"hostname": host,
|
|
||||||
"message": line,
|
|
||||||
"ddtags": ",".join(t for t in tags if t and "None" not in t),
|
|
||||||
"timestamp": int((rec.get("ts") or time.time()) * 1000),
|
|
||||||
"level": level,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _telemetry_record_to_metrics(
|
|
||||||
rec: dict[str, Any], host: str
|
|
||||||
) -> list[dict[str, Any]]:
|
|
||||||
fields = rec.get("fields") or {}
|
|
||||||
if not isinstance(fields, dict):
|
|
||||||
return []
|
|
||||||
variant = rec.get("variant") or "unknown"
|
|
||||||
ts = int(rec.get("ts") or time.time())
|
|
||||||
out: list[dict[str, Any]] = []
|
|
||||||
tags = []
|
|
||||||
if rec.get("port"):
|
|
||||||
tags.append(f"port:{rec['port']}")
|
|
||||||
if rec.get("role"):
|
|
||||||
tags.append(f"role:{rec['role']}")
|
|
||||||
if rec.get("from_node"):
|
|
||||||
tags.append(f"from_node:{rec['from_node']}")
|
|
||||||
tags.append(f"variant:{variant}")
|
|
||||||
for field, value in fields.items():
|
|
||||||
if not isinstance(value, (int, float)) or isinstance(value, bool):
|
|
||||||
continue
|
|
||||||
metric = f"mesh.{variant}.{_metric_safe(field)}"
|
|
||||||
out.append(
|
|
||||||
{
|
|
||||||
"metric": metric,
|
|
||||||
"type": 3, # GAUGE
|
|
||||||
"points": [{"timestamp": ts, "value": float(value)}],
|
|
||||||
"tags": tags,
|
|
||||||
"resources": [{"type": "host", "name": host}],
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _metric_safe(name: str) -> str:
|
|
||||||
# Lowercase, replace non-alnum with underscore for safe metric names.
|
|
||||||
return "".join(c.lower() if c.isalnum() else "_" for c in name)
|
|
||||||
|
|
||||||
|
|
||||||
# --- main loop ---------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def run(
|
|
||||||
log_dir: Path,
|
|
||||||
*,
|
|
||||||
once: bool,
|
|
||||||
since_seconds: float | None,
|
|
||||||
poll_interval: float,
|
|
||||||
dd: _DDSession,
|
|
||||||
) -> int:
|
|
||||||
cursor_path = log_dir / ".dd-cursor.json"
|
|
||||||
cursors = _load_cursor(cursor_path)
|
|
||||||
|
|
||||||
# `--since` overrides cursor: rewind to (now-since) timestamp.
|
|
||||||
# We can't seek by timestamp directly (cursor is byte position), so
|
|
||||||
# we just reset cursors to 0 and let the time filter in iter_new
|
|
||||||
# drop older records.
|
|
||||||
cutoff_ts: float | None = None
|
|
||||||
if since_seconds is not None:
|
|
||||||
cursors = {}
|
|
||||||
cutoff_ts = time.time() - since_seconds
|
|
||||||
|
|
||||||
sent_total = {"logs": 0, "telemetry": 0}
|
|
||||||
|
|
||||||
while True:
|
|
||||||
# logs.jsonl → DD logs
|
|
||||||
log_cursor = cursors.setdefault("logs", {})
|
|
||||||
log_batch: list[dict[str, Any]] = []
|
|
||||||
for rec in _StreamReader(log_dir / "logs.jsonl", log_cursor).iter_new_records():
|
|
||||||
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
|
|
||||||
continue
|
|
||||||
log_batch.append(_log_record_to_dd(rec, dd.hostname))
|
|
||||||
if log_batch:
|
|
||||||
n = dd.send_logs(log_batch)
|
|
||||||
sent_total["logs"] += n
|
|
||||||
print(f"logs: sent {n}/{len(log_batch)}")
|
|
||||||
|
|
||||||
# telemetry.jsonl → DD metrics
|
|
||||||
telem_cursor = cursors.setdefault("telemetry", {})
|
|
||||||
metric_series: list[dict[str, Any]] = []
|
|
||||||
for rec in _StreamReader(
|
|
||||||
log_dir / "telemetry.jsonl", telem_cursor
|
|
||||||
).iter_new_records():
|
|
||||||
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
|
|
||||||
continue
|
|
||||||
metric_series.extend(_telemetry_record_to_metrics(rec, dd.hostname))
|
|
||||||
if metric_series:
|
|
||||||
n = dd.send_metrics(metric_series)
|
|
||||||
sent_total["telemetry"] += n
|
|
||||||
print(f"telemetry: sent {n}/{len(metric_series)} metric points")
|
|
||||||
|
|
||||||
_save_cursor(cursor_path, cursors)
|
|
||||||
|
|
||||||
if once:
|
|
||||||
print(f"done. logs={sent_total['logs']} metrics={sent_total['telemetry']}")
|
|
||||||
return 0
|
|
||||||
time.sleep(poll_interval)
|
|
||||||
|
|
||||||
|
|
||||||
def main(argv: list[str] | None = None) -> int:
|
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
|
||||||
parser.add_argument(
|
|
||||||
"--log-dir",
|
|
||||||
default=str(_DEFAULT_LOG_DIR),
|
|
||||||
help="Path to .mtlog/ (default: mcp-server/.mtlog)",
|
|
||||||
)
|
|
||||||
mode = parser.add_mutually_exclusive_group()
|
|
||||||
mode.add_argument("--once", action="store_true", help="Catch up then exit")
|
|
||||||
mode.add_argument(
|
|
||||||
"--tail",
|
|
||||||
action="store_true",
|
|
||||||
help="Daemon: poll forever (default)",
|
|
||||||
)
|
|
||||||
parser.add_argument(
|
|
||||||
"--since",
|
|
||||||
type=float,
|
|
||||||
default=None,
|
|
||||||
help="Backfill last N seconds. Resets cursor.",
|
|
||||||
)
|
|
||||||
parser.add_argument(
|
|
||||||
"--poll-interval",
|
|
||||||
type=float,
|
|
||||||
default=5.0,
|
|
||||||
help="Seconds between tail polls (default 5)",
|
|
||||||
)
|
|
||||||
parser.add_argument(
|
|
||||||
"--site",
|
|
||||||
default=os.environ.get("DD_SITE", "us5.datadoghq.com"),
|
|
||||||
help=(
|
|
||||||
"Datadog site. Default is the team's instance (us5.datadoghq.com). "
|
|
||||||
"Override via DD_SITE env var or this flag."
|
|
||||||
),
|
|
||||||
)
|
|
||||||
parser.add_argument(
|
|
||||||
"--host",
|
|
||||||
default=socket.gethostname(),
|
|
||||||
help="Hostname tag (default: socket.gethostname())",
|
|
||||||
)
|
|
||||||
args = parser.parse_args(argv)
|
|
||||||
|
|
||||||
api_key = os.environ.get("DD_API_KEY")
|
|
||||||
if not api_key:
|
|
||||||
print("DD_API_KEY env var required.", file=sys.stderr)
|
|
||||||
return 2
|
|
||||||
|
|
||||||
log_dir = Path(args.log_dir)
|
|
||||||
if not log_dir.exists():
|
|
||||||
print(
|
|
||||||
f"log dir {log_dir} does not exist - start the mcp-server first.",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
return 2
|
|
||||||
|
|
||||||
dd = _DDSession(api_key=api_key, site=args.site, hostname=args.host)
|
|
||||||
once = args.once and not args.tail
|
|
||||||
return run(
|
|
||||||
log_dir,
|
|
||||||
once=once,
|
|
||||||
since_seconds=args.since,
|
|
||||||
poll_interval=args.poll_interval,
|
|
||||||
dd=dd,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
sys.exit(main())
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
"""Meshtastic MCP server - device discovery, PlatformIO tooling, and device admin."""
|
|
||||||
|
|
||||||
__version__ = "0.1.0"
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
"""Entry point for `python -m meshtastic_mcp`."""
|
|
||||||
|
|
||||||
from meshtastic_mcp.server import app
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
app.run()
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,417 +0,0 @@
|
|||||||
"""Device administration: owner, config, channels, messaging, admin actions.
|
|
||||||
|
|
||||||
All operations use the same `connect()` context manager so port selection,
|
|
||||||
port-busy detection, and cleanup are handled uniformly.
|
|
||||||
|
|
||||||
Config writes use a dot-path: the first segment names a section (e.g.
|
|
||||||
`"lora"` in LocalConfig or `"mqtt"` in LocalModuleConfig), remaining segments
|
|
||||||
walk protobuf fields. Enum fields accept their string names (`"US"` for
|
|
||||||
`lora.region`) so callers don't need to know the numeric values.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from google.protobuf import descriptor as pb_descriptor
|
|
||||||
from google.protobuf import json_format
|
|
||||||
from meshtastic.protobuf import localonly_pb2
|
|
||||||
|
|
||||||
from .connection import connect
|
|
||||||
|
|
||||||
|
|
||||||
class AdminError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
LOCAL_CONFIG_SECTIONS = {f.name for f in localonly_pb2.LocalConfig.DESCRIPTOR.fields}
|
|
||||||
MODULE_CONFIG_SECTIONS = {
|
|
||||||
f.name for f in localonly_pb2.LocalModuleConfig.DESCRIPTOR.fields
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _require_confirm(confirm: bool, operation: str) -> None:
|
|
||||||
if not confirm:
|
|
||||||
raise AdminError(f"{operation} is destructive and requires confirm=True.")
|
|
||||||
|
|
||||||
|
|
||||||
def _message_to_dict(msg: Any) -> dict[str, Any]:
|
|
||||||
# `including_default_value_fields` was renamed to
|
|
||||||
# `always_print_fields_with_no_presence` in protobuf 5.26+. Pick whichever
|
|
||||||
# kwarg the installed version accepts so we work against both.
|
|
||||||
kwargs: dict[str, Any] = {"preserving_proto_field_name": True}
|
|
||||||
import inspect
|
|
||||||
|
|
||||||
sig = inspect.signature(json_format.MessageToDict)
|
|
||||||
if "always_print_fields_with_no_presence" in sig.parameters:
|
|
||||||
kwargs["always_print_fields_with_no_presence"] = False
|
|
||||||
elif "including_default_value_fields" in sig.parameters:
|
|
||||||
kwargs["including_default_value_fields"] = False
|
|
||||||
return json_format.MessageToDict(msg, **kwargs)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- owner ----------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def set_owner(
|
|
||||||
long_name: str,
|
|
||||||
short_name: str | None = None,
|
|
||||||
port: str | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
if short_name is not None and len(short_name) > 4:
|
|
||||||
raise AdminError("short_name must be 4 characters or fewer")
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
iface.localNode.setOwner(long_name=long_name, short_name=short_name)
|
|
||||||
return {
|
|
||||||
"ok": True,
|
|
||||||
"long_name": long_name,
|
|
||||||
"short_name": short_name,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- config reads ---------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _section_container(node, section: str) -> tuple[Any, str]:
|
|
||||||
"""Return (container_message, parent_name) for a section name.
|
|
||||||
|
|
||||||
Parent is 'localConfig' or 'moduleConfig' so callers know where to call
|
|
||||||
writeConfig() after mutating.
|
|
||||||
"""
|
|
||||||
if section in LOCAL_CONFIG_SECTIONS:
|
|
||||||
return getattr(node.localConfig, section), "localConfig"
|
|
||||||
if section in MODULE_CONFIG_SECTIONS:
|
|
||||||
return getattr(node.moduleConfig, section), "moduleConfig"
|
|
||||||
raise AdminError(
|
|
||||||
f"Unknown config section: {section!r}. "
|
|
||||||
f"Valid sections: {sorted(LOCAL_CONFIG_SECTIONS | MODULE_CONFIG_SECTIONS)}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def get_config(section: str | None = None, port: str | None = None) -> dict[str, Any]:
|
|
||||||
"""Read one or all config sections.
|
|
||||||
|
|
||||||
`section` may be any name in LocalConfig (device, lora, position, power,
|
|
||||||
network, display, bluetooth, security) or LocalModuleConfig (mqtt, serial,
|
|
||||||
telemetry, ...). Omit `section` or pass `"all"` for everything.
|
|
||||||
"""
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
node = iface.localNode
|
|
||||||
if section in (None, "all"):
|
|
||||||
lc = _message_to_dict(node.localConfig)
|
|
||||||
mc = _message_to_dict(node.moduleConfig)
|
|
||||||
return {
|
|
||||||
"config": {
|
|
||||||
"localConfig": lc,
|
|
||||||
"moduleConfig": mc,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
container, _parent = _section_container(node, section)
|
|
||||||
return {"config": {section: _message_to_dict(container)}}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- config writes --------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _coerce_enum(field: pb_descriptor.FieldDescriptor, value: Any) -> int:
|
|
||||||
"""Accept an enum value as either its int or its string name."""
|
|
||||||
enum_type = field.enum_type
|
|
||||||
if isinstance(value, bool):
|
|
||||||
raise AdminError(f"{field.name}: expected enum {enum_type.name}, got bool")
|
|
||||||
if isinstance(value, int):
|
|
||||||
if enum_type.values_by_number.get(value) is None:
|
|
||||||
raise AdminError(
|
|
||||||
f"{field.name}: {value} is not a valid {enum_type.name} value"
|
|
||||||
)
|
|
||||||
return value
|
|
||||||
if isinstance(value, str):
|
|
||||||
upper = value.upper()
|
|
||||||
ev = enum_type.values_by_name.get(upper)
|
|
||||||
if ev is None:
|
|
||||||
valid = sorted(enum_type.values_by_name.keys())
|
|
||||||
raise AdminError(
|
|
||||||
f"{field.name}: {value!r} is not a valid {enum_type.name}. "
|
|
||||||
f"Valid: {valid}"
|
|
||||||
)
|
|
||||||
return ev.number
|
|
||||||
raise AdminError(
|
|
||||||
f"{field.name}: expected enum {enum_type.name}, got {type(value).__name__}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _coerce_scalar(field: pb_descriptor.FieldDescriptor, value: Any) -> Any:
|
|
||||||
t = field.type
|
|
||||||
FT = pb_descriptor.FieldDescriptor
|
|
||||||
if t == FT.TYPE_ENUM:
|
|
||||||
return _coerce_enum(field, value)
|
|
||||||
if t == FT.TYPE_BOOL:
|
|
||||||
if isinstance(value, bool):
|
|
||||||
return value
|
|
||||||
if isinstance(value, str):
|
|
||||||
return value.strip().lower() in ("true", "yes", "1", "on")
|
|
||||||
if isinstance(value, int):
|
|
||||||
return bool(value)
|
|
||||||
if t in (
|
|
||||||
FT.TYPE_INT32,
|
|
||||||
FT.TYPE_INT64,
|
|
||||||
FT.TYPE_UINT32,
|
|
||||||
FT.TYPE_UINT64,
|
|
||||||
FT.TYPE_SINT32,
|
|
||||||
FT.TYPE_SINT64,
|
|
||||||
FT.TYPE_FIXED32,
|
|
||||||
FT.TYPE_FIXED64,
|
|
||||||
):
|
|
||||||
return int(value)
|
|
||||||
if t in (FT.TYPE_FLOAT, FT.TYPE_DOUBLE):
|
|
||||||
return float(value)
|
|
||||||
if t == FT.TYPE_STRING:
|
|
||||||
return str(value)
|
|
||||||
if t == FT.TYPE_BYTES:
|
|
||||||
if isinstance(value, (bytes, bytearray)):
|
|
||||||
return bytes(value)
|
|
||||||
return str(value).encode("utf-8")
|
|
||||||
raise AdminError(
|
|
||||||
f"{field.name}: unsupported field type {t}. Use raw protobuf for this field."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _walk_to_field(
|
|
||||||
root_msg: Any, path_segments: list[str]
|
|
||||||
) -> tuple[Any, pb_descriptor.FieldDescriptor]:
|
|
||||||
"""Walk `root_msg` by field names until the leaf; return (parent_msg, leaf_field_descriptor)."""
|
|
||||||
msg = root_msg
|
|
||||||
for i, name in enumerate(path_segments):
|
|
||||||
desc = msg.DESCRIPTOR
|
|
||||||
field = desc.fields_by_name.get(name)
|
|
||||||
if field is None:
|
|
||||||
trail = ".".join(path_segments[:i] or ["<root>"])
|
|
||||||
valid = [f.name for f in desc.fields]
|
|
||||||
raise AdminError(f"No field {name!r} in {trail}. Valid: {valid}")
|
|
||||||
is_last = i == len(path_segments) - 1
|
|
||||||
if is_last:
|
|
||||||
return msg, field
|
|
||||||
if field.type != pb_descriptor.FieldDescriptor.TYPE_MESSAGE:
|
|
||||||
raise AdminError(
|
|
||||||
f"{'.'.join(path_segments[:i+1])} is a scalar; cannot descend into it"
|
|
||||||
)
|
|
||||||
msg = getattr(msg, name)
|
|
||||||
# path_segments was empty
|
|
||||||
raise AdminError("Empty config path")
|
|
||||||
|
|
||||||
|
|
||||||
def set_config(path: str, value: Any, port: str | None = None) -> dict[str, Any]:
|
|
||||||
"""Set a single config field by dot-path and write it to the device.
|
|
||||||
|
|
||||||
Examples:
|
|
||||||
set_config("lora.region", "US")
|
|
||||||
set_config("lora.modem_preset", "LONG_FAST")
|
|
||||||
set_config("device.role", "ROUTER")
|
|
||||||
set_config("mqtt.enabled", True)
|
|
||||||
set_config("mqtt.address", "mqtt.example.com")
|
|
||||||
|
|
||||||
"""
|
|
||||||
segments = [s for s in path.split(".") if s]
|
|
||||||
if not segments:
|
|
||||||
raise AdminError("path cannot be empty")
|
|
||||||
section = segments[0]
|
|
||||||
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
node = iface.localNode
|
|
||||||
container, parent_name = _section_container(node, section)
|
|
||||||
|
|
||||||
# Treat the section as the root; the rest of the path walks into it.
|
|
||||||
leaf_parent, field = _walk_to_field(container, segments[1:] or [])
|
|
||||||
# Use `is_repeated` (modern upb protobuf API) rather than the
|
|
||||||
# deprecated `label == LABEL_REPEATED` check - the C-extension
|
|
||||||
# FieldDescriptor in protobuf >= 5.x doesn't expose `.label` at
|
|
||||||
# all, and `is_repeated` is the supported replacement that works
|
|
||||||
# across both the pure-python and upb backends.
|
|
||||||
if field.is_repeated:
|
|
||||||
raise AdminError(
|
|
||||||
f"{path!r} is a repeated field; v1 only supports scalar sets. "
|
|
||||||
"Use the raw meshtastic CLI for now."
|
|
||||||
)
|
|
||||||
old_raw = getattr(leaf_parent, field.name)
|
|
||||||
coerced = _coerce_scalar(field, value)
|
|
||||||
try:
|
|
||||||
setattr(leaf_parent, field.name, coerced)
|
|
||||||
except (TypeError, ValueError) as exc:
|
|
||||||
raise AdminError(f"{path}: {exc}") from exc
|
|
||||||
|
|
||||||
node.writeConfig(section)
|
|
||||||
|
|
||||||
# Stringify enums for the response (so the caller can see the change in
|
|
||||||
# the same vocabulary they used to set it).
|
|
||||||
if field.type == pb_descriptor.FieldDescriptor.TYPE_ENUM:
|
|
||||||
try:
|
|
||||||
old_display = field.enum_type.values_by_number[old_raw].name
|
|
||||||
new_display = field.enum_type.values_by_number[coerced].name
|
|
||||||
except Exception:
|
|
||||||
old_display, new_display = old_raw, coerced
|
|
||||||
else:
|
|
||||||
old_display, new_display = old_raw, coerced
|
|
||||||
|
|
||||||
return {
|
|
||||||
"ok": True,
|
|
||||||
"path": path,
|
|
||||||
"section": section,
|
|
||||||
"parent": parent_name,
|
|
||||||
"old_value": old_display,
|
|
||||||
"new_value": new_display,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- channels -------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def get_channel_url(
|
|
||||||
include_all: bool = False, port: str | None = None
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
url = iface.localNode.getURL(includeAll=include_all)
|
|
||||||
return {"url": url}
|
|
||||||
|
|
||||||
|
|
||||||
def set_channel_url(url: str, port: str | None = None) -> dict[str, Any]:
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
# setURL replaces the channel set from the URL's contents. It does not
|
|
||||||
# return a count; we infer by counting non-DISABLED channels after.
|
|
||||||
iface.localNode.setURL(url)
|
|
||||||
channels = iface.localNode.channels or []
|
|
||||||
active = sum(1 for c in channels if getattr(c, "role", 0) != 0)
|
|
||||||
return {"ok": True, "channels_imported": active}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- messaging ------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def send_text(
|
|
||||||
text: str,
|
|
||||||
to: str | int | None = None,
|
|
||||||
channel_index: int = 0,
|
|
||||||
want_ack: bool = False,
|
|
||||||
port: str | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
destination = to if to is not None else "^all"
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
packet = iface.sendText(
|
|
||||||
text,
|
|
||||||
destinationId=destination,
|
|
||||||
wantAck=want_ack,
|
|
||||||
channelIndex=channel_index,
|
|
||||||
)
|
|
||||||
packet_id = getattr(packet, "id", None)
|
|
||||||
return {"ok": True, "packet_id": packet_id, "destination": destination}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- diagnostics ----------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def set_debug_log_api(enabled: bool, port: str | None = None) -> dict[str, Any]:
|
|
||||||
"""Toggle `config.security.debug_log_api_enabled` on the local node.
|
|
||||||
|
|
||||||
When enabled, firmware emits log lines as protobuf `LogRecord` messages
|
|
||||||
over the StreamAPI instead of raw text. meshtastic-python surfaces them
|
|
||||||
on pubsub topic `meshtastic.log.line`, which flows through the SAME
|
|
||||||
SerialInterface our tests already hold open - no `pio device monitor`
|
|
||||||
needed, no port-contention with admin/info calls.
|
|
||||||
|
|
||||||
Firmware gate: `src/SerialConsole.cpp` (`usingProtobufs &&
|
|
||||||
config.security.debug_log_api_enabled`). Setting persists in NVS; it
|
|
||||||
survives reboot. `factory_reset(full=False)` clears it unless it's
|
|
||||||
re-applied after reset.
|
|
||||||
|
|
||||||
Previously-documented concurrency hazard (emitLogRecord sharing the
|
|
||||||
main packet-emission buffers) has been fixed - see `StreamAPI.h`
|
|
||||||
where the log path now owns dedicated `fromRadioScratchLog` /
|
|
||||||
`txBufLog` buffers, and `StreamAPI::emitTxBuffer` +
|
|
||||||
`StreamAPI::emitLogRecord` both serialize their `stream->write`
|
|
||||||
calls via `streamLock`. Leaving the flag on under traffic is safe.
|
|
||||||
"""
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
sec = iface.localNode.localConfig.security
|
|
||||||
sec.debug_log_api_enabled = bool(enabled)
|
|
||||||
iface.localNode.writeConfig("security")
|
|
||||||
return {"ok": True, "debug_log_api_enabled": bool(enabled)}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- admin actions --------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def reboot(
|
|
||||||
port: str | None = None, confirm: bool = False, seconds: int = 10
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
_require_confirm(confirm, "reboot")
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
iface.localNode.reboot(secs=seconds)
|
|
||||||
return {"ok": True, "rebooting_in_s": seconds}
|
|
||||||
|
|
||||||
|
|
||||||
def shutdown(
|
|
||||||
port: str | None = None, confirm: bool = False, seconds: int = 10
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
_require_confirm(confirm, "shutdown")
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
iface.localNode.shutdown(secs=seconds)
|
|
||||||
return {"ok": True, "shutting_down_in_s": seconds}
|
|
||||||
|
|
||||||
|
|
||||||
def send_input_event(
|
|
||||||
event_code: int | str,
|
|
||||||
kb_char: int = 0,
|
|
||||||
touch_x: int = 0,
|
|
||||||
touch_y: int = 0,
|
|
||||||
port: str | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Inject an InputBroker event (button press / key / gesture) into the UI.
|
|
||||||
|
|
||||||
Wraps `AdminMessage.send_input_event` (handled in firmware at
|
|
||||||
src/modules/AdminModule.cpp::handleSendInputEvent). Local-only - no PKI
|
|
||||||
warmup needed since the admin message is addressed to `my_node_num`.
|
|
||||||
|
|
||||||
`event_code` accepts an int, a case-insensitive name
|
|
||||||
(`"RIGHT"` / `"input_broker_right"`), or an `InputEventCode`. The
|
|
||||||
firmware-side enum lives in src/input/InputBroker.h and is mirrored in
|
|
||||||
`meshtastic_mcp.input_events`.
|
|
||||||
"""
|
|
||||||
from meshtastic.protobuf import admin_pb2 # type: ignore[import-untyped]
|
|
||||||
|
|
||||||
from .input_events import coerce_event_code
|
|
||||||
|
|
||||||
code = coerce_event_code(event_code)
|
|
||||||
if not 0 <= kb_char <= 255:
|
|
||||||
raise ValueError(f"kb_char out of u8 range: {kb_char}")
|
|
||||||
if not 0 <= touch_x <= 65535:
|
|
||||||
raise ValueError(f"touch_x out of u16 range: {touch_x}")
|
|
||||||
if not 0 <= touch_y <= 65535:
|
|
||||||
raise ValueError(f"touch_y out of u16 range: {touch_y}")
|
|
||||||
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
msg = admin_pb2.AdminMessage()
|
|
||||||
msg.send_input_event.event_code = code
|
|
||||||
msg.send_input_event.kb_char = kb_char
|
|
||||||
msg.send_input_event.touch_x = touch_x
|
|
||||||
msg.send_input_event.touch_y = touch_y
|
|
||||||
iface.localNode._sendAdmin(msg)
|
|
||||||
return {"ok": True, "event_code": code, "kb_char": kb_char}
|
|
||||||
|
|
||||||
|
|
||||||
def factory_reset(
|
|
||||||
port: str | None = None, confirm: bool = False, full: bool = False
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Tell the node to factory-reset its config.
|
|
||||||
|
|
||||||
Works around a meshtastic-python 2.7.8 bug: `Node.factoryReset(full=True)`
|
|
||||||
internally does `p.factory_reset_config = True` where the field is
|
|
||||||
int32. protobuf 5.x rejects bool→int assignment as a TypeError. We build
|
|
||||||
the AdminMessage directly with int values (1=non-full, 2=full) and call
|
|
||||||
`_sendAdmin` to sidestep the SDK bug entirely.
|
|
||||||
"""
|
|
||||||
_require_confirm(confirm, "factory_reset")
|
|
||||||
from meshtastic.protobuf import admin_pb2 # type: ignore[import-untyped]
|
|
||||||
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
msg = admin_pb2.AdminMessage()
|
|
||||||
msg.factory_reset_config = 2 if full else 1
|
|
||||||
iface.localNode._sendAdmin(msg)
|
|
||||||
return {"ok": True, "full": full}
|
|
||||||
@@ -1,159 +0,0 @@
|
|||||||
"""Board / PlatformIO env enumeration.
|
|
||||||
|
|
||||||
Parses `pio project config --json-output` - a nested list of
|
|
||||||
`[section_name, [[key, value], ...]]` pairs - into a dict keyed by env name,
|
|
||||||
extracting the `custom_meshtastic_*` metadata the firmware variants expose.
|
|
||||||
|
|
||||||
The parsed config is cached and invalidated when `platformio.ini`'s mtime
|
|
||||||
changes, so subsequent calls don't pay the 1-2s pio startup cost.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import threading
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from . import config, pio
|
|
||||||
|
|
||||||
_CACHE_LOCK = threading.Lock()
|
|
||||||
_CACHE: dict[str, Any] = {"mtime": None, "envs": None}
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_bool(value: Any) -> bool:
|
|
||||||
if isinstance(value, bool):
|
|
||||||
return value
|
|
||||||
if isinstance(value, str):
|
|
||||||
return value.strip().lower() in ("true", "yes", "1", "on")
|
|
||||||
return bool(value)
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_int(value: Any) -> int | None:
|
|
||||||
try:
|
|
||||||
return int(value)
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_tags(value: Any) -> list[str]:
|
|
||||||
if value is None:
|
|
||||||
return []
|
|
||||||
if isinstance(value, list):
|
|
||||||
return [str(v).strip() for v in value if str(v).strip()]
|
|
||||||
return [t.strip() for t in str(value).replace(",", " ").split() if t.strip()]
|
|
||||||
|
|
||||||
|
|
||||||
def _env_record(env_name: str, items: list[list[Any]]) -> dict[str, Any]:
|
|
||||||
"""Build a normalized dict for one env section."""
|
|
||||||
d = dict(items)
|
|
||||||
return {
|
|
||||||
"env": env_name,
|
|
||||||
"architecture": d.get("custom_meshtastic_architecture"),
|
|
||||||
"hw_model": _parse_int(d.get("custom_meshtastic_hw_model")),
|
|
||||||
"hw_model_slug": d.get("custom_meshtastic_hw_model_slug"),
|
|
||||||
"display_name": d.get("custom_meshtastic_display_name"),
|
|
||||||
"actively_supported": _parse_bool(
|
|
||||||
d.get("custom_meshtastic_actively_supported")
|
|
||||||
),
|
|
||||||
"support_level": _parse_int(d.get("custom_meshtastic_support_level")),
|
|
||||||
"board_level": d.get("board_level"), # "pr", "extra", or None
|
|
||||||
"tags": _parse_tags(d.get("custom_meshtastic_tags")),
|
|
||||||
"images": _parse_tags(d.get("custom_meshtastic_images")),
|
|
||||||
"board": d.get("board"),
|
|
||||||
"upload_speed": _parse_int(d.get("upload_speed")),
|
|
||||||
"upload_protocol": d.get("upload_protocol"),
|
|
||||||
"monitor_speed": _parse_int(d.get("monitor_speed")),
|
|
||||||
"monitor_filters": d.get("monitor_filters") or [],
|
|
||||||
"_raw": d, # Full dict for get_board
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _load_all() -> dict[str, dict[str, Any]]:
|
|
||||||
"""Parse `pio project config` into `{env_name: record}`."""
|
|
||||||
raw = pio.run_json(["project", "config"], timeout=pio.TIMEOUT_PROJECT_CONFIG)
|
|
||||||
result: dict[str, dict[str, Any]] = {}
|
|
||||||
for section_name, items in raw:
|
|
||||||
if not isinstance(section_name, str) or not section_name.startswith("env:"):
|
|
||||||
continue
|
|
||||||
env_name = section_name.split(":", 1)[1]
|
|
||||||
result[env_name] = _env_record(env_name, items)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def _get_cached() -> dict[str, dict[str, Any]]:
|
|
||||||
root = config.firmware_root()
|
|
||||||
platformio_ini = root / "platformio.ini"
|
|
||||||
try:
|
|
||||||
mtime = platformio_ini.stat().st_mtime
|
|
||||||
except FileNotFoundError:
|
|
||||||
mtime = None
|
|
||||||
|
|
||||||
with _CACHE_LOCK:
|
|
||||||
if _CACHE["envs"] is not None and _CACHE["mtime"] == mtime:
|
|
||||||
return _CACHE["envs"]
|
|
||||||
envs = _load_all()
|
|
||||||
_CACHE["envs"] = envs
|
|
||||||
_CACHE["mtime"] = mtime
|
|
||||||
return envs
|
|
||||||
|
|
||||||
|
|
||||||
def invalidate_cache() -> None:
|
|
||||||
with _CACHE_LOCK:
|
|
||||||
_CACHE["envs"] = None
|
|
||||||
_CACHE["mtime"] = None
|
|
||||||
|
|
||||||
|
|
||||||
def _public_record(rec: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
"""Strip the `_raw` field for list outputs."""
|
|
||||||
return {k: v for k, v in rec.items() if not k.startswith("_")}
|
|
||||||
|
|
||||||
|
|
||||||
def list_boards(
|
|
||||||
architecture: str | None = None,
|
|
||||||
actively_supported_only: bool = False,
|
|
||||||
query: str | None = None,
|
|
||||||
board_level: str | None = None, # "release" | "pr" | "extra"
|
|
||||||
) -> list[dict[str, Any]]:
|
|
||||||
"""Enumerate PlatformIO envs with Meshtastic metadata.
|
|
||||||
|
|
||||||
Filters are cumulative (AND). `board_level="release"` means envs with no
|
|
||||||
explicit `board_level` set (the default release targets).
|
|
||||||
"""
|
|
||||||
envs = _get_cached()
|
|
||||||
q = query.lower().strip() if query else None
|
|
||||||
|
|
||||||
out = []
|
|
||||||
for rec in envs.values():
|
|
||||||
if architecture and rec.get("architecture") != architecture:
|
|
||||||
continue
|
|
||||||
if actively_supported_only and not rec.get("actively_supported"):
|
|
||||||
continue
|
|
||||||
if board_level is not None:
|
|
||||||
rec_level = rec.get("board_level")
|
|
||||||
if board_level == "release":
|
|
||||||
if rec_level not in (None, ""):
|
|
||||||
continue
|
|
||||||
elif rec_level != board_level:
|
|
||||||
continue
|
|
||||||
if q:
|
|
||||||
display = (rec.get("display_name") or "").lower()
|
|
||||||
env_name = rec.get("env", "").lower()
|
|
||||||
slug = (rec.get("hw_model_slug") or "").lower()
|
|
||||||
if q not in display and q not in env_name and q not in slug:
|
|
||||||
continue
|
|
||||||
out.append(_public_record(rec))
|
|
||||||
|
|
||||||
out.sort(key=lambda r: (r.get("architecture") or "", r.get("env")))
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def get_board(env: str) -> dict[str, Any]:
|
|
||||||
"""Full metadata for one env, including the raw pio config dict."""
|
|
||||||
envs = _get_cached()
|
|
||||||
rec = envs.get(env)
|
|
||||||
if rec is None:
|
|
||||||
raise KeyError(
|
|
||||||
f"Unknown env: {env!r}. Use list_boards() to see available envs."
|
|
||||||
)
|
|
||||||
public = _public_record(rec)
|
|
||||||
public["raw_config"] = rec["_raw"]
|
|
||||||
return public
|
|
||||||
@@ -1,286 +0,0 @@
|
|||||||
"""Cross-platform USB-webcam capture for UI tests + the `capture_screen` tool.
|
|
||||||
|
|
||||||
Backends:
|
|
||||||
- `opencv` - cv2.VideoCapture (AVFoundation on macOS, V4L2 on Linux).
|
|
||||||
- `ffmpeg` - subprocess shelling out to the system `ffmpeg` binary. Slower
|
|
||||||
per frame, but zero Python deps beyond stdlib.
|
|
||||||
- `null` - no-op stub returning a 1×1 black PNG. Used when no camera is
|
|
||||||
configured; keeps code paths alive without forcing every operator to
|
|
||||||
hook up hardware.
|
|
||||||
|
|
||||||
Environment variables (read at `get_camera()` call time):
|
|
||||||
- `MESHTASTIC_UI_CAMERA_BACKEND` - one of `opencv` / `ffmpeg` / `null` /
|
|
||||||
`auto` (default). `auto` picks opencv if `cv2` imports, else ffmpeg if
|
|
||||||
`ffmpeg --version` resolves, else null.
|
|
||||||
- `MESHTASTIC_UI_CAMERA_DEVICE` - generic default (index or path).
|
|
||||||
- `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` - per-role override, e.g.
|
|
||||||
`MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0` for the OLED-bearing heltec-v3.
|
|
||||||
Role suffix is uppercased before lookup.
|
|
||||||
|
|
||||||
Dependencies land in the optional `[ui]` extra; imports are lazy so clients
|
|
||||||
without `opencv-python-headless` installed can still import this module.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import io
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
import warnings
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Protocol
|
|
||||||
|
|
||||||
|
|
||||||
class CameraError(RuntimeError):
|
|
||||||
"""Raised when a camera backend fails to initialize or capture."""
|
|
||||||
|
|
||||||
|
|
||||||
class CameraBackend(Protocol):
|
|
||||||
name: str
|
|
||||||
|
|
||||||
def capture(self) -> bytes:
|
|
||||||
"""Return one PNG-encoded frame."""
|
|
||||||
...
|
|
||||||
|
|
||||||
def close(self) -> None: ...
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- OpenCV backend -------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class OpenCVBackend:
|
|
||||||
name = "opencv"
|
|
||||||
|
|
||||||
def __init__(self, device: int | str, warmup_frames: int = 5) -> None:
|
|
||||||
try:
|
|
||||||
import cv2 # type: ignore[import-untyped] # noqa: PLC0415
|
|
||||||
except ImportError as exc:
|
|
||||||
raise CameraError(
|
|
||||||
"opencv backend requested but `cv2` is not installed. "
|
|
||||||
"Install the mcp-server [ui] extra: pip install -e '.[ui]'"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
self._cv2 = cv2
|
|
||||||
device_arg: int | str
|
|
||||||
if isinstance(device, str) and device.isdigit():
|
|
||||||
device_arg = int(device)
|
|
||||||
else:
|
|
||||||
device_arg = device
|
|
||||||
self._cap = cv2.VideoCapture(device_arg)
|
|
||||||
if not self._cap.isOpened():
|
|
||||||
raise CameraError(
|
|
||||||
f"cv2.VideoCapture({device_arg!r}) failed to open. "
|
|
||||||
"On macOS check TCC Camera permission; on Linux check /dev/video* and v4l2 access."
|
|
||||||
)
|
|
||||||
|
|
||||||
# Drop the first few frames - auto-exposure + white-balance settle.
|
|
||||||
for _ in range(warmup_frames):
|
|
||||||
self._cap.read()
|
|
||||||
# Detect a stuck black-frame camera early rather than silently
|
|
||||||
# producing all-black captures.
|
|
||||||
ok, frame = self._cap.read()
|
|
||||||
if not ok or frame is None:
|
|
||||||
self._cap.release()
|
|
||||||
raise CameraError(f"camera {device_arg!r} opened but returned no frames")
|
|
||||||
|
|
||||||
def capture(self) -> bytes:
|
|
||||||
cv2 = self._cv2
|
|
||||||
ok, frame = self._cap.read()
|
|
||||||
if not ok or frame is None:
|
|
||||||
raise CameraError("cv2.VideoCapture.read() returned no frame")
|
|
||||||
success, buf = cv2.imencode(".png", frame)
|
|
||||||
if not success:
|
|
||||||
raise CameraError("cv2.imencode('.png', ...) failed")
|
|
||||||
return bytes(buf)
|
|
||||||
|
|
||||||
def close(self) -> None:
|
|
||||||
try:
|
|
||||||
self._cap.release()
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- ffmpeg subprocess backend --------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class FfmpegBackend:
|
|
||||||
name = "ffmpeg"
|
|
||||||
|
|
||||||
def __init__(self, device: int | str) -> None:
|
|
||||||
if shutil.which("ffmpeg") is None:
|
|
||||||
raise CameraError("ffmpeg backend requested but `ffmpeg` is not on PATH")
|
|
||||||
|
|
||||||
self._device = str(device)
|
|
||||||
# Platform-specific -f flag:
|
|
||||||
# macOS → avfoundation (index like "0")
|
|
||||||
# Linux → v4l2 (device like "/dev/video0" or "0")
|
|
||||||
if sys.platform == "darwin":
|
|
||||||
self._input_format = "avfoundation"
|
|
||||||
self._input_spec = self._device # bare index for avfoundation
|
|
||||||
else:
|
|
||||||
self._input_format = "v4l2"
|
|
||||||
self._input_spec = (
|
|
||||||
self._device
|
|
||||||
if self._device.startswith("/dev/")
|
|
||||||
else f"/dev/video{self._device}"
|
|
||||||
)
|
|
||||||
|
|
||||||
def capture(self) -> bytes:
|
|
||||||
cmd = [
|
|
||||||
"ffmpeg",
|
|
||||||
"-hide_banner",
|
|
||||||
"-loglevel",
|
|
||||||
"error",
|
|
||||||
"-f",
|
|
||||||
self._input_format,
|
|
||||||
"-i",
|
|
||||||
self._input_spec,
|
|
||||||
"-frames:v",
|
|
||||||
"1",
|
|
||||||
"-f",
|
|
||||||
"image2pipe",
|
|
||||||
"-vcodec",
|
|
||||||
"png",
|
|
||||||
"-",
|
|
||||||
]
|
|
||||||
try:
|
|
||||||
out = subprocess.run(
|
|
||||||
cmd, capture_output=True, check=True, timeout=15 # noqa: S603
|
|
||||||
)
|
|
||||||
except subprocess.CalledProcessError as exc:
|
|
||||||
raise CameraError(
|
|
||||||
f"ffmpeg capture failed (rc={exc.returncode}): {exc.stderr.decode(errors='replace')[:200]}"
|
|
||||||
) from exc
|
|
||||||
except subprocess.TimeoutExpired as exc:
|
|
||||||
raise CameraError("ffmpeg capture timed out after 15s") from exc
|
|
||||||
return out.stdout
|
|
||||||
|
|
||||||
def close(self) -> None:
|
|
||||||
pass # stateless - each capture spawns a new process
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Null backend ---------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
# A tiny valid 1×1 transparent PNG so callers always get a decodable image.
|
|
||||||
_BLACK_1X1_PNG = bytes.fromhex(
|
|
||||||
"89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c489"
|
|
||||||
"0000000d49444154789c6300010000000500010d0a2db40000000049454e44ae426082"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class NullBackend:
|
|
||||||
name = "null"
|
|
||||||
|
|
||||||
def capture(self) -> bytes:
|
|
||||||
return _BLACK_1X1_PNG
|
|
||||||
|
|
||||||
def close(self) -> None:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Factory --------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_device(role: str | None) -> str | None:
|
|
||||||
if role:
|
|
||||||
specific = os.environ.get(f"MESHTASTIC_UI_CAMERA_DEVICE_{role.upper()}")
|
|
||||||
if specific:
|
|
||||||
return specific
|
|
||||||
return os.environ.get("MESHTASTIC_UI_CAMERA_DEVICE")
|
|
||||||
|
|
||||||
|
|
||||||
def get_camera(role: str | None = None) -> CameraBackend:
|
|
||||||
"""Return a CameraBackend for the given device role (e.g. `"esp32s3"`).
|
|
||||||
|
|
||||||
Falls back to `NullBackend` if no camera is configured or the selected
|
|
||||||
backend fails to init - tests should treat captures as best-effort
|
|
||||||
evidence, not a blocker.
|
|
||||||
"""
|
|
||||||
backend = os.environ.get("MESHTASTIC_UI_CAMERA_BACKEND", "auto").lower()
|
|
||||||
device = _resolve_device(role)
|
|
||||||
|
|
||||||
if backend in ("null", "none") or device is None:
|
|
||||||
return NullBackend()
|
|
||||||
|
|
||||||
if backend == "auto":
|
|
||||||
# Prefer opencv if importable; fall back to ffmpeg; else null.
|
|
||||||
try:
|
|
||||||
import cv2 # type: ignore[import-untyped] # noqa: F401,PLC0415
|
|
||||||
|
|
||||||
backend = "opencv"
|
|
||||||
except ImportError:
|
|
||||||
backend = "ffmpeg" if shutil.which("ffmpeg") else "null"
|
|
||||||
|
|
||||||
if backend == "opencv":
|
|
||||||
try:
|
|
||||||
return OpenCVBackend(device)
|
|
||||||
except CameraError as exc:
|
|
||||||
warnings.warn(
|
|
||||||
f"camera backend {backend!r} failed to initialize for device "
|
|
||||||
f"{device!r}: {exc}; falling back to null backend",
|
|
||||||
RuntimeWarning,
|
|
||||||
stacklevel=2,
|
|
||||||
)
|
|
||||||
return NullBackend()
|
|
||||||
if backend == "ffmpeg":
|
|
||||||
try:
|
|
||||||
return FfmpegBackend(device)
|
|
||||||
except CameraError as exc:
|
|
||||||
warnings.warn(
|
|
||||||
f"camera backend {backend!r} failed to initialize for device "
|
|
||||||
f"{device!r}: {exc}; falling back to null backend",
|
|
||||||
RuntimeWarning,
|
|
||||||
stacklevel=2,
|
|
||||||
)
|
|
||||||
return NullBackend()
|
|
||||||
if backend == "null":
|
|
||||||
return NullBackend()
|
|
||||||
|
|
||||||
raise CameraError(f"unknown MESHTASTIC_UI_CAMERA_BACKEND: {backend!r}")
|
|
||||||
|
|
||||||
|
|
||||||
def save_capture(png_bytes: bytes, path: Path) -> None:
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
path.write_bytes(png_bytes)
|
|
||||||
|
|
||||||
|
|
||||||
def capture_to_file(role: str | None, path: Path) -> dict[str, object]:
|
|
||||||
"""One-shot: open camera, capture, write PNG, close. Returns metadata."""
|
|
||||||
started = time.monotonic()
|
|
||||||
cam = get_camera(role)
|
|
||||||
try:
|
|
||||||
data = cam.capture()
|
|
||||||
finally:
|
|
||||||
cam.close()
|
|
||||||
save_capture(data, path)
|
|
||||||
return {
|
|
||||||
"backend": cam.name,
|
|
||||||
"path": str(path),
|
|
||||||
"bytes": len(data),
|
|
||||||
"elapsed_s": round(time.monotonic() - started, 3),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _is_png(data: bytes) -> bool:
|
|
||||||
return data.startswith(b"\x89PNG\r\n\x1a\n")
|
|
||||||
|
|
||||||
|
|
||||||
# Exposed so callers can sanity-check a capture without a full PIL import.
|
|
||||||
__all__ = [
|
|
||||||
"CameraBackend",
|
|
||||||
"CameraError",
|
|
||||||
"FfmpegBackend",
|
|
||||||
"NullBackend",
|
|
||||||
"OpenCVBackend",
|
|
||||||
"capture_to_file",
|
|
||||||
"get_camera",
|
|
||||||
"save_capture",
|
|
||||||
]
|
|
||||||
|
|
||||||
# Keep `io` import used (pyflakes is picky) via a small guard used at import
|
|
||||||
# time to normalize stdin/stdout if a subclass ever needs it.
|
|
||||||
_ = io.BytesIO # noqa: SLF001
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
"""Command-line entry points that sit alongside the MCP server.
|
|
||||||
|
|
||||||
Modules here are loaded on-demand by `[project.scripts]` entries in
|
|
||||||
`pyproject.toml`. They are NOT imported by `meshtastic_mcp.server` or the
|
|
||||||
admin/info tool surface - the MCP server stays pure stdio JSON-RPC.
|
|
||||||
"""
|
|
||||||
@@ -1,73 +0,0 @@
|
|||||||
"""Flash progress log tailer for ``meshtastic-mcp-test-tui``.
|
|
||||||
|
|
||||||
``pio.py`` / ``hw_tools.py`` tee subprocess output (``pio run -t upload``,
|
|
||||||
``esptool erase_flash``, ``nrfutil dfu``, etc.) to ``tests/flash.log``
|
|
||||||
line-by-line as it arrives - controlled by the ``MESHTASTIC_MCP_FLASH_LOG``
|
|
||||||
env var that ``run-tests.sh`` sets. The TUI tails that file so the operator
|
|
||||||
sees live flash progress in the pytest pane instead of 3 minutes of silence
|
|
||||||
during ``test_00_bake``.
|
|
||||||
|
|
||||||
Separate from ``_fwlog.py`` because that one parses JSONL, this one
|
|
||||||
streams plain text lines. Same daemon-thread + EOF-backoff structure.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import pathlib
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from typing import Callable
|
|
||||||
|
|
||||||
|
|
||||||
class FlashLogTailer(threading.Thread):
|
|
||||||
"""Tail a plain-text log file, publish each stripped line via ``post``.
|
|
||||||
|
|
||||||
``post`` is invoked with a single ``str`` for every new line. Lines are
|
|
||||||
stripped of trailing newlines; empty lines after stripping are dropped.
|
|
||||||
|
|
||||||
The file may not exist yet when this thread starts - it's truncated by
|
|
||||||
``run-tests.sh`` at session start, but if the tailer races the shell,
|
|
||||||
we tolerate FileNotFoundError for up to ``wait_s`` seconds.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
path: pathlib.Path,
|
|
||||||
post: Callable[[str], None],
|
|
||||||
stop: threading.Event,
|
|
||||||
*,
|
|
||||||
wait_s: float = 30.0,
|
|
||||||
) -> None:
|
|
||||||
super().__init__(daemon=True, name="flashlog-tail")
|
|
||||||
self._path = path
|
|
||||||
self._post = post
|
|
||||||
self._stop = stop
|
|
||||||
self._wait_s = wait_s
|
|
||||||
|
|
||||||
def run(self) -> None:
|
|
||||||
deadline = time.monotonic() + self._wait_s
|
|
||||||
while not self._path.is_file():
|
|
||||||
if self._stop.is_set() or time.monotonic() > deadline:
|
|
||||||
return
|
|
||||||
time.sleep(0.1)
|
|
||||||
try:
|
|
||||||
fh = self._path.open("r", encoding="utf-8", errors="replace")
|
|
||||||
except OSError:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
while not self._stop.is_set():
|
|
||||||
line = fh.readline()
|
|
||||||
if not line:
|
|
||||||
time.sleep(0.05)
|
|
||||||
continue
|
|
||||||
line = line.rstrip("\r\n")
|
|
||||||
if not line:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
self._post(line)
|
|
||||||
except Exception:
|
|
||||||
# A post failure (e.g. closed app) is terminal for this
|
|
||||||
# thread but we still want to close the file handle.
|
|
||||||
return
|
|
||||||
finally:
|
|
||||||
fh.close()
|
|
||||||
@@ -1,96 +0,0 @@
|
|||||||
"""Firmware log tail worker for ``meshtastic-mcp-test-tui``.
|
|
||||||
|
|
||||||
Complements v1's reportlog-tail worker. ``tests/conftest.py`` owns a
|
|
||||||
session-scoped autouse fixture (``_firmware_log_stream``) that mirrors
|
|
||||||
every ``meshtastic.log.line`` pubsub event to ``tests/fwlog.jsonl`` -
|
|
||||||
one JSON object per line:
|
|
||||||
|
|
||||||
{"ts": 1729100000.123, "port": "/dev/cu.usbmodem1101", "line": "..."}
|
|
||||||
|
|
||||||
The TUI tails that file from a worker thread; each new line becomes a
|
|
||||||
:class:`FirmwareLogLine` message posted to the App. Same pattern as the
|
|
||||||
reportlog tail worker - truncate on launch, tolerate missing file for
|
|
||||||
30 s, back off at EOF.
|
|
||||||
|
|
||||||
Kept in its own module so the (large) ``test_tui.py`` stays focused on
|
|
||||||
the Textual App shell.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import pathlib
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from typing import Any, Callable
|
|
||||||
|
|
||||||
|
|
||||||
class FirmwareLogTailer(threading.Thread):
|
|
||||||
"""Tail ``tests/fwlog.jsonl``, publish parsed records via ``post``.
|
|
||||||
|
|
||||||
``post`` is the App's ``post_message`` (or any callable that accepts a
|
|
||||||
single payload arg). We pass parsed dicts rather than constructing
|
|
||||||
Textual Message objects here - keeps this module free of the
|
|
||||||
textual dependency so it's unit-testable in a bare venv.
|
|
||||||
|
|
||||||
Parameters
|
|
||||||
----------
|
|
||||||
path:
|
|
||||||
Path to ``tests/fwlog.jsonl``. The file may not exist yet at
|
|
||||||
startup - pytest only creates it once the session fixture runs.
|
|
||||||
post:
|
|
||||||
Callable invoked with a dict ``{"ts", "port", "line"}`` for every
|
|
||||||
new line parsed from the file.
|
|
||||||
stop:
|
|
||||||
An event the App sets to signal shutdown.
|
|
||||||
wait_s:
|
|
||||||
How long to poll for the file's creation before giving up. Default
|
|
||||||
30 s; pytest collection on a cold cache can be slow.
|
|
||||||
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
path: pathlib.Path,
|
|
||||||
post: Callable[[dict[str, Any]], None],
|
|
||||||
stop: threading.Event,
|
|
||||||
*,
|
|
||||||
wait_s: float = 30.0,
|
|
||||||
) -> None:
|
|
||||||
super().__init__(daemon=True, name="fwlog-tail")
|
|
||||||
self._path = path
|
|
||||||
self._post = post
|
|
||||||
self._stop = stop
|
|
||||||
self._wait_s = wait_s
|
|
||||||
|
|
||||||
def run(self) -> None:
|
|
||||||
deadline = time.monotonic() + self._wait_s
|
|
||||||
while not self._path.is_file():
|
|
||||||
if self._stop.is_set() or time.monotonic() > deadline:
|
|
||||||
return
|
|
||||||
time.sleep(0.1)
|
|
||||||
try:
|
|
||||||
fh = self._path.open("r", encoding="utf-8")
|
|
||||||
except OSError:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
while not self._stop.is_set():
|
|
||||||
line = fh.readline()
|
|
||||||
if not line:
|
|
||||||
time.sleep(0.05)
|
|
||||||
continue
|
|
||||||
line = line.strip()
|
|
||||||
if not line:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
record = json.loads(line)
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
continue
|
|
||||||
# Defensive: require the three fields we rely on.
|
|
||||||
if not isinstance(record, dict):
|
|
||||||
continue
|
|
||||||
if "line" not in record:
|
|
||||||
continue
|
|
||||||
self._post(record)
|
|
||||||
finally:
|
|
||||||
fh.close()
|
|
||||||
@@ -1,127 +0,0 @@
|
|||||||
"""Cross-run history for ``meshtastic-mcp-test-tui``.
|
|
||||||
|
|
||||||
Persists one JSON object per pytest run to
|
|
||||||
``mcp-server/tests/.history/runs.jsonl``. The TUI reads the last N
|
|
||||||
entries on launch to render a duration sparkline in the header - a
|
|
||||||
quick read on whether the suite is slowing down over time.
|
|
||||||
|
|
||||||
Schema (keep small; the file can grow for months):
|
|
||||||
|
|
||||||
{"run": 42, "ts": 1729100000.0, "duration_s": 387.2,
|
|
||||||
"passed": 52, "failed": 0, "skipped": 23, "exit_code": 0,
|
|
||||||
"seed": "mcp-user-host"}
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import pathlib
|
|
||||||
import time
|
|
||||||
from dataclasses import asdict, dataclass
|
|
||||||
from typing import Iterable
|
|
||||||
|
|
||||||
# Sparkline glyphs, low → high. 8 levels is the Unicode convention.
|
|
||||||
_SPARK_BLOCKS = "▁▂▃▄▅▆▇█"
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class RunRecord:
|
|
||||||
run: int
|
|
||||||
ts: float
|
|
||||||
duration_s: float
|
|
||||||
passed: int
|
|
||||||
failed: int
|
|
||||||
skipped: int
|
|
||||||
exit_code: int
|
|
||||||
seed: str
|
|
||||||
|
|
||||||
|
|
||||||
class HistoryStore:
|
|
||||||
"""Append-only JSONL store with bounded read.
|
|
||||||
|
|
||||||
Writes are fsynced after each append (the file is tiny; fsync cost
|
|
||||||
is negligible and protects against truncation on a crash).
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, path: pathlib.Path, *, keep_last: int = 50) -> None:
|
|
||||||
self._path = path
|
|
||||||
self._keep_last = keep_last
|
|
||||||
|
|
||||||
def append(self, record: RunRecord) -> None:
|
|
||||||
try:
|
|
||||||
self._path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
with self._path.open("a", encoding="utf-8") as fh:
|
|
||||||
fh.write(json.dumps(asdict(record)) + "\n")
|
|
||||||
fh.flush()
|
|
||||||
except Exception:
|
|
||||||
# Non-fatal: history is cosmetic.
|
|
||||||
pass
|
|
||||||
|
|
||||||
def read_recent(self) -> list[RunRecord]:
|
|
||||||
"""Return the last ``keep_last`` records in chronological order."""
|
|
||||||
if not self._path.is_file():
|
|
||||||
return []
|
|
||||||
try:
|
|
||||||
lines = self._path.read_text(encoding="utf-8").splitlines()
|
|
||||||
except OSError:
|
|
||||||
return []
|
|
||||||
out: list[RunRecord] = []
|
|
||||||
# Parse tail-first so we don't waste work on a huge history.
|
|
||||||
for line in lines[-self._keep_last :]:
|
|
||||||
line = line.strip()
|
|
||||||
if not line:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
raw = json.loads(line)
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
out.append(RunRecord(**raw))
|
|
||||||
except TypeError:
|
|
||||||
# Schema drift; skip the record rather than crash.
|
|
||||||
continue
|
|
||||||
return out
|
|
||||||
|
|
||||||
def record_run(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
run: int,
|
|
||||||
duration_s: float,
|
|
||||||
passed: int,
|
|
||||||
failed: int,
|
|
||||||
skipped: int,
|
|
||||||
exit_code: int,
|
|
||||||
seed: str,
|
|
||||||
) -> RunRecord:
|
|
||||||
rec = RunRecord(
|
|
||||||
run=run,
|
|
||||||
ts=time.time(),
|
|
||||||
duration_s=float(duration_s),
|
|
||||||
passed=int(passed),
|
|
||||||
failed=int(failed),
|
|
||||||
skipped=int(skipped),
|
|
||||||
exit_code=int(exit_code),
|
|
||||||
seed=seed,
|
|
||||||
)
|
|
||||||
self.append(rec)
|
|
||||||
return rec
|
|
||||||
|
|
||||||
|
|
||||||
def sparkline(values: Iterable[float], *, width: int = 20) -> str:
|
|
||||||
"""Render a Unicode block-character sparkline from the last ``width`` values.
|
|
||||||
|
|
||||||
Returns an empty string for empty input so the header handles
|
|
||||||
"no history yet" gracefully.
|
|
||||||
"""
|
|
||||||
buf = [v for v in values if v >= 0][-width:]
|
|
||||||
if not buf:
|
|
||||||
return ""
|
|
||||||
lo, hi = min(buf), max(buf)
|
|
||||||
if hi - lo < 1e-9:
|
|
||||||
return _SPARK_BLOCKS[len(_SPARK_BLOCKS) // 2] * len(buf)
|
|
||||||
n = len(_SPARK_BLOCKS) - 1
|
|
||||||
out = []
|
|
||||||
for v in buf:
|
|
||||||
idx = int(round((v - lo) / (hi - lo) * n))
|
|
||||||
out.append(_SPARK_BLOCKS[max(0, min(n, idx))])
|
|
||||||
return "".join(out)
|
|
||||||
@@ -1,214 +0,0 @@
|
|||||||
"""Reproducer bundle builder for ``meshtastic-mcp-test-tui``.
|
|
||||||
|
|
||||||
When the operator presses ``x`` on a failed test leaf, we package the
|
|
||||||
minimum viable failure context into a tarball under
|
|
||||||
``mcp-server/tests/reproducers/``:
|
|
||||||
|
|
||||||
::
|
|
||||||
|
|
||||||
repro-<ts>-<short_nodeid>.tar.gz
|
|
||||||
├── README.md human-readable overview
|
|
||||||
├── test_report.json the failing TestReport event from reportlog
|
|
||||||
├── fwlog.jsonl firmware log filtered to the failure window
|
|
||||||
├── devices.json per-device device_info + lora config snapshot
|
|
||||||
└── env.json seed, run #, pytest version, platform, hostname
|
|
||||||
|
|
||||||
Separate module so the logic can be unit-tested without Textual. The
|
|
||||||
TUI glue is thin - one key binding calls :func:`build_reproducer_bundle`
|
|
||||||
with the focused test's state and shows the path in a modal.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import io
|
|
||||||
import json
|
|
||||||
import pathlib
|
|
||||||
import platform
|
|
||||||
import re
|
|
||||||
import socket
|
|
||||||
import tarfile
|
|
||||||
import time
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Any, Iterable
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class ReproContext:
|
|
||||||
"""Everything :func:`build_reproducer_bundle` needs. Shaped to map
|
|
||||||
cleanly onto the state the TUI already tracks - no extra data
|
|
||||||
collection required at export time."""
|
|
||||||
|
|
||||||
nodeid: str
|
|
||||||
longrepr: str
|
|
||||||
sections: list[tuple[str, str]]
|
|
||||||
start_ts: float | None
|
|
||||||
stop_ts: float | None
|
|
||||||
seed: str
|
|
||||||
run_number: int
|
|
||||||
exit_code: int | None
|
|
||||||
fwlog_path: pathlib.Path
|
|
||||||
output_dir: pathlib.Path
|
|
||||||
extra_device_rows: list[dict[str, Any]] # [{role, port, info, ...}, ...]
|
|
||||||
|
|
||||||
|
|
||||||
def _short_nodeid(nodeid: str) -> str:
|
|
||||||
"""Collapse a pytest nodeid into a filename-safe slug (<= 60 chars)."""
|
|
||||||
# Drop the file path prefix; keep test name + parametrization.
|
|
||||||
tail = nodeid.split("::", 1)[-1] if "::" in nodeid else nodeid
|
|
||||||
slug = re.sub(r"[^A-Za-z0-9_.\-]", "_", tail)
|
|
||||||
return slug[:60].strip("_.-") or "test"
|
|
||||||
|
|
||||||
|
|
||||||
def _filtered_fwlog(
|
|
||||||
fwlog_path: pathlib.Path,
|
|
||||||
start_ts: float | None,
|
|
||||||
stop_ts: float | None,
|
|
||||||
*,
|
|
||||||
pad_s: float = 5.0,
|
|
||||||
) -> bytes:
|
|
||||||
"""Return fwlog.jsonl lines whose ``ts`` lies in [start-pad, stop+pad]."""
|
|
||||||
if not fwlog_path.is_file():
|
|
||||||
return b""
|
|
||||||
if start_ts is None or stop_ts is None:
|
|
||||||
# Without a time window, include the whole file - rare; happens
|
|
||||||
# when a test fails in setup before pytest emitted a start ts.
|
|
||||||
try:
|
|
||||||
return fwlog_path.read_bytes()
|
|
||||||
except OSError:
|
|
||||||
return b""
|
|
||||||
lo, hi = start_ts - pad_s, stop_ts + pad_s
|
|
||||||
out = io.BytesIO()
|
|
||||||
try:
|
|
||||||
with fwlog_path.open("r", encoding="utf-8") as fh:
|
|
||||||
for line in fh:
|
|
||||||
stripped = line.strip()
|
|
||||||
if not stripped:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
record = json.loads(stripped)
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
continue
|
|
||||||
ts = record.get("ts")
|
|
||||||
if not isinstance(ts, (int, float)):
|
|
||||||
continue
|
|
||||||
if lo <= ts <= hi:
|
|
||||||
out.write(line.encode("utf-8"))
|
|
||||||
except OSError:
|
|
||||||
return b""
|
|
||||||
return out.getvalue()
|
|
||||||
|
|
||||||
|
|
||||||
def _readme(ctx: ReproContext) -> str:
|
|
||||||
t = time.strftime("%Y-%m-%d %H:%M:%S %Z", time.localtime())
|
|
||||||
return f"""# Reproducer bundle
|
|
||||||
|
|
||||||
Exported by `meshtastic-mcp-test-tui` on {t}.
|
|
||||||
|
|
||||||
## Failing test
|
|
||||||
|
|
||||||
- **nodeid:** `{ctx.nodeid}`
|
|
||||||
- **seed:** `{ctx.seed}`
|
|
||||||
- **run #:** {ctx.run_number}
|
|
||||||
- **suite exit code (at export time):** {ctx.exit_code if ctx.exit_code is not None else "in progress"}
|
|
||||||
|
|
||||||
## Files in this archive
|
|
||||||
|
|
||||||
| File | Contents |
|
|
||||||
|---|---|
|
|
||||||
| `test_report.json` | The pytest-reportlog `TestReport` event for the failing test - includes `longrepr`, captured `sections` (stdout/stderr/log), `duration`, `location`, `keywords`. |
|
|
||||||
| `fwlog.jsonl` | Firmware log lines (from `meshtastic.log.line` pubsub) filtered to [start−5s, stop+5s] around the test's run window. Each line is `{{ts, port, line}}`. |
|
|
||||||
| `devices.json` | Per-device snapshot at export time: `device_info` + `lora` config per detected role. |
|
|
||||||
| `env.json` | Python version, platform, hostname, seed, run number. |
|
|
||||||
|
|
||||||
## How to triage
|
|
||||||
|
|
||||||
1. Open `test_report.json` and read `longrepr` + `sections` - most failures explain themselves there.
|
|
||||||
2. If the failure is a mesh/telemetry assertion, `fwlog.jsonl` is where the answer usually lives. Grep for `Error=`, `NAK`, `PKI_UNKNOWN_PUBKEY`, `Skip send`, `Guru Meditation`, or the uptime timestamps around the assertion event.
|
|
||||||
3. Compare `devices.json` against the expected state (e.g. `num_nodes >= 2`, `primary_channel == "McpTest"`, `region == "US"`). If fields disagree with the seed-derived USERPREFS profile, the device probably wasn't baked with this session's profile.
|
|
||||||
|
|
||||||
## Reproducing locally
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd mcp-server
|
|
||||||
MESHTASTIC_MCP_SEED='{ctx.seed}' .venv/bin/pytest '{ctx.nodeid}' --tb=long -v
|
|
||||||
```
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
def build_reproducer_bundle(ctx: ReproContext) -> pathlib.Path:
|
|
||||||
"""Build a tarball under ``ctx.output_dir`` and return its path.
|
|
||||||
|
|
||||||
Parent dirs are created as needed. Errors during optional sections
|
|
||||||
(devices, env) are swallowed - the bundle is still useful without
|
|
||||||
them; refusing to export because the device poller had a hiccup
|
|
||||||
would be worse than the export missing a file.
|
|
||||||
"""
|
|
||||||
ctx.output_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
ts = int(time.time())
|
|
||||||
slug = _short_nodeid(ctx.nodeid)
|
|
||||||
archive_path = ctx.output_dir / f"repro-{ts}-{slug}.tar.gz"
|
|
||||||
|
|
||||||
with tarfile.open(archive_path, "w:gz") as tar:
|
|
||||||
|
|
||||||
def _add(name: str, data: bytes) -> None:
|
|
||||||
info = tarfile.TarInfo(name=name)
|
|
||||||
info.size = len(data)
|
|
||||||
info.mtime = ts
|
|
||||||
tar.addfile(info, io.BytesIO(data))
|
|
||||||
|
|
||||||
# README
|
|
||||||
_add("README.md", _readme(ctx).encode("utf-8"))
|
|
||||||
|
|
||||||
# test_report.json - reconstruct from the fields the TUI stashes.
|
|
||||||
test_report = {
|
|
||||||
"nodeid": ctx.nodeid,
|
|
||||||
"outcome": "failed",
|
|
||||||
"longrepr": ctx.longrepr,
|
|
||||||
"sections": [list(s) for s in ctx.sections],
|
|
||||||
"start": ctx.start_ts,
|
|
||||||
"stop": ctx.stop_ts,
|
|
||||||
}
|
|
||||||
_add(
|
|
||||||
"test_report.json",
|
|
||||||
json.dumps(test_report, indent=2, default=str).encode("utf-8"),
|
|
||||||
)
|
|
||||||
|
|
||||||
# fwlog.jsonl (filtered)
|
|
||||||
_add("fwlog.jsonl", _filtered_fwlog(ctx.fwlog_path, ctx.start_ts, ctx.stop_ts))
|
|
||||||
|
|
||||||
# devices.json
|
|
||||||
try:
|
|
||||||
devices_payload = json.dumps(
|
|
||||||
ctx.extra_device_rows or [], indent=2, default=str
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
devices_payload = "[]"
|
|
||||||
_add("devices.json", devices_payload.encode("utf-8"))
|
|
||||||
|
|
||||||
# env.json
|
|
||||||
try:
|
|
||||||
from importlib.metadata import version as _pkg_version
|
|
||||||
|
|
||||||
pytest_version = _pkg_version("pytest")
|
|
||||||
except Exception:
|
|
||||||
pytest_version = "unknown"
|
|
||||||
env_payload = {
|
|
||||||
"seed": ctx.seed,
|
|
||||||
"run": ctx.run_number,
|
|
||||||
"exit_code": ctx.exit_code,
|
|
||||||
"export_ts": ts,
|
|
||||||
"python": platform.python_version(),
|
|
||||||
"pytest": pytest_version,
|
|
||||||
"platform": f"{platform.system()} {platform.release()} {platform.machine()}",
|
|
||||||
"hostname": socket.gethostname(),
|
|
||||||
}
|
|
||||||
_add("env.json", json.dumps(env_payload, indent=2).encode("utf-8"))
|
|
||||||
|
|
||||||
return archive_path
|
|
||||||
|
|
||||||
|
|
||||||
def iter_entries(archive_path: pathlib.Path) -> Iterable[str]:
|
|
||||||
"""Yield member names - used by callers that want to confirm the bundle shape."""
|
|
||||||
with tarfile.open(archive_path, "r:gz") as tar:
|
|
||||||
for m in tar.getmembers():
|
|
||||||
yield m.name
|
|
||||||
@@ -1,83 +0,0 @@
|
|||||||
"""UI-capture transcript tailer for ``meshtastic-mcp-test-tui``.
|
|
||||||
|
|
||||||
Watches ``tests/ui_captures/<session_seed>/`` for new transcript lines
|
|
||||||
(one per ``frame_capture()`` call from the UI tier) and posts them to
|
|
||||||
the TUI. Enabled by ``MESHTASTIC_UI_TUI_CAMERA=1``.
|
|
||||||
|
|
||||||
Design mirrors ``_flashlog.py``:
|
|
||||||
- Daemon thread, cooperative stop via ``threading.Event``.
|
|
||||||
- Tolerates the captures directory not existing yet (UI tier hasn't run).
|
|
||||||
- Per-file seek state so we only forward genuinely-new lines.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import pathlib
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from typing import Callable
|
|
||||||
|
|
||||||
|
|
||||||
class UiCaptureTailer(threading.Thread):
|
|
||||||
"""Recursively watch a captures root for new `transcript.md` lines.
|
|
||||||
|
|
||||||
Invokes ``post(test_id, line)`` for each new line, where ``test_id``
|
|
||||||
is derived from the path - the sanitized nodeid directory name.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
root: pathlib.Path,
|
|
||||||
post: Callable[[str, str], None],
|
|
||||||
stop: threading.Event,
|
|
||||||
*,
|
|
||||||
poll_interval: float = 0.5,
|
|
||||||
) -> None:
|
|
||||||
super().__init__(daemon=True, name="uicap-tail")
|
|
||||||
self._root = root
|
|
||||||
self._post = post
|
|
||||||
self._stop = stop
|
|
||||||
self._poll_interval = poll_interval
|
|
||||||
# path → byte offset we've already read through
|
|
||||||
self._offsets: dict[pathlib.Path, int] = {}
|
|
||||||
|
|
||||||
def run(self) -> None:
|
|
||||||
while not self._stop.is_set():
|
|
||||||
try:
|
|
||||||
self._scan_once()
|
|
||||||
except Exception:
|
|
||||||
# Best-effort tailer - never bring down the TUI because a
|
|
||||||
# directory vanished mid-scan.
|
|
||||||
pass
|
|
||||||
time.sleep(self._poll_interval)
|
|
||||||
|
|
||||||
def _scan_once(self) -> None:
|
|
||||||
if not self._root.is_dir():
|
|
||||||
return
|
|
||||||
for transcript in self._root.rglob("transcript.md"):
|
|
||||||
test_id = transcript.parent.name
|
|
||||||
offset = self._offsets.get(transcript, 0)
|
|
||||||
try:
|
|
||||||
size = transcript.stat().st_size
|
|
||||||
except OSError:
|
|
||||||
continue
|
|
||||||
if size < offset:
|
|
||||||
# File truncated / rewritten - reset and re-emit.
|
|
||||||
offset = 0
|
|
||||||
if size == offset:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
with transcript.open("rb") as fh:
|
|
||||||
fh.seek(offset)
|
|
||||||
chunk = fh.read(size - offset).decode("utf-8", errors="replace")
|
|
||||||
except OSError:
|
|
||||||
continue
|
|
||||||
for line in chunk.splitlines():
|
|
||||||
line = line.rstrip()
|
|
||||||
if not line or line.startswith("#"):
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
self._post(test_id, line)
|
|
||||||
except Exception:
|
|
||||||
return
|
|
||||||
self._offsets[transcript] = size
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,148 +0,0 @@
|
|||||||
"""Resolves the firmware repo root and the binaries we invoke.
|
|
||||||
|
|
||||||
Everything that needs a path (the firmware root, `pio`, `esptool`, etc.) goes
|
|
||||||
through this module so the rest of the package never calls `shutil.which` or
|
|
||||||
parses environment variables directly.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Iterable
|
|
||||||
|
|
||||||
|
|
||||||
class ConfigError(RuntimeError):
|
|
||||||
"""Raised when a required path or binary cannot be resolved."""
|
|
||||||
|
|
||||||
|
|
||||||
def firmware_root() -> Path:
|
|
||||||
"""Resolve the root of the Meshtastic firmware repo.
|
|
||||||
|
|
||||||
Resolution order:
|
|
||||||
1. `MESHTASTIC_FIRMWARE_ROOT` env var.
|
|
||||||
2. Walk up from `cwd` looking for a directory with `platformio.ini`.
|
|
||||||
"""
|
|
||||||
env = os.environ.get("MESHTASTIC_FIRMWARE_ROOT")
|
|
||||||
if env:
|
|
||||||
root = Path(env).expanduser().resolve()
|
|
||||||
if not (root / "platformio.ini").is_file():
|
|
||||||
raise ConfigError(
|
|
||||||
f"MESHTASTIC_FIRMWARE_ROOT={env!r} does not contain platformio.ini"
|
|
||||||
)
|
|
||||||
return root
|
|
||||||
|
|
||||||
cur = Path.cwd().resolve()
|
|
||||||
for candidate in (cur, *cur.parents):
|
|
||||||
if (candidate / "platformio.ini").is_file():
|
|
||||||
return candidate
|
|
||||||
raise ConfigError(
|
|
||||||
"Could not locate Meshtastic firmware root. Set MESHTASTIC_FIRMWARE_ROOT "
|
|
||||||
"to the directory containing platformio.ini."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _first_existing(paths: Iterable[Path]) -> Path | None:
|
|
||||||
for p in paths:
|
|
||||||
if p and p.is_file() and os.access(p, os.X_OK):
|
|
||||||
return p
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def pio_bin() -> Path:
|
|
||||||
"""Resolve the `pio` binary.
|
|
||||||
|
|
||||||
Order: MESHTASTIC_PIO_BIN → ~/.platformio/penv/bin/pio (PlatformIO keeps
|
|
||||||
this one current) → `pio` on PATH → `platformio` on PATH.
|
|
||||||
"""
|
|
||||||
env = os.environ.get("MESHTASTIC_PIO_BIN")
|
|
||||||
if env:
|
|
||||||
p = Path(env).expanduser()
|
|
||||||
if p.is_file() and os.access(p, os.X_OK):
|
|
||||||
return p
|
|
||||||
raise ConfigError(f"MESHTASTIC_PIO_BIN={env!r} is not an executable file")
|
|
||||||
|
|
||||||
penv = Path.home() / ".platformio" / "penv" / "bin" / "pio"
|
|
||||||
if penv.is_file() and os.access(penv, os.X_OK):
|
|
||||||
return penv
|
|
||||||
|
|
||||||
for name in ("pio", "platformio"):
|
|
||||||
w = shutil.which(name)
|
|
||||||
if w:
|
|
||||||
return Path(w)
|
|
||||||
|
|
||||||
raise ConfigError(
|
|
||||||
"Could not find `pio`. Install PlatformIO (https://platformio.org/install/cli) "
|
|
||||||
"or set MESHTASTIC_PIO_BIN."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _hw_tool(env_var: str, names: tuple[str, ...], install_hint: str) -> Path:
|
|
||||||
"""Shared resolver for esptool / nrfutil / picotool.
|
|
||||||
|
|
||||||
Prefers the firmware repo's own `.venv/bin/<name>` (esptool lives there),
|
|
||||||
then PATH.
|
|
||||||
"""
|
|
||||||
env = os.environ.get(env_var)
|
|
||||||
if env:
|
|
||||||
p = Path(env).expanduser()
|
|
||||||
if p.is_file() and os.access(p, os.X_OK):
|
|
||||||
return p
|
|
||||||
raise ConfigError(f"{env_var}={env!r} is not an executable file")
|
|
||||||
|
|
||||||
try:
|
|
||||||
venv_bin = firmware_root() / ".venv" / "bin"
|
|
||||||
except ConfigError:
|
|
||||||
venv_bin = None
|
|
||||||
|
|
||||||
for name in names:
|
|
||||||
if venv_bin is not None:
|
|
||||||
p = venv_bin / name
|
|
||||||
if p.is_file() and os.access(p, os.X_OK):
|
|
||||||
return p
|
|
||||||
|
|
||||||
for name in names:
|
|
||||||
w = shutil.which(name)
|
|
||||||
if w:
|
|
||||||
return Path(w)
|
|
||||||
|
|
||||||
raise ConfigError(
|
|
||||||
f"Could not find `{names[0]}`. {install_hint} "
|
|
||||||
f"Or set {env_var} to an absolute path."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def esptool_bin() -> Path:
|
|
||||||
return _hw_tool(
|
|
||||||
"MESHTASTIC_ESPTOOL_BIN",
|
|
||||||
("esptool", "esptool.py"),
|
|
||||||
"Install via `pip install esptool`.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def nrfutil_bin() -> Path:
|
|
||||||
return _hw_tool(
|
|
||||||
"MESHTASTIC_NRFUTIL_BIN",
|
|
||||||
("nrfutil", "adafruit-nrfutil"),
|
|
||||||
"Install via `pip install adafruit-nrfutil` or download Nordic nRF Util.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def picotool_bin() -> Path:
|
|
||||||
return _hw_tool(
|
|
||||||
"MESHTASTIC_PICOTOOL_BIN",
|
|
||||||
("picotool",),
|
|
||||||
"Install via `brew install picotool` or build from https://github.com/raspberrypi/picotool.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def uhubctl_bin() -> Path:
|
|
||||||
return _hw_tool(
|
|
||||||
"MESHTASTIC_UHUBCTL_BIN",
|
|
||||||
("uhubctl",),
|
|
||||||
"Install via `brew install uhubctl` (macOS) or `apt install uhubctl` "
|
|
||||||
"(Debian/Ubuntu). On Linux without the udev rules, or on older macOS "
|
|
||||||
"with certain hubs, you may need to run via `sudo`: "
|
|
||||||
"https://github.com/mvp/uhubctl#linux-usb-permissions",
|
|
||||||
)
|
|
||||||
@@ -1,229 +0,0 @@
|
|||||||
"""Context manager for meshtastic interface connections (serial + TCP).
|
|
||||||
|
|
||||||
Every info/admin tool goes through `connect(port)` so we have a single place
|
|
||||||
that:
|
|
||||||
- auto-selects the port when one likely_meshtastic device is present,
|
|
||||||
- fails fast if a serial_session is already holding the port,
|
|
||||||
- guarantees `.close()` is called, even on exception.
|
|
||||||
|
|
||||||
Two transports:
|
|
||||||
- Serial: USB-attached firmware on `/dev/cu.*` / `/dev/ttyUSB*` / `COM*`.
|
|
||||||
- TCP: a `meshtasticd` daemon (e.g. the native macOS / Linux Portduino
|
|
||||||
headless build) addressed as `tcp://host[:port]` (default port 4403).
|
|
||||||
Surfaced by `devices.list_devices()` when `MESHTASTIC_MCP_TCP_HOST` is
|
|
||||||
set, so `resolve_port(None)` auto-selects it like a USB candidate.
|
|
||||||
|
|
||||||
Both `SerialInterface` and `TCPInterface` block on construction waiting for
|
|
||||||
the node database; that's fine for v1 since every tool is a short-lived
|
|
||||||
request.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from contextlib import contextmanager
|
|
||||||
from typing import Iterator
|
|
||||||
|
|
||||||
from . import devices, registry
|
|
||||||
|
|
||||||
DEFAULT_TCP_PORT = 4403
|
|
||||||
TCP_SCHEME = "tcp://"
|
|
||||||
TCP_HOST_ENV = "MESHTASTIC_MCP_TCP_HOST"
|
|
||||||
|
|
||||||
|
|
||||||
class ConnectionError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def is_tcp_port(port: str | None) -> bool:
|
|
||||||
return bool(port) and port.startswith(TCP_SCHEME)
|
|
||||||
|
|
||||||
|
|
||||||
def parse_tcp_port(port: str) -> tuple[str, int]:
|
|
||||||
"""Parse `tcp://host[:port]` → (host, port). Defaults to 4403.
|
|
||||||
|
|
||||||
Validates host shape (non-empty, no path separators) and port range
|
|
||||||
(1..65535). Raises `ConnectionError` on malformed input - never lets
|
|
||||||
a raw `ValueError` bubble up to a tool surface.
|
|
||||||
"""
|
|
||||||
if not port.startswith(TCP_SCHEME):
|
|
||||||
raise ConnectionError(
|
|
||||||
f"Invalid TCP endpoint {port!r}: expected '{TCP_SCHEME}host[:port]'."
|
|
||||||
)
|
|
||||||
rest = port[len(TCP_SCHEME) :]
|
|
||||||
if ":" in rest:
|
|
||||||
host, port_str = rest.rsplit(":", 1)
|
|
||||||
try:
|
|
||||||
tcp_port = int(port_str)
|
|
||||||
except ValueError as e:
|
|
||||||
raise ConnectionError(
|
|
||||||
f"Invalid TCP endpoint {port!r}: port {port_str!r} is not an integer."
|
|
||||||
) from e
|
|
||||||
else:
|
|
||||||
host, tcp_port = rest, DEFAULT_TCP_PORT
|
|
||||||
if not host:
|
|
||||||
raise ConnectionError(f"Invalid TCP endpoint {port!r}: empty host.")
|
|
||||||
if any(c in host for c in ("/", "\\")):
|
|
||||||
raise ConnectionError(
|
|
||||||
f"Invalid TCP endpoint {port!r}: host {host!r} contains a path "
|
|
||||||
"separator. TCP hostnames cannot contain '/' or '\\' - did you "
|
|
||||||
"pass a serial port path or a Windows drive path by mistake?"
|
|
||||||
)
|
|
||||||
if not (1 <= tcp_port <= 65535):
|
|
||||||
raise ConnectionError(
|
|
||||||
f"Invalid TCP endpoint {port!r}: port {tcp_port} out of range "
|
|
||||||
"(must be 1..65535)."
|
|
||||||
)
|
|
||||||
return host, tcp_port
|
|
||||||
|
|
||||||
|
|
||||||
def normalize_tcp_endpoint(endpoint: str) -> str:
|
|
||||||
r"""Normalize `host`, `host:port`, or `tcp://host[:port]` → canonical
|
|
||||||
`tcp://host:port` form. One place that owns the lock-key shape.
|
|
||||||
|
|
||||||
Defers all validation to `parse_tcp_port`, so path-like inputs
|
|
||||||
(`/dev/cu.foo`, `C:\Windows\…`), empty hosts, non-integer ports,
|
|
||||||
and out-of-range ports raise `ConnectionError` here too.
|
|
||||||
"""
|
|
||||||
if endpoint.startswith(TCP_SCHEME):
|
|
||||||
canonical = endpoint
|
|
||||||
elif ":" in endpoint:
|
|
||||||
canonical = f"{TCP_SCHEME}{endpoint}"
|
|
||||||
else:
|
|
||||||
canonical = f"{TCP_SCHEME}{endpoint}:{DEFAULT_TCP_PORT}"
|
|
||||||
host, port = parse_tcp_port(canonical)
|
|
||||||
return f"{TCP_SCHEME}{host}:{port}"
|
|
||||||
|
|
||||||
|
|
||||||
def reject_if_tcp(port: str | None, tool_name: str) -> None:
|
|
||||||
"""Raise if `port` is a TCP endpoint - for tools that need real USB
|
|
||||||
hardware (flash, bootloader, vendor escape hatches, serial monitor).
|
|
||||||
|
|
||||||
Only checks the explicit arg; auto-selection via env var is the caller's
|
|
||||||
responsibility to handle if it matters.
|
|
||||||
"""
|
|
||||||
if is_tcp_port(port):
|
|
||||||
raise ConnectionError(
|
|
||||||
f"{tool_name} is not applicable to TCP/native nodes ({port}). "
|
|
||||||
"This tool requires USB-attached hardware."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_port(port: str | None) -> str:
|
|
||||||
"""Pick a port: explicit > sole likely_meshtastic candidate > error.
|
|
||||||
|
|
||||||
A `tcp://` string passes through (after canonicalization). When `port`
|
|
||||||
is None and no USB candidates are present, `MESHTASTIC_MCP_TCP_HOST`
|
|
||||||
is consulted via `devices.list_devices()`.
|
|
||||||
"""
|
|
||||||
if port:
|
|
||||||
if is_tcp_port(port):
|
|
||||||
return normalize_tcp_endpoint(port)
|
|
||||||
return port
|
|
||||||
candidates = [d for d in devices.list_devices() if d["likely_meshtastic"]]
|
|
||||||
if not candidates:
|
|
||||||
raise ConnectionError(
|
|
||||||
"No Meshtastic devices detected. Plug one in, set "
|
|
||||||
f"{TCP_HOST_ENV}=<host[:port]> for a meshtasticd daemon, "
|
|
||||||
"or pass `port` explicitly. Run `list_devices` with "
|
|
||||||
"include_unknown=True to see all serial ports."
|
|
||||||
)
|
|
||||||
if len(candidates) > 1:
|
|
||||||
ports = ", ".join(c["port"] for c in candidates)
|
|
||||||
raise ConnectionError(
|
|
||||||
f"Multiple Meshtastic devices detected ({ports}). "
|
|
||||||
"Specify `port` explicitly."
|
|
||||||
)
|
|
||||||
return candidates[0]["port"]
|
|
||||||
|
|
||||||
|
|
||||||
@contextmanager
|
|
||||||
def connect(port: str | None = None, timeout_s: float = 8.0) -> Iterator:
|
|
||||||
"""Open a meshtastic interface (serial or TCP) and always close it.
|
|
||||||
|
|
||||||
For serial: raises `ConnectionError` immediately if another serial
|
|
||||||
session holds the port (a `pio device monitor` in `serial_sessions/`).
|
|
||||||
For TCP: no exclusive-access requirement, so the serial-session check
|
|
||||||
is skipped - but the `port_lock` still serializes parallel `connect()`
|
|
||||||
calls to the same daemon endpoint.
|
|
||||||
|
|
||||||
`timeout_s` is plumbed through to both `SerialInterface(timeout=...)`
|
|
||||||
and `TCPInterface(timeout=...)`. The meshtastic library uses the value
|
|
||||||
as the reply-wait deadline for `localNode.waitForConfig()` during
|
|
||||||
construction and for any subsequent admin RPC. `int()`-converted at
|
|
||||||
the boundary because the upstream API expects whole seconds.
|
|
||||||
"""
|
|
||||||
resolved = resolve_port(port)
|
|
||||||
timeout = int(timeout_s)
|
|
||||||
|
|
||||||
if is_tcp_port(resolved):
|
|
||||||
from meshtastic.tcp_interface import (
|
|
||||||
TCPInterface, # type: ignore[import-untyped]
|
|
||||||
)
|
|
||||||
|
|
||||||
host, tcp_port = parse_tcp_port(resolved)
|
|
||||||
lock = registry.port_lock(resolved)
|
|
||||||
if not lock.acquire(blocking=False):
|
|
||||||
raise ConnectionError(
|
|
||||||
f"TCP endpoint {resolved} is busy - another device operation "
|
|
||||||
"is in flight. Retry shortly."
|
|
||||||
)
|
|
||||||
|
|
||||||
iface = None
|
|
||||||
try:
|
|
||||||
iface = TCPInterface(
|
|
||||||
hostname=host,
|
|
||||||
portNumber=tcp_port,
|
|
||||||
connectNow=True,
|
|
||||||
noProto=False,
|
|
||||||
timeout=timeout,
|
|
||||||
)
|
|
||||||
yield iface
|
|
||||||
finally:
|
|
||||||
if iface is not None:
|
|
||||||
try:
|
|
||||||
iface.close()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
try:
|
|
||||||
lock.release()
|
|
||||||
except RuntimeError:
|
|
||||||
pass
|
|
||||||
return
|
|
||||||
|
|
||||||
from meshtastic.serial_interface import (
|
|
||||||
SerialInterface, # type: ignore[import-untyped]
|
|
||||||
)
|
|
||||||
|
|
||||||
active = registry.active_session_for_port(resolved)
|
|
||||||
if active is not None:
|
|
||||||
raise ConnectionError(
|
|
||||||
f"Port {resolved} is held by serial session {active.id}. "
|
|
||||||
"Call `serial_close` first."
|
|
||||||
)
|
|
||||||
|
|
||||||
lock = registry.port_lock(resolved)
|
|
||||||
if not lock.acquire(blocking=False):
|
|
||||||
raise ConnectionError(
|
|
||||||
f"Port {resolved} is busy - another device operation is in flight. "
|
|
||||||
"Retry shortly."
|
|
||||||
)
|
|
||||||
|
|
||||||
iface = None
|
|
||||||
try:
|
|
||||||
iface = SerialInterface(
|
|
||||||
devPath=resolved,
|
|
||||||
connectNow=True,
|
|
||||||
noProto=False,
|
|
||||||
timeout=timeout,
|
|
||||||
)
|
|
||||||
yield iface
|
|
||||||
finally:
|
|
||||||
if iface is not None:
|
|
||||||
try:
|
|
||||||
iface.close()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
try:
|
|
||||||
lock.release()
|
|
||||||
except RuntimeError:
|
|
||||||
pass
|
|
||||||
@@ -1,135 +0,0 @@
|
|||||||
"""USB/serial + TCP device discovery.
|
|
||||||
|
|
||||||
Combines the canonical `meshtastic.util.findPorts()` allowlist/blocklist with
|
|
||||||
the richer metadata (`serial.tools.list_ports.comports()`) so callers see
|
|
||||||
VID/PID, descriptions, and manufacturer strings alongside the "is this likely
|
|
||||||
a Meshtastic device" signal.
|
|
||||||
|
|
||||||
If `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` is set, a synthetic entry for the
|
|
||||||
`meshtasticd` daemon at that endpoint is prepended to the result, so
|
|
||||||
`resolve_port(None)` auto-selects it like a USB candidate.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from serial.tools import list_ports
|
|
||||||
|
|
||||||
|
|
||||||
def _to_hex(value: int | None) -> str | None:
|
|
||||||
if value is None:
|
|
||||||
return None
|
|
||||||
return f"0x{value:04x}"
|
|
||||||
|
|
||||||
|
|
||||||
def _tcp_endpoint_from_env() -> dict[str, Any] | None:
|
|
||||||
"""Synthesize a TCP device entry from MESHTASTIC_MCP_TCP_HOST, if set.
|
|
||||||
|
|
||||||
If the env var is malformed (non-integer port, path-like host, etc.),
|
|
||||||
return an entry with `likely_meshtastic=False` and the parser error in
|
|
||||||
the description, rather than raising - `list_devices` is the diagnostic
|
|
||||||
tool a user reaches for when their env var isn't working, so it must
|
|
||||||
not crash on misconfiguration.
|
|
||||||
"""
|
|
||||||
host = os.environ.get("MESHTASTIC_MCP_TCP_HOST")
|
|
||||||
if not host:
|
|
||||||
return None
|
|
||||||
# Lazy import to avoid a circular dependency (connection imports devices).
|
|
||||||
from . import connection
|
|
||||||
|
|
||||||
try:
|
|
||||||
port = connection.normalize_tcp_endpoint(host)
|
|
||||||
description = "meshtasticd (TCP)"
|
|
||||||
likely = True
|
|
||||||
except connection.ConnectionError as e:
|
|
||||||
# Surface the raw env-var value plus the parser's reason so the
|
|
||||||
# user can see exactly what they set and why it was rejected.
|
|
||||||
# Don't double the scheme if the user already prefixed `tcp://`.
|
|
||||||
port = host if host.startswith(connection.TCP_SCHEME) else f"tcp://{host}"
|
|
||||||
description = f"meshtasticd (TCP) - invalid MESHTASTIC_MCP_TCP_HOST: {e}"
|
|
||||||
likely = False
|
|
||||||
return {
|
|
||||||
"port": port,
|
|
||||||
"vid": None,
|
|
||||||
"pid": None,
|
|
||||||
"description": description,
|
|
||||||
"manufacturer": None,
|
|
||||||
"product": None,
|
|
||||||
"serial_number": None,
|
|
||||||
"likely_meshtastic": likely,
|
|
||||||
"blacklisted": False,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def list_devices(include_unknown: bool = False) -> list[dict[str, Any]]:
|
|
||||||
"""Return enriched info for serial ports, flagging Meshtastic candidates.
|
|
||||||
|
|
||||||
`likely_meshtastic` is True when the port's USB VID matches the Meshtastic
|
|
||||||
allowlist (`0x239a` Adafruit/RAK, `0x303a` Espressif). When no allowlisted
|
|
||||||
ports are present, ports whose VID is NOT in the blocklist (J-Link, ST-LINK,
|
|
||||||
PPK2, etc.) are surfaced as `likely_meshtastic=False` candidates.
|
|
||||||
|
|
||||||
With `include_unknown=False` (default), we return only ports that are
|
|
||||||
plausibly Meshtastic. With `include_unknown=True`, every serial port the
|
|
||||||
OS knows about is returned (useful for debugging "why isn't my board
|
|
||||||
detected").
|
|
||||||
"""
|
|
||||||
# Import lazily so the module loads even without the `meshtastic` package
|
|
||||||
# (useful for introspection / schema generation).
|
|
||||||
from meshtastic import util as mt_util # type: ignore[import-untyped]
|
|
||||||
|
|
||||||
meshtastic_ports: set[str] = set(mt_util.findPorts(eliminate_duplicates=True))
|
|
||||||
whitelist = getattr(mt_util, "whitelistVids", {})
|
|
||||||
blacklist = getattr(mt_util, "blacklistVids", {})
|
|
||||||
|
|
||||||
results: list[dict[str, Any]] = []
|
|
||||||
for info in list_ports.comports():
|
|
||||||
port_path = info.device
|
|
||||||
vid = info.vid
|
|
||||||
in_whitelist = vid is not None and vid in whitelist
|
|
||||||
in_blacklist = vid is not None and vid in blacklist
|
|
||||||
|
|
||||||
likely = port_path in meshtastic_ports and in_whitelist
|
|
||||||
# If no allowlisted ports were found, findPorts falls back to
|
|
||||||
# everything-not-in-blacklist; treat those as plausible candidates
|
|
||||||
# but not "likely".
|
|
||||||
fallback_candidate = port_path in meshtastic_ports and not in_whitelist
|
|
||||||
|
|
||||||
if not likely and not fallback_candidate and not include_unknown:
|
|
||||||
continue
|
|
||||||
|
|
||||||
results.append(
|
|
||||||
{
|
|
||||||
"port": port_path,
|
|
||||||
"vid": _to_hex(vid),
|
|
||||||
"pid": _to_hex(info.pid),
|
|
||||||
"description": info.description or None,
|
|
||||||
"manufacturer": info.manufacturer or None,
|
|
||||||
"product": info.product or None,
|
|
||||||
"serial_number": info.serial_number or None,
|
|
||||||
"likely_meshtastic": likely,
|
|
||||||
"blacklisted": in_blacklist,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
# Append the TCP endpoint (if env var set) and sort everything together.
|
|
||||||
tcp_entry = _tcp_endpoint_from_env()
|
|
||||||
if tcp_entry is not None:
|
|
||||||
results.append(tcp_entry)
|
|
||||||
|
|
||||||
# Stable ordering: likely_meshtastic first; within rank, TCP wins over
|
|
||||||
# USB (explicit env-var configuration takes precedence over USB
|
|
||||||
# enumeration); then by port path. A misconfigured TCP entry has
|
|
||||||
# likely_meshtastic=False and lands among the other ignored entries -
|
|
||||||
# it does NOT pre-empt real USB devices at the top of the list.
|
|
||||||
results.sort(
|
|
||||||
key=lambda r: (
|
|
||||||
not r["likely_meshtastic"],
|
|
||||||
not r["port"].startswith("tcp://"),
|
|
||||||
r["port"],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
return results
|
|
||||||
@@ -1,382 +0,0 @@
|
|||||||
"""Fake NodeDB fixture push - Portduino file copy + hardware XModem upload.
|
|
||||||
|
|
||||||
The fixture pipeline is two-stage:
|
|
||||||
1. `bin/gen-fake-nodedb-seed.py` produces a deterministic JSONL describing N
|
|
||||||
fake-but-realistic peers. Committed under `test/fixtures/nodedb/`.
|
|
||||||
2. `bin/seed-json-to-proto.py` compiles JSONL → binary v25 NodeDatabase
|
|
||||||
protobuf with fresh wall-clock timestamps.
|
|
||||||
|
|
||||||
This module exposes `push_fake_nodedb(...)`, the MCP tool that:
|
|
||||||
- target="portduino": compiles the JSONL into the device's prefs dir on
|
|
||||||
the local filesystem (`~/.portduino/<config>/prefs/nodes.proto`).
|
|
||||||
- target="hardware": compiles to a temp file, then streams it over the
|
|
||||||
XModem protocol (via the meshtastic SerialInterface/BLEInterface +
|
|
||||||
`meshtastic.xmodempacket` pubsub topic) to `/prefs/nodes.proto` on the
|
|
||||||
device. Triggers a reboot so the firmware loads the new state on next
|
|
||||||
boot.
|
|
||||||
|
|
||||||
XModem wire details (mirrors firmware impl at src/xmodem.cpp:115-260):
|
|
||||||
* 128-byte chunks; final chunk padded to 128 B with 0x1A (SUB) bytes.
|
|
||||||
* CRC16-CCITT (poly 0x1021, init 0x0000).
|
|
||||||
* SOH/seq=0 carries the destination filename in `buffer.bytes`. ACK if
|
|
||||||
`FSCom.open(filename, FILE_O_WRITE)` succeeds; NAK otherwise.
|
|
||||||
* SOH/seq≥1 carries a 128-byte chunk. ACK = advance; NAK = retransmit.
|
|
||||||
* EOT after the last chunk flushes + closes the file on-device.
|
|
||||||
|
|
||||||
Hardware push requires `confirm=True` (mirrors factory_reset / erase_and_flash
|
|
||||||
in the .github/copilot-instructions.md "never do these without asking" list).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import dataclasses
|
|
||||||
import hashlib
|
|
||||||
import pathlib
|
|
||||||
import queue
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import tempfile
|
|
||||||
import time
|
|
||||||
from typing import Any, Literal
|
|
||||||
|
|
||||||
from .connection import connect, is_tcp_port
|
|
||||||
|
|
||||||
# Resolve repo root so the tool works regardless of mcp-server cwd.
|
|
||||||
_REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
|
|
||||||
_SEED_DIR = _REPO_ROOT / "test" / "fixtures" / "nodedb"
|
|
||||||
_COMPILE_SCRIPT = _REPO_ROOT / "bin" / "seed-json-to-proto.py"
|
|
||||||
|
|
||||||
_DEFAULT_NODES_FILENAME = "/prefs/nodes.proto"
|
|
||||||
_XMODEM_CHUNK = 128
|
|
||||||
_XMODEM_SUB = 0x1A
|
|
||||||
_ACK_TIMEOUT_INIT_S = 5.0
|
|
||||||
_ACK_TIMEOUT_CHUNK_S = 2.0
|
|
||||||
_MAX_CHUNK_RETRIES = 5
|
|
||||||
|
|
||||||
_VALID_SIZES = (250, 500, 1000, 2000)
|
|
||||||
|
|
||||||
|
|
||||||
class FixtureError(RuntimeError):
|
|
||||||
"""Raised for any fixture-push failure (compile, transport, ack timeout, …)."""
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# CRC16-CCITT (poly 0x1021, init 0x0000). Matches the firmware's `crc16_ccitt`.
|
|
||||||
# Hand-rolled to avoid the optional `crcmod` dep.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
def _crc16_ccitt(data: bytes, *, init: int = 0x0000) -> int:
|
|
||||||
crc = init
|
|
||||||
for b in data:
|
|
||||||
crc ^= b << 8
|
|
||||||
for _ in range(8):
|
|
||||||
if crc & 0x8000:
|
|
||||||
crc = ((crc << 1) ^ 0x1021) & 0xFFFF
|
|
||||||
else:
|
|
||||||
crc = (crc << 1) & 0xFFFF
|
|
||||||
return crc
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Compile step - shells out to bin/seed-json-to-proto.py so the MCP module
|
|
||||||
# doesn't have to duplicate the proto-encoding logic.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
def _compile_proto(jsonl_path: pathlib.Path, out_path: pathlib.Path) -> None:
|
|
||||||
if not _COMPILE_SCRIPT.is_file():
|
|
||||||
raise FixtureError(f"compile script missing at {_COMPILE_SCRIPT}")
|
|
||||||
cmd = [
|
|
||||||
sys.executable,
|
|
||||||
str(_COMPILE_SCRIPT),
|
|
||||||
"--in",
|
|
||||||
str(jsonl_path),
|
|
||||||
"--out",
|
|
||||||
str(out_path),
|
|
||||||
]
|
|
||||||
try:
|
|
||||||
subprocess.run(cmd, check=True, capture_output=True, text=True)
|
|
||||||
except subprocess.CalledProcessError as exc:
|
|
||||||
raise FixtureError(
|
|
||||||
f"seed-json-to-proto.py failed (exit {exc.returncode}):\n"
|
|
||||||
f" stdout: {exc.stdout}\n stderr: {exc.stderr}"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_seed_jsonl(size: int, custom: str | None) -> pathlib.Path:
|
|
||||||
if custom is not None:
|
|
||||||
p = pathlib.Path(custom).expanduser().resolve()
|
|
||||||
if not p.is_file():
|
|
||||||
raise FixtureError(f"custom_seed_jsonl not found: {p}")
|
|
||||||
return p
|
|
||||||
p = _SEED_DIR / f"seed_v25_{size:04d}.jsonl"
|
|
||||||
if not p.is_file():
|
|
||||||
raise FixtureError(
|
|
||||||
f"missing committed seed at {p}. "
|
|
||||||
f"Run `./bin/regen-fake-nodedbs.sh` to generate it."
|
|
||||||
)
|
|
||||||
return p
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Portduino push - file copy into ~/.portduino/<config>/prefs/
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
def _portduino_prefs_dir(config_name: str) -> pathlib.Path:
|
|
||||||
home = pathlib.Path.home()
|
|
||||||
return home / ".portduino" / config_name / "prefs"
|
|
||||||
|
|
||||||
|
|
||||||
def _push_portduino(
|
|
||||||
size: int,
|
|
||||||
jsonl: pathlib.Path,
|
|
||||||
portduino_config: str,
|
|
||||||
backup_existing: bool,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
prefs = _portduino_prefs_dir(portduino_config)
|
|
||||||
prefs.mkdir(parents=True, exist_ok=True)
|
|
||||||
target = prefs / "nodes.proto"
|
|
||||||
backed_up_to: str | None = None
|
|
||||||
if backup_existing and target.is_file():
|
|
||||||
ts = int(time.time())
|
|
||||||
backup = prefs / f"nodes.proto.bak.{ts}"
|
|
||||||
shutil.move(str(target), str(backup))
|
|
||||||
backed_up_to = str(backup)
|
|
||||||
_compile_proto(jsonl, target)
|
|
||||||
raw = target.read_bytes()
|
|
||||||
return {
|
|
||||||
"transport": "portduino",
|
|
||||||
"path": str(target),
|
|
||||||
"bytes": len(raw),
|
|
||||||
"sha256": hashlib.sha256(raw).hexdigest(),
|
|
||||||
"jsonl_source": str(jsonl),
|
|
||||||
"backed_up_to": backed_up_to,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Hardware push - XModem over BLE/serial via the meshtastic Python interface.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
@dataclasses.dataclass
|
|
||||||
class _AckEvent:
|
|
||||||
control: int
|
|
||||||
seq: int
|
|
||||||
|
|
||||||
|
|
||||||
def _wait_for_response(q: "queue.Queue[_AckEvent]", timeout_s: float) -> _AckEvent:
|
|
||||||
try:
|
|
||||||
return q.get(timeout=timeout_s)
|
|
||||||
except queue.Empty as exc:
|
|
||||||
raise FixtureError(
|
|
||||||
f"XModem response timeout after {timeout_s:.1f}s - device not responding"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
|
|
||||||
def _push_hardware(
|
|
||||||
size: int,
|
|
||||||
jsonl: pathlib.Path,
|
|
||||||
port: str | None,
|
|
||||||
reboot_after: bool,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
# Lazy imports so the module loads even when the meshtastic deps aren't
|
|
||||||
# available (e.g. CI in a Python env without the package installed).
|
|
||||||
try:
|
|
||||||
from meshtastic.protobuf import mesh_pb2, xmodem_pb2
|
|
||||||
from pubsub import pub
|
|
||||||
except ImportError as exc: # pragma: no cover - dep missing
|
|
||||||
raise FixtureError(
|
|
||||||
f"hardware push requires the meshtastic + pypubsub packages: {exc}"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
if is_tcp_port(port):
|
|
||||||
raise FixtureError(
|
|
||||||
"hardware push over TCP/portduino is not supported - use "
|
|
||||||
"target='portduino' to drop the fixture directly into the prefs dir."
|
|
||||||
)
|
|
||||||
|
|
||||||
# Compile the fixture to a temp file with fresh timestamps.
|
|
||||||
with tempfile.NamedTemporaryFile(suffix=".proto", delete=False) as tf:
|
|
||||||
proto_path = pathlib.Path(tf.name)
|
|
||||||
try:
|
|
||||||
_compile_proto(jsonl, proto_path)
|
|
||||||
payload = proto_path.read_bytes()
|
|
||||||
finally:
|
|
||||||
proto_path.unlink(missing_ok=True)
|
|
||||||
|
|
||||||
sha256 = hashlib.sha256(payload).hexdigest()
|
|
||||||
total_bytes = len(payload)
|
|
||||||
|
|
||||||
# Subscribe to XModem responses BEFORE we open the interface, so we don't
|
|
||||||
# race the first ACK that arrives during the SOH/seq=0 handshake.
|
|
||||||
#
|
|
||||||
# NB: the signature MUST declare every kwarg pypubsub will see for this
|
|
||||||
# topic, or pubsub locks the topic spec to a smaller set (whichever
|
|
||||||
# subscribe arrives first) and then *rejects* the meshtastic library's
|
|
||||||
# publish call with `SenderUnknownMsgDataError: unknown ... interface`.
|
|
||||||
# The meshtastic lib publishes both `packet=` and `interface=`
|
|
||||||
# (mesh_interface.py:1389-1395), so both must appear here.
|
|
||||||
response_q: "queue.Queue[_AckEvent]" = queue.Queue()
|
|
||||||
|
|
||||||
def _on_xmodem(packet: Any = None, interface: Any = None, **_kw: Any) -> None:
|
|
||||||
if packet is None:
|
|
||||||
return
|
|
||||||
response_q.put(_AckEvent(control=int(packet.control), seq=int(packet.seq)))
|
|
||||||
|
|
||||||
pub.subscribe(_on_xmodem, "meshtastic.xmodempacket")
|
|
||||||
|
|
||||||
chunks_sent = 0
|
|
||||||
retried = 0
|
|
||||||
rebooted = False
|
|
||||||
|
|
||||||
XMC = xmodem_pb2.XModem.Control
|
|
||||||
try:
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
# 1) Send the filename (SOH, seq=0).
|
|
||||||
init_pkt = xmodem_pb2.XModem(
|
|
||||||
control=XMC.Value("SOH"),
|
|
||||||
seq=0,
|
|
||||||
buffer=_DEFAULT_NODES_FILENAME.encode("utf-8"),
|
|
||||||
)
|
|
||||||
iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=init_pkt))
|
|
||||||
ack = _wait_for_response(response_q, _ACK_TIMEOUT_INIT_S)
|
|
||||||
if ack.control != XMC.Value("ACK"):
|
|
||||||
raise FixtureError(
|
|
||||||
f"device refused filename {_DEFAULT_NODES_FILENAME!r} "
|
|
||||||
f"(got control={ack.control}, expected ACK). "
|
|
||||||
f"Filesystem full or permissions issue?"
|
|
||||||
)
|
|
||||||
|
|
||||||
# 2) Stream the payload in 128 B chunks.
|
|
||||||
for offset in range(0, total_bytes, _XMODEM_CHUNK):
|
|
||||||
chunk = payload[offset : offset + _XMODEM_CHUNK]
|
|
||||||
if len(chunk) < _XMODEM_CHUNK:
|
|
||||||
# Pad final chunk to 128 B with SUB. The trailing 0x1A bytes
|
|
||||||
# become part of the file on-device, but nanopb ignores
|
|
||||||
# bytes past the end of the top-level message.
|
|
||||||
chunk = chunk + bytes([_XMODEM_SUB] * (_XMODEM_CHUNK - len(chunk)))
|
|
||||||
seq = ((offset // _XMODEM_CHUNK) + 1) % 256
|
|
||||||
# Retry loop on NAK / timeout.
|
|
||||||
attempts = 0
|
|
||||||
while True:
|
|
||||||
pkt = xmodem_pb2.XModem(
|
|
||||||
control=XMC.Value("SOH"),
|
|
||||||
seq=seq,
|
|
||||||
buffer=chunk,
|
|
||||||
crc16=_crc16_ccitt(chunk),
|
|
||||||
)
|
|
||||||
iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=pkt))
|
|
||||||
ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
|
|
||||||
if ack.control == XMC.Value("ACK"):
|
|
||||||
chunks_sent += 1
|
|
||||||
break
|
|
||||||
if ack.control == XMC.Value("NAK"):
|
|
||||||
attempts += 1
|
|
||||||
retried += 1
|
|
||||||
if attempts >= _MAX_CHUNK_RETRIES:
|
|
||||||
# Abort: send CAN so the firmware removes the half-
|
|
||||||
# written file via FSCom.remove(filename).
|
|
||||||
iface._sendToRadio(
|
|
||||||
mesh_pb2.ToRadio(
|
|
||||||
xmodemPacket=xmodem_pb2.XModem(
|
|
||||||
control=XMC.Value("CAN")
|
|
||||||
)
|
|
||||||
)
|
|
||||||
)
|
|
||||||
raise FixtureError(
|
|
||||||
f"chunk seq={seq} NAK'd {attempts} times; "
|
|
||||||
f"aborted transfer (file removed on-device)."
|
|
||||||
)
|
|
||||||
continue # retry the same chunk
|
|
||||||
raise FixtureError(
|
|
||||||
f"unexpected XModem control={ack.control} on seq={seq}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# 3) Tell the device we're done.
|
|
||||||
iface._sendToRadio(
|
|
||||||
mesh_pb2.ToRadio(
|
|
||||||
xmodemPacket=xmodem_pb2.XModem(control=XMC.Value("EOT"))
|
|
||||||
)
|
|
||||||
)
|
|
||||||
ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
|
|
||||||
if ack.control != XMC.Value("ACK"):
|
|
||||||
raise FixtureError(f"EOT not ACKed (got control={ack.control})")
|
|
||||||
|
|
||||||
# 4) Reboot so loadFromDisk picks up the new file.
|
|
||||||
if reboot_after:
|
|
||||||
iface.localNode.reboot(secs=1)
|
|
||||||
rebooted = True
|
|
||||||
finally:
|
|
||||||
try:
|
|
||||||
pub.unsubscribe(_on_xmodem, "meshtastic.xmodempacket")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
return {
|
|
||||||
"transport": "hardware",
|
|
||||||
"port": port,
|
|
||||||
"filename_on_device": _DEFAULT_NODES_FILENAME,
|
|
||||||
"bytes": total_bytes,
|
|
||||||
"chunks_sent": chunks_sent,
|
|
||||||
"retried": retried,
|
|
||||||
"sha256": sha256,
|
|
||||||
"jsonl_source": str(jsonl),
|
|
||||||
"rebooted": rebooted,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Public entry point - registered as an MCP tool in server.py.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
def push_fake_nodedb(
|
|
||||||
size: int,
|
|
||||||
target: Literal["portduino", "hardware"] = "portduino",
|
|
||||||
*,
|
|
||||||
port: str | None = None,
|
|
||||||
portduino_config: str = "default",
|
|
||||||
backup_existing: bool = True,
|
|
||||||
confirm: bool = False,
|
|
||||||
reboot_after: bool = True,
|
|
||||||
custom_seed_jsonl: str | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Compile a fresh-timestamp NodeDatabase fixture and push it to a device.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
size: 250, 500, 1000, or 2000 - selects which committed seed JSONL to use.
|
|
||||||
target: "portduino" (file copy to ~/.portduino/<config>/prefs/) or
|
|
||||||
"hardware" (XModem upload to /prefs/nodes.proto + reboot).
|
|
||||||
port: required for target="hardware". Serial path (e.g. /dev/cu.usbmodemXXXX)
|
|
||||||
or BLE identifier. TCP endpoints are rejected - use target="portduino"
|
|
||||||
instead.
|
|
||||||
portduino_config: which Portduino instance dir under ~/.portduino/. Default "default".
|
|
||||||
backup_existing: portduino only. Move nodes.proto -> nodes.proto.bak.<ts>
|
|
||||||
if present, so you can roll back.
|
|
||||||
confirm: required True for target="hardware" (writes flash + reboots).
|
|
||||||
reboot_after: hardware only. If True, send a 1-second reboot after the
|
|
||||||
final ACK so loadFromDisk picks up the new file at next boot.
|
|
||||||
custom_seed_jsonl: override the committed JSONL. Use to push a hand-edited
|
|
||||||
test scenario.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
dict with transport, bytes, sha256, etc. - depends on target.
|
|
||||||
|
|
||||||
"""
|
|
||||||
if size not in _VALID_SIZES:
|
|
||||||
raise FixtureError(
|
|
||||||
f"size must be one of {_VALID_SIZES}; got {size!r}. "
|
|
||||||
f"Add a new committed seed if you need a different cardinality."
|
|
||||||
)
|
|
||||||
|
|
||||||
jsonl = _resolve_seed_jsonl(size, custom_seed_jsonl)
|
|
||||||
|
|
||||||
if target == "portduino":
|
|
||||||
return _push_portduino(size, jsonl, portduino_config, backup_existing)
|
|
||||||
|
|
||||||
if target == "hardware":
|
|
||||||
if not confirm:
|
|
||||||
raise FixtureError(
|
|
||||||
"hardware push writes flash and triggers a reboot - pass confirm=True."
|
|
||||||
)
|
|
||||||
if not port:
|
|
||||||
raise FixtureError(
|
|
||||||
"target='hardware' requires a port (e.g. /dev/cu.usbmodemXXXX)."
|
|
||||||
)
|
|
||||||
return _push_hardware(size, jsonl, port, reboot_after)
|
|
||||||
|
|
||||||
raise FixtureError(f"unknown target {target!r}; expected 'portduino' or 'hardware'")
|
|
||||||
@@ -1,507 +0,0 @@
|
|||||||
"""Build, clean, flash, and bootloader-entry operations.
|
|
||||||
|
|
||||||
Design: pio is the preferred path for every architecture via `flash()`. For
|
|
||||||
ESP32 factory flashes we shell out to `bin/device-install.sh` (which knows
|
|
||||||
about partition offsets and the OTA/littlefs partitions); for ESP32 OTA
|
|
||||||
updates we use `bin/device-update.sh`. Both scripts require the build
|
|
||||||
artifacts to exist, so these tools build first if needed.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import subprocess
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import serial
|
|
||||||
|
|
||||||
from . import boards, config, connection, devices, pio, userprefs
|
|
||||||
|
|
||||||
# Meshtastic variants use both `esp32s3` and `esp32-s3` style names across
|
|
||||||
# variants/*/platformio.ini (no consistency enforced). Accept both spellings.
|
|
||||||
ESP32_ARCHES = {
|
|
||||||
"esp32",
|
|
||||||
"esp32s2",
|
|
||||||
"esp32-s2",
|
|
||||||
"esp32s3",
|
|
||||||
"esp32-s3",
|
|
||||||
"esp32c3",
|
|
||||||
"esp32-c3",
|
|
||||||
"esp32c6",
|
|
||||||
"esp32-c6",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class FlashError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _require_confirm(confirm: bool, operation: str) -> None:
|
|
||||||
if not confirm:
|
|
||||||
raise FlashError(
|
|
||||||
f"{operation} is destructive and requires confirm=True. "
|
|
||||||
"This will overwrite firmware on the device."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _reject_native_env(env: str, operation: str) -> None:
|
|
||||||
"""`native*` envs build a host executable, not firmware - there's no
|
|
||||||
upload step. The user wants `build` (or just runs the binary directly).
|
|
||||||
"""
|
|
||||||
if env.startswith("native"):
|
|
||||||
raise FlashError(
|
|
||||||
f"{operation} is not applicable for env {env!r}: native envs "
|
|
||||||
"produce a host executable, not flashable firmware. Use `build` "
|
|
||||||
"instead, then run the resulting binary directly."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _artifacts_for(env: str) -> list[Path]:
|
|
||||||
build_dir = config.firmware_root() / ".pio" / "build" / env
|
|
||||||
if not build_dir.is_dir():
|
|
||||||
return []
|
|
||||||
patterns = (
|
|
||||||
"firmware*.bin",
|
|
||||||
"firmware*.uf2",
|
|
||||||
"firmware*.hex",
|
|
||||||
"firmware*.zip",
|
|
||||||
"firmware*.elf",
|
|
||||||
"*.mt.json",
|
|
||||||
"littlefs-*.bin",
|
|
||||||
)
|
|
||||||
out: list[Path] = []
|
|
||||||
for pat in patterns:
|
|
||||||
out.extend(sorted(build_dir.glob(pat)))
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _factory_bin_for(env: str) -> Path | None:
|
|
||||||
build_dir = config.firmware_root() / ".pio" / "build" / env
|
|
||||||
if not build_dir.is_dir():
|
|
||||||
return None
|
|
||||||
matches = sorted(build_dir.glob("firmware-*.factory.bin"))
|
|
||||||
return matches[0] if matches else None
|
|
||||||
|
|
||||||
|
|
||||||
def _firmware_bin_for(env: str) -> Path | None:
|
|
||||||
"""Return the OTA-update firmware binary (app partition only)."""
|
|
||||||
build_dir = config.firmware_root() / ".pio" / "build" / env
|
|
||||||
if not build_dir.is_dir():
|
|
||||||
return None
|
|
||||||
# device-update.sh expects firmware-<env>-<version>.bin (not .factory.bin)
|
|
||||||
matches = sorted(
|
|
||||||
p
|
|
||||||
for p in build_dir.glob("firmware-*.bin")
|
|
||||||
if not p.name.endswith(".factory.bin")
|
|
||||||
)
|
|
||||||
return matches[0] if matches else None
|
|
||||||
|
|
||||||
|
|
||||||
def _userprefs_summary(active: dict[str, str]) -> dict[str, Any]:
|
|
||||||
"""Compact summary of which USERPREFS_* are baked into the build."""
|
|
||||||
return {"count": len(active), "keys": sorted(active.keys())}
|
|
||||||
|
|
||||||
|
|
||||||
def build(
|
|
||||||
env: str,
|
|
||||||
with_manifest: bool = True,
|
|
||||||
userprefs_overrides: dict[str, Any] | None = None,
|
|
||||||
build_flags: dict[str, Any] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Run `pio run -e <env>` and return artifact paths.
|
|
||||||
|
|
||||||
`userprefs_overrides` (optional): dict of `USERPREFS_<KEY>: value` to inject
|
|
||||||
into userPrefs.jsonc for this build only. File is restored byte-for-byte
|
|
||||||
on exit. Use `userprefs_set()` for persistent changes.
|
|
||||||
|
|
||||||
`build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros to set for
|
|
||||||
this build only via `PLATFORMIO_BUILD_FLAGS`. Common useful flag:
|
|
||||||
`{"DEBUG_HEAP": 1}` enables per-thread leak detection + `[heap N]`
|
|
||||||
prefix on every log line. Combines with the recorder so heap shows
|
|
||||||
up at log cadence (much higher resolution than the ~60 s LocalStats
|
|
||||||
packet) - see `recorder/parsers.py:_HEAP_PREFIX_RE`. Bool values
|
|
||||||
expand to bare `-D<NAME>` (presence-only flags).
|
|
||||||
"""
|
|
||||||
args = ["run", "-e", env]
|
|
||||||
if with_manifest:
|
|
||||||
args.extend(["-t", "mtjson"])
|
|
||||||
extra_env = _build_flags_env(build_flags) if build_flags else None
|
|
||||||
with userprefs.temporary_overrides(userprefs_overrides) as effective:
|
|
||||||
result = pio.run(
|
|
||||||
args,
|
|
||||||
timeout=pio.TIMEOUT_BUILD,
|
|
||||||
check=False,
|
|
||||||
extra_env=extra_env,
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"exit_code": result.returncode,
|
|
||||||
"artifacts": [str(p) for p in _artifacts_for(env)],
|
|
||||||
"stdout_tail": pio.tail_lines(result.stdout, 200),
|
|
||||||
"stderr_tail": pio.tail_lines(result.stderr, 200),
|
|
||||||
"duration_s": round(result.duration_s, 2),
|
|
||||||
"userprefs": _userprefs_summary(effective),
|
|
||||||
"build_flags": dict(build_flags) if build_flags else None,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _build_flags_env(build_flags: dict[str, Any]) -> dict[str, str]:
|
|
||||||
"""Translate `{"DEBUG_HEAP": 1, "FOO": "bar"}` → `{"PLATFORMIO_BUILD_FLAGS":
|
|
||||||
"-DDEBUG_HEAP=1 -DFOO=bar"}`. Bool True → bare `-D<NAME>`; False/None drop
|
|
||||||
the flag entirely. Other types stringify."""
|
|
||||||
parts: list[str] = []
|
|
||||||
for key, value in build_flags.items():
|
|
||||||
if value is False or value is None:
|
|
||||||
continue
|
|
||||||
if value is True:
|
|
||||||
parts.append(f"-D{key}")
|
|
||||||
else:
|
|
||||||
parts.append(f"-D{key}={value}")
|
|
||||||
if not parts:
|
|
||||||
return {}
|
|
||||||
return {"PLATFORMIO_BUILD_FLAGS": " ".join(parts)}
|
|
||||||
|
|
||||||
|
|
||||||
def clean(env: str) -> dict[str, Any]:
|
|
||||||
"""Run `pio run -e <env> -t clean`."""
|
|
||||||
result = pio.run(["run", "-e", env, "-t", "clean"], timeout=120, check=False)
|
|
||||||
return {
|
|
||||||
"exit_code": result.returncode,
|
|
||||||
"stdout_tail": pio.tail_lines(result.stdout, 200),
|
|
||||||
"stderr_tail": pio.tail_lines(result.stderr, 200),
|
|
||||||
"duration_s": round(result.duration_s, 2),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def flash(
|
|
||||||
env: str,
|
|
||||||
port: str,
|
|
||||||
confirm: bool = False,
|
|
||||||
userprefs_overrides: dict[str, Any] | None = None,
|
|
||||||
build_flags: dict[str, Any] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""`pio run -e <env> -t upload --upload-port <port>`. All architectures.
|
|
||||||
|
|
||||||
`userprefs_overrides` (optional): see `build()` - the rebuild-before-upload
|
|
||||||
that pio performs will pick up the injected values.
|
|
||||||
|
|
||||||
`build_flags` (optional): same shape as `build()` - `PLATFORMIO_BUILD_FLAGS`
|
|
||||||
is exported for the rebuild-before-upload, so the uploaded firmware
|
|
||||||
actually carries the flags. Without this propagation, `pio run -t upload`
|
|
||||||
would relink without the env var and silently drop them. Common use:
|
|
||||||
`build_flags={"DEBUG_HEAP": 1}` for the leak-hunt path.
|
|
||||||
"""
|
|
||||||
_require_confirm(confirm, "flash")
|
|
||||||
_reject_native_env(env, "flash")
|
|
||||||
connection.reject_if_tcp(port, "flash")
|
|
||||||
extra_env = _build_flags_env(build_flags) if build_flags else None
|
|
||||||
with userprefs.temporary_overrides(userprefs_overrides) as effective:
|
|
||||||
result = pio.run(
|
|
||||||
["run", "-e", env, "-t", "upload", "--upload-port", port],
|
|
||||||
timeout=pio.TIMEOUT_UPLOAD,
|
|
||||||
check=False,
|
|
||||||
extra_env=extra_env,
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"exit_code": result.returncode,
|
|
||||||
"stdout_tail": pio.tail_lines(result.stdout, 200),
|
|
||||||
"stderr_tail": pio.tail_lines(result.stderr, 200),
|
|
||||||
"duration_s": round(result.duration_s, 2),
|
|
||||||
"userprefs": _userprefs_summary(effective),
|
|
||||||
"build_flags": dict(build_flags) if build_flags else None,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _check_esp32_env(env: str) -> str:
|
|
||||||
rec = boards.get_board(env)
|
|
||||||
arch = rec.get("architecture")
|
|
||||||
if arch not in ESP32_ARCHES:
|
|
||||||
raise FlashError(
|
|
||||||
f"Env {env!r} has architecture {arch!r}, not ESP32. "
|
|
||||||
"Use `flash` for non-ESP32 boards."
|
|
||||||
)
|
|
||||||
return arch
|
|
||||||
|
|
||||||
|
|
||||||
def _run_install_script(script: Path, port: str, binary: Path) -> dict[str, Any]:
|
|
||||||
"""Invoke bin/device-install.sh or bin/device-update.sh."""
|
|
||||||
t0 = time.monotonic()
|
|
||||||
proc = subprocess.run(
|
|
||||||
[str(script), "-p", port, "-f", str(binary)],
|
|
||||||
cwd=str(config.firmware_root()),
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
timeout=pio.TIMEOUT_UPLOAD,
|
|
||||||
)
|
|
||||||
duration = time.monotonic() - t0
|
|
||||||
return {
|
|
||||||
"exit_code": proc.returncode,
|
|
||||||
"stdout_tail": pio.tail_lines(proc.stdout, 200),
|
|
||||||
"stderr_tail": pio.tail_lines(proc.stderr, 200),
|
|
||||||
"duration_s": round(duration, 2),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def erase_and_flash(
|
|
||||||
env: str,
|
|
||||||
port: str,
|
|
||||||
confirm: bool = False,
|
|
||||||
skip_build: bool = False,
|
|
||||||
userprefs_overrides: dict[str, Any] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""ESP32-only: full erase + factory flash via bin/device-install.sh.
|
|
||||||
|
|
||||||
`userprefs_overrides`: baked into the factory.bin via a fresh build. If
|
|
||||||
overrides are provided we always force a rebuild (skip_build=True errors
|
|
||||||
in that case) since a cached factory.bin would not reflect the new prefs.
|
|
||||||
"""
|
|
||||||
_require_confirm(confirm, "erase_and_flash")
|
|
||||||
connection.reject_if_tcp(port, "erase_and_flash")
|
|
||||||
_check_esp32_env(env)
|
|
||||||
|
|
||||||
if userprefs_overrides and skip_build:
|
|
||||||
raise FlashError(
|
|
||||||
"userprefs_overrides forces a rebuild so the factory.bin reflects "
|
|
||||||
"the new values; skip_build=True is incompatible."
|
|
||||||
)
|
|
||||||
|
|
||||||
with userprefs.temporary_overrides(userprefs_overrides) as effective:
|
|
||||||
# If overrides were provided, always build; otherwise only build if
|
|
||||||
# no factory.bin is present.
|
|
||||||
factory = _factory_bin_for(env)
|
|
||||||
if factory is None or userprefs_overrides:
|
|
||||||
if skip_build:
|
|
||||||
raise FlashError(
|
|
||||||
f"No factory.bin found for env {env!r} and skip_build=True. "
|
|
||||||
"Run `build` first or set skip_build=False."
|
|
||||||
)
|
|
||||||
build_args = ["run", "-e", env, "-t", "mtjson"]
|
|
||||||
build_result = pio.run(build_args, timeout=pio.TIMEOUT_BUILD, check=False)
|
|
||||||
if build_result.returncode != 0:
|
|
||||||
return {
|
|
||||||
"exit_code": build_result.returncode,
|
|
||||||
"stdout_tail": pio.tail_lines(build_result.stdout, 200),
|
|
||||||
"stderr_tail": pio.tail_lines(build_result.stderr, 200),
|
|
||||||
"duration_s": round(build_result.duration_s, 2),
|
|
||||||
"error": "build failed before erase_and_flash could run",
|
|
||||||
"userprefs": _userprefs_summary(effective),
|
|
||||||
}
|
|
||||||
factory = _factory_bin_for(env)
|
|
||||||
if factory is None:
|
|
||||||
raise FlashError(
|
|
||||||
f"Build succeeded but no factory.bin appeared in .pio/build/{env}/"
|
|
||||||
)
|
|
||||||
|
|
||||||
script = config.firmware_root() / "bin" / "device-install.sh"
|
|
||||||
if not script.is_file():
|
|
||||||
raise FlashError(f"device-install.sh not found at {script}")
|
|
||||||
result = _run_install_script(script, port, factory)
|
|
||||||
|
|
||||||
result["userprefs"] = _userprefs_summary(effective)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def update_flash(
|
|
||||||
env: str,
|
|
||||||
port: str,
|
|
||||||
confirm: bool = False,
|
|
||||||
skip_build: bool = False,
|
|
||||||
userprefs_overrides: dict[str, Any] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""ESP32-only: OTA app-partition update via bin/device-update.sh.
|
|
||||||
|
|
||||||
`userprefs_overrides`: baked into the firmware.bin via a fresh build. If
|
|
||||||
overrides are provided we always force a rebuild.
|
|
||||||
"""
|
|
||||||
_require_confirm(confirm, "update_flash")
|
|
||||||
connection.reject_if_tcp(port, "update_flash")
|
|
||||||
_check_esp32_env(env)
|
|
||||||
|
|
||||||
if userprefs_overrides and skip_build:
|
|
||||||
raise FlashError(
|
|
||||||
"userprefs_overrides forces a rebuild so the firmware.bin reflects "
|
|
||||||
"the new values; skip_build=True is incompatible."
|
|
||||||
)
|
|
||||||
|
|
||||||
with userprefs.temporary_overrides(userprefs_overrides) as effective:
|
|
||||||
firmware = _firmware_bin_for(env)
|
|
||||||
if firmware is None or userprefs_overrides:
|
|
||||||
if skip_build:
|
|
||||||
raise FlashError(
|
|
||||||
f"No firmware.bin found for env {env!r} and skip_build=True. "
|
|
||||||
"Run `build` first or set skip_build=False."
|
|
||||||
)
|
|
||||||
build_args = ["run", "-e", env, "-t", "mtjson"]
|
|
||||||
build_result = pio.run(build_args, timeout=pio.TIMEOUT_BUILD, check=False)
|
|
||||||
if build_result.returncode != 0:
|
|
||||||
return {
|
|
||||||
"exit_code": build_result.returncode,
|
|
||||||
"stdout_tail": pio.tail_lines(build_result.stdout, 200),
|
|
||||||
"stderr_tail": pio.tail_lines(build_result.stderr, 200),
|
|
||||||
"duration_s": round(build_result.duration_s, 2),
|
|
||||||
"error": "build failed before update_flash could run",
|
|
||||||
"userprefs": _userprefs_summary(effective),
|
|
||||||
}
|
|
||||||
firmware = _firmware_bin_for(env)
|
|
||||||
if firmware is None:
|
|
||||||
raise FlashError(
|
|
||||||
f"Build succeeded but no firmware.bin appeared in .pio/build/{env}/"
|
|
||||||
)
|
|
||||||
|
|
||||||
script = config.firmware_root() / "bin" / "device-update.sh"
|
|
||||||
if not script.is_file():
|
|
||||||
raise FlashError(f"device-update.sh not found at {script}")
|
|
||||||
result = _run_install_script(script, port, firmware)
|
|
||||||
|
|
||||||
result["userprefs"] = _userprefs_summary(effective)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def _do_1200bps_touch(port: str, settle_ms: int, touch_timeout_s: float = 3.0) -> None:
|
|
||||||
"""Open port at 1200 baud and close, bounded by a worker thread.
|
|
||||||
|
|
||||||
Both the open and the close can block on a busy CDC device - we wrap the
|
|
||||||
whole thing in a worker so the caller returns in at most `touch_timeout_s`
|
|
||||||
regardless. The touch is signal-only: the USB configuration change to
|
|
||||||
1200 baud alone is enough to trip the Adafruit bootloader's reset, so a
|
|
||||||
worker that's still blocked in the background after timeout has already
|
|
||||||
delivered the signal.
|
|
||||||
"""
|
|
||||||
errors: list[BaseException] = []
|
|
||||||
|
|
||||||
def _inner() -> None:
|
|
||||||
try:
|
|
||||||
s = serial.Serial(port, 1200)
|
|
||||||
except serial.SerialException as exc:
|
|
||||||
if "No such file" in str(exc) or "could not open" in str(exc).lower():
|
|
||||||
raise
|
|
||||||
return # other serial errors mid-open are expected during DFU entry
|
|
||||||
try:
|
|
||||||
time.sleep(settle_ms / 1000.0)
|
|
||||||
finally:
|
|
||||||
try:
|
|
||||||
s.close()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def _runner() -> None:
|
|
||||||
try:
|
|
||||||
_inner()
|
|
||||||
except BaseException as exc: # re-raised on caller thread after join
|
|
||||||
errors.append(exc)
|
|
||||||
|
|
||||||
worker = threading.Thread(target=_runner, daemon=True)
|
|
||||||
worker.start()
|
|
||||||
worker.join(timeout=touch_timeout_s)
|
|
||||||
if worker.is_alive():
|
|
||||||
return # signal already delivered; allow daemon worker to finish/exit
|
|
||||||
if errors:
|
|
||||||
raise errors[0]
|
|
||||||
|
|
||||||
|
|
||||||
# Adafruit nRF52 bootloader VID/PID (BOTH RAK4631 and most Feather nRF52 boards).
|
|
||||||
# See https://github.com/adafruit/Adafruit_nRF52_Bootloader
|
|
||||||
_NRF52_BOOTLOADER_VID = 0x239A
|
|
||||||
_NRF52_BOOTLOADER_PIDS = {
|
|
||||||
0x0029, # Adafruit nRF52 bootloader (generic, used by RAK4631)
|
|
||||||
0x002A, # Adafruit Feather Express bootloader variant
|
|
||||||
0x4029, # alt seen on some boards
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _find_nrf52_bootloader_port() -> dict[str, Any] | None:
|
|
||||||
"""Return a dict for any currently-enumerated nRF52 bootloader port, or None."""
|
|
||||||
for d in devices.list_devices(include_unknown=True):
|
|
||||||
vid_str = d.get("vid")
|
|
||||||
pid_str = d.get("pid")
|
|
||||||
if vid_str is None or pid_str is None:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
vid = int(vid_str, 16) if isinstance(vid_str, str) else int(vid_str)
|
|
||||||
pid = int(pid_str, 16) if isinstance(pid_str, str) else int(pid_str)
|
|
||||||
except ValueError:
|
|
||||||
continue
|
|
||||||
if vid == _NRF52_BOOTLOADER_VID and pid in _NRF52_BOOTLOADER_PIDS:
|
|
||||||
return d
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def touch_1200bps(
|
|
||||||
port: str,
|
|
||||||
settle_ms: int = 250,
|
|
||||||
poll_timeout_s: float = 8.0,
|
|
||||||
retries: int = 2,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Open port at 1200 baud, close immediately - triggers USB CDC bootloader.
|
|
||||||
|
|
||||||
Works for: nRF52840 (Adafruit bootloader), ESP32-S3 (native USB download
|
|
||||||
mode), RP2040 (when built with 1200bps-reset stdio), Arduino Leonardo/Micro.
|
|
||||||
|
|
||||||
For nRF52 specifically: after the touch, polls for the Adafruit bootloader
|
|
||||||
VID/PID (0x239A / 0x0029) for up to `poll_timeout_s` seconds. Adafruit's
|
|
||||||
bootloader docs note a touch sometimes needs to be repeated, so this
|
|
||||||
retries up to `retries` times. The returned `new_port` is the bootloader
|
|
||||||
port (distinct from the app port) - exactly what's needed for `pio run
|
|
||||||
-t upload` to drive nrfutil.
|
|
||||||
|
|
||||||
For non-nRF52 devices (ESP32-S3, RP2040, Arduino), falls back to
|
|
||||||
"any-new-port appeared" detection.
|
|
||||||
|
|
||||||
Returns `{ok, former_port, new_port, new_port_vid_pid, attempts}`.
|
|
||||||
"""
|
|
||||||
connection.reject_if_tcp(port, "touch_1200bps")
|
|
||||||
before_list = devices.list_devices(include_unknown=True)
|
|
||||||
before_ports = {d["port"] for d in before_list}
|
|
||||||
|
|
||||||
attempts = 0
|
|
||||||
new_port_info: dict[str, Any] | None = None
|
|
||||||
|
|
||||||
for attempt in range(1, retries + 1):
|
|
||||||
attempts = attempt
|
|
||||||
_do_1200bps_touch(port, settle_ms=settle_ms, touch_timeout_s=3.0)
|
|
||||||
|
|
||||||
# Poll for either (a) the nRF52 bootloader VID/PID appearing, or
|
|
||||||
# (b) a brand-new port appearing that wasn't there before.
|
|
||||||
deadline = time.monotonic() + poll_timeout_s
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
time.sleep(0.2)
|
|
||||||
|
|
||||||
bootloader = _find_nrf52_bootloader_port()
|
|
||||||
if bootloader is not None:
|
|
||||||
new_port_info = bootloader
|
|
||||||
break
|
|
||||||
|
|
||||||
current = devices.list_devices(include_unknown=True)
|
|
||||||
current_paths = {d["port"] for d in current}
|
|
||||||
added = current_paths - before_ports
|
|
||||||
if added:
|
|
||||||
added_record = next((d for d in current if d["port"] in added), None)
|
|
||||||
if added_record:
|
|
||||||
new_port_info = added_record
|
|
||||||
break
|
|
||||||
|
|
||||||
if new_port_info is not None:
|
|
||||||
break
|
|
||||||
# No bootloader appeared; try touching again (Adafruit recommends
|
|
||||||
# sometimes requiring two touches for reliability).
|
|
||||||
|
|
||||||
if new_port_info is not None:
|
|
||||||
return {
|
|
||||||
"ok": True,
|
|
||||||
"former_port": port,
|
|
||||||
"new_port": new_port_info["port"],
|
|
||||||
"new_port_vid_pid": (
|
|
||||||
new_port_info.get("vid"),
|
|
||||||
new_port_info.get("pid"),
|
|
||||||
),
|
|
||||||
"attempts": attempts,
|
|
||||||
}
|
|
||||||
|
|
||||||
return {
|
|
||||||
"ok": False,
|
|
||||||
"former_port": port,
|
|
||||||
"new_port": None,
|
|
||||||
"new_port_vid_pid": (None, None),
|
|
||||||
"attempts": attempts,
|
|
||||||
}
|
|
||||||
@@ -1,248 +0,0 @@
|
|||||||
"""Direct wrappers around vendor flashing tools: esptool, nrfutil, picotool.
|
|
||||||
|
|
||||||
These are escape hatches. Prefer the pio-based tools in flash.py when they
|
|
||||||
cover the operation - pio knows the correct offsets, protocols, and filters
|
|
||||||
for every supported board. Use these when pio doesn't: to erase a bricked
|
|
||||||
ESP32, DFU-flash an nRF52 zip package, or inspect an RP2040's bootloader.
|
|
||||||
|
|
||||||
Every destructive `*_raw` subcommand is gated by `confirm=True` so callers
|
|
||||||
can't accidentally `--write-flash` from freeform args.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Sequence
|
|
||||||
|
|
||||||
from . import config, connection, pio
|
|
||||||
|
|
||||||
_TIMEOUT_SHORT = 30
|
|
||||||
_TIMEOUT_LONG = 600
|
|
||||||
|
|
||||||
|
|
||||||
class ToolError(RuntimeError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _run(
|
|
||||||
binary: Path,
|
|
||||||
args: Sequence[str],
|
|
||||||
*,
|
|
||||||
timeout: float = _TIMEOUT_LONG,
|
|
||||||
cwd: Path | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
# Shared with pio.run(): if `MESHTASTIC_MCP_FLASH_LOG` is set, each line
|
|
||||||
# of output is tee'd to that file as it arrives so the TUI can show live
|
|
||||||
# esptool/nrfutil/picotool progress instead of 3 minutes of silence.
|
|
||||||
full = [str(binary), *args]
|
|
||||||
try:
|
|
||||||
rc, stdout, stderr, duration = pio._run_capturing(
|
|
||||||
full,
|
|
||||||
cwd=cwd,
|
|
||||||
timeout=timeout,
|
|
||||||
tee_header=f"{binary.name} {' '.join(args)}",
|
|
||||||
)
|
|
||||||
except subprocess.TimeoutExpired as exc:
|
|
||||||
raise ToolError(
|
|
||||||
f"{binary.name} {' '.join(args)} timed out after {timeout}s"
|
|
||||||
) from exc
|
|
||||||
return {
|
|
||||||
"exit_code": rc,
|
|
||||||
"stdout": stdout,
|
|
||||||
"stderr": stderr,
|
|
||||||
"stdout_tail": pio.tail_lines(stdout, 200),
|
|
||||||
"stderr_tail": pio.tail_lines(stderr, 200),
|
|
||||||
"duration_s": round(duration, 2),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _require_confirm(confirm: bool, what: str) -> None:
|
|
||||||
if not confirm:
|
|
||||||
raise ToolError(f"{what} is destructive and requires confirm=True.")
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- esptool --------------------------------------------------------
|
|
||||||
|
|
||||||
ESPTOOL_DESTRUCTIVE = {
|
|
||||||
"write_flash",
|
|
||||||
"write-flash",
|
|
||||||
"erase_flash",
|
|
||||||
"erase-flash",
|
|
||||||
"erase_region",
|
|
||||||
"erase-region",
|
|
||||||
"merge_bin",
|
|
||||||
"merge-bin",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_esptool_chip_info(stdout: str) -> dict[str, Any]:
|
|
||||||
"""Parse `esptool chip_id` / `flash_id` output into structured fields."""
|
|
||||||
result: dict[str, Any] = {
|
|
||||||
"chip": None,
|
|
||||||
"mac": None,
|
|
||||||
"crystal_mhz": None,
|
|
||||||
"flash_size": None,
|
|
||||||
"features": [],
|
|
||||||
}
|
|
||||||
for line in stdout.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
if m := re.match(r"Chip is (.+)", line):
|
|
||||||
result["chip"] = m.group(1).strip()
|
|
||||||
elif m := re.match(r"MAC: ([0-9a-fA-F:]+)", line):
|
|
||||||
result["mac"] = m.group(1)
|
|
||||||
elif m := re.match(r"Crystal is (\d+)MHz", line):
|
|
||||||
result["crystal_mhz"] = int(m.group(1))
|
|
||||||
elif m := re.match(r"Detected flash size: (\S+)", line):
|
|
||||||
result["flash_size"] = m.group(1)
|
|
||||||
elif m := re.match(r"Features: (.+)", line):
|
|
||||||
result["features"] = [f.strip() for f in m.group(1).split(",") if f.strip()]
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def esptool_chip_info(port: str) -> dict[str, Any]:
|
|
||||||
connection.reject_if_tcp(port, "esptool_chip_info")
|
|
||||||
binary = config.esptool_bin()
|
|
||||||
# `chip_id` prints chip + mac + crystal + features. `flash_id` adds flash.
|
|
||||||
combined = _run(binary, ["--port", port, "flash_id"], timeout=_TIMEOUT_SHORT)
|
|
||||||
if combined["exit_code"] != 0:
|
|
||||||
raise ToolError(
|
|
||||||
f"esptool failed (exit {combined['exit_code']}):\n{combined['stderr_tail']}"
|
|
||||||
)
|
|
||||||
parsed = _parse_esptool_chip_info(combined["stdout"])
|
|
||||||
return {**parsed, "raw_stdout_tail": combined["stdout_tail"]}
|
|
||||||
|
|
||||||
|
|
||||||
def esptool_erase_flash(port: str, confirm: bool = False) -> dict[str, Any]:
|
|
||||||
"""Full-chip erase. Leaves the device unbootable until reflashed."""
|
|
||||||
_require_confirm(confirm, "esptool_erase_flash")
|
|
||||||
connection.reject_if_tcp(port, "esptool_erase_flash")
|
|
||||||
binary = config.esptool_bin()
|
|
||||||
# esptool v5 uses `erase-flash`, older uses `erase_flash`. Try the new name
|
|
||||||
# first; if it fails with unknown command, retry old.
|
|
||||||
res = _run(binary, ["--port", port, "erase-flash"], timeout=_TIMEOUT_LONG)
|
|
||||||
if (
|
|
||||||
res["exit_code"] != 0
|
|
||||||
and "unrecognized" in (res["stderr"] or res["stdout"]).lower()
|
|
||||||
):
|
|
||||||
res = _run(binary, ["--port", port, "erase_flash"], timeout=_TIMEOUT_LONG)
|
|
||||||
return res
|
|
||||||
|
|
||||||
|
|
||||||
def esptool_raw(
|
|
||||||
args: list[str], port: str | None = None, confirm: bool = False
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Raw esptool passthrough. Destructive subcommands require confirm=True."""
|
|
||||||
if not args:
|
|
||||||
raise ToolError("args must not be empty")
|
|
||||||
connection.reject_if_tcp(port, "esptool_raw")
|
|
||||||
# Find the first non-flag arg (the subcommand).
|
|
||||||
subcommand = next((a for a in args if not a.startswith("-")), None)
|
|
||||||
if subcommand and subcommand.replace("-", "_") in {
|
|
||||||
s.replace("-", "_") for s in ESPTOOL_DESTRUCTIVE
|
|
||||||
}:
|
|
||||||
_require_confirm(confirm, f"esptool {subcommand}")
|
|
||||||
|
|
||||||
binary = config.esptool_bin()
|
|
||||||
full_args: list[str] = []
|
|
||||||
if port:
|
|
||||||
full_args.extend(["--port", port])
|
|
||||||
full_args.extend(args)
|
|
||||||
return _run(binary, full_args, timeout=_TIMEOUT_LONG)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- nrfutil --------------------------------------------------------
|
|
||||||
|
|
||||||
NRFUTIL_DESTRUCTIVE = {"dfu", "settings"}
|
|
||||||
|
|
||||||
|
|
||||||
def nrfutil_dfu(port: str, package_path: str, confirm: bool = False) -> dict[str, Any]:
|
|
||||||
_require_confirm(confirm, "nrfutil_dfu")
|
|
||||||
connection.reject_if_tcp(port, "nrfutil_dfu")
|
|
||||||
pkg = Path(package_path).expanduser()
|
|
||||||
if not pkg.is_file():
|
|
||||||
raise ToolError(f"Package not found: {pkg}")
|
|
||||||
binary = config.nrfutil_bin()
|
|
||||||
return _run(
|
|
||||||
binary,
|
|
||||||
["dfu", "serial", "--package", str(pkg), "--port", port, "-b", "115200"],
|
|
||||||
timeout=_TIMEOUT_LONG,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def nrfutil_raw(args: list[str], confirm: bool = False) -> dict[str, Any]:
|
|
||||||
if not args:
|
|
||||||
raise ToolError("args must not be empty")
|
|
||||||
subcommand = next((a for a in args if not a.startswith("-")), None)
|
|
||||||
if subcommand in NRFUTIL_DESTRUCTIVE:
|
|
||||||
_require_confirm(confirm, f"nrfutil {subcommand}")
|
|
||||||
binary = config.nrfutil_bin()
|
|
||||||
return _run(binary, args, timeout=_TIMEOUT_LONG)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- picotool -------------------------------------------------------
|
|
||||||
|
|
||||||
PICOTOOL_DESTRUCTIVE = {"load", "reboot", "save", "erase"}
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_picotool_info(stdout: str) -> dict[str, Any]:
|
|
||||||
result: dict[str, Any] = {
|
|
||||||
"vendor": None,
|
|
||||||
"product": None,
|
|
||||||
"serial": None,
|
|
||||||
"flash_size": None,
|
|
||||||
"program_name": None,
|
|
||||||
"program_version": None,
|
|
||||||
}
|
|
||||||
for line in stdout.splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
if m := re.match(r"Program information:", line):
|
|
||||||
continue
|
|
||||||
if m := re.match(r"name:\s*(.+)", line):
|
|
||||||
result["program_name"] = m.group(1).strip()
|
|
||||||
elif m := re.match(r"version:\s*(.+)", line):
|
|
||||||
result["program_version"] = m.group(1).strip()
|
|
||||||
elif m := re.match(r"vendor:\s*(.+)", line):
|
|
||||||
result["vendor"] = m.group(1).strip()
|
|
||||||
elif m := re.match(r"product:\s*(.+)", line):
|
|
||||||
result["product"] = m.group(1).strip()
|
|
||||||
elif m := re.match(r"serial number:\s*(.+)", line):
|
|
||||||
result["serial"] = m.group(1).strip()
|
|
||||||
elif m := re.match(r"flash size:\s*(.+)", line):
|
|
||||||
result["flash_size"] = m.group(1).strip()
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def picotool_info(port: str | None = None) -> dict[str, Any]:
|
|
||||||
"""Read device info from a Pico in BOOTSEL mode. `port` is informational
|
|
||||||
only - picotool auto-detects."""
|
|
||||||
connection.reject_if_tcp(port, "picotool_info")
|
|
||||||
binary = config.picotool_bin()
|
|
||||||
res = _run(binary, ["info", "-a"], timeout=_TIMEOUT_SHORT)
|
|
||||||
if res["exit_code"] != 0:
|
|
||||||
raise ToolError(
|
|
||||||
f"picotool info failed (exit {res['exit_code']}): "
|
|
||||||
"is the Pico in BOOTSEL mode?\n" + res["stderr_tail"]
|
|
||||||
)
|
|
||||||
parsed = _parse_picotool_info(res["stdout"])
|
|
||||||
return {**parsed, "raw_stdout_tail": res["stdout_tail"]}
|
|
||||||
|
|
||||||
|
|
||||||
def picotool_load(uf2_path: str, confirm: bool = False) -> dict[str, Any]:
|
|
||||||
_require_confirm(confirm, "picotool_load")
|
|
||||||
uf2 = Path(uf2_path).expanduser()
|
|
||||||
if not uf2.is_file():
|
|
||||||
raise ToolError(f"UF2 not found: {uf2}")
|
|
||||||
binary = config.picotool_bin()
|
|
||||||
return _run(binary, ["load", "-x", "-t", "uf2", str(uf2)], timeout=_TIMEOUT_LONG)
|
|
||||||
|
|
||||||
|
|
||||||
def picotool_raw(args: list[str], confirm: bool = False) -> dict[str, Any]:
|
|
||||||
if not args:
|
|
||||||
raise ToolError("args must not be empty")
|
|
||||||
subcommand = next((a for a in args if not a.startswith("-")), None)
|
|
||||||
if subcommand in PICOTOOL_DESTRUCTIVE:
|
|
||||||
_require_confirm(confirm, f"picotool {subcommand}")
|
|
||||||
binary = config.picotool_bin()
|
|
||||||
return _run(binary, args, timeout=_TIMEOUT_LONG)
|
|
||||||
@@ -1,103 +0,0 @@
|
|||||||
"""Read-only device queries via meshtastic.SerialInterface."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from .connection import connect
|
|
||||||
|
|
||||||
|
|
||||||
def _primary_channel_name(iface) -> str | None:
|
|
||||||
try:
|
|
||||||
channels = iface.localNode.channels or []
|
|
||||||
except AttributeError:
|
|
||||||
return None
|
|
||||||
for ch in channels:
|
|
||||||
role = getattr(ch, "role", None)
|
|
||||||
# Role enum: 0 DISABLED, 1 PRIMARY, 2 SECONDARY
|
|
||||||
if role == 1:
|
|
||||||
name = getattr(getattr(ch, "settings", None), "name", None)
|
|
||||||
return name or "(default)"
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def device_info(port: str | None = None, timeout_s: float = 8.0) -> dict[str, Any]:
|
|
||||||
"""Return summary info for the connected device."""
|
|
||||||
with connect(port=port, timeout_s=timeout_s) as iface:
|
|
||||||
my = iface.myInfo
|
|
||||||
meta = iface.metadata
|
|
||||||
local = iface.localNode
|
|
||||||
|
|
||||||
# Owner (long/short name) is on the local node's user record
|
|
||||||
long_name: str | None = None
|
|
||||||
short_name: str | None = None
|
|
||||||
hw_model: str | int | None = None
|
|
||||||
if iface.nodesByNum and my is not None:
|
|
||||||
local_rec = iface.nodesByNum.get(my.my_node_num, {})
|
|
||||||
user = local_rec.get("user") or {}
|
|
||||||
long_name = user.get("longName")
|
|
||||||
short_name = user.get("shortName")
|
|
||||||
hw_model = user.get("hwModel")
|
|
||||||
|
|
||||||
region = None
|
|
||||||
if local is not None and local.localConfig is not None:
|
|
||||||
try:
|
|
||||||
lora = local.localConfig.lora
|
|
||||||
# region is an enum; get its string name
|
|
||||||
region = (
|
|
||||||
lora.DESCRIPTOR.fields_by_name["region"]
|
|
||||||
.enum_type.values_by_number[lora.region]
|
|
||||||
.name
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
region = None
|
|
||||||
|
|
||||||
return {
|
|
||||||
"port": iface.devPath if hasattr(iface, "devPath") else port,
|
|
||||||
"my_node_num": getattr(my, "my_node_num", None),
|
|
||||||
"long_name": long_name,
|
|
||||||
"short_name": short_name,
|
|
||||||
"firmware_version": getattr(meta, "firmware_version", None),
|
|
||||||
"hw_model": hw_model,
|
|
||||||
"region": region,
|
|
||||||
"num_nodes": len(iface.nodesByNum) if iface.nodesByNum else 0,
|
|
||||||
"primary_channel": _primary_channel_name(iface),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _node_record(node_dict: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
user = node_dict.get("user") or {}
|
|
||||||
position = node_dict.get("position") or None
|
|
||||||
device_metrics = node_dict.get("deviceMetrics") or {}
|
|
||||||
return {
|
|
||||||
"node_num": node_dict.get("num"),
|
|
||||||
"user": {
|
|
||||||
"long_name": user.get("longName"),
|
|
||||||
"short_name": user.get("shortName"),
|
|
||||||
"hw_model": user.get("hwModel"),
|
|
||||||
"role": user.get("role"),
|
|
||||||
},
|
|
||||||
"position": (
|
|
||||||
{
|
|
||||||
"latitude": position.get("latitude"),
|
|
||||||
"longitude": position.get("longitude"),
|
|
||||||
"altitude": position.get("altitude"),
|
|
||||||
"time": position.get("time"),
|
|
||||||
}
|
|
||||||
if position
|
|
||||||
else None
|
|
||||||
),
|
|
||||||
"snr": node_dict.get("snr"),
|
|
||||||
"rssi": node_dict.get("rssi"),
|
|
||||||
"last_heard": node_dict.get("lastHeard"),
|
|
||||||
"battery_level": device_metrics.get("batteryLevel"),
|
|
||||||
"is_favorite": bool(node_dict.get("isFavorite", False)),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def list_nodes(port: str | None = None, timeout_s: float = 8.0) -> list[dict[str, Any]]:
|
|
||||||
"""Return the device's node database."""
|
|
||||||
with connect(port=port, timeout_s=timeout_s) as iface:
|
|
||||||
if not iface.nodesByNum:
|
|
||||||
return []
|
|
||||||
return [_node_record(n) for n in iface.nodesByNum.values()]
|
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
"""Python mirror of firmware `enum input_broker_event` (src/input/InputBroker.h).
|
|
||||||
|
|
||||||
Used by `admin.send_input_event` + `tests/ui/` so callers can say
|
|
||||||
`InputEventCode.RIGHT` instead of hard-coding 20. Values MUST stay in sync
|
|
||||||
with the firmware enum - unit test `tests/unit/test_input_event_codes.py`
|
|
||||||
pins the mapping.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from enum import IntEnum
|
|
||||||
|
|
||||||
|
|
||||||
class InputEventCode(IntEnum):
|
|
||||||
"""Button / key / gesture events dispatched by the firmware InputBroker."""
|
|
||||||
|
|
||||||
NONE = 0
|
|
||||||
SELECT = 10
|
|
||||||
SELECT_LONG = 11
|
|
||||||
UP_LONG = 12
|
|
||||||
DOWN_LONG = 13
|
|
||||||
UP = 17
|
|
||||||
DOWN = 18
|
|
||||||
LEFT = 19
|
|
||||||
RIGHT = 20
|
|
||||||
CANCEL = 24
|
|
||||||
BACK = 27
|
|
||||||
# Auto-incremented values in the C enum (27 + 1, +2, +3):
|
|
||||||
USER_PRESS = 28
|
|
||||||
ALT_PRESS = 29
|
|
||||||
ALT_LONG = 30
|
|
||||||
SHUTDOWN = 0x9B
|
|
||||||
GPS_TOGGLE = 0x9E
|
|
||||||
SEND_PING = 0xAF
|
|
||||||
FN_F1 = 0xF1
|
|
||||||
FN_F2 = 0xF2
|
|
||||||
FN_F3 = 0xF3
|
|
||||||
FN_F4 = 0xF4
|
|
||||||
FN_F5 = 0xF5
|
|
||||||
MATRIXKEY = 0xFE
|
|
||||||
ANYKEY = 0xFF
|
|
||||||
|
|
||||||
|
|
||||||
def coerce_event_code(value: int | str | InputEventCode) -> int:
|
|
||||||
"""Accept an int, a case-insensitive name, or an `InputEventCode` and return
|
|
||||||
the u8 wire value. Raises ValueError on unknown names / out-of-range ints.
|
|
||||||
"""
|
|
||||||
if isinstance(value, InputEventCode):
|
|
||||||
return int(value)
|
|
||||||
if isinstance(value, int):
|
|
||||||
if not 0 <= value <= 255:
|
|
||||||
raise ValueError(f"event_code out of u8 range: {value}")
|
|
||||||
return value
|
|
||||||
if isinstance(value, str):
|
|
||||||
key = value.upper().replace("-", "_")
|
|
||||||
if key.startswith("INPUT_BROKER_"):
|
|
||||||
key = key[len("INPUT_BROKER_") :]
|
|
||||||
try:
|
|
||||||
return int(InputEventCode[key])
|
|
||||||
except KeyError as exc:
|
|
||||||
known = ", ".join(m.name for m in InputEventCode)
|
|
||||||
raise ValueError(
|
|
||||||
f"unknown event code name {value!r}; known: {known}"
|
|
||||||
) from exc
|
|
||||||
raise TypeError(
|
|
||||||
f"event_code must be int|str|InputEventCode, got {type(value).__name__}"
|
|
||||||
)
|
|
||||||
@@ -1,410 +0,0 @@
|
|||||||
"""Read-side queries over the recorder's JSONL streams.
|
|
||||||
|
|
||||||
Pure functions over `mcp-server/.mtlog/`. Streaming JSONL reader: never
|
|
||||||
loads a whole file. Time-bound queries short-circuit as soon as `ts`
|
|
||||||
exceeds the requested end. The recorder writes monotonically, so a
|
|
||||||
forward scan is cheap; we don't need an index.
|
|
||||||
|
|
||||||
All time arguments accept:
|
|
||||||
- epoch seconds (int/float)
|
|
||||||
- relative strings: "-15m", "-2h", "-3d", "now"
|
|
||||||
- ISO-ish absolute strings: "2026-05-07T14:30:00" (naive timestamps are
|
|
||||||
treated as UTC)
|
|
||||||
|
|
||||||
Tools that return data ALWAYS cap their output (max_lines / max_points
|
|
||||||
/ max), and report whether more matched than was returned.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import gzip
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
import statistics
|
|
||||||
import time
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Iterator
|
|
||||||
|
|
||||||
from .recorder.recorder import get_recorder
|
|
||||||
|
|
||||||
_REL_RE = re.compile(r"^\s*-\s*(\d+(?:\.\d+)?)\s*([smhd])\s*$")
|
|
||||||
_REGEX_PREVIEW_MAX = 100
|
|
||||||
_REGEX_PREVIEW_TRUNCATE = 97
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_time(value: Any, *, now: float | None = None) -> float:
|
|
||||||
"""Coerce to epoch seconds. Defaults `now` to `time.time()`."""
|
|
||||||
if value is None:
|
|
||||||
return time.time()
|
|
||||||
if isinstance(value, (int, float)):
|
|
||||||
return float(value)
|
|
||||||
if not isinstance(value, str):
|
|
||||||
raise ValueError(f"invalid time: {value!r}")
|
|
||||||
s = value.strip().lower()
|
|
||||||
if s in ("", "now"):
|
|
||||||
return time.time() if now is None else now
|
|
||||||
m = _REL_RE.match(s)
|
|
||||||
if m:
|
|
||||||
n = float(m.group(1))
|
|
||||||
unit = m.group(2)
|
|
||||||
secs = n * {"s": 1, "m": 60, "h": 3600, "d": 86400}[unit]
|
|
||||||
base = time.time() if now is None else now
|
|
||||||
return base - secs
|
|
||||||
# Try ISO 8601. Accept naive (assume UTC) and Z-suffixed.
|
|
||||||
try:
|
|
||||||
if s.endswith("z"):
|
|
||||||
s = s[:-1] + "+00:00"
|
|
||||||
dt = datetime.fromisoformat(s)
|
|
||||||
if dt.tzinfo is None:
|
|
||||||
dt = dt.replace(tzinfo=timezone.utc)
|
|
||||||
return dt.timestamp()
|
|
||||||
except ValueError as e:
|
|
||||||
raise ValueError(f"unparseable time: {value!r}") from e
|
|
||||||
|
|
||||||
|
|
||||||
def _iter_jsonl(path: Path, *, since: float, until: float) -> Iterator[dict[str, Any]]:
|
|
||||||
"""Stream records in chronological order: rotated archives first
|
|
||||||
(oldest → newest by lex sort, which is chronological for our
|
|
||||||
`YYYYMMDD-HHMMSS-uuuuuu-NNNNN` archive naming), then the live file
|
|
||||||
last. The "keep last N" pop-front logic in the window queries
|
|
||||||
relies on records arriving in time order across files.
|
|
||||||
"""
|
|
||||||
files: list[Path] = []
|
|
||||||
# Gzipped archives are named "<stem>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz".
|
|
||||||
for archive in sorted(path.parent.glob(f"{path.stem}.*.jsonl.gz")):
|
|
||||||
files.append(archive)
|
|
||||||
if path.exists():
|
|
||||||
files.append(path)
|
|
||||||
for f in files:
|
|
||||||
opener = gzip.open if f.suffix == ".gz" else open
|
|
||||||
try:
|
|
||||||
with opener(f, "rt", encoding="utf-8") as fh: # type: ignore[arg-type]
|
|
||||||
for line in fh:
|
|
||||||
line = line.strip()
|
|
||||||
if not line:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
rec = json.loads(line)
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
continue
|
|
||||||
ts = rec.get("ts")
|
|
||||||
if not isinstance(ts, (int, float)):
|
|
||||||
continue
|
|
||||||
if ts < since:
|
|
||||||
continue
|
|
||||||
if ts > until:
|
|
||||||
# Records are append-monotonic within a file, so
|
|
||||||
# the rest of this file is also past `until`.
|
|
||||||
# Archives can still overlap each other, so only
|
|
||||||
# short-circuit this file, not the whole scan.
|
|
||||||
break
|
|
||||||
yield rec
|
|
||||||
except (FileNotFoundError, OSError):
|
|
||||||
continue
|
|
||||||
|
|
||||||
|
|
||||||
# -- queries ------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def logs_window(
|
|
||||||
start: Any = "-15m",
|
|
||||||
end: Any = "now",
|
|
||||||
*,
|
|
||||||
grep: str | None = None,
|
|
||||||
level: str | None = None,
|
|
||||||
tag: str | None = None,
|
|
||||||
port: str | None = None,
|
|
||||||
max_lines: int = 200,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Recent firmware log lines, filtered.
|
|
||||||
|
|
||||||
`level` accepts a single level name or pipe-separated set
|
|
||||||
("WARN|ERROR|CRIT"). `grep` is a regex (Python re) over the raw
|
|
||||||
`line` field. Returns the last `max_lines` matches.
|
|
||||||
"""
|
|
||||||
s = _parse_time(start)
|
|
||||||
e = _parse_time(end)
|
|
||||||
levels = _split_set(level)
|
|
||||||
if grep:
|
|
||||||
try:
|
|
||||||
grep_re = re.compile(grep)
|
|
||||||
except re.error as exc:
|
|
||||||
preview = (
|
|
||||||
grep
|
|
||||||
if len(grep) <= _REGEX_PREVIEW_MAX
|
|
||||||
else f"{grep[:_REGEX_PREVIEW_TRUNCATE]}..."
|
|
||||||
)
|
|
||||||
raise ValueError(f"invalid grep regex {preview!r}: {exc}") from exc
|
|
||||||
else:
|
|
||||||
grep_re = None
|
|
||||||
|
|
||||||
base = get_recorder().base_dir
|
|
||||||
matched = 0
|
|
||||||
out: list[dict[str, Any]] = []
|
|
||||||
for rec in _iter_jsonl(base / "logs.jsonl", since=s, until=e):
|
|
||||||
if levels and rec.get("level") not in levels:
|
|
||||||
continue
|
|
||||||
if tag and rec.get("tag") != tag:
|
|
||||||
continue
|
|
||||||
if port and rec.get("port") != port:
|
|
||||||
continue
|
|
||||||
if grep_re and not grep_re.search(rec.get("line") or ""):
|
|
||||||
continue
|
|
||||||
matched += 1
|
|
||||||
out.append(rec)
|
|
||||||
if len(out) > max_lines:
|
|
||||||
out.pop(0) # keep the most recent N
|
|
||||||
return {
|
|
||||||
"lines": out,
|
|
||||||
"total_matched": matched,
|
|
||||||
"dropped": max(0, matched - max_lines),
|
|
||||||
"window": {"start": s, "end": e},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def telemetry_timeline(
|
|
||||||
window: Any = "1h",
|
|
||||||
*,
|
|
||||||
variant: str = "local",
|
|
||||||
field: str = "free_heap",
|
|
||||||
port: str | None = None,
|
|
||||||
max_points: int = 200,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Timeseries of one telemetry field, downsampled.
|
|
||||||
|
|
||||||
`field` matches both the protobuf snake_case name (`free_heap`,
|
|
||||||
`heap_free_bytes`, `battery_level`) and camelCase (`freeHeap`).
|
|
||||||
Server-side bucket-mean downsamples to ≤ `max_points`. Returns
|
|
||||||
`slope_per_min` (linear regression slope, units/min) so a leak
|
|
||||||
detector can read one number.
|
|
||||||
"""
|
|
||||||
end = time.time()
|
|
||||||
if isinstance(window, (int, float)):
|
|
||||||
# Numeric `window` is a duration in seconds - "last N seconds".
|
|
||||||
# Without this branch, `_parse_time(-N)` would treat -N as an
|
|
||||||
# absolute epoch timestamp (i.e., Jan 1 1970 minus N seconds),
|
|
||||||
# producing a wildly negative `start` and matching nothing.
|
|
||||||
start = end - float(window)
|
|
||||||
elif isinstance(window, str) and not window.startswith("-"):
|
|
||||||
# Bare string like "1h" is sugar for "-1h".
|
|
||||||
start = _parse_time(f"-{window}", now=end)
|
|
||||||
else:
|
|
||||||
start = _parse_time(window, now=end)
|
|
||||||
|
|
||||||
base = get_recorder().base_dir
|
|
||||||
raw: list[tuple[float, float]] = []
|
|
||||||
field_aliases = _field_aliases(field)
|
|
||||||
for rec in _iter_jsonl(base / "telemetry.jsonl", since=start, until=end):
|
|
||||||
if rec.get("variant") != variant:
|
|
||||||
continue
|
|
||||||
if port and rec.get("port") != port:
|
|
||||||
continue
|
|
||||||
fields = rec.get("fields") or {}
|
|
||||||
value: Any = None
|
|
||||||
for alias in field_aliases:
|
|
||||||
if alias in fields:
|
|
||||||
value = fields[alias]
|
|
||||||
break
|
|
||||||
if not isinstance(value, (int, float)):
|
|
||||||
continue
|
|
||||||
raw.append((float(rec["ts"]), float(value)))
|
|
||||||
|
|
||||||
if not raw:
|
|
||||||
return {
|
|
||||||
"points": [],
|
|
||||||
"samples": 0,
|
|
||||||
"min": None,
|
|
||||||
"max": None,
|
|
||||||
"slope_per_min": None,
|
|
||||||
"window": {"start": start, "end": end, "variant": variant, "field": field},
|
|
||||||
}
|
|
||||||
|
|
||||||
points = _downsample(raw, max_points=max_points)
|
|
||||||
values = [v for _, v in raw]
|
|
||||||
return {
|
|
||||||
"points": [{"ts": ts, "value": v} for ts, v in points],
|
|
||||||
"samples": len(raw),
|
|
||||||
"min": min(values),
|
|
||||||
"max": max(values),
|
|
||||||
"slope_per_min": _slope_per_min(raw),
|
|
||||||
"window": {"start": start, "end": end, "variant": variant, "field": field},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def packets_window(
|
|
||||||
start: Any = "-5m",
|
|
||||||
end: Any = "now",
|
|
||||||
*,
|
|
||||||
portnum: str | None = None,
|
|
||||||
from_node: str | None = None,
|
|
||||||
to_node: str | None = None,
|
|
||||||
max: int = 200,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
s = _parse_time(start)
|
|
||||||
e = _parse_time(end)
|
|
||||||
portnums = _split_set(portnum)
|
|
||||||
base = get_recorder().base_dir
|
|
||||||
matched = 0
|
|
||||||
out: list[dict[str, Any]] = []
|
|
||||||
for rec in _iter_jsonl(base / "packets.jsonl", since=s, until=e):
|
|
||||||
if portnums and rec.get("portnum") not in portnums:
|
|
||||||
continue
|
|
||||||
if from_node and str(rec.get("from_node")) != str(from_node):
|
|
||||||
continue
|
|
||||||
if to_node and str(rec.get("to_node")) != str(to_node):
|
|
||||||
continue
|
|
||||||
matched += 1
|
|
||||||
out.append(rec)
|
|
||||||
if len(out) > max:
|
|
||||||
out.pop(0)
|
|
||||||
return {
|
|
||||||
"packets": out,
|
|
||||||
"total_matched": matched,
|
|
||||||
"dropped": matched - max if matched > max else 0,
|
|
||||||
"window": {"start": s, "end": e},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def events_window(
|
|
||||||
start: Any = "-1h",
|
|
||||||
end: Any = "now",
|
|
||||||
*,
|
|
||||||
kind: str | None = None,
|
|
||||||
max: int = 200,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
s = _parse_time(start)
|
|
||||||
e = _parse_time(end)
|
|
||||||
kinds = _split_set(kind)
|
|
||||||
base = get_recorder().base_dir
|
|
||||||
matched = 0
|
|
||||||
out: list[dict[str, Any]] = []
|
|
||||||
for rec in _iter_jsonl(base / "events.jsonl", since=s, until=e):
|
|
||||||
if kinds and rec.get("kind") not in kinds:
|
|
||||||
continue
|
|
||||||
matched += 1
|
|
||||||
out.append(rec)
|
|
||||||
if len(out) > max:
|
|
||||||
out.pop(0)
|
|
||||||
return {
|
|
||||||
"events": out,
|
|
||||||
"total_matched": matched,
|
|
||||||
"dropped": matched - max if matched > max else 0,
|
|
||||||
"window": {"start": s, "end": e},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def export(
|
|
||||||
start: Any,
|
|
||||||
end: Any,
|
|
||||||
dest_dir: str,
|
|
||||||
*,
|
|
||||||
streams: list[str] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Bundle a slice of each requested stream into `dest_dir`.
|
|
||||||
|
|
||||||
For a notebook, a bug report, or a Datadog backfill. Output files
|
|
||||||
are uncompressed JSONL (callers gzip themselves if they want to).
|
|
||||||
"""
|
|
||||||
s = _parse_time(start)
|
|
||||||
e = _parse_time(end)
|
|
||||||
selected = streams or ["logs", "telemetry", "packets", "events"]
|
|
||||||
dest = Path(dest_dir)
|
|
||||||
dest.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
base = get_recorder().base_dir
|
|
||||||
paths: dict[str, str] = {}
|
|
||||||
for stream in selected:
|
|
||||||
src = base / f"{stream}.jsonl"
|
|
||||||
if not src.exists() and not list(base.glob(f"{stream}.*.jsonl.gz")):
|
|
||||||
continue
|
|
||||||
out_path = dest / f"{stream}.jsonl"
|
|
||||||
n = 0
|
|
||||||
with out_path.open("w", encoding="utf-8") as fh:
|
|
||||||
for rec in _iter_jsonl(src, since=s, until=e):
|
|
||||||
fh.write(json.dumps(rec, separators=(",", ":")) + "\n")
|
|
||||||
n += 1
|
|
||||||
paths[stream] = str(out_path)
|
|
||||||
paths[f"{stream}_count"] = str(n)
|
|
||||||
return {"dest_dir": str(dest), "paths": paths, "window": {"start": s, "end": e}}
|
|
||||||
|
|
||||||
|
|
||||||
# -- helpers ------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _split_set(value: str | None) -> set[str] | None:
|
|
||||||
if not value:
|
|
||||||
return None
|
|
||||||
return {v.strip() for v in value.split("|") if v.strip()}
|
|
||||||
|
|
||||||
|
|
||||||
def _field_aliases(field: str) -> list[str]:
|
|
||||||
"""Accept snake_case OR camelCase, plus a few legacy aliases."""
|
|
||||||
snake = field
|
|
||||||
camel = _snake_to_camel(field)
|
|
||||||
aliases = {snake, camel}
|
|
||||||
# Old protobuf fields (pre-LocalStats) used different names
|
|
||||||
legacy = {
|
|
||||||
"free_heap": ["free_heap", "freeHeap", "heap_free_bytes", "heapFreeBytes"],
|
|
||||||
"heap_free_bytes": [
|
|
||||||
"heap_free_bytes",
|
|
||||||
"heapFreeBytes",
|
|
||||||
"free_heap",
|
|
||||||
"freeHeap",
|
|
||||||
],
|
|
||||||
"total_heap": ["total_heap", "totalHeap", "heap_total_bytes", "heapTotalBytes"],
|
|
||||||
"heap_total_bytes": [
|
|
||||||
"heap_total_bytes",
|
|
||||||
"heapTotalBytes",
|
|
||||||
"total_heap",
|
|
||||||
"totalHeap",
|
|
||||||
],
|
|
||||||
}
|
|
||||||
if field in legacy:
|
|
||||||
aliases.update(legacy[field])
|
|
||||||
return list(aliases)
|
|
||||||
|
|
||||||
|
|
||||||
def _snake_to_camel(name: str) -> str:
|
|
||||||
parts = name.split("_")
|
|
||||||
return parts[0] + "".join(p.title() for p in parts[1:])
|
|
||||||
|
|
||||||
|
|
||||||
def _downsample(
|
|
||||||
points: list[tuple[float, float]], *, max_points: int
|
|
||||||
) -> list[tuple[float, float]]:
|
|
||||||
if len(points) <= max_points:
|
|
||||||
return points
|
|
||||||
# Even-bucket mean. Preserves shape better than nth-sample picking.
|
|
||||||
n = len(points)
|
|
||||||
bucket = n / max_points
|
|
||||||
out: list[tuple[float, float]] = []
|
|
||||||
i = 0
|
|
||||||
for k in range(max_points):
|
|
||||||
end = int((k + 1) * bucket)
|
|
||||||
end = min(end, n)
|
|
||||||
if end <= i:
|
|
||||||
continue
|
|
||||||
chunk = points[i:end]
|
|
||||||
ts = chunk[len(chunk) // 2][0]
|
|
||||||
val = statistics.fmean(v for _, v in chunk)
|
|
||||||
out.append((ts, val))
|
|
||||||
i = end
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _slope_per_min(points: list[tuple[float, float]]) -> float | None:
|
|
||||||
"""Least-squares slope (units per minute). None if too few points."""
|
|
||||||
if len(points) < 2:
|
|
||||||
return None
|
|
||||||
xs = [t for t, _ in points]
|
|
||||||
ys = [v for _, v in points]
|
|
||||||
n = len(xs)
|
|
||||||
mean_x = sum(xs) / n
|
|
||||||
mean_y = sum(ys) / n
|
|
||||||
num = sum((xs[i] - mean_x) * (ys[i] - mean_y) for i in range(n))
|
|
||||||
den = sum((x - mean_x) ** 2 for x in xs)
|
|
||||||
if den == 0:
|
|
||||||
return None
|
|
||||||
slope_per_sec = num / den
|
|
||||||
return slope_per_sec * 60.0
|
|
||||||
@@ -1,147 +0,0 @@
|
|||||||
"""OCR wrapper for UI tests + the `capture_screen` tool.
|
|
||||||
|
|
||||||
Auto-selects a reader in priority order:
|
|
||||||
1. `easyocr` (deep-learning, high quality on OLED screens - but ~100 MB
|
|
||||||
model download on first use).
|
|
||||||
2. `pytesseract` (requires system `tesseract` binary on PATH).
|
|
||||||
3. `null` - returns `""` with a warning. Tests fall back to log + image
|
|
||||||
evidence when OCR is unavailable.
|
|
||||||
|
|
||||||
Override via `MESHTASTIC_UI_OCR_BACKEND=easyocr|pytesseract|null|auto`
|
|
||||||
(default `auto`).
|
|
||||||
|
|
||||||
`ocr_text(png_bytes) -> str` is the only public entry point. The reader is
|
|
||||||
constructed lazily on first call and cached, so the easyocr cold-start cost
|
|
||||||
only hits once per process.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import functools
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
import sys
|
|
||||||
from typing import Callable
|
|
||||||
|
|
||||||
log = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
def _backend_choice() -> str:
|
|
||||||
return os.environ.get("MESHTASTIC_UI_OCR_BACKEND", "auto").lower()
|
|
||||||
|
|
||||||
|
|
||||||
@functools.lru_cache(maxsize=1)
|
|
||||||
def _reader() -> tuple[str, Callable[[bytes], str]]:
|
|
||||||
"""Return `(backend_name, callable)` for whichever OCR is available."""
|
|
||||||
choice = _backend_choice()
|
|
||||||
|
|
||||||
def _easyocr() -> tuple[str, Callable[[bytes], str]]:
|
|
||||||
import easyocr # type: ignore[import-untyped] # noqa: PLC0415
|
|
||||||
import numpy as np # type: ignore[import-untyped] # noqa: PLC0415
|
|
||||||
|
|
||||||
reader = easyocr.Reader(["en"], gpu=False, verbose=False)
|
|
||||||
|
|
||||||
def _run(png: bytes) -> str:
|
|
||||||
try:
|
|
||||||
import cv2 # type: ignore[import-untyped] # noqa: PLC0415
|
|
||||||
|
|
||||||
arr = np.frombuffer(png, dtype=np.uint8)
|
|
||||||
img = cv2.imdecode(arr, cv2.IMREAD_COLOR)
|
|
||||||
except ImportError:
|
|
||||||
# Fall back to PIL if cv2 isn't around.
|
|
||||||
from io import BytesIO # noqa: PLC0415
|
|
||||||
|
|
||||||
from PIL import Image # type: ignore[import-untyped] # noqa: PLC0415
|
|
||||||
|
|
||||||
img = np.array(Image.open(BytesIO(png)).convert("RGB"))
|
|
||||||
try:
|
|
||||||
results = reader.readtext(img, detail=0, paragraph=True)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
log.warning("easyocr failed: %s", exc)
|
|
||||||
return ""
|
|
||||||
return "\n".join(str(r) for r in results)
|
|
||||||
|
|
||||||
return "easyocr", _run
|
|
||||||
|
|
||||||
def _pytesseract() -> tuple[str, Callable[[bytes], str]]:
|
|
||||||
from io import BytesIO # noqa: PLC0415
|
|
||||||
|
|
||||||
import pytesseract # type: ignore[import-untyped] # noqa: PLC0415
|
|
||||||
from PIL import Image # type: ignore[import-untyped] # noqa: PLC0415
|
|
||||||
|
|
||||||
if shutil.which("tesseract") is None:
|
|
||||||
raise ImportError("`tesseract` binary not on PATH")
|
|
||||||
|
|
||||||
def _run(png: bytes) -> str:
|
|
||||||
try:
|
|
||||||
return str(pytesseract.image_to_string(Image.open(BytesIO(png))))
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
log.warning("pytesseract failed: %s", exc)
|
|
||||||
return ""
|
|
||||||
|
|
||||||
return "pytesseract", _run
|
|
||||||
|
|
||||||
def _null() -> tuple[str, Callable[[bytes], str]]:
|
|
||||||
log.warning(
|
|
||||||
"OCR backend is null; install easyocr or tesseract for text extraction"
|
|
||||||
)
|
|
||||||
return "null", lambda _png: ""
|
|
||||||
|
|
||||||
if choice == "easyocr":
|
|
||||||
return _easyocr()
|
|
||||||
if choice == "pytesseract":
|
|
||||||
return _pytesseract()
|
|
||||||
if choice == "null":
|
|
||||||
return _null()
|
|
||||||
if choice != "auto":
|
|
||||||
print(
|
|
||||||
f"[ocr] unknown MESHTASTIC_UI_OCR_BACKEND={choice!r}; falling back to auto",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
|
|
||||||
# auto mode
|
|
||||||
try:
|
|
||||||
return _easyocr()
|
|
||||||
except ImportError:
|
|
||||||
pass
|
|
||||||
try:
|
|
||||||
return _pytesseract()
|
|
||||||
except ImportError:
|
|
||||||
pass
|
|
||||||
return _null()
|
|
||||||
|
|
||||||
|
|
||||||
def ocr_text(png_bytes: bytes) -> str:
|
|
||||||
"""Run OCR on a PNG-encoded image and return the decoded text (possibly empty)."""
|
|
||||||
if not png_bytes:
|
|
||||||
return ""
|
|
||||||
_, run = _reader()
|
|
||||||
return run(png_bytes)
|
|
||||||
|
|
||||||
|
|
||||||
def backend_name() -> str:
|
|
||||||
"""Return the currently-selected backend name, initializing if necessary."""
|
|
||||||
name, _ = _reader()
|
|
||||||
return name
|
|
||||||
|
|
||||||
|
|
||||||
def warm() -> None:
|
|
||||||
"""Run one dummy inference so the easyocr cold-start cost is paid upfront.
|
|
||||||
|
|
||||||
Pytest session fixture calls this once so the first real capture doesn't
|
|
||||||
eat the model-load latency.
|
|
||||||
"""
|
|
||||||
# A 64×32 white PNG - decodes clean, no text to extract.
|
|
||||||
white_png = bytes.fromhex(
|
|
||||||
"89504e470d0a1a0a0000000d49484452000000400000002008060000007ccac28e"
|
|
||||||
"0000001c49444154785eedc1010d000000c2a0f74f6d0d370000000000000080"
|
|
||||||
"0b010000ffff030000000000000049454e44ae426082"
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
ocr_text(white_png)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
log.warning("ocr.warm() failed: %s", exc)
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = ["backend_name", "ocr_text", "warm"]
|
|
||||||
@@ -1,310 +0,0 @@
|
|||||||
"""Subprocess wrappers around the `pio` CLI.
|
|
||||||
|
|
||||||
Every PlatformIO interaction in this package funnels through `run()` so we
|
|
||||||
have a single place that owns timeouts, buffer sizes, JSON parsing, and the
|
|
||||||
"stderr on exit-0 is informational" convention.
|
|
||||||
|
|
||||||
`run()` has two execution paths:
|
|
||||||
|
|
||||||
* Fast path (default): `subprocess.run(capture_output=True)` - buffered, one
|
|
||||||
return; fine for sub-second pio calls like `pio --version` or
|
|
||||||
`pio project config --json-output`.
|
|
||||||
* Streaming path: when the `MESHTASTIC_MCP_FLASH_LOG` env var is set, each
|
|
||||||
output line is tee'd to that file as it arrives via a threaded reader.
|
|
||||||
The TUI tails the file to give live flash progress - otherwise a 3-minute
|
|
||||||
`pio run -t upload` is completely silent to the operator.
|
|
||||||
|
|
||||||
`hw_tools.py` shares the streaming helper via `pio._run_capturing()` so
|
|
||||||
esptool/nrfutil/picotool output also streams when the env var is set.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Sequence, TextIO
|
|
||||||
|
|
||||||
from . import config
|
|
||||||
|
|
||||||
# 10 MB matches the reference impl (jl-codes/platformio-mcp). Build output can
|
|
||||||
# be hundreds of KB; we'd rather keep it in memory than truncate.
|
|
||||||
_MAX_BUFFER = 10 * 1024 * 1024
|
|
||||||
|
|
||||||
# Per-operation defaults (seconds). None = no timeout.
|
|
||||||
TIMEOUT_DEFAULT = 120
|
|
||||||
TIMEOUT_PROJECT_CONFIG = 60
|
|
||||||
TIMEOUT_DEVICE_LIST = 15
|
|
||||||
TIMEOUT_BUILD = 900
|
|
||||||
TIMEOUT_UPLOAD = 600
|
|
||||||
|
|
||||||
|
|
||||||
class PioError(RuntimeError):
|
|
||||||
"""pio exited non-zero."""
|
|
||||||
|
|
||||||
def __init__(self, args: Sequence[str], returncode: int, stdout: str, stderr: str):
|
|
||||||
self.args = list(args)
|
|
||||||
self.returncode = returncode
|
|
||||||
self.stdout = stdout
|
|
||||||
self.stderr = stderr
|
|
||||||
tail = (stderr or stdout).strip().splitlines()[-20:]
|
|
||||||
super().__init__(
|
|
||||||
f"pio {' '.join(args)} failed with exit {returncode}:\n" + "\n".join(tail)
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class PioTimeout(RuntimeError):
|
|
||||||
"""pio did not return within the timeout."""
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class PioResult:
|
|
||||||
args: list[str]
|
|
||||||
returncode: int
|
|
||||||
stdout: str
|
|
||||||
stderr: str
|
|
||||||
duration_s: float
|
|
||||||
|
|
||||||
|
|
||||||
_FLASH_LOG_ENV = "MESHTASTIC_MCP_FLASH_LOG"
|
|
||||||
|
|
||||||
|
|
||||||
def _flash_log_path() -> Path | None:
|
|
||||||
"""Return the path to tee subprocess output to, or None if streaming off.
|
|
||||||
|
|
||||||
Controlled by `MESHTASTIC_MCP_FLASH_LOG`. `run-tests.sh` sets this to
|
|
||||||
`tests/flash.log`; the TUI tails that file so `pio run -t upload` shows
|
|
||||||
live progress in the pytest pane.
|
|
||||||
"""
|
|
||||||
raw = os.environ.get(_FLASH_LOG_ENV)
|
|
||||||
if not raw:
|
|
||||||
return None
|
|
||||||
return Path(raw)
|
|
||||||
|
|
||||||
|
|
||||||
def _run_capturing(
|
|
||||||
argv: Sequence[str],
|
|
||||||
*,
|
|
||||||
cwd: Path | None = None,
|
|
||||||
timeout: float | None = None,
|
|
||||||
tee_header: str | None = None,
|
|
||||||
extra_env: dict[str, str] | None = None,
|
|
||||||
) -> tuple[int, str, str, float]:
|
|
||||||
"""Run a subprocess, capture stdout+stderr, optionally tee to the flash log.
|
|
||||||
|
|
||||||
Returns `(returncode, stdout_str, stderr_str, duration_s)`. Raises
|
|
||||||
`subprocess.TimeoutExpired` on timeout (callers map this to their own
|
|
||||||
domain-specific error).
|
|
||||||
|
|
||||||
`extra_env` merges into the subprocess environment (parent env stays
|
|
||||||
intact). Used for `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar.
|
|
||||||
|
|
||||||
Fast path: `subprocess.run(capture_output=True)` when no flash log is
|
|
||||||
configured (unchanged behavior).
|
|
||||||
|
|
||||||
Streaming path: `Popen` with line-buffered stdout+stderr pipes; two
|
|
||||||
reader threads accumulate into result strings AND append each line to
|
|
||||||
the flash log file. Stdout and stderr stay separate in the return value
|
|
||||||
(so `stderr_tail` still means stderr), but are interleaved in the log
|
|
||||||
file in the order they arrived - that's what a human wants to read.
|
|
||||||
"""
|
|
||||||
log_path = _flash_log_path()
|
|
||||||
t0 = time.monotonic()
|
|
||||||
env = None
|
|
||||||
if extra_env:
|
|
||||||
env = {**os.environ, **extra_env}
|
|
||||||
|
|
||||||
if log_path is None:
|
|
||||||
# Fast path - unchanged.
|
|
||||||
proc = subprocess.run(
|
|
||||||
list(argv),
|
|
||||||
cwd=str(cwd) if cwd else None,
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
timeout=timeout,
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
return (
|
|
||||||
proc.returncode,
|
|
||||||
proc.stdout or "",
|
|
||||||
proc.stderr or "",
|
|
||||||
time.monotonic() - t0,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Streaming path: line-buffered Popen, threaded readers, tee to file.
|
|
||||||
# Ensure parent directory exists so the first tee write doesn't fail.
|
|
||||||
log_path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
log_fh: TextIO | None = None
|
|
||||||
try:
|
|
||||||
log_fh = log_path.open("a", encoding="utf-8")
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
# Append mode: the TUI truncates on startup, the session may produce
|
|
||||||
# many tee'd commands (erase + flash + factory-reset response), and
|
|
||||||
# we want all of them chronologically in one log.
|
|
||||||
proc = subprocess.Popen( # noqa: S603
|
|
||||||
list(argv),
|
|
||||||
cwd=str(cwd) if cwd else None,
|
|
||||||
stdout=subprocess.PIPE,
|
|
||||||
stderr=subprocess.PIPE,
|
|
||||||
text=True,
|
|
||||||
bufsize=1, # line-buffered
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
stdout_chunks: list[str] = []
|
|
||||||
stderr_chunks: list[str] = []
|
|
||||||
log_lock = threading.Lock()
|
|
||||||
|
|
||||||
def _append_log(line: str) -> None:
|
|
||||||
# Hold the lock briefly to serialize interleaved stdout/stderr writes
|
|
||||||
# so a half-written line from one stream doesn't get garbled by the
|
|
||||||
# other.
|
|
||||||
nonlocal log_fh
|
|
||||||
with log_lock:
|
|
||||||
if log_fh is None:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
log_fh.write(line)
|
|
||||||
log_fh.flush()
|
|
||||||
except OSError:
|
|
||||||
# Log file disappeared (umount, operator deleted the dir).
|
|
||||||
# Don't let that bubble up - the subprocess output is still
|
|
||||||
# collected in-memory for the return value.
|
|
||||||
try:
|
|
||||||
log_fh.close()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
log_fh = None
|
|
||||||
|
|
||||||
def _tee(stream, sink: list[str]) -> None:
|
|
||||||
try:
|
|
||||||
for line in stream:
|
|
||||||
sink.append(line)
|
|
||||||
_append_log(line)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
# Header line so the operator can tell commands apart in the log.
|
|
||||||
if tee_header:
|
|
||||||
_append_log(f"\n--- {tee_header} (start)\n")
|
|
||||||
|
|
||||||
assert proc.stdout is not None and proc.stderr is not None
|
|
||||||
t_out = threading.Thread(
|
|
||||||
target=_tee, args=(proc.stdout, stdout_chunks), daemon=True
|
|
||||||
)
|
|
||||||
t_err = threading.Thread(
|
|
||||||
target=_tee, args=(proc.stderr, stderr_chunks), daemon=True
|
|
||||||
)
|
|
||||||
t_out.start()
|
|
||||||
t_err.start()
|
|
||||||
|
|
||||||
# `Popen.wait` with a timeout is the cleanest way to get TimeoutExpired.
|
|
||||||
try:
|
|
||||||
proc.wait(timeout=timeout)
|
|
||||||
except subprocess.TimeoutExpired:
|
|
||||||
proc.kill()
|
|
||||||
proc.wait()
|
|
||||||
# Drain readers before re-raising so we don't leave threads behind.
|
|
||||||
t_out.join(timeout=2)
|
|
||||||
t_err.join(timeout=2)
|
|
||||||
raise
|
|
||||||
|
|
||||||
t_out.join()
|
|
||||||
t_err.join()
|
|
||||||
duration = time.monotonic() - t0
|
|
||||||
|
|
||||||
if tee_header:
|
|
||||||
_append_log(f"--- {tee_header} (exit {proc.returncode} in {duration:.1f}s)\n")
|
|
||||||
|
|
||||||
try:
|
|
||||||
return (
|
|
||||||
proc.returncode,
|
|
||||||
"".join(stdout_chunks),
|
|
||||||
"".join(stderr_chunks),
|
|
||||||
duration,
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
if log_fh is not None:
|
|
||||||
try:
|
|
||||||
log_fh.close()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def run(
|
|
||||||
args: Sequence[str],
|
|
||||||
*,
|
|
||||||
cwd: Path | None = None,
|
|
||||||
timeout: float | None = TIMEOUT_DEFAULT,
|
|
||||||
check: bool = True,
|
|
||||||
extra_env: dict[str, str] | None = None,
|
|
||||||
) -> PioResult:
|
|
||||||
"""Invoke `pio <args>` and return captured output.
|
|
||||||
|
|
||||||
`cwd` defaults to the firmware root. `check=True` raises `PioError` on
|
|
||||||
non-zero exit; set `check=False` to inspect `returncode` manually.
|
|
||||||
|
|
||||||
`extra_env` merges into the subprocess environment - used for
|
|
||||||
`PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar build-time
|
|
||||||
toggles that can't be expressed as command-line args.
|
|
||||||
|
|
||||||
If `MESHTASTIC_MCP_FLASH_LOG` is set, output is also tee'd to that file
|
|
||||||
line-by-line as it arrives (for live flash progress in the TUI).
|
|
||||||
"""
|
|
||||||
binary = str(config.pio_bin())
|
|
||||||
work_dir = cwd or config.firmware_root()
|
|
||||||
full = [binary, *args]
|
|
||||||
try:
|
|
||||||
rc, stdout, stderr, duration = _run_capturing(
|
|
||||||
full,
|
|
||||||
cwd=work_dir,
|
|
||||||
timeout=timeout,
|
|
||||||
tee_header=f"pio {' '.join(args)}",
|
|
||||||
extra_env=extra_env,
|
|
||||||
)
|
|
||||||
except subprocess.TimeoutExpired as exc:
|
|
||||||
raise PioTimeout(f"pio {' '.join(args)} timed out after {timeout}s") from exc
|
|
||||||
|
|
||||||
result = PioResult(
|
|
||||||
args=list(args),
|
|
||||||
returncode=rc,
|
|
||||||
stdout=stdout,
|
|
||||||
stderr=stderr,
|
|
||||||
duration_s=duration,
|
|
||||||
)
|
|
||||||
if check and result.returncode != 0:
|
|
||||||
raise PioError(args, result.returncode, result.stdout, result.stderr)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def run_json(
|
|
||||||
args: Sequence[str],
|
|
||||||
*,
|
|
||||||
cwd: Path | None = None,
|
|
||||||
timeout: float | None = TIMEOUT_DEFAULT,
|
|
||||||
):
|
|
||||||
"""Run pio with `--json-output` appended and parse the result."""
|
|
||||||
full = list(args)
|
|
||||||
if "--json-output" not in full:
|
|
||||||
full.append("--json-output")
|
|
||||||
res = run(full, cwd=cwd, timeout=timeout, check=True)
|
|
||||||
if not res.stdout.strip():
|
|
||||||
raise PioError(args, 0, res.stdout, res.stderr or "pio returned empty output")
|
|
||||||
try:
|
|
||||||
return json.loads(res.stdout)
|
|
||||||
except json.JSONDecodeError as exc:
|
|
||||||
raise PioError(
|
|
||||||
args, 0, res.stdout[:2000], f"invalid JSON from pio: {exc}"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
|
|
||||||
def tail_lines(text: str, n: int = 200) -> str:
|
|
||||||
"""Last `n` lines of `text`, joined with newlines. Empty string stays empty."""
|
|
||||||
if not text:
|
|
||||||
return ""
|
|
||||||
lines = text.splitlines()
|
|
||||||
return "\n".join(lines[-n:])
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
"""Persistent device-log capture.
|
|
||||||
|
|
||||||
Singleton `Recorder` subscribes once to the meshtastic pubsub fan-out
|
|
||||||
(`meshtastic.log.line`, `meshtastic.receive.*`, `meshtastic.connection.*`)
|
|
||||||
and appends to four JSONL files under `mcp-server/.mtlog/`. Pubsub is
|
|
||||||
process-global so a single subscription captures every active interface
|
|
||||||
(serial / TCP / BLE) without any per-connection bookkeeping.
|
|
||||||
|
|
||||||
The recorder is opt-in-by-import: importing this package is a no-op; call
|
|
||||||
`get_recorder().start()` (which `server.py` does at FastMCP app init) to
|
|
||||||
begin writing. `pause()` / `resume()` exist for the rare case the user
|
|
||||||
wants a clean stretch of file (e.g. capturing a known-good baseline).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from .recorder import Recorder, get_recorder
|
|
||||||
|
|
||||||
__all__ = ["Recorder", "get_recorder"]
|
|
||||||
@@ -1,309 +0,0 @@
|
|||||||
"""Best-effort parsers for log lines and telemetry packets.
|
|
||||||
|
|
||||||
Two flavors of log line cross our pubsub subscription:
|
|
||||||
1. Text-mode path (debug_log_api disabled): the meshtastic Python lib
|
|
||||||
accumulates bytes between protobuf frames and emits the full
|
|
||||||
firmware-formatted line, e.g.
|
|
||||||
"INFO | 12:34:56 12345 [Main] Booting"
|
|
||||||
- level, HH:MM:SS, uptime seconds, thread bracket, then message.
|
|
||||||
2. LogRecord protobuf path (debug_log_api enabled): the lib calls
|
|
||||||
`_handleLogLine(record.message)` with ONLY the message body. The
|
|
||||||
level/source/time fields on the LogRecord are dropped before
|
|
||||||
pubsub fan-out. We get e.g. just "Booting".
|
|
||||||
|
|
||||||
Both arrive on `meshtastic.log.line`. The parser tries to recover a
|
|
||||||
level + thread when the prefix is present and falls back to level=None
|
|
||||||
otherwise. Consumers who want level filtering on protobuf-mode hosts
|
|
||||||
should grep the raw `line` field instead.
|
|
||||||
|
|
||||||
Telemetry: `meshtastic.receive.telemetry` packets carry one of several
|
|
||||||
metric variants in `packet["decoded"]["telemetry"]`. We flatten the
|
|
||||||
chosen variant into a {field: value} dict so callers don't have to
|
|
||||||
know the protobuf shape.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
# Match: LEVEL | HH:MM:SS UPTIME [Thread] message
|
|
||||||
# HH:MM:SS may be ??:??:?? when RTC isn't valid. The level alternation
|
|
||||||
# below is the canonical list - DebugConfiguration.h's MESHTASTIC_LOG_LEVEL_*
|
|
||||||
# macros must stay in sync with these strings.
|
|
||||||
_LINE_RE = re.compile(
|
|
||||||
r"""
|
|
||||||
^
|
|
||||||
(?P<level>DEBUG|INFO\ |WARN\ |ERROR|CRIT\ |TRACE|HEAP\ )
|
|
||||||
\s*\|\s*
|
|
||||||
(?P<clock>(?:\d{2}:\d{2}:\d{2})|(?:\?{2}:\?{2}:\?{2}))
|
|
||||||
\s+
|
|
||||||
(?P<uptime>\d+)
|
|
||||||
\s+
|
|
||||||
(?:\[(?P<thread>[^\]]+)\]\s+)?
|
|
||||||
(?P<msg>.*)
|
|
||||||
$
|
|
||||||
""",
|
|
||||||
re.VERBOSE,
|
|
||||||
)
|
|
||||||
|
|
||||||
# DEBUG_HEAP build prepends `[heap N] ` to every message body, AFTER the
|
|
||||||
# thread bracket. See src/RedirectablePrint.cpp:175.
|
|
||||||
_HEAP_PREFIX_RE = re.compile(r"^\[heap\s+(?P<heap>\d+)\]\s+(?P<rest>.*)$")
|
|
||||||
|
|
||||||
# OSThread leak/free detection. See src/concurrency/OSThread.cpp:89-91.
|
|
||||||
# Format: "------ Thread NAME leaked heap A -> B (delta) ------"
|
|
||||||
# "++++++ Thread NAME freed heap A -> B (delta) ++++++"
|
|
||||||
_THREAD_HEAP_RE = re.compile(
|
|
||||||
r"""
|
|
||||||
^[\-+]+\s*
|
|
||||||
Thread\s+(?P<thread>\S+)\s+
|
|
||||||
(?P<kind>leaked|freed)\s+heap\s+
|
|
||||||
(?P<before>-?\d+)\s*->\s*(?P<after>-?\d+)\s+
|
|
||||||
\((?P<delta>-?\d+)\)
|
|
||||||
""",
|
|
||||||
re.VERBOSE,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Power.cpp:908 periodic heap status (DEBUG_HEAP only).
|
|
||||||
# Format: "Heap status: FREE/TOTAL bytes free (DELTA), running R/N threads"
|
|
||||||
_HEAP_STATUS_RE = re.compile(
|
|
||||||
r"""
|
|
||||||
Heap\s+status:\s+
|
|
||||||
(?P<free>\d+)\s*/\s*(?P<total>\d+)\s+bytes\s+free
|
|
||||||
(?:\s+\((?P<delta>-?\d+)\))?
|
|
||||||
""",
|
|
||||||
re.VERBOSE,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
|
|
||||||
_HEAP_BRACKET_RE = re.compile(r"^heap\s+(?P<heap>\d+)$")
|
|
||||||
|
|
||||||
|
|
||||||
def parse_log_line(line: str) -> dict[str, Any]:
|
|
||||||
"""Best-effort decompose a raw firmware log line.
|
|
||||||
|
|
||||||
Returns a dict with at least `line` (the original, unmodified - ANSI
|
|
||||||
codes preserved for fidelity). Adds `level`, `tag`, `clock`,
|
|
||||||
`uptime_s`, and `msg` when the full prefix is present.
|
|
||||||
|
|
||||||
Handles two firmware quirks:
|
|
||||||
- LogRecord.message can carry ANSI color escapes from RedirectablePrint
|
|
||||||
(the BLE/StreamAPI path inherited the colored body in some builds).
|
|
||||||
We strip ANSI before regex matching so the prefix survives.
|
|
||||||
- DEBUG_HEAP injects `[heap N]` after the thread bracket. When NO
|
|
||||||
thread name is set, the heap takes the thread bracket position -
|
|
||||||
looks like `[heap 12345] msg`. We detect that shape and move it
|
|
||||||
out of `tag` and into `heap_free`.
|
|
||||||
|
|
||||||
DEBUG_HEAP-build extras (when `[heap N]` is injected): `heap_free`
|
|
||||||
(bytes), and when a `Thread X leaked|freed heap` line is recognized,
|
|
||||||
`heap_event` = {kind, thread, before, after, delta}.
|
|
||||||
|
|
||||||
Never raises.
|
|
||||||
"""
|
|
||||||
out: dict[str, Any] = {"line": line}
|
|
||||||
if not line:
|
|
||||||
return out
|
|
||||||
|
|
||||||
# Strip ANSI escapes BEFORE any regex matching. The original `line`
|
|
||||||
# stays in `out["line"]` for fidelity / future grep.
|
|
||||||
clean = _ANSI_RE.sub("", line)
|
|
||||||
|
|
||||||
m = _LINE_RE.match(clean)
|
|
||||||
msg: str | None = None
|
|
||||||
if m:
|
|
||||||
level = m.group("level").rstrip()
|
|
||||||
out["level"] = level
|
|
||||||
out["clock"] = m.group("clock")
|
|
||||||
try:
|
|
||||||
out["uptime_s"] = int(m.group("uptime"))
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
out["uptime_s"] = None
|
|
||||||
thread = m.group("thread")
|
|
||||||
if thread:
|
|
||||||
# If "thread" is actually the heap prefix taking the bracket
|
|
||||||
# position (DEBUG_HEAP build, no thread set), capture heap
|
|
||||||
# and leave tag unset.
|
|
||||||
hb = _HEAP_BRACKET_RE.match(thread.strip())
|
|
||||||
if hb:
|
|
||||||
try:
|
|
||||||
out["heap_free"] = int(hb.group("heap"))
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
pass
|
|
||||||
else:
|
|
||||||
out["tag"] = thread
|
|
||||||
msg = m.group("msg")
|
|
||||||
out["msg"] = msg
|
|
||||||
else:
|
|
||||||
# No prefix - bare LogRecord.message body. Inspect the whole
|
|
||||||
# line for DEBUG_HEAP-style content; the heap-prefix and
|
|
||||||
# thread-leak patterns can survive on either path.
|
|
||||||
msg = clean
|
|
||||||
|
|
||||||
# DEBUG_HEAP per-line heap prefix: `[heap 92344] message`.
|
|
||||||
# Sits AFTER the thread bracket and BEFORE the message body, but
|
|
||||||
# for bare LogRecord lines it's at the start. Match it at the
|
|
||||||
# head of `msg`.
|
|
||||||
if msg:
|
|
||||||
hp = _HEAP_PREFIX_RE.match(msg)
|
|
||||||
if hp:
|
|
||||||
try:
|
|
||||||
out["heap_free"] = int(hp.group("heap"))
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
pass
|
|
||||||
else:
|
|
||||||
# Strip the prefix from `msg` so a grep on the message
|
|
||||||
# body doesn't have to know about it.
|
|
||||||
out["msg"] = hp.group("rest")
|
|
||||||
msg = hp.group("rest")
|
|
||||||
|
|
||||||
# Thread-level leak/free detection.
|
|
||||||
thr = _THREAD_HEAP_RE.search(msg)
|
|
||||||
if thr:
|
|
||||||
try:
|
|
||||||
out["heap_event"] = {
|
|
||||||
"kind": thr.group("kind"),
|
|
||||||
"thread": thr.group("thread"),
|
|
||||||
"before": int(thr.group("before")),
|
|
||||||
"after": int(thr.group("after")),
|
|
||||||
"delta": int(thr.group("delta")),
|
|
||||||
}
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
# Power.cpp periodic "Heap status: F/T bytes free (D), running ..."
|
|
||||||
hs = _HEAP_STATUS_RE.search(msg)
|
|
||||||
if hs:
|
|
||||||
try:
|
|
||||||
out["heap_free"] = int(hs.group("free"))
|
|
||||||
out["heap_total"] = int(hs.group("total"))
|
|
||||||
if hs.group("delta") is not None:
|
|
||||||
out["heap_delta"] = int(hs.group("delta"))
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
# -- Telemetry ----------------------------------------------------------
|
|
||||||
|
|
||||||
# Order matters: meshtastic-python decoded packets use the protobuf
|
|
||||||
# `oneof variant` field name (snake_case) as the dict key.
|
|
||||||
_TELEMETRY_VARIANTS = (
|
|
||||||
("device_metrics", "device"),
|
|
||||||
("local_stats", "local"),
|
|
||||||
("environment_metrics", "environment"),
|
|
||||||
("power_metrics", "power"),
|
|
||||||
("air_quality_metrics", "airQuality"),
|
|
||||||
("health_metrics", "health"),
|
|
||||||
("host_metrics", "host"),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def extract_telemetry(packet: dict[str, Any]) -> dict[str, Any] | None:
|
|
||||||
"""Pull the telemetry variant + flat fields out of a `meshtastic.receive.telemetry`
|
|
||||||
packet. Returns None when the shape isn't what we expect - so the
|
|
||||||
caller can fall back to a generic packets.jsonl row.
|
|
||||||
"""
|
|
||||||
if not isinstance(packet, dict):
|
|
||||||
return None
|
|
||||||
decoded = packet.get("decoded")
|
|
||||||
if not isinstance(decoded, dict):
|
|
||||||
return None
|
|
||||||
telem = decoded.get("telemetry")
|
|
||||||
if not isinstance(telem, dict):
|
|
||||||
return None
|
|
||||||
# The Python lib produces dict-of-camelCase keys via MessageToDict.
|
|
||||||
# Try both camelCase and snake_case to be robust to lib version drift.
|
|
||||||
for snake, label in _TELEMETRY_VARIANTS:
|
|
||||||
camel = _snake_to_camel(snake)
|
|
||||||
for key in (snake, camel):
|
|
||||||
value = telem.get(key)
|
|
||||||
if isinstance(value, dict):
|
|
||||||
return {
|
|
||||||
"variant": label,
|
|
||||||
"fields": {k: _scalarize(v) for k, v in value.items()},
|
|
||||||
"time": telem.get("time"),
|
|
||||||
}
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _snake_to_camel(name: str) -> str:
|
|
||||||
parts = name.split("_")
|
|
||||||
return parts[0] + "".join(p.title() for p in parts[1:])
|
|
||||||
|
|
||||||
|
|
||||||
def _scalarize(value: Any) -> Any:
|
|
||||||
"""Keep telemetry fields JSON-friendly. Lists/dicts pass through
|
|
||||||
untouched; bytes -> hex string; protobuf enums occasionally arrive
|
|
||||||
as ints (fine) or strings (also fine)."""
|
|
||||||
if isinstance(value, (bytes, bytearray, memoryview)):
|
|
||||||
return bytes(value).hex()
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
# -- Generic packet summary ---------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def summarize_packet(
|
|
||||||
packet: dict[str, Any], *, payload_hex_len: int = 64
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Reduce a packet dict to a stable, queryable summary. Drops the
|
|
||||||
full payload bytes - the recorder records summaries, not pcaps.
|
|
||||||
"""
|
|
||||||
if not isinstance(packet, dict):
|
|
||||||
return {"raw_type": type(packet).__name__}
|
|
||||||
decoded = packet.get("decoded") if isinstance(packet.get("decoded"), dict) else {}
|
|
||||||
portnum = decoded.get("portnum") if isinstance(decoded, dict) else None
|
|
||||||
payload = decoded.get("payload") if isinstance(decoded, dict) else None
|
|
||||||
payload_hex = None
|
|
||||||
payload_size = None
|
|
||||||
if isinstance(payload, (bytes, bytearray, memoryview)):
|
|
||||||
b = bytes(payload)
|
|
||||||
payload_size = len(b)
|
|
||||||
payload_hex = b[:payload_hex_len].hex() if b else ""
|
|
||||||
elif isinstance(payload, str):
|
|
||||||
# Some decoded payloads (text messages) come as decoded strings.
|
|
||||||
payload_size = len(payload)
|
|
||||||
payload_hex = None # not bytes
|
|
||||||
return {
|
|
||||||
"from_node": packet.get("fromId") or packet.get("from"),
|
|
||||||
"to_node": packet.get("toId") or packet.get("to"),
|
|
||||||
"portnum": portnum,
|
|
||||||
"hop_limit": packet.get("hopLimit"),
|
|
||||||
"want_ack": packet.get("wantAck"),
|
|
||||||
"rx_rssi": packet.get("rxRssi"),
|
|
||||||
"rx_snr": packet.get("rxSnr"),
|
|
||||||
"channel": packet.get("channel"),
|
|
||||||
"id": packet.get("id"),
|
|
||||||
"payload_size": payload_size,
|
|
||||||
"payload_hex_prefix": payload_hex,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# -- Interface identification ------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def interface_label(interface: Any) -> dict[str, Any]:
|
|
||||||
"""Stable identifier for the meshtastic interface that emitted an event.
|
|
||||||
|
|
||||||
Used as the `port`/`role` tag on every recorded row. SerialInterface
|
|
||||||
has `devPath`; TCPInterface has `hostname`+`portNumber`; BLEInterface
|
|
||||||
has `address`. Falls back to the class name when none of those exist.
|
|
||||||
"""
|
|
||||||
if interface is None:
|
|
||||||
return {"port": None, "role": None}
|
|
||||||
dev_path = getattr(interface, "devPath", None)
|
|
||||||
if dev_path:
|
|
||||||
return {"port": str(dev_path), "role": "serial"}
|
|
||||||
hostname = getattr(interface, "hostname", None)
|
|
||||||
if hostname:
|
|
||||||
port_num = getattr(interface, "portNumber", None)
|
|
||||||
endpoint = f"tcp://{hostname}:{port_num}" if port_num else f"tcp://{hostname}"
|
|
||||||
return {"port": endpoint, "role": "tcp"}
|
|
||||||
address = getattr(interface, "address", None)
|
|
||||||
if address:
|
|
||||||
return {"port": str(address), "role": "ble"}
|
|
||||||
return {"port": type(interface).__name__, "role": None}
|
|
||||||
@@ -1,467 +0,0 @@
|
|||||||
"""Process-global recorder singleton.
|
|
||||||
|
|
||||||
Subscribes once to the meshtastic pubsub fan-out and writes four append-only
|
|
||||||
JSONL streams under `mcp-server/.mtlog/`. The pubsub fan-out is
|
|
||||||
process-global - a single subscription captures every active interface
|
|
||||||
without per-connection bookkeeping.
|
|
||||||
|
|
||||||
Files:
|
|
||||||
logs.jsonl - every `meshtastic.log.line` event (best-effort prefix
|
|
||||||
parsed for level/tag/uptime; raw `line` always preserved)
|
|
||||||
telemetry.jsonl - `meshtastic.receive.telemetry` packets, flattened by
|
|
||||||
variant (device / local / environment / power / etc.)
|
|
||||||
packets.jsonl - every other `meshtastic.receive.*` packet, summarized
|
|
||||||
(portnum, hops, RSSI/SNR, payload size + 64-byte hex)
|
|
||||||
events.jsonl - connection lifecycle, node-DB updates, and manual
|
|
||||||
`mark_event` rows. Lower volume; useful for aligning
|
|
||||||
timelines.
|
|
||||||
|
|
||||||
Pause/resume: `pause()` flips a flag; subscriptions stay registered. The
|
|
||||||
write methods short-circuit when paused, so we don't lose ordering when
|
|
||||||
resumed (we just have a gap). No queueing.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from . import parsers
|
|
||||||
from .rotating import _RotatingJsonl
|
|
||||||
|
|
||||||
_DEFAULT_DIR = Path(__file__).resolve().parents[3] / ".mtlog"
|
|
||||||
log = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
class Recorder:
|
|
||||||
"""Singleton write-side of the persistent log capture system."""
|
|
||||||
|
|
||||||
def __init__(self, base_dir: Path | None = None) -> None:
|
|
||||||
self.base_dir = Path(base_dir) if base_dir else _DEFAULT_DIR
|
|
||||||
self._lock = threading.RLock()
|
|
||||||
self._started = False
|
|
||||||
self._paused = False
|
|
||||||
self._pause_reason: str | None = None
|
|
||||||
self._started_at: float | None = None
|
|
||||||
self._handlers: list[tuple[str, Any]] = []
|
|
||||||
self._files: dict[str, _RotatingJsonl] = {}
|
|
||||||
|
|
||||||
# -- lifecycle ----------------------------------------------------
|
|
||||||
|
|
||||||
def start(self) -> None:
|
|
||||||
"""Idempotent. Safe to call from FastMCP app startup."""
|
|
||||||
with self._lock:
|
|
||||||
if self._started:
|
|
||||||
return
|
|
||||||
self.base_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
self._files = {
|
|
||||||
"logs": _RotatingJsonl(self.base_dir / "logs.jsonl"),
|
|
||||||
"telemetry": _RotatingJsonl(self.base_dir / "telemetry.jsonl"),
|
|
||||||
"packets": _RotatingJsonl(self.base_dir / "packets.jsonl"),
|
|
||||||
"events": _RotatingJsonl(self.base_dir / "events.jsonl"),
|
|
||||||
}
|
|
||||||
self._wire_pubsub()
|
|
||||||
self._started = True
|
|
||||||
self._started_at = time.time()
|
|
||||||
# Write the recorder_start marker after the initialization block.
|
|
||||||
# `_write_event()` re-checks recorder state via `_files_snapshot()`,
|
|
||||||
# so keeping this out of the setup block avoids nested lifecycle work.
|
|
||||||
self._write_event(kind="recorder_start", label="recorder_started")
|
|
||||||
|
|
||||||
def stop(self) -> None:
|
|
||||||
with self._lock:
|
|
||||||
if not self._started:
|
|
||||||
return
|
|
||||||
self._unwire_pubsub()
|
|
||||||
for f in self._files.values():
|
|
||||||
f.close()
|
|
||||||
self._files = {}
|
|
||||||
self._started = False
|
|
||||||
|
|
||||||
def pause(self, reason: str | None = None) -> None:
|
|
||||||
# Write the pause marker BEFORE flipping the flag - `_write_event`
|
|
||||||
# short-circuits when paused, so the order matters for this event
|
|
||||||
# to actually land in events.jsonl.
|
|
||||||
self._write_event(
|
|
||||||
kind="recorder_pause",
|
|
||||||
label="paused",
|
|
||||||
note=reason,
|
|
||||||
)
|
|
||||||
with self._lock:
|
|
||||||
self._paused = True
|
|
||||||
self._pause_reason = reason
|
|
||||||
|
|
||||||
def resume(self) -> None:
|
|
||||||
# Mirror of `pause()`: clear the flag first, then write the marker
|
|
||||||
# so it isn't suppressed by the still-paused short-circuit.
|
|
||||||
with self._lock:
|
|
||||||
self._paused = False
|
|
||||||
self._pause_reason = None
|
|
||||||
self._write_event(kind="recorder_resume", label="resumed")
|
|
||||||
|
|
||||||
# -- pubsub wiring ------------------------------------------------
|
|
||||||
|
|
||||||
def _wire_pubsub(self) -> None:
|
|
||||||
from pubsub import pub # type: ignore[import-untyped]
|
|
||||||
|
|
||||||
# Subscribers - one per topic. Each pubsub publisher sends
|
|
||||||
# keyword args matching its handler's signature; pubsub
|
|
||||||
# introspects the function signature to route args.
|
|
||||||
bindings = [
|
|
||||||
("meshtastic.log.line", self._on_log_line),
|
|
||||||
("meshtastic.serial.line", self._on_serial_line),
|
|
||||||
("meshtastic.receive", self._on_receive),
|
|
||||||
("meshtastic.receive.telemetry", self._on_telemetry),
|
|
||||||
("meshtastic.connection.established", self._on_connection_established),
|
|
||||||
("meshtastic.connection.lost", self._on_connection_lost),
|
|
||||||
("meshtastic.node.updated", self._on_node_updated),
|
|
||||||
]
|
|
||||||
for topic, handler in bindings:
|
|
||||||
try:
|
|
||||||
pub.subscribe(handler, topic)
|
|
||||||
self._handlers.append((topic, handler))
|
|
||||||
except Exception as exc:
|
|
||||||
# If pubsub refuses one binding (signature mismatch on
|
|
||||||
# an old lib version), log it and keep the rest.
|
|
||||||
log.warning("Recorder failed to subscribe to %s: %s", topic, exc)
|
|
||||||
|
|
||||||
def _unwire_pubsub(self) -> None:
|
|
||||||
from pubsub import pub # type: ignore[import-untyped]
|
|
||||||
|
|
||||||
for topic, handler in self._handlers:
|
|
||||||
try:
|
|
||||||
pub.unsubscribe(handler, topic)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
self._handlers.clear()
|
|
||||||
|
|
||||||
# -- handlers -----------------------------------------------------
|
|
||||||
#
|
|
||||||
# Pubsub callbacks must never raise. Every handler is wrapped in a
|
|
||||||
# try/except that swallows so a bug here can't take down the
|
|
||||||
# SerialInterface receive thread.
|
|
||||||
#
|
|
||||||
# Threading: handlers fire on whatever thread the meshtastic library
|
|
||||||
# dispatches from (varies by interface), while `stop()` clears
|
|
||||||
# `self._files` under `self._lock`. We snapshot `_files` under the
|
|
||||||
# lock at the top of each handler so a concurrent stop can't
|
|
||||||
# KeyError us mid-write. The actual file write goes through
|
|
||||||
# `_RotatingJsonl` which has its own lock.
|
|
||||||
|
|
||||||
def _files_snapshot(self) -> dict[str, _RotatingJsonl] | None:
|
|
||||||
"""Atomic-ish view of `self._files`. Returns None when the recorder
|
|
||||||
is paused or stopped, so handlers can early-exit cleanly without
|
|
||||||
racing `stop()`'s clear."""
|
|
||||||
with self._lock:
|
|
||||||
if not self._started or self._paused:
|
|
||||||
return None
|
|
||||||
return dict(self._files)
|
|
||||||
|
|
||||||
def _on_log_line(self, line: str, interface: Any = None) -> None:
|
|
||||||
files = self._files_snapshot()
|
|
||||||
if files is None:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
tags = parsers.interface_label(interface)
|
|
||||||
parsed = parsers.parse_log_line(str(line))
|
|
||||||
ts = time.time()
|
|
||||||
record: dict[str, Any] = {
|
|
||||||
"ts": ts,
|
|
||||||
"port": tags["port"],
|
|
||||||
"role": tags["role"],
|
|
||||||
"level": parsed.get("level"),
|
|
||||||
"tag": parsed.get("tag"),
|
|
||||||
"uptime_s": parsed.get("uptime_s"),
|
|
||||||
"line": parsed["line"],
|
|
||||||
}
|
|
||||||
# DEBUG_HEAP enrichments (only present when the firmware
|
|
||||||
# was built with -DDEBUG_HEAP=1). Surface as first-class
|
|
||||||
# fields so logs_window can grep/filter on them and so
|
|
||||||
# heap_free synthesizes a telemetry point below.
|
|
||||||
if "heap_free" in parsed:
|
|
||||||
record["heap_free"] = parsed["heap_free"]
|
|
||||||
if "heap_total" in parsed:
|
|
||||||
record["heap_total"] = parsed["heap_total"]
|
|
||||||
if "heap_delta" in parsed:
|
|
||||||
record["heap_delta"] = parsed["heap_delta"]
|
|
||||||
heap_event = parsed.get("heap_event")
|
|
||||||
if heap_event:
|
|
||||||
record["heap_event"] = heap_event
|
|
||||||
files["logs"].write(record)
|
|
||||||
|
|
||||||
# If the line carried a heap snapshot, also write it as a
|
|
||||||
# synthesized LocalStats-shaped row so telemetry_timeline
|
|
||||||
# picks it up at log cadence (much higher resolution than
|
|
||||||
# the ~60 s LocalStats packet). Tagged source=debug_heap so
|
|
||||||
# consumers can filter if mixing scales is unwanted.
|
|
||||||
heap_free = parsed.get("heap_free")
|
|
||||||
if isinstance(heap_free, int):
|
|
||||||
fields: dict[str, Any] = {"heap_free_bytes": heap_free}
|
|
||||||
heap_total = parsed.get("heap_total")
|
|
||||||
if isinstance(heap_total, int):
|
|
||||||
fields["heap_total_bytes"] = heap_total
|
|
||||||
files["telemetry"].write(
|
|
||||||
{
|
|
||||||
"ts": ts,
|
|
||||||
"port": tags["port"],
|
|
||||||
"role": tags["role"],
|
|
||||||
"from_node": None,
|
|
||||||
"variant": "local",
|
|
||||||
"fields": fields,
|
|
||||||
"source": "debug_heap",
|
|
||||||
}
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def _on_serial_line(self, line: str, port: str | None = None) -> None:
|
|
||||||
"""Text-mode passive tap. Fired from `serial_session._drain` when a
|
|
||||||
`pio device monitor` subprocess is running.
|
|
||||||
|
|
||||||
Same parse + heap-synthesis path as `_on_log_line`, but receives
|
|
||||||
the raw text-formatted line (full level/clock/uptime/thread/`[heap N]`/
|
|
||||||
body). On DEBUG_HEAP builds in text mode this gives us per-log-line
|
|
||||||
heap data - far higher cadence than LocalStats, and works without
|
|
||||||
protobuf API mode (no SerialInterface required).
|
|
||||||
"""
|
|
||||||
files = self._files_snapshot()
|
|
||||||
if files is None:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
parsed = parsers.parse_log_line(str(line))
|
|
||||||
ts = time.time()
|
|
||||||
record: dict[str, Any] = {
|
|
||||||
"ts": ts,
|
|
||||||
"port": port,
|
|
||||||
"role": "serial_session",
|
|
||||||
"level": parsed.get("level"),
|
|
||||||
"tag": parsed.get("tag"),
|
|
||||||
"uptime_s": parsed.get("uptime_s"),
|
|
||||||
"line": parsed["line"],
|
|
||||||
}
|
|
||||||
if "heap_free" in parsed:
|
|
||||||
record["heap_free"] = parsed["heap_free"]
|
|
||||||
if "heap_total" in parsed:
|
|
||||||
record["heap_total"] = parsed["heap_total"]
|
|
||||||
if "heap_delta" in parsed:
|
|
||||||
record["heap_delta"] = parsed["heap_delta"]
|
|
||||||
heap_event = parsed.get("heap_event")
|
|
||||||
if heap_event:
|
|
||||||
record["heap_event"] = heap_event
|
|
||||||
files["logs"].write(record)
|
|
||||||
|
|
||||||
# Synthesize a heap_free telemetry sample whenever the line
|
|
||||||
# carries one - same logic as _on_log_line, tagged source so
|
|
||||||
# consumers can distinguish text-mode tap from protobuf path.
|
|
||||||
heap_free = parsed.get("heap_free")
|
|
||||||
if isinstance(heap_free, int):
|
|
||||||
fields: dict[str, Any] = {"heap_free_bytes": heap_free}
|
|
||||||
heap_total = parsed.get("heap_total")
|
|
||||||
if isinstance(heap_total, int):
|
|
||||||
fields["heap_total_bytes"] = heap_total
|
|
||||||
files["telemetry"].write(
|
|
||||||
{
|
|
||||||
"ts": ts,
|
|
||||||
"port": port,
|
|
||||||
"role": "serial_session",
|
|
||||||
"from_node": None,
|
|
||||||
"variant": "local",
|
|
||||||
"fields": fields,
|
|
||||||
"source": "debug_heap_serial",
|
|
||||||
}
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def _on_telemetry(self, packet: dict[str, Any], interface: Any = None) -> None:
|
|
||||||
files = self._files_snapshot()
|
|
||||||
if files is None:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
tags = parsers.interface_label(interface)
|
|
||||||
extracted = parsers.extract_telemetry(packet)
|
|
||||||
if extracted is None:
|
|
||||||
# Couldn't extract a known variant - fall through to the
|
|
||||||
# generic `_on_receive` path, which will still fire for
|
|
||||||
# this packet via the parent topic.
|
|
||||||
return
|
|
||||||
record = {
|
|
||||||
"ts": time.time(),
|
|
||||||
"port": tags["port"],
|
|
||||||
"role": tags["role"],
|
|
||||||
"from_node": packet.get("fromId") or packet.get("from"),
|
|
||||||
"variant": extracted["variant"],
|
|
||||||
"fields": extracted["fields"],
|
|
||||||
"device_time": extracted.get("time"),
|
|
||||||
}
|
|
||||||
files["telemetry"].write(record)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def _on_receive(self, packet: dict[str, Any], interface: Any = None) -> None:
|
|
||||||
# Generic-receive fires for EVERY packet. Telemetry packets get
|
|
||||||
# recorded twice (here and in _on_telemetry) - that's intentional:
|
|
||||||
# packets.jsonl is the universal record, telemetry.jsonl is the
|
|
||||||
# structured timeseries view.
|
|
||||||
files = self._files_snapshot()
|
|
||||||
if files is None:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
tags = parsers.interface_label(interface)
|
|
||||||
summary = parsers.summarize_packet(packet)
|
|
||||||
record = {
|
|
||||||
"ts": time.time(),
|
|
||||||
"port": tags["port"],
|
|
||||||
"role": tags["role"],
|
|
||||||
**summary,
|
|
||||||
}
|
|
||||||
files["packets"].write(record)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def _on_connection_established(self, interface: Any = None) -> None:
|
|
||||||
self._write_event(
|
|
||||||
kind="connection_established",
|
|
||||||
interface=interface,
|
|
||||||
)
|
|
||||||
|
|
||||||
def _on_connection_lost(self, interface: Any = None) -> None:
|
|
||||||
self._write_event(
|
|
||||||
kind="connection_lost",
|
|
||||||
interface=interface,
|
|
||||||
)
|
|
||||||
|
|
||||||
def _on_node_updated(
|
|
||||||
self, node: dict[str, Any] | None = None, interface: Any = None
|
|
||||||
) -> None:
|
|
||||||
# Lower-volume than packets but informative - node ID, hops away,
|
|
||||||
# last heard. Skip the user dict if absent.
|
|
||||||
try:
|
|
||||||
user = (node or {}).get("user") if isinstance(node, dict) else None
|
|
||||||
self._write_event(
|
|
||||||
kind="node_updated",
|
|
||||||
interface=interface,
|
|
||||||
data={
|
|
||||||
"num": (node or {}).get("num"),
|
|
||||||
"id": (user or {}).get("id"),
|
|
||||||
"short": (user or {}).get("shortName"),
|
|
||||||
"long": (user or {}).get("longName"),
|
|
||||||
"hops_away": (node or {}).get("hopsAway"),
|
|
||||||
"snr": (node or {}).get("snr"),
|
|
||||||
"last_heard": (node or {}).get("lastHeard"),
|
|
||||||
},
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
# -- public write helpers -----------------------------------------
|
|
||||||
|
|
||||||
def mark_event(
|
|
||||||
self,
|
|
||||||
label: str,
|
|
||||||
note: str | None = None,
|
|
||||||
data: dict[str, Any] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""User-facing marker. Writes to events.jsonl AND emits a
|
|
||||||
synthetic logs.jsonl row tagged level=MARK so timelines align.
|
|
||||||
"""
|
|
||||||
ts = self._write_event(kind="mark", label=label, note=note, data=data)
|
|
||||||
# Mirror into logs so a single logs_window grep finds it.
|
|
||||||
files = self._files_snapshot()
|
|
||||||
if files is not None:
|
|
||||||
try:
|
|
||||||
files["logs"].write(
|
|
||||||
{
|
|
||||||
"ts": ts,
|
|
||||||
"port": None,
|
|
||||||
"role": "marker",
|
|
||||||
"level": "MARK",
|
|
||||||
"tag": "mark_event",
|
|
||||||
"line": f"[mark] {label}" + (f" - {note}" if note else ""),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
return {"ts": ts, "label": label}
|
|
||||||
|
|
||||||
def _write_event(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
kind: str,
|
|
||||||
label: str | None = None,
|
|
||||||
note: str | None = None,
|
|
||||||
interface: Any = None,
|
|
||||||
data: dict[str, Any] | None = None,
|
|
||||||
) -> float:
|
|
||||||
ts = time.time()
|
|
||||||
# Lifecycle markers (recorder_start, recorder_pause, recorder_resume)
|
|
||||||
# arrive at choreographed moments - `pause()` writes BEFORE flipping
|
|
||||||
# the flag and `resume()` writes AFTER clearing it, so those calls
|
|
||||||
# see _paused=False here. Other event kinds short-circuit when
|
|
||||||
# paused via the snapshot guard below.
|
|
||||||
files = self._files_snapshot()
|
|
||||||
if files is None:
|
|
||||||
return ts
|
|
||||||
try:
|
|
||||||
tags = parsers.interface_label(interface)
|
|
||||||
files["events"].write(
|
|
||||||
{
|
|
||||||
"ts": ts,
|
|
||||||
"kind": kind,
|
|
||||||
"label": label,
|
|
||||||
"note": note,
|
|
||||||
"port": tags["port"],
|
|
||||||
"role": tags["role"],
|
|
||||||
"data": data,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
return ts
|
|
||||||
|
|
||||||
# -- introspection ------------------------------------------------
|
|
||||||
|
|
||||||
def status(self) -> dict[str, Any]:
|
|
||||||
with self._lock:
|
|
||||||
return {
|
|
||||||
"running": self._started,
|
|
||||||
"paused": self._paused,
|
|
||||||
"pause_reason": self._pause_reason,
|
|
||||||
"started_at": self._started_at,
|
|
||||||
"base_dir": str(self.base_dir),
|
|
||||||
"files": {name: f.status() for name, f in self._files.items()},
|
|
||||||
}
|
|
||||||
|
|
||||||
def force_rotate_all(self) -> dict[str, Any]:
|
|
||||||
"""Test/admin hook: rotate every stream right now."""
|
|
||||||
with self._lock:
|
|
||||||
files = list(self._files.values())
|
|
||||||
for f in files:
|
|
||||||
f.force_rotate()
|
|
||||||
# `status()` re-acquires `self._lock`; release before calling it.
|
|
||||||
return self.status()
|
|
||||||
|
|
||||||
|
|
||||||
# -- module-level singleton accessor ------------------------------------
|
|
||||||
|
|
||||||
_INSTANCE_LOCK = threading.Lock()
|
|
||||||
_INSTANCE: Recorder | None = None
|
|
||||||
|
|
||||||
|
|
||||||
def get_recorder() -> Recorder:
|
|
||||||
"""Return the process-global Recorder. Created on first call.
|
|
||||||
|
|
||||||
Honors `MESHTASTIC_MCP_LOG_DIR` env var for the base directory
|
|
||||||
(used by tests to redirect to a tmpdir).
|
|
||||||
"""
|
|
||||||
global _INSTANCE
|
|
||||||
with _INSTANCE_LOCK:
|
|
||||||
if _INSTANCE is None:
|
|
||||||
override = os.environ.get("MESHTASTIC_MCP_LOG_DIR")
|
|
||||||
base = Path(override) if override else None
|
|
||||||
_INSTANCE = Recorder(base_dir=base)
|
|
||||||
return _INSTANCE
|
|
||||||
@@ -1,163 +0,0 @@
|
|||||||
"""Append-only JSONL writer with size-capped rotation.
|
|
||||||
|
|
||||||
A `_RotatingJsonl` owns one live `.jsonl` file. Writes are line-delimited
|
|
||||||
JSON objects (one row per call). When the live file exceeds `max_bytes`,
|
|
||||||
it is closed, gzipped to `<name>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`,
|
|
||||||
and the live file resets to empty. Old archives past `keep_archives` are
|
|
||||||
unlinked oldest-first.
|
|
||||||
|
|
||||||
Size check is amortized - `os.fstat` runs every `check_every` writes,
|
|
||||||
not per-write, so the hot path stays at one `fh.write` + one `fh.flush`.
|
|
||||||
|
|
||||||
Threading: every public method acquires `self._lock`. The recorder runs
|
|
||||||
several pubsub handlers on whatever thread the meshtastic library
|
|
||||||
dispatches from (varies by interface), and queries from MCP tool calls
|
|
||||||
arrive on the FastMCP request thread, so this lock is not optional.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import gzip
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
import threading
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
|
|
||||||
class _RotatingJsonl:
|
|
||||||
"""Append-only JSONL with size rotation. Thread-safe."""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
path: Path,
|
|
||||||
*,
|
|
||||||
max_bytes: int = 100 * 1024 * 1024,
|
|
||||||
keep_archives: int = 5,
|
|
||||||
check_every: int = 1000,
|
|
||||||
) -> None:
|
|
||||||
self.path = path
|
|
||||||
self.max_bytes = max_bytes
|
|
||||||
self.keep_archives = keep_archives
|
|
||||||
self.check_every = check_every
|
|
||||||
self._lock = threading.Lock()
|
|
||||||
self._fh: Any = None
|
|
||||||
self._writes_since_check = 0
|
|
||||||
self._rotations = 0
|
|
||||||
self._lines_written = 0
|
|
||||||
self._last_ts: float | None = None
|
|
||||||
self._open()
|
|
||||||
|
|
||||||
# -- lifecycle ----------------------------------------------------
|
|
||||||
|
|
||||||
def _open(self) -> None:
|
|
||||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
self._fh = self.path.open("a", encoding="utf-8")
|
|
||||||
|
|
||||||
def close(self) -> None:
|
|
||||||
with self._lock:
|
|
||||||
if self._fh is not None:
|
|
||||||
try:
|
|
||||||
self._fh.close()
|
|
||||||
finally:
|
|
||||||
self._fh = None
|
|
||||||
|
|
||||||
# -- write --------------------------------------------------------
|
|
||||||
|
|
||||||
def write(self, record: dict[str, Any]) -> None:
|
|
||||||
"""Append one JSON object as a line. Triggers rotation if oversized."""
|
|
||||||
line = json.dumps(record, separators=(",", ":"), default=str) + "\n"
|
|
||||||
with self._lock:
|
|
||||||
if self._fh is None:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
self._fh.write(line)
|
|
||||||
self._fh.flush()
|
|
||||||
except Exception:
|
|
||||||
# Best-effort: a failed write must not crash the pubsub
|
|
||||||
# handler. Caller has no way to react anyway.
|
|
||||||
return
|
|
||||||
self._lines_written += 1
|
|
||||||
ts = record.get("ts")
|
|
||||||
if isinstance(ts, (int, float)):
|
|
||||||
self._last_ts = float(ts)
|
|
||||||
self._writes_since_check += 1
|
|
||||||
if self._writes_since_check >= self.check_every:
|
|
||||||
self._writes_since_check = 0
|
|
||||||
self._maybe_rotate()
|
|
||||||
|
|
||||||
# -- rotation -----------------------------------------------------
|
|
||||||
|
|
||||||
def _maybe_rotate(self) -> None:
|
|
||||||
# Caller holds self._lock.
|
|
||||||
try:
|
|
||||||
size = os.fstat(self._fh.fileno()).st_size
|
|
||||||
except OSError:
|
|
||||||
return
|
|
||||||
if size < self.max_bytes:
|
|
||||||
return
|
|
||||||
self._rotate_locked()
|
|
||||||
|
|
||||||
def _rotate_locked(self) -> None:
|
|
||||||
# Close, gzip-rename, reopen empty, prune oldest archives.
|
|
||||||
try:
|
|
||||||
self._fh.close()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
self._fh = None
|
|
||||||
# Microsecond-resolution timestamp + per-instance counter so back-
|
|
||||||
# to-back rotations (small max_bytes, repeated `force_rotate()`,
|
|
||||||
# or chatty test loops) get unique archive filenames. The lex
|
|
||||||
# sort order of `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` is chronological,
|
|
||||||
# which `_prune_archives()` and `log_query._iter_jsonl()` both
|
|
||||||
# rely on.
|
|
||||||
stamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S-%f")
|
|
||||||
archive = self.path.with_suffix(f".{stamp}-{self._rotations:05d}.jsonl.gz")
|
|
||||||
try:
|
|
||||||
with self.path.open("rb") as src, gzip.open(archive, "wb") as dst:
|
|
||||||
shutil.copyfileobj(src, dst, length=1024 * 1024)
|
|
||||||
self.path.unlink()
|
|
||||||
except Exception:
|
|
||||||
# Rotation is best-effort. If gzip fails, leave the file
|
|
||||||
# in place and re-open it; we'll try again next check.
|
|
||||||
pass
|
|
||||||
self._open()
|
|
||||||
self._rotations += 1
|
|
||||||
self._prune_archives()
|
|
||||||
|
|
||||||
def _prune_archives(self) -> None:
|
|
||||||
# Match siblings of self.path.name with `.jsonl.gz` suffix.
|
|
||||||
prefix = self.path.stem # "logs" for "logs.jsonl"
|
|
||||||
# Archive filenames are already lexicographically chronological.
|
|
||||||
# Prune by name, not mtime, so copied/restored files don't reorder.
|
|
||||||
archives = sorted(self.path.parent.glob(f"{prefix}.*.jsonl.gz"))
|
|
||||||
excess = len(archives) - self.keep_archives
|
|
||||||
for old in archives[: max(0, excess)]:
|
|
||||||
try:
|
|
||||||
old.unlink()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def force_rotate(self) -> None:
|
|
||||||
"""Test/admin hook: rotate immediately regardless of size."""
|
|
||||||
with self._lock:
|
|
||||||
if self._fh is not None:
|
|
||||||
self._rotate_locked()
|
|
||||||
|
|
||||||
# -- introspection ------------------------------------------------
|
|
||||||
|
|
||||||
def status(self) -> dict[str, Any]:
|
|
||||||
with self._lock:
|
|
||||||
try:
|
|
||||||
size = os.fstat(self._fh.fileno()).st_size if self._fh else 0
|
|
||||||
except OSError:
|
|
||||||
size = 0
|
|
||||||
return {
|
|
||||||
"path": str(self.path),
|
|
||||||
"size": size,
|
|
||||||
"lines": self._lines_written,
|
|
||||||
"last_ts": self._last_ts,
|
|
||||||
"rotations": self._rotations,
|
|
||||||
}
|
|
||||||
@@ -1,98 +0,0 @@
|
|||||||
"""In-memory registry of active serial monitor sessions and port locks.
|
|
||||||
|
|
||||||
Two things live here so the rest of the package has a single place to reach
|
|
||||||
them:
|
|
||||||
1. `sessions`: `{session_id: SerialSession}` for pio device monitor subprocs.
|
|
||||||
2. `port_locks`: `{port: threading.Lock}` so admin/info tools can fail fast
|
|
||||||
when a serial monitor or another meshtastic client already owns a port.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import threading
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from .serial_session import SerialSession, close_session
|
|
||||||
|
|
||||||
_LOCK = threading.Lock()
|
|
||||||
_sessions: dict[str, SerialSession] = {}
|
|
||||||
_port_locks: dict[str, threading.Lock] = {}
|
|
||||||
|
|
||||||
|
|
||||||
def register_session(session: SerialSession) -> None:
|
|
||||||
with _LOCK:
|
|
||||||
_sessions[session.id] = session
|
|
||||||
|
|
||||||
|
|
||||||
def get_session(session_id: str) -> SerialSession:
|
|
||||||
with _LOCK:
|
|
||||||
session = _sessions.get(session_id)
|
|
||||||
if session is None:
|
|
||||||
raise KeyError(f"Unknown session_id: {session_id!r}")
|
|
||||||
return session
|
|
||||||
|
|
||||||
|
|
||||||
def remove_session(session_id: str) -> SerialSession | None:
|
|
||||||
with _LOCK:
|
|
||||||
return _sessions.pop(session_id, None)
|
|
||||||
|
|
||||||
|
|
||||||
def active_session_for_port(port: str) -> SerialSession | None:
|
|
||||||
"""Find any active (non-eof) session owning `port`."""
|
|
||||||
sweep_dead()
|
|
||||||
with _LOCK:
|
|
||||||
for s in _sessions.values():
|
|
||||||
if s.port == port and s.proc.poll() is None:
|
|
||||||
return s
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def all_sessions() -> list[SerialSession]:
|
|
||||||
with _LOCK:
|
|
||||||
return list(_sessions.values())
|
|
||||||
|
|
||||||
|
|
||||||
def sweep_dead() -> int:
|
|
||||||
"""Remove sessions whose subprocess has exited. Returns count removed."""
|
|
||||||
removed_sessions: list[SerialSession] = []
|
|
||||||
with _LOCK:
|
|
||||||
for sid, s in list(_sessions.items()):
|
|
||||||
if s.proc.poll() is not None:
|
|
||||||
removed_sessions.append(_sessions.pop(sid))
|
|
||||||
for session in removed_sessions:
|
|
||||||
try:
|
|
||||||
close_session(session)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
return len(removed_sessions)
|
|
||||||
|
|
||||||
|
|
||||||
def shutdown_all() -> None:
|
|
||||||
"""Close every live session (called on server exit)."""
|
|
||||||
with _LOCK:
|
|
||||||
items = list(_sessions.items())
|
|
||||||
_sessions.clear()
|
|
||||||
for _sid, session in items:
|
|
||||||
try:
|
|
||||||
close_session(session)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def port_lock(port: str) -> threading.Lock:
|
|
||||||
"""Per-port lock for SerialInterface / admin tool serialization."""
|
|
||||||
with _LOCK:
|
|
||||||
lock = _port_locks.get(port)
|
|
||||||
if lock is None:
|
|
||||||
lock = threading.Lock()
|
|
||||||
_port_locks[port] = lock
|
|
||||||
return lock
|
|
||||||
|
|
||||||
|
|
||||||
def snapshot() -> dict[str, Any]:
|
|
||||||
"""Debug dump: session count, port lock count."""
|
|
||||||
with _LOCK:
|
|
||||||
return {
|
|
||||||
"sessions": len(_sessions),
|
|
||||||
"port_locks": len(_port_locks),
|
|
||||||
}
|
|
||||||
@@ -1,246 +0,0 @@
|
|||||||
"""Long-running serial monitor sessions via `pio device monitor`.
|
|
||||||
|
|
||||||
Why pio instead of raw pyserial: pio applies the board's monitor_filters -
|
|
||||||
`esp32_exception_decoder` symbolicates crash stacks, `time` adds timestamps,
|
|
||||||
etc. Raw pyserial would give us bytes; pio gives us developer-grade logs.
|
|
||||||
|
|
||||||
Each session runs `pio device monitor` in a subprocess, with a daemon reader
|
|
||||||
thread draining stdout into a bounded ring buffer. Callers pull lines via
|
|
||||||
`serial_read` using a cursor that survives across calls.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import collections
|
|
||||||
import subprocess
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
import uuid
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from . import boards, config
|
|
||||||
|
|
||||||
_BUFFER_MAX_LINES = 10_000
|
|
||||||
_POLL_NEW_PORT_TIMEOUT_S = 3.0
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class SerialSession:
|
|
||||||
id: str
|
|
||||||
port: str
|
|
||||||
baud: int
|
|
||||||
filters: list[str]
|
|
||||||
env: str | None
|
|
||||||
proc: subprocess.Popen
|
|
||||||
buffer: collections.deque = field(
|
|
||||||
default_factory=lambda: collections.deque(maxlen=_BUFFER_MAX_LINES)
|
|
||||||
)
|
|
||||||
# Total lines seen (not bounded by buffer maxlen). `dropped = total - len(buffer)`
|
|
||||||
# if the reader has advanced past buffer head.
|
|
||||||
total_lines: int = 0
|
|
||||||
started_at: float = field(default_factory=time.time)
|
|
||||||
stopped_at: float | None = None
|
|
||||||
lock: threading.Lock = field(default_factory=threading.Lock)
|
|
||||||
_thread: threading.Thread | None = None
|
|
||||||
|
|
||||||
|
|
||||||
def _drain(session: SerialSession) -> None:
|
|
||||||
"""Reader thread: line-by-line pull stdout into buffer.
|
|
||||||
|
|
||||||
Each line is also published to the `meshtastic.serial.line` pubsub
|
|
||||||
topic so the persistent recorder can capture it without holding its
|
|
||||||
own port. This is the text-mode tap path: when no SerialInterface is
|
|
||||||
open, the firmware emits full formatted lines (level + clock + uptime
|
|
||||||
+ thread + `[heap N]` prefix on DEBUG_HEAP builds + body), and we
|
|
||||||
fan them out to whoever is listening. Pubsub is best-effort -
|
|
||||||
publish failures must never block the reader.
|
|
||||||
"""
|
|
||||||
# Lazy import: pubsub isn't required just to import this module
|
|
||||||
# (e.g., during static analysis), and we want a clean test surface.
|
|
||||||
try:
|
|
||||||
from pubsub import pub # type: ignore[import-untyped]
|
|
||||||
except Exception: # pragma: no cover - defensive
|
|
||||||
pub = None
|
|
||||||
|
|
||||||
assert session.proc.stdout is not None
|
|
||||||
try:
|
|
||||||
for line in session.proc.stdout:
|
|
||||||
line_stripped = line.rstrip("\r\n")
|
|
||||||
with session.lock:
|
|
||||||
session.buffer.append(line_stripped)
|
|
||||||
session.total_lines += 1
|
|
||||||
if pub is not None:
|
|
||||||
try:
|
|
||||||
pub.sendMessage(
|
|
||||||
"meshtastic.serial.line",
|
|
||||||
line=line_stripped,
|
|
||||||
port=session.port,
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
# A subscriber raising must not break the reader.
|
|
||||||
pass
|
|
||||||
except Exception: # pragma: no cover - defensive
|
|
||||||
pass
|
|
||||||
finally:
|
|
||||||
session.stopped_at = time.time()
|
|
||||||
|
|
||||||
|
|
||||||
def open_session(
|
|
||||||
port: str,
|
|
||||||
baud: int = 115200,
|
|
||||||
env: str | None = None,
|
|
||||||
filters: list[str] | None = None,
|
|
||||||
) -> SerialSession:
|
|
||||||
"""Spawn `pio device monitor` and return a SerialSession.
|
|
||||||
|
|
||||||
If `env` is supplied, pio resolves baud and filters from platformio.ini.
|
|
||||||
Otherwise uses the supplied `baud` and `filters` (default `['direct']`).
|
|
||||||
"""
|
|
||||||
# Lazy import to avoid circular: registry imports serial_session.
|
|
||||||
from . import connection
|
|
||||||
|
|
||||||
connection.reject_if_tcp(port, "serial_open")
|
|
||||||
args = ["device", "monitor", "--port", port, "--no-reconnect"]
|
|
||||||
effective_filters: list[str]
|
|
||||||
effective_baud: int = baud
|
|
||||||
if env is not None:
|
|
||||||
args.extend(["-e", env])
|
|
||||||
raw_config: dict[str, Any] = {}
|
|
||||||
try:
|
|
||||||
raw = boards.get_board(env).get("raw_config")
|
|
||||||
if isinstance(raw, dict):
|
|
||||||
raw_config = raw
|
|
||||||
except Exception:
|
|
||||||
raw_config = {}
|
|
||||||
|
|
||||||
monitor_speed = raw_config.get("monitor_speed")
|
|
||||||
has_board_speed = False
|
|
||||||
if monitor_speed is not None:
|
|
||||||
try:
|
|
||||||
effective_baud = int(str(monitor_speed).strip())
|
|
||||||
has_board_speed = True
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
pass
|
|
||||||
|
|
||||||
monitor_filters_raw = raw_config.get("monitor_filters")
|
|
||||||
parsed_board_filters: list[str] = []
|
|
||||||
if isinstance(monitor_filters_raw, str):
|
|
||||||
for token in monitor_filters_raw.replace("\n", ",").split(","):
|
|
||||||
item = token.strip()
|
|
||||||
if item:
|
|
||||||
parsed_board_filters.append(item)
|
|
||||||
elif isinstance(monitor_filters_raw, list):
|
|
||||||
parsed_board_filters = [
|
|
||||||
str(item).strip() for item in monitor_filters_raw if str(item).strip()
|
|
||||||
]
|
|
||||||
|
|
||||||
has_board_filters = len(parsed_board_filters) > 0
|
|
||||||
effective_filters = (
|
|
||||||
parsed_board_filters if has_board_filters else (filters or [])
|
|
||||||
)
|
|
||||||
|
|
||||||
if not has_board_speed:
|
|
||||||
args.extend(["--baud", str(effective_baud)])
|
|
||||||
if not has_board_filters:
|
|
||||||
for f in effective_filters:
|
|
||||||
args.extend(["--filter", f])
|
|
||||||
else:
|
|
||||||
args.extend(["--baud", str(baud)])
|
|
||||||
effective_filters = filters or ["direct"]
|
|
||||||
for f in effective_filters:
|
|
||||||
args.extend(["--filter", f])
|
|
||||||
|
|
||||||
binary = str(config.pio_bin())
|
|
||||||
work_dir = str(config.firmware_root())
|
|
||||||
|
|
||||||
proc = subprocess.Popen(
|
|
||||||
[binary, *args],
|
|
||||||
cwd=work_dir,
|
|
||||||
stdout=subprocess.PIPE,
|
|
||||||
stderr=subprocess.STDOUT,
|
|
||||||
text=True,
|
|
||||||
bufsize=1, # line-buffered
|
|
||||||
)
|
|
||||||
|
|
||||||
session = SerialSession(
|
|
||||||
id=uuid.uuid4().hex,
|
|
||||||
port=port,
|
|
||||||
baud=effective_baud,
|
|
||||||
filters=effective_filters,
|
|
||||||
env=env,
|
|
||||||
proc=proc,
|
|
||||||
)
|
|
||||||
t = threading.Thread(target=_drain, args=(session,), daemon=True)
|
|
||||||
t.start()
|
|
||||||
session._thread = t
|
|
||||||
return session
|
|
||||||
|
|
||||||
|
|
||||||
def read_session(
|
|
||||||
session: SerialSession, max_lines: int = 200, since_cursor: int | None = None
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Snapshot recent lines from the buffer.
|
|
||||||
|
|
||||||
Cursor semantics: the global cursor is `total_lines` at read time. Pass
|
|
||||||
`since_cursor` from a previous response's `new_cursor` to page forward.
|
|
||||||
`since_cursor=0` reads everything still in the ring buffer.
|
|
||||||
"""
|
|
||||||
with session.lock:
|
|
||||||
total = session.total_lines
|
|
||||||
buf_len = len(session.buffer)
|
|
||||||
head_cursor = total - buf_len # cursor value at buffer[0]
|
|
||||||
current_buffer = list(session.buffer)
|
|
||||||
|
|
||||||
if since_cursor is None:
|
|
||||||
since_cursor = head_cursor
|
|
||||||
|
|
||||||
# Clamp: never read what's aged out of the buffer.
|
|
||||||
effective_start = max(since_cursor, head_cursor)
|
|
||||||
# Number of lines skipped because they aged out between reads.
|
|
||||||
dropped = max(0, head_cursor - since_cursor) if since_cursor < head_cursor else 0
|
|
||||||
|
|
||||||
start_idx = effective_start - head_cursor
|
|
||||||
end_idx = min(start_idx + max_lines, buf_len)
|
|
||||||
lines = current_buffer[start_idx:end_idx]
|
|
||||||
new_cursor = effective_start + len(lines)
|
|
||||||
|
|
||||||
eof = session.proc.poll() is not None
|
|
||||||
return {
|
|
||||||
"lines": lines,
|
|
||||||
"new_cursor": new_cursor,
|
|
||||||
"eof": eof,
|
|
||||||
"dropped": dropped,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def close_session(session: SerialSession) -> bool:
|
|
||||||
"""Terminate the subprocess and join the reader thread."""
|
|
||||||
proc = session.proc
|
|
||||||
if proc.poll() is None:
|
|
||||||
try:
|
|
||||||
proc.terminate()
|
|
||||||
proc.wait(timeout=3)
|
|
||||||
except subprocess.TimeoutExpired:
|
|
||||||
proc.kill()
|
|
||||||
proc.wait(timeout=3)
|
|
||||||
if session._thread is not None:
|
|
||||||
session._thread.join(timeout=3)
|
|
||||||
session.stopped_at = session.stopped_at or time.time()
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
def session_summary(session: SerialSession) -> dict[str, Any]:
|
|
||||||
with session.lock:
|
|
||||||
line_count = session.total_lines
|
|
||||||
return {
|
|
||||||
"session_id": session.id,
|
|
||||||
"port": session.port,
|
|
||||||
"baud": session.baud,
|
|
||||||
"filters": session.filters,
|
|
||||||
"env": session.env,
|
|
||||||
"started_at": session.started_at,
|
|
||||||
"stopped_at": session.stopped_at,
|
|
||||||
"line_count": line_count,
|
|
||||||
"eof": session.proc.poll() is not None,
|
|
||||||
}
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,321 +0,0 @@
|
|||||||
"""USB hub power control via `uhubctl` - hard-recovery for wedged devices +
|
|
||||||
deliberate offline-peer simulation for mesh tests.
|
|
||||||
|
|
||||||
Why: when a Meshtastic device's serial port wedges (stuck in a boot loop,
|
|
||||||
frozen USB CDC, crashed firmware that didn't reboot), the only recovery is
|
|
||||||
a physical unplug. uhubctl toggles VBUS per-port on any hub with Per-Port
|
|
||||||
Power Switching (PPPS) support - which is most externally-powered hubs
|
|
||||||
from the last ~5 years - so the harness can power-cycle a device
|
|
||||||
programmatically.
|
|
||||||
|
|
||||||
Architecture:
|
|
||||||
- `list_hubs()` parses `uhubctl` default output into structured records.
|
|
||||||
- `find_port_for_vid(vid)` walks the hubs to find which location+port
|
|
||||||
hosts a given USB VID.
|
|
||||||
- `resolve_target(role)` is the public entry for callers that know a role
|
|
||||||
(`nrf52`, `esp32s3`) but not a hub location: env-var pins win, VID
|
|
||||||
auto-detect falls back.
|
|
||||||
- `power_on`, `power_off`, `cycle` wrap the corresponding `uhubctl -a`
|
|
||||||
invocations, routed through `hw_tools._run` so they share tee-to-flash-
|
|
||||||
log + timeout handling with esptool / nrfutil / picotool.
|
|
||||||
|
|
||||||
Sudo policy: **fail fast**. Modern macOS + most PPPS-capable hubs work
|
|
||||||
without root, but Linux without udev rules (or old macOS with specific
|
|
||||||
driver quirks) still needs it. We run uhubctl non-root; if stderr
|
|
||||||
matches the classic permission pattern we raise `UhubctlError` with an
|
|
||||||
install hint pointing at the uhubctl docs. Auto-wrapping with `sudo`
|
|
||||||
would prompt in the middle of test runs - bad for CI.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import re
|
|
||||||
from typing import Any, Sequence
|
|
||||||
|
|
||||||
from . import config, hw_tools
|
|
||||||
|
|
||||||
# ---------- Parser ---------------------------------------------------------
|
|
||||||
|
|
||||||
# Hub descriptor line:
|
|
||||||
# Current status for hub 1-1.3 [2109:2817 VIA Labs, Inc. USB2.0 Hub, USB 2.10, 4 ports, ppps]
|
|
||||||
_HUB_RE = re.compile(
|
|
||||||
r"^Current status for hub (?P<location>\S+)\s+\[(?P<descriptor>.+)\]\s*$"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Port line:
|
|
||||||
# " Port 2: 0103 power enable connect [239a:8029 RAKwireless ...]"
|
|
||||||
# The bracketed section is absent for empty ports.
|
|
||||||
_PORT_RE = re.compile(
|
|
||||||
r"^\s+Port\s+(?P<port>\d+):\s+(?P<status>\S+)\s+(?P<flags>.*?)"
|
|
||||||
r"(?:\s+\[(?P<device_vid>[0-9a-fA-F]{4}):(?P<device_pid>[0-9a-fA-F]{4})(?:\s+(?P<device_desc>.+))?\])?\s*$"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class UhubctlError(RuntimeError):
|
|
||||||
"""Raised on uhubctl-specific failures: parse errors, permission denied,
|
|
||||||
hub-not-found, or PPPS not supported."""
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Role → VID map -------------------------------------------------
|
|
||||||
|
|
||||||
# Mirrors the default hub_profile in `mcp-server/tests/conftest.py:335`.
|
|
||||||
# Note: esp32s3 and esp32s3_alt share a logical role - we search both.
|
|
||||||
ROLE_VIDS: dict[str, tuple[int, ...]] = {
|
|
||||||
"nrf52": (0x239A,),
|
|
||||||
"esp32s3": (0x303A, 0x10C4),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_role(role: str) -> str:
|
|
||||||
"""Collapse `esp32s3_alt` → `esp32s3` to match the tier conventions."""
|
|
||||||
return role.split("_alt", 1)[0].lower()
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Core subprocess runner -----------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
# If uhubctl hits a permission problem - most commonly Linux without the
|
|
||||||
# udev rules, or a macOS variant where the kernel holds the hub driver -
|
|
||||||
# it prints something like "Permission denied. Try running as root".
|
|
||||||
# Linux error text varies; we match a broad substring rather than exact.
|
|
||||||
_PERM_ERROR_PATTERNS = (
|
|
||||||
"permission denied",
|
|
||||||
"operation not permitted",
|
|
||||||
"try running as root",
|
|
||||||
"need root",
|
|
||||||
"requires root",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _run_uhubctl(args: Sequence[str], *, timeout: float = 30.0) -> dict[str, Any]:
|
|
||||||
"""Invoke uhubctl with the given args. Returns `hw_tools._run`'s dict.
|
|
||||||
|
|
||||||
Translates permission-denied failures into a `UhubctlError` with the
|
|
||||||
install hint, so callers don't have to match stderr themselves. Other
|
|
||||||
non-zero exits are returned as-is for the caller to interpret.
|
|
||||||
"""
|
|
||||||
binary = config.uhubctl_bin()
|
|
||||||
result = hw_tools._run(binary, args, timeout=timeout) # noqa: SLF001
|
|
||||||
if result["exit_code"] != 0:
|
|
||||||
combined = (result.get("stderr") or "") + "\n" + (result.get("stdout") or "")
|
|
||||||
lower = combined.lower()
|
|
||||||
if any(pat in lower for pat in _PERM_ERROR_PATTERNS):
|
|
||||||
raise UhubctlError(
|
|
||||||
"uhubctl exited with a permission error. Install the udev "
|
|
||||||
"rules on Linux, or try `sudo` as a fallback: "
|
|
||||||
"https://github.com/mvp/uhubctl#linux-usb-permissions\n"
|
|
||||||
f"stderr: {result.get('stderr_tail')!r}"
|
|
||||||
)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- List / parse ---------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def parse_list_output(output: str) -> list[dict[str, Any]]:
|
|
||||||
"""Parse the default `uhubctl` stdout into structured hubs.
|
|
||||||
|
|
||||||
Each hub: {
|
|
||||||
"location": "1-1.3",
|
|
||||||
"descriptor": "2109:2817 VIA Labs ...",
|
|
||||||
"vid": 0x2109,
|
|
||||||
"pid": 0x2817,
|
|
||||||
"ppps": bool,
|
|
||||||
"ports": [{"port": int, "status": str, "flags": str,
|
|
||||||
"device_vid": int | None, "device_pid": int | None,
|
|
||||||
"device_desc": str | None}, ...],
|
|
||||||
}
|
|
||||||
"""
|
|
||||||
hubs: list[dict[str, Any]] = []
|
|
||||||
current: dict[str, Any] | None = None
|
|
||||||
|
|
||||||
for line in output.splitlines():
|
|
||||||
hm = _HUB_RE.match(line)
|
|
||||||
if hm:
|
|
||||||
descriptor = hm.group("descriptor")
|
|
||||||
hub_vid, hub_pid = None, None
|
|
||||||
vid_match = re.match(r"([0-9a-fA-F]{4}):([0-9a-fA-F]{4})", descriptor)
|
|
||||||
if vid_match:
|
|
||||||
hub_vid = int(vid_match.group(1), 16)
|
|
||||||
hub_pid = int(vid_match.group(2), 16)
|
|
||||||
current = {
|
|
||||||
"location": hm.group("location"),
|
|
||||||
"descriptor": descriptor,
|
|
||||||
"vid": hub_vid,
|
|
||||||
"pid": hub_pid,
|
|
||||||
"ppps": ", ppps" in descriptor or descriptor.endswith("ppps"),
|
|
||||||
"ports": [],
|
|
||||||
}
|
|
||||||
hubs.append(current)
|
|
||||||
continue
|
|
||||||
|
|
||||||
pm = _PORT_RE.match(line)
|
|
||||||
if pm and current is not None:
|
|
||||||
device_vid = pm.group("device_vid")
|
|
||||||
device_pid = pm.group("device_pid")
|
|
||||||
current["ports"].append(
|
|
||||||
{
|
|
||||||
"port": int(pm.group("port")),
|
|
||||||
"status": pm.group("status"),
|
|
||||||
"flags": (pm.group("flags") or "").strip(),
|
|
||||||
"device_vid": int(device_vid, 16) if device_vid else None,
|
|
||||||
"device_pid": int(device_pid, 16) if device_pid else None,
|
|
||||||
"device_desc": (pm.group("device_desc") or "").strip() or None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return hubs
|
|
||||||
|
|
||||||
|
|
||||||
def list_hubs() -> list[dict[str, Any]]:
|
|
||||||
"""Enumerate every hub uhubctl can see, with per-port device attachments.
|
|
||||||
|
|
||||||
Pure read - no power state changes. Useful as a pre-flight check before
|
|
||||||
a destructive `power_off` call.
|
|
||||||
"""
|
|
||||||
result = _run_uhubctl([], timeout=15.0)
|
|
||||||
if result["exit_code"] != 0:
|
|
||||||
raise UhubctlError(
|
|
||||||
f"uhubctl list failed (exit {result['exit_code']}): {result.get('stderr_tail')!r}"
|
|
||||||
)
|
|
||||||
return parse_list_output(result["stdout"])
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Lookup / resolution -------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def find_port_for_vid(
|
|
||||||
vid: int, pid: int | None = None, *, only_ppps: bool = True
|
|
||||||
) -> list[tuple[str, int]]:
|
|
||||||
"""Return ALL (location, port) matches for a device VID (optionally +PID).
|
|
||||||
|
|
||||||
`only_ppps=True` filters out hubs that don't advertise PPPS - we can't
|
|
||||||
control them anyway. Callers that want to diagnose a missing device can
|
|
||||||
pass `only_ppps=False` to see if the device is on a non-controllable
|
|
||||||
hub (and raise a clearer error).
|
|
||||||
"""
|
|
||||||
hubs = list_hubs()
|
|
||||||
matches: list[tuple[str, int]] = []
|
|
||||||
for hub in hubs:
|
|
||||||
if only_ppps and not hub["ppps"]:
|
|
||||||
continue
|
|
||||||
for port in hub["ports"]:
|
|
||||||
if port["device_vid"] != vid:
|
|
||||||
continue
|
|
||||||
if pid is not None and port["device_pid"] != pid:
|
|
||||||
continue
|
|
||||||
matches.append((hub["location"], port["port"]))
|
|
||||||
return matches
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_target(role: str) -> tuple[str, int]:
|
|
||||||
"""Resolve a Meshtastic role to (hub_location, port_number).
|
|
||||||
|
|
||||||
Priority:
|
|
||||||
1. Env vars `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` + `_PORT_<ROLE>`
|
|
||||||
(e.g. `MESHTASTIC_UHUBCTL_LOCATION_NRF52=1-1.3`, `_PORT_NRF52=2`).
|
|
||||||
2. VID auto-detect against `ROLE_VIDS[role]`, taking the first PPPS
|
|
||||||
match.
|
|
||||||
|
|
||||||
Raises `UhubctlError` on ambiguity (multiple matches) or no-match. The
|
|
||||||
env-var path exists specifically to disambiguate when two devices share
|
|
||||||
a VID.
|
|
||||||
"""
|
|
||||||
role = _normalize_role(role)
|
|
||||||
env_key_loc = f"MESHTASTIC_UHUBCTL_LOCATION_{role.upper()}"
|
|
||||||
env_key_port = f"MESHTASTIC_UHUBCTL_PORT_{role.upper()}"
|
|
||||||
loc = os.environ.get(env_key_loc)
|
|
||||||
port_str = os.environ.get(env_key_port)
|
|
||||||
if loc and port_str:
|
|
||||||
try:
|
|
||||||
return (loc, int(port_str))
|
|
||||||
except ValueError as exc:
|
|
||||||
raise UhubctlError(
|
|
||||||
f"{env_key_port}={port_str!r} is not a valid integer"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
if role not in ROLE_VIDS:
|
|
||||||
raise UhubctlError(
|
|
||||||
f"unknown role {role!r}; known roles: {sorted(ROLE_VIDS)}. "
|
|
||||||
f"Set {env_key_loc} + {env_key_port} to pin manually."
|
|
||||||
)
|
|
||||||
|
|
||||||
matches: list[tuple[str, int]] = []
|
|
||||||
for vid in ROLE_VIDS[role]:
|
|
||||||
matches.extend(find_port_for_vid(vid))
|
|
||||||
|
|
||||||
if not matches:
|
|
||||||
vids = ", ".join(f"0x{v:04x}" for v in ROLE_VIDS[role])
|
|
||||||
raise UhubctlError(
|
|
||||||
f"no controllable hub hosts a device with VID in {{{vids}}} "
|
|
||||||
f"for role={role!r}. Check the device is plugged into a "
|
|
||||||
f"PPPS-capable hub, or pin manually via {env_key_loc} + {env_key_port}."
|
|
||||||
)
|
|
||||||
if len(matches) > 1:
|
|
||||||
shown = ", ".join(f"{loc}:port{p}" for loc, p in matches)
|
|
||||||
raise UhubctlError(
|
|
||||||
f"ambiguous: multiple devices match role={role!r} ({shown}). "
|
|
||||||
f"Pin the target via {env_key_loc} + {env_key_port}."
|
|
||||||
)
|
|
||||||
return matches[0]
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Power actions --------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _action(
|
|
||||||
action: str,
|
|
||||||
location: str,
|
|
||||||
port: int,
|
|
||||||
*,
|
|
||||||
delay_s: int | None = None,
|
|
||||||
timeout: float = 30.0,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
args: list[str] = ["-a", action, "-l", location, "-p", str(port)]
|
|
||||||
if delay_s is not None:
|
|
||||||
args.extend(["-d", str(delay_s)])
|
|
||||||
# Suppress verbose "before" printout so our parser doesn't have to skip it.
|
|
||||||
args.append("-N")
|
|
||||||
result = _run_uhubctl(args, timeout=timeout)
|
|
||||||
if result["exit_code"] != 0:
|
|
||||||
raise UhubctlError(
|
|
||||||
f"uhubctl -a {action} -l {location} -p {port} failed "
|
|
||||||
f"(exit {result['exit_code']}): {result.get('stderr_tail')!r}"
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"action": action,
|
|
||||||
"location": location,
|
|
||||||
"port": port,
|
|
||||||
"delay_s": delay_s,
|
|
||||||
"duration_s": result["duration_s"],
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def power_on(location: str, port: int) -> dict[str, Any]:
|
|
||||||
"""Drive the port VBUS high. Device re-enumerates in 1-3 s on healthy hubs."""
|
|
||||||
return _action("on", location, port)
|
|
||||||
|
|
||||||
|
|
||||||
def power_off(location: str, port: int) -> dict[str, Any]:
|
|
||||||
"""Drive the port VBUS low. Device disappears from `list_devices` immediately."""
|
|
||||||
return _action("off", location, port)
|
|
||||||
|
|
||||||
|
|
||||||
def cycle(location: str, port: int, delay_s: int = 2) -> dict[str, Any]:
|
|
||||||
"""Off → wait `delay_s` → on. The common hard-reset pattern."""
|
|
||||||
# uhubctl's own `-a cycle` handles the delay internally; we use a
|
|
||||||
# slightly longer timeout to accommodate delay_s + enumeration.
|
|
||||||
return _action("cycle", location, port, delay_s=delay_s, timeout=30.0 + delay_s * 2)
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"ROLE_VIDS",
|
|
||||||
"UhubctlError",
|
|
||||||
"cycle",
|
|
||||||
"find_port_for_vid",
|
|
||||||
"list_hubs",
|
|
||||||
"parse_list_output",
|
|
||||||
"power_off",
|
|
||||||
"power_on",
|
|
||||||
"resolve_target",
|
|
||||||
]
|
|
||||||
@@ -1,540 +0,0 @@
|
|||||||
"""USERPREFS: build-time constants baked into the firmware binary.
|
|
||||||
|
|
||||||
The firmware repo has `userPrefs.jsonc` at its root - a JSONC file with every
|
|
||||||
available USERPREFS_* key listed, most commented out. At build time,
|
|
||||||
`bin/platformio-custom.py` reads it, strips comments, and emits
|
|
||||||
`-DUSERPREFS_<KEY>=<value>` build flags into the compile step. Firmware code
|
|
||||||
uses `#ifdef USERPREFS_<KEY>` to pick up the baked-in defaults for channels,
|
|
||||||
owner name, LoRa region, OEM branding, MQTT credentials, etc.
|
|
||||||
|
|
||||||
This module:
|
|
||||||
1. Parses `userPrefs.jsonc` (preserving which keys are active vs commented)
|
|
||||||
2. Greps `src/` for the set of keys the firmware actually consumes (the
|
|
||||||
real discovery manifest - anything here that isn't in the jsonc is still
|
|
||||||
a valid override)
|
|
||||||
3. Provides a context manager for temporarily swapping in overrides during
|
|
||||||
a build/flash, then restoring the original file
|
|
||||||
4. Provides persistent `set` / `reset` for when the caller wants the change
|
|
||||||
to stick across multiple builds
|
|
||||||
|
|
||||||
The firmware's platformio-custom.py value-type detection mirrors what we need
|
|
||||||
for serialization: dict-like `{...}` (byte arrays, enum lists), digit-like
|
|
||||||
(ints and floats), `true`/`false`, `meshtastic_*` enum constants, and
|
|
||||||
everything else gets string-wrapped via `env.StringifyMacro`. We store the
|
|
||||||
raw string values exactly as they'd appear in the jsonc to avoid round-trip
|
|
||||||
surprises.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
import shutil
|
|
||||||
import time
|
|
||||||
from contextlib import contextmanager
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Iterator
|
|
||||||
|
|
||||||
from . import config
|
|
||||||
|
|
||||||
USERPREFS_FILE = "userPrefs.jsonc"
|
|
||||||
BACKUP_SUFFIX = ".mcp.bak"
|
|
||||||
|
|
||||||
# Pattern for lines like `// "USERPREFS_FOO": "value",` or `"USERPREFS_FOO": "v"`
|
|
||||||
_ACTIVE_LINE = re.compile(r'^\s*"(USERPREFS_[A-Z0-9_]+)"\s*:\s*"((?:[^"\\]|\\.)*)"')
|
|
||||||
_COMMENTED_LINE = re.compile(
|
|
||||||
r'^\s*//\s*"(USERPREFS_[A-Z0-9_]+)"\s*:\s*"((?:[^"\\]|\\.)*)"'
|
|
||||||
)
|
|
||||||
# Inline comment stripper (matches platformio-custom.py:219)
|
|
||||||
_LINE_COMMENT = re.compile(r"//.*")
|
|
||||||
|
|
||||||
# USERPREFS_* usage in firmware source (#ifdef, #if defined, direct refs)
|
|
||||||
_USAGE_PATTERN = re.compile(r"\bUSERPREFS_[A-Z0-9_]+\b")
|
|
||||||
|
|
||||||
|
|
||||||
def jsonc_path() -> Path:
|
|
||||||
return config.firmware_root() / USERPREFS_FILE
|
|
||||||
|
|
||||||
|
|
||||||
def _read_file(path: Path) -> str:
|
|
||||||
return path.read_text(encoding="utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_jsonc_state(text: str) -> dict[str, Any]:
|
|
||||||
"""Parse userPrefs.jsonc while preserving comment state per key.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
{
|
|
||||||
"active": {key: string_value, ...}, # uncommented
|
|
||||||
"commented": {key: string_value, ...}, # commented examples
|
|
||||||
"order": [key, ...] # source order for round-trip
|
|
||||||
}
|
|
||||||
|
|
||||||
"""
|
|
||||||
active: dict[str, str] = {}
|
|
||||||
commented: dict[str, str] = {}
|
|
||||||
order: list[str] = []
|
|
||||||
for line in text.splitlines():
|
|
||||||
if m := _COMMENTED_LINE.match(line):
|
|
||||||
key, val = m.group(1), m.group(2)
|
|
||||||
commented[key] = val
|
|
||||||
order.append(key)
|
|
||||||
elif m := _ACTIVE_LINE.match(line):
|
|
||||||
key, val = m.group(1), m.group(2)
|
|
||||||
active[key] = val
|
|
||||||
order.append(key)
|
|
||||||
return {"active": active, "commented": commented, "order": order}
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_jsonc_active(text: str) -> dict[str, str]:
|
|
||||||
"""Parse active-only values by stripping line comments + feeding to json."""
|
|
||||||
stripped = "\n".join(_LINE_COMMENT.sub("", line) for line in text.splitlines())
|
|
||||||
try:
|
|
||||||
return {k: str(v) for k, v in json.loads(stripped).items()}
|
|
||||||
except json.JSONDecodeError as exc:
|
|
||||||
raise ValueError(f"userPrefs.jsonc is not valid JSONC: {exc}") from exc
|
|
||||||
|
|
||||||
|
|
||||||
def read_state() -> dict[str, Any]:
|
|
||||||
"""Return {active, commented, order, path}."""
|
|
||||||
path = jsonc_path()
|
|
||||||
if not path.is_file():
|
|
||||||
return {"active": {}, "commented": {}, "order": [], "path": str(path)}
|
|
||||||
state = _parse_jsonc_state(_read_file(path))
|
|
||||||
state["path"] = str(path)
|
|
||||||
return state
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Manifest ------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _scan_consumed_keys() -> dict[str, list[str]]:
|
|
||||||
"""Grep firmware src/ for USERPREFS_* references.
|
|
||||||
|
|
||||||
Returns {key: [relative_file_paths]} - only includes files under `src/`.
|
|
||||||
"""
|
|
||||||
src_dir = config.firmware_root() / "src"
|
|
||||||
if not src_dir.is_dir():
|
|
||||||
return {}
|
|
||||||
out: dict[str, set[str]] = {}
|
|
||||||
for path in src_dir.rglob("*"):
|
|
||||||
if not path.is_file() or path.suffix.lower() not in {
|
|
||||||
".c",
|
|
||||||
".cc",
|
|
||||||
".cpp",
|
|
||||||
".h",
|
|
||||||
".hpp",
|
|
||||||
".ipp",
|
|
||||||
".inl",
|
|
||||||
}:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
text = path.read_text(encoding="utf-8", errors="ignore")
|
|
||||||
except Exception:
|
|
||||||
continue
|
|
||||||
for m in _USAGE_PATTERN.finditer(text):
|
|
||||||
key = m.group(0)
|
|
||||||
# Skip our own "_USERPREFS_" artifacts (reserve-word guard from build-userprefs-json.py)
|
|
||||||
if key.startswith("_USERPREFS_"):
|
|
||||||
continue
|
|
||||||
out.setdefault(key, set()).add(
|
|
||||||
str(path.relative_to(config.firmware_root()))
|
|
||||||
)
|
|
||||||
return {k: sorted(v) for k, v in sorted(out.items())}
|
|
||||||
|
|
||||||
|
|
||||||
def build_manifest() -> dict[str, Any]:
|
|
||||||
"""Build the discovery manifest.
|
|
||||||
|
|
||||||
Every known USERPREFS_* key appears exactly once with:
|
|
||||||
- `value` (current active value, if any)
|
|
||||||
- `example` (commented default from jsonc, if any)
|
|
||||||
- `active` bool
|
|
||||||
- `declared_in_jsonc` bool (key appears anywhere in userPrefs.jsonc)
|
|
||||||
- `consumed_by` list of source files that reference it
|
|
||||||
- `inferred_type`: one of "brace", "number", "bool", "enum", "string"
|
|
||||||
- matches platformio-custom.py's value-wrapping switch
|
|
||||||
"""
|
|
||||||
state = read_state()
|
|
||||||
consumed = _scan_consumed_keys()
|
|
||||||
|
|
||||||
all_keys = set(state["active"]) | set(state["commented"]) | set(consumed)
|
|
||||||
records = []
|
|
||||||
for key in sorted(all_keys):
|
|
||||||
example = state["commented"].get(key)
|
|
||||||
value = state["active"].get(key)
|
|
||||||
records.append(
|
|
||||||
{
|
|
||||||
"key": key,
|
|
||||||
"active": key in state["active"],
|
|
||||||
"value": value,
|
|
||||||
"example": example,
|
|
||||||
"declared_in_jsonc": key in state["active"]
|
|
||||||
or key in state["commented"],
|
|
||||||
"consumed_by": consumed.get(key, []),
|
|
||||||
"inferred_type": infer_type(value if value is not None else example),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"path": state["path"],
|
|
||||||
"active_count": len(state["active"]),
|
|
||||||
"commented_count": len(state["commented"]),
|
|
||||||
"consumed_key_count": len(consumed),
|
|
||||||
"total_keys": len(records),
|
|
||||||
"entries": records,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def infer_type(value: str | None) -> str:
|
|
||||||
"""Classify a raw value string the way platformio-custom.py does.
|
|
||||||
|
|
||||||
Mirrors the branch order in `bin/platformio-custom.py:222-235`.
|
|
||||||
"""
|
|
||||||
if value is None:
|
|
||||||
return "unknown"
|
|
||||||
v = value.strip()
|
|
||||||
if v.startswith("{"):
|
|
||||||
return "brace" # byte array / enum init list
|
|
||||||
if v.lstrip("-").replace(".", "", 1).isdigit():
|
|
||||||
return "number"
|
|
||||||
if v in ("true", "false"):
|
|
||||||
return "bool"
|
|
||||||
if v.startswith("meshtastic_"):
|
|
||||||
return "enum"
|
|
||||||
return "string"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Writing -------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _format_jsonc_line(key: str, value: str, commented: bool) -> str:
|
|
||||||
prefix = " // " if commented else " "
|
|
||||||
# Escape backslashes and quotes inside value the way platformio-custom.py
|
|
||||||
# expects - the original jsonc uses raw strings for most content. Keep it
|
|
||||||
# literal; callers are responsible for correct escaping if they pass
|
|
||||||
# dict/enum-init values that contain quotes.
|
|
||||||
return f'{prefix}"{key}": "{value}",'
|
|
||||||
|
|
||||||
|
|
||||||
def _render_jsonc(
|
|
||||||
active: dict[str, str], commented: dict[str, str], order: list[str]
|
|
||||||
) -> str:
|
|
||||||
"""Render userPrefs.jsonc preserving source order and comment state."""
|
|
||||||
seen: set[str] = set()
|
|
||||||
lines = ["{"]
|
|
||||||
for key in order:
|
|
||||||
if key in seen:
|
|
||||||
continue
|
|
||||||
seen.add(key)
|
|
||||||
if key in active:
|
|
||||||
lines.append(_format_jsonc_line(key, active[key], commented=False))
|
|
||||||
elif key in commented:
|
|
||||||
lines.append(_format_jsonc_line(key, commented[key], commented=True))
|
|
||||||
# Append any newly-added keys (not in original order) at the end, active.
|
|
||||||
for key, value in active.items():
|
|
||||||
if key in seen:
|
|
||||||
continue
|
|
||||||
seen.add(key)
|
|
||||||
lines.append(_format_jsonc_line(key, value, commented=False))
|
|
||||||
# Strip trailing comma on the last data line (valid JSONC allows it, but
|
|
||||||
# strict `json.loads` after comment-stripping does not; the loader in
|
|
||||||
# platformio-custom.py uses json.loads).
|
|
||||||
if len(lines) > 1 and lines[-1].endswith(","):
|
|
||||||
lines[-1] = lines[-1].rstrip(",")
|
|
||||||
lines.append("}")
|
|
||||||
lines.append("") # trailing newline
|
|
||||||
return "\n".join(lines)
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_after_write(text: str) -> None:
|
|
||||||
"""Ensure the rendered text still parses the way platformio-custom.py does."""
|
|
||||||
stripped = "\n".join(_LINE_COMMENT.sub("", line) for line in text.splitlines())
|
|
||||||
json.loads(stripped) # raises on any error
|
|
||||||
|
|
||||||
|
|
||||||
def write_state(
|
|
||||||
active: dict[str, str], commented: dict[str, str], order: list[str]
|
|
||||||
) -> None:
|
|
||||||
path = jsonc_path()
|
|
||||||
text = _render_jsonc(active, commented, order)
|
|
||||||
_validate_after_write(text)
|
|
||||||
path.write_text(text, encoding="utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def merge_active(overrides: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
"""Merge `overrides` into the active set and persist.
|
|
||||||
|
|
||||||
Existing active values not in `overrides` are kept. Example/commented
|
|
||||||
values are preserved. Returns {before_active, after_active, path}.
|
|
||||||
"""
|
|
||||||
state = read_state()
|
|
||||||
before = dict(state["active"])
|
|
||||||
after = dict(before)
|
|
||||||
commented = dict(state["commented"])
|
|
||||||
order = list(state["order"])
|
|
||||||
for key, raw in overrides.items():
|
|
||||||
if not key.startswith("USERPREFS_"):
|
|
||||||
raise ValueError(f"key {key!r} must start with USERPREFS_")
|
|
||||||
after[key] = _stringify(raw)
|
|
||||||
# If the key was commented, uncommenting it means removing from commented set.
|
|
||||||
commented.pop(key, None)
|
|
||||||
if key not in order:
|
|
||||||
order.append(key)
|
|
||||||
write_state(after, commented, order)
|
|
||||||
return {"before_active": before, "after_active": after, "path": str(jsonc_path())}
|
|
||||||
|
|
||||||
|
|
||||||
def _stringify(value: Any) -> str:
|
|
||||||
"""Convert a Python value to the string form userPrefs.jsonc expects.
|
|
||||||
|
|
||||||
bool → "true" / "false"; int/float → str(); anything else → str(value).
|
|
||||||
Callers passing brace-init strings (`"{ 0x01, 0x02, ... }"`) must format
|
|
||||||
them themselves - this function doesn't try to synthesize them.
|
|
||||||
"""
|
|
||||||
if isinstance(value, bool):
|
|
||||||
return "true" if value else "false"
|
|
||||||
if isinstance(value, (int, float)):
|
|
||||||
return str(value)
|
|
||||||
return str(value)
|
|
||||||
|
|
||||||
|
|
||||||
def reset() -> dict[str, Any]:
|
|
||||||
"""Restore userPrefs.jsonc from the MCP backup if present.
|
|
||||||
|
|
||||||
Returns {restored: bool, path, backup_path}.
|
|
||||||
"""
|
|
||||||
path = jsonc_path()
|
|
||||||
backup = path.with_suffix(path.suffix + BACKUP_SUFFIX)
|
|
||||||
if backup.is_file():
|
|
||||||
shutil.copy2(backup, path)
|
|
||||||
backup.unlink()
|
|
||||||
return {"restored": True, "path": str(path), "backup_path": str(backup)}
|
|
||||||
return {"restored": False, "path": str(path), "backup_path": str(backup)}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Transient override (for build/flash) --------------------------
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Pre-baked profiles --------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _psk_from_bytes(data: bytes) -> str:
|
|
||||||
"""Format 32 bytes as a C-style brace-init list for USERPREFS_CHANNEL_*_PSK.
|
|
||||||
|
|
||||||
Matches the exact format used in userPrefs.jsonc:
|
|
||||||
{ 0x38, 0x4b, 0xbc, ... }
|
|
||||||
"""
|
|
||||||
if len(data) != 32:
|
|
||||||
raise ValueError(f"PSK must be exactly 32 bytes, got {len(data)}")
|
|
||||||
return "{ " + ", ".join(f"0x{b:02x}" for b in data) + " }"
|
|
||||||
|
|
||||||
|
|
||||||
def generate_psk(seed: str | None = None) -> str:
|
|
||||||
"""Generate a 32-byte PSK as a brace-init string.
|
|
||||||
|
|
||||||
If `seed` is provided, the PSK is deterministic (derived via SHA-256 of
|
|
||||||
the seed); otherwise it's cryptographically random. Use a seed for
|
|
||||||
automated testing so every device in a test run shares the same key.
|
|
||||||
"""
|
|
||||||
if seed is None:
|
|
||||||
import secrets
|
|
||||||
|
|
||||||
raw = secrets.token_bytes(32)
|
|
||||||
else:
|
|
||||||
import hashlib
|
|
||||||
|
|
||||||
raw = hashlib.sha256(seed.encode("utf-8")).digest()
|
|
||||||
return _psk_from_bytes(raw)
|
|
||||||
|
|
||||||
|
|
||||||
# Meshtastic region enum name → short description (for the manifest tool).
|
|
||||||
# Not exhaustive; these are the regions a US-based test lab is likely to pick.
|
|
||||||
KNOWN_REGIONS = {
|
|
||||||
"US": "meshtastic_Config_LoRaConfig_RegionCode_US",
|
|
||||||
"EU_433": "meshtastic_Config_LoRaConfig_RegionCode_EU_433",
|
|
||||||
"EU_868": "meshtastic_Config_LoRaConfig_RegionCode_EU_868",
|
|
||||||
"CN": "meshtastic_Config_LoRaConfig_RegionCode_CN",
|
|
||||||
"JP": "meshtastic_Config_LoRaConfig_RegionCode_JP",
|
|
||||||
"ANZ": "meshtastic_Config_LoRaConfig_RegionCode_ANZ",
|
|
||||||
"KR": "meshtastic_Config_LoRaConfig_RegionCode_KR",
|
|
||||||
"TW": "meshtastic_Config_LoRaConfig_RegionCode_TW",
|
|
||||||
"RU": "meshtastic_Config_LoRaConfig_RegionCode_RU",
|
|
||||||
"IN": "meshtastic_Config_LoRaConfig_RegionCode_IN",
|
|
||||||
"NZ_865": "meshtastic_Config_LoRaConfig_RegionCode_NZ_865",
|
|
||||||
"TH": "meshtastic_Config_LoRaConfig_RegionCode_TH",
|
|
||||||
"UA_433": "meshtastic_Config_LoRaConfig_RegionCode_UA_433",
|
|
||||||
"UA_868": "meshtastic_Config_LoRaConfig_RegionCode_UA_868",
|
|
||||||
"MY_433": "meshtastic_Config_LoRaConfig_RegionCode_MY_433",
|
|
||||||
"MY_919": "meshtastic_Config_LoRaConfig_RegionCode_MY_919",
|
|
||||||
"SG_923": "meshtastic_Config_LoRaConfig_RegionCode_SG_923",
|
|
||||||
"LORA_24": "meshtastic_Config_LoRaConfig_RegionCode_LORA_24",
|
|
||||||
}
|
|
||||||
|
|
||||||
KNOWN_MODEM_PRESETS = {
|
|
||||||
"LONG_FAST": "meshtastic_Config_LoRaConfig_ModemPreset_LONG_FAST",
|
|
||||||
"LONG_SLOW": "meshtastic_Config_LoRaConfig_ModemPreset_LONG_SLOW",
|
|
||||||
"LONG_MODERATE": "meshtastic_Config_LoRaConfig_ModemPreset_LONG_MODERATE",
|
|
||||||
"VERY_LONG_SLOW": "meshtastic_Config_LoRaConfig_ModemPreset_VERY_LONG_SLOW",
|
|
||||||
"MEDIUM_SLOW": "meshtastic_Config_LoRaConfig_ModemPreset_MEDIUM_SLOW",
|
|
||||||
"MEDIUM_FAST": "meshtastic_Config_LoRaConfig_ModemPreset_MEDIUM_FAST",
|
|
||||||
"SHORT_SLOW": "meshtastic_Config_LoRaConfig_ModemPreset_SHORT_SLOW",
|
|
||||||
"SHORT_FAST": "meshtastic_Config_LoRaConfig_ModemPreset_SHORT_FAST",
|
|
||||||
"SHORT_TURBO": "meshtastic_Config_LoRaConfig_ModemPreset_SHORT_TURBO",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def build_testing_profile(
|
|
||||||
psk_seed: str | None = None,
|
|
||||||
channel_name: str = "McpTest",
|
|
||||||
channel_num: int = 88,
|
|
||||||
region: str = "US",
|
|
||||||
modem_preset: str = "LONG_FAST",
|
|
||||||
short_name: str | None = None,
|
|
||||||
long_name: str | None = None,
|
|
||||||
disable_mqtt: bool = True,
|
|
||||||
disable_position: bool = False,
|
|
||||||
enable_ui_log: bool = False,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Build a USERPREFS dict for an isolated test-mesh device.
|
|
||||||
|
|
||||||
Defaults: US region, LONG_FAST modem, channel slot 88 (well away from the
|
|
||||||
default `hash("LongFast") % numChannels` slot that production devices use),
|
|
||||||
and a private PSK. Devices baked with the same `psk_seed` land on the same
|
|
||||||
isolated mesh.
|
|
||||||
|
|
||||||
See `src/mesh/RadioInterface.cpp:849` for the slot-selection math:
|
|
||||||
`slot = (channel_num ? channel_num - 1 : hash(name)) % numChannels`.
|
|
||||||
Setting `channel_num` explicitly (non-zero) forces a deterministic slot.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
psk_seed: seed for deterministic PSK generation. `None` = random (fine
|
|
||||||
for one-off bakes, useless for multi-device test clusters).
|
|
||||||
channel_name: primary channel name. Must differ from defaults
|
|
||||||
("LongFast", "MediumFast", etc.) so production devices don't
|
|
||||||
accidentally match after the PSK check.
|
|
||||||
channel_num: 1-indexed LoRa slot (1..numChannels). 88 is mid-upper US
|
|
||||||
band. Set to 0 to fall back to name-hash (not recommended for
|
|
||||||
isolation).
|
|
||||||
region: short code from `KNOWN_REGIONS`.
|
|
||||||
modem_preset: short code from `KNOWN_MODEM_PRESETS`.
|
|
||||||
short_name: optional owner short-name stamp (≤4 chars). None = unset.
|
|
||||||
long_name: optional owner long-name stamp. None = unset.
|
|
||||||
disable_mqtt: if True (default), disables the MQTT module and the
|
|
||||||
uplink/downlink bridge on the primary channel - so private test
|
|
||||||
traffic never leaks to a public broker.
|
|
||||||
disable_position: if True, disables GPS + position broadcasts - useful
|
|
||||||
when test devices sit on a bench without antennas.
|
|
||||||
enable_ui_log: if True, stamps `USERPREFS_UI_TEST_LOG=true` so the
|
|
||||||
firmware emits one `Screen: frame N/M name=... reason=...` log
|
|
||||||
line per frame transition. Test-only; off by default because the
|
|
||||||
log is chatty (multiple times per second during UI interaction).
|
|
||||||
|
|
||||||
"""
|
|
||||||
if region not in KNOWN_REGIONS:
|
|
||||||
raise ValueError(
|
|
||||||
f"Unknown region {region!r}. Known: {sorted(KNOWN_REGIONS.keys())}"
|
|
||||||
)
|
|
||||||
if modem_preset not in KNOWN_MODEM_PRESETS:
|
|
||||||
raise ValueError(
|
|
||||||
f"Unknown modem_preset {modem_preset!r}. Known: {sorted(KNOWN_MODEM_PRESETS.keys())}"
|
|
||||||
)
|
|
||||||
if not (0 <= channel_num <= 255):
|
|
||||||
raise ValueError(f"channel_num must be 0..255, got {channel_num}")
|
|
||||||
if len(channel_name) > 11:
|
|
||||||
raise ValueError(
|
|
||||||
f"channel_name {channel_name!r} exceeds Meshtastic's 11-char max"
|
|
||||||
)
|
|
||||||
if short_name is not None and len(short_name) > 4:
|
|
||||||
raise ValueError(f"short_name must be ≤4 chars, got {len(short_name)}")
|
|
||||||
|
|
||||||
psk = generate_psk(seed=psk_seed)
|
|
||||||
|
|
||||||
prefs: dict[str, Any] = {
|
|
||||||
# --- LoRa ---
|
|
||||||
"USERPREFS_CONFIG_LORA_REGION": KNOWN_REGIONS[region],
|
|
||||||
"USERPREFS_LORACONFIG_MODEM_PRESET": KNOWN_MODEM_PRESETS[modem_preset],
|
|
||||||
"USERPREFS_LORACONFIG_CHANNEL_NUM": channel_num,
|
|
||||||
# --- Primary channel (isolated from public default) ---
|
|
||||||
"USERPREFS_CHANNELS_TO_WRITE": 1,
|
|
||||||
"USERPREFS_CHANNEL_0_NAME": channel_name,
|
|
||||||
"USERPREFS_CHANNEL_0_PSK": psk,
|
|
||||||
"USERPREFS_CHANNEL_0_PRECISION": 14,
|
|
||||||
}
|
|
||||||
if disable_mqtt:
|
|
||||||
prefs.update(
|
|
||||||
{
|
|
||||||
"USERPREFS_CONFIG_LORA_IGNORE_MQTT": True,
|
|
||||||
"USERPREFS_MQTT_ENABLED": 0,
|
|
||||||
"USERPREFS_CHANNEL_0_UPLINK_ENABLED": False,
|
|
||||||
"USERPREFS_CHANNEL_0_DOWNLINK_ENABLED": False,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
if disable_position:
|
|
||||||
prefs.update(
|
|
||||||
{
|
|
||||||
"USERPREFS_CONFIG_GPS_MODE": "meshtastic_Config_PositionConfig_GpsMode_DISABLED",
|
|
||||||
"USERPREFS_CONFIG_SMART_POSITION_ENABLED": False,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
if long_name is not None:
|
|
||||||
prefs["USERPREFS_CONFIG_OWNER_LONG_NAME"] = long_name
|
|
||||||
if short_name is not None:
|
|
||||||
prefs["USERPREFS_CONFIG_OWNER_SHORT_NAME"] = short_name
|
|
||||||
if enable_ui_log:
|
|
||||||
# Consumed by `#ifdef USERPREFS_UI_TEST_LOG` in src/graphics/Screen.cpp.
|
|
||||||
prefs["USERPREFS_UI_TEST_LOG"] = True
|
|
||||||
|
|
||||||
return prefs
|
|
||||||
|
|
||||||
|
|
||||||
@contextmanager
|
|
||||||
def temporary_overrides(overrides: dict[str, Any] | None) -> Iterator[dict[str, str]]:
|
|
||||||
"""Apply `overrides` to userPrefs.jsonc for the duration of the context.
|
|
||||||
|
|
||||||
Yields a dict of the *effective* active values (original active merged
|
|
||||||
with overrides). Always restores the original file on exit, even on
|
|
||||||
exception. If `overrides` is None or empty, this is a no-op.
|
|
||||||
|
|
||||||
The restore writes the original file content byte-for-byte, so there's no
|
|
||||||
round-trip ambiguity even if the file had unusual whitespace.
|
|
||||||
"""
|
|
||||||
if not overrides:
|
|
||||||
state = read_state()
|
|
||||||
yield dict(state["active"])
|
|
||||||
return
|
|
||||||
|
|
||||||
path = jsonc_path()
|
|
||||||
if not path.is_file():
|
|
||||||
raise FileNotFoundError(f"userPrefs.jsonc not found at {path}")
|
|
||||||
|
|
||||||
original_bytes = path.read_bytes()
|
|
||||||
original_stat = path.stat()
|
|
||||||
|
|
||||||
# Merge and write
|
|
||||||
state = _parse_jsonc_state(original_bytes.decode("utf-8"))
|
|
||||||
effective = dict(state["active"])
|
|
||||||
commented = dict(state["commented"])
|
|
||||||
order = list(state["order"])
|
|
||||||
for key, raw in overrides.items():
|
|
||||||
if not key.startswith("USERPREFS_"):
|
|
||||||
raise ValueError(f"key {key!r} must start with USERPREFS_")
|
|
||||||
effective[key] = _stringify(raw)
|
|
||||||
commented.pop(key, None)
|
|
||||||
if key not in order:
|
|
||||||
order.append(key)
|
|
||||||
|
|
||||||
rendered = _render_jsonc(effective, commented, order)
|
|
||||||
_validate_after_write(rendered)
|
|
||||||
path.write_text(rendered, encoding="utf-8")
|
|
||||||
# pio watches file mtimes to invalidate build cache; force the modification
|
|
||||||
# time to now so a pre-existing `.pio/build/<env>/` cache is discarded.
|
|
||||||
now = time.time()
|
|
||||||
import os
|
|
||||||
|
|
||||||
os.utime(path, (now, now))
|
|
||||||
|
|
||||||
try:
|
|
||||||
yield effective
|
|
||||||
finally:
|
|
||||||
path.write_bytes(original_bytes)
|
|
||||||
os.utime(path, (original_stat.st_atime, original_stat.st_mtime))
|
|
||||||
@@ -1,116 +0,0 @@
|
|||||||
# Meshtastic MCP Server - Test Harness
|
|
||||||
|
|
||||||
Automated test suite for the MCP server, organized around real operator
|
|
||||||
concerns rather than generic "unit vs hardware".
|
|
||||||
|
|
||||||
## Tiers
|
|
||||||
|
|
||||||
| Dir | Hardware | Question this tier answers |
|
|
||||||
| --------------- | ----------------------- | --------------------------------------------------------------------- |
|
|
||||||
| `unit/` | none | Do the parsing / filtering / profile-generation primitives work? |
|
|
||||||
| `provisioning/` | 1 device, per-test bake | Did my pre-bake recipe stick? Does it survive a factory reset? |
|
|
||||||
| `admin/` | 1 device, shared bake | Do my daily admin ops (owner, channel URL, config writes) round-trip? |
|
|
||||||
| `mesh/` | 2 devices, shared bake | Do my devices actually form a mesh? Send + receive? ACKs? |
|
|
||||||
| `telemetry/` | 2 devices, shared bake | Is telemetry reporting? Is position broadcast correct? |
|
|
||||||
| `monitor/` | 1 device, shared bake | Is the boot log clean (no panics)? |
|
|
||||||
| `fleet/` | varies | Are my CI runs isolated from each other? Are reflashes idempotent? |
|
|
||||||
|
|
||||||
## Quick start
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd mcp-server
|
|
||||||
pip install -e ".[test]"
|
|
||||||
|
|
||||||
# No hardware - 33 unit tests, ~3 seconds
|
|
||||||
pytest tests/unit -v
|
|
||||||
|
|
||||||
# Hub attached (nRF52840 + ESP32-S3) - first run bakes, then exercises everything
|
|
||||||
pytest tests/ --html=report.html
|
|
||||||
|
|
||||||
# Hub already baked with session profile (dev loop) - skip bake
|
|
||||||
pytest tests/ --assume-baked --html=report.html
|
|
||||||
|
|
||||||
# Force a rebake (new firmware, new seed, etc.)
|
|
||||||
pytest tests/ --force-bake --html=report.html
|
|
||||||
```
|
|
||||||
|
|
||||||
## CLI flags
|
|
||||||
|
|
||||||
- `--force-bake` - always reflash both roles at session start, even if the
|
|
||||||
current state matches the session profile.
|
|
||||||
- `--assume-baked` - skip `test_00_bake.py` entirely. Use when you know the
|
|
||||||
devices are already baked and want a fast dev loop.
|
|
||||||
- `--hub-profile=<yaml>` - point at a YAML file for non-default hub hardware.
|
|
||||||
Default targets VID `0x239a` (nRF52) and `0x303a`/`0x10c4` (ESP32-S3).
|
|
||||||
- `--no-teardown-rebake` - skip the session-end rebake that `provisioning/`
|
|
||||||
and `fleet/` tests perform. Useful in rapid iteration.
|
|
||||||
|
|
||||||
## Environment variables
|
|
||||||
|
|
||||||
- `MESHTASTIC_FIRMWARE_ROOT` - firmware repo path (defaults to `../` from tests/)
|
|
||||||
- `MESHTASTIC_MCP_ENV_NRF52` - PlatformIO env for the nRF52 role (default
|
|
||||||
`rak4631`)
|
|
||||||
- `MESHTASTIC_MCP_ENV_ESP32S3` - PlatformIO env for the ESP32-S3 role (default
|
|
||||||
`heltec-v3`)
|
|
||||||
- `MESHTASTIC_MCP_SEED` - override the session PSK seed (default:
|
|
||||||
`pytest-<unix-ts>`). Set this to reproduce a specific failing run.
|
|
||||||
|
|
||||||
## Fixtures you'll use when adding tests
|
|
||||||
|
|
||||||
All defined in `conftest.py`:
|
|
||||||
|
|
||||||
- **`hub_devices`** → `{"nrf52": "/dev/cu.X", "esp32s3": "/dev/cu.Y"}`. Auto-
|
|
||||||
skips the test if a required role isn't present.
|
|
||||||
- **`test_profile`** → USERPREFS dict for the session (`build_testing_profile`).
|
|
||||||
- **`no_region_profile`** → variant without `USERPREFS_CONFIG_LORA_REGION`.
|
|
||||||
- **`baked_mesh`** → verifies both devices are baked with the session profile
|
|
||||||
(does NOT reflash - that's `test_00_bake.py`'s job).
|
|
||||||
- **`baked_single`** → single verified baked device; parametrize `request.param`
|
|
||||||
to pick role.
|
|
||||||
- **`serial_capture`** → factory; `cap = serial_capture("esp32s3")` starts a
|
|
||||||
pio device monitor session, drains into a per-test buffer, attaches the
|
|
||||||
buffer to the pytest-html report on failure.
|
|
||||||
- **`wait_until`** → exponential-backoff polling helper; `wait_until(lambda:
|
|
||||||
predicate(), timeout=60)` replaces flaky `time.sleep()` patterns.
|
|
||||||
|
|
||||||
## Reports
|
|
||||||
|
|
||||||
`pytest --html=report.html` produces a self-contained HTML with:
|
|
||||||
|
|
||||||
- Per-test pass/fail/skip with timings
|
|
||||||
- On failure: serial log capture from any `serial_capture` fixture used
|
|
||||||
- On failure: `device_info` + lora config JSON for every role on the hub
|
|
||||||
- Session seed and session start time (for reproducibility)
|
|
||||||
|
|
||||||
`pytest --junitxml=junit.xml` produces CI-integration XML.
|
|
||||||
|
|
||||||
`tool_coverage.json` is emitted at session end in the tests directory - shows
|
|
||||||
which of the 38 MCP tools the run exercised. Useful for closing test gaps.
|
|
||||||
|
|
||||||
## Adding a new test
|
|
||||||
|
|
||||||
1. Pick the category that matches the operator concern (not the technical
|
|
||||||
surface). "Does my fleet's owner name persist" is `admin/`, not `unit/`.
|
|
||||||
2. If you need both devices, depend on `baked_mesh`. If you need one, depend
|
|
||||||
on `baked_single`. If you need to mutate hardware state, put it in
|
|
||||||
`provisioning/` or `fleet/` and add a `try/finally` teardown that re-bakes
|
|
||||||
the session profile.
|
|
||||||
3. Use `wait_until` for anything involving LoRa timing - fixed `sleep()`
|
|
||||||
produces flakes.
|
|
||||||
4. Use `serial_capture` when you need to observe firmware log output (e.g.
|
|
||||||
"did the packet get decoded?").
|
|
||||||
5. Add a `@pytest.mark.timeout(N)` - mesh tests routinely hit LoRa-airtime
|
|
||||||
waits; default pytest timeout is infinite.
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
- **All hardware tests SKIP** → hub not detected. Plug in the USB hub, verify
|
|
||||||
with `pytest tests/ --collect-only` or `python -c "from meshtastic_mcp import
|
|
||||||
devices; print(devices.list_devices())"`.
|
|
||||||
- **`baked_mesh` fails with "devices not baked"** → run `pytest
|
|
||||||
tests/test_00_bake.py` first, or pass `--force-bake` on the full run.
|
|
||||||
- **Mesh formation tests time out** → check that both devices are on the same
|
|
||||||
session profile (`--force-bake` forces both to the current seed).
|
|
||||||
- **Provisioning tests leave device in bad state** → teardowns re-bake, but
|
|
||||||
if a test crashes between "bake broken state" and "bake good state", run
|
|
||||||
`pytest tests/test_00_bake.py --force-bake` to recover.
|
|
||||||
@@ -1,118 +0,0 @@
|
|||||||
"""Role-to-port rediscovery after USB CDC re-enumeration.
|
|
||||||
|
|
||||||
Used by tests that mutate device identity in ways macOS treats as a
|
|
||||||
"new device" - notably ``factory_reset(full=False)`` on the nRF52840 and
|
|
||||||
any operation that kicks the device through its bootloader. Both cases
|
|
||||||
cause the kernel to re-assign the ``/dev/cu.usbmodem*`` path; a test that
|
|
||||||
captured the pre-operation port and reuses it after will fail with
|
|
||||||
``FileNotFoundError``.
|
|
||||||
|
|
||||||
The helper polls :func:`meshtastic_mcp.devices.list_devices` (the same API
|
|
||||||
``run-tests.sh`` and ``conftest.py::hub_devices`` use for initial hub
|
|
||||||
detection) filtered by the role's canonical USB VID. Returns the first
|
|
||||||
matching port - equivalent to "give me the single nRF52 (or ESP32-S3) on
|
|
||||||
the bench right now, whichever `cu.*` path it happens to be at".
|
|
||||||
|
|
||||||
Test-harness-local (not exported from ``meshtastic_mcp``): a thin wrapper
|
|
||||||
over public ``devices.list_devices`` with no extra moving parts. If a
|
|
||||||
non-test caller ever needs this, it's trivial to promote.
|
|
||||||
|
|
||||||
Caveat: the session-scoped ``hub_devices`` fixture snapshots ports at
|
|
||||||
session start and is dict-keyed - it doesn't learn about re-enumerations.
|
|
||||||
Tests that call ``resolve_port_by_role`` should use the returned port
|
|
||||||
locally for the rest of the test body rather than expecting
|
|
||||||
``hub_devices[role]`` to update.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
|
|
||||||
from meshtastic_mcp import devices as devices_module
|
|
||||||
|
|
||||||
# Role → canonical VID(s). Kept in sync with:
|
|
||||||
# - `mcp-server/run-tests.sh` (ROLE_BY_VID)
|
|
||||||
# - `mcp-server/tests/conftest.py::hub_profile`
|
|
||||||
# If any of those change, this must too.
|
|
||||||
_ROLE_VIDS: dict[str, tuple[int, ...]] = {
|
|
||||||
"nrf52": (0x239A,), # Adafruit / RAK nRF52840 native USB
|
|
||||||
"esp32s3": (0x303A, 0x10C4), # Espressif native USB + CP2102 USB-UART
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _coerce_vid(raw: object) -> int | None:
|
|
||||||
"""`devices.list_devices` returns vid as either '0x239a' or an int;
|
|
||||||
normalize to int. None on un-parseable input (matches the same fault-
|
|
||||||
tolerance `run-tests.sh` uses for its role detection)."""
|
|
||||||
if raw is None:
|
|
||||||
return None
|
|
||||||
if isinstance(raw, int):
|
|
||||||
return raw
|
|
||||||
if isinstance(raw, str):
|
|
||||||
try:
|
|
||||||
return int(raw, 16) if raw.lower().startswith("0x") else int(raw)
|
|
||||||
except ValueError:
|
|
||||||
return None
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_port_by_role(
|
|
||||||
role: str,
|
|
||||||
*,
|
|
||||||
timeout_s: float = 30.0,
|
|
||||||
poll_start: float = 0.5,
|
|
||||||
poll_max: float = 5.0,
|
|
||||||
) -> str:
|
|
||||||
"""Return the current ``/dev/cu.*`` path for ``role`` once one appears.
|
|
||||||
|
|
||||||
Polls ``devices.list_devices(include_unknown=True)`` every ``poll_start``
|
|
||||||
seconds (1.5× backoff, capped at ``poll_max``) until a device matching
|
|
||||||
``role``'s VID appears. Returns the first matching port.
|
|
||||||
|
|
||||||
On timeout raises :class:`AssertionError` with the list of devices that
|
|
||||||
WERE seen - helpful when debugging "wrong board connected" vs. "no
|
|
||||||
board connected" vs. "still re-enumerating".
|
|
||||||
|
|
||||||
Args:
|
|
||||||
role: ``"nrf52"`` or ``"esp32s3"`` (keys of ``_ROLE_VIDS``).
|
|
||||||
timeout_s: upper bound on how long to wait for the device to
|
|
||||||
re-appear. Default 30 s - nRF52 factory_reset observed at
|
|
||||||
2-12 s on a healthy lab hub.
|
|
||||||
poll_start: initial poll interval in seconds. Default 0.5 s.
|
|
||||||
poll_max: cap on poll interval after backoff. Default 5 s.
|
|
||||||
|
|
||||||
Raises:
|
|
||||||
AssertionError: if no matching device appears within ``timeout_s``.
|
|
||||||
ValueError: if ``role`` is not in ``_ROLE_VIDS``.
|
|
||||||
|
|
||||||
"""
|
|
||||||
if role not in _ROLE_VIDS:
|
|
||||||
raise ValueError(f"unknown role {role!r}; expected one of {sorted(_ROLE_VIDS)}")
|
|
||||||
wanted_vids = _ROLE_VIDS[role]
|
|
||||||
|
|
||||||
deadline = time.monotonic() + timeout_s
|
|
||||||
delay = poll_start
|
|
||||||
last_seen: list[dict] = []
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
try:
|
|
||||||
last_seen = devices_module.list_devices(include_unknown=True)
|
|
||||||
except Exception as exc:
|
|
||||||
# list_devices is wrapped by meshtastic_mcp.devices and
|
|
||||||
# shouldn't raise on normal enumeration - but a kernel-level
|
|
||||||
# USB hiccup during re-enumeration can bubble up briefly.
|
|
||||||
# Treat as "nothing seen this round" and retry.
|
|
||||||
last_seen = [{"error": repr(exc)}]
|
|
||||||
for dev in last_seen:
|
|
||||||
vid = _coerce_vid(dev.get("vid"))
|
|
||||||
if vid is not None and vid in wanted_vids and dev.get("port"):
|
|
||||||
return dev["port"]
|
|
||||||
time.sleep(delay)
|
|
||||||
delay = min(delay * 1.5, poll_max)
|
|
||||||
|
|
||||||
# Timeout path - include what we saw so the operator can tell
|
|
||||||
# "nothing plugged in" from "wrong VID" from "transient USB error".
|
|
||||||
raise AssertionError(
|
|
||||||
f"no device matching role {role!r} (VIDs "
|
|
||||||
f"{[hex(v) for v in wanted_vids]}) appeared within {timeout_s:.0f}s. "
|
|
||||||
f"Last enumeration: {last_seen!r}"
|
|
||||||
)
|
|
||||||
@@ -1,112 +0,0 @@
|
|||||||
"""USB hub power control for tests - thin composition of the `uhubctl`
|
|
||||||
module + `_port_discovery.resolve_port_by_role`.
|
|
||||||
|
|
||||||
Why separate from the production module:
|
|
||||||
- `meshtastic_mcp.uhubctl.cycle` returns as soon as uhubctl exits (VBUS is
|
|
||||||
back on, but the device hasn't finished enumerating as a CDC port yet).
|
|
||||||
- Tests that want to immediately issue a `connect(port=...)` need the NEW
|
|
||||||
`/dev/cu.*` path, which can differ from the pre-cycle path on nRF52
|
|
||||||
boards (CDC re-enumeration assigns a fresh `cu.usbmodemNNNN`).
|
|
||||||
- `resolve_port_by_role` already handles that wait + path-resolution for
|
|
||||||
the `factory_reset` flow. Composing the two gives a one-call helper.
|
|
||||||
|
|
||||||
Also exposes `is_uhubctl_available()` so fixtures can skip cleanly when
|
|
||||||
uhubctl isn't installed - we never want "no uhubctl" to look like a test
|
|
||||||
failure.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from meshtastic_mcp import config as config_mod
|
|
||||||
from meshtastic_mcp import uhubctl as uhubctl_mod
|
|
||||||
|
|
||||||
from ._port_discovery import resolve_port_by_role
|
|
||||||
|
|
||||||
|
|
||||||
def is_uhubctl_available() -> bool:
|
|
||||||
"""Return True iff `config.uhubctl_bin()` resolves AND the binary is callable.
|
|
||||||
|
|
||||||
Soft-fails silently - fixtures use this to `pytest.skip` with an
|
|
||||||
actionable message when the operator hasn't installed uhubctl.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
config_mod.uhubctl_bin()
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
return False
|
|
||||||
# Do NOT actually invoke uhubctl here - on macOS a non-sudo run would
|
|
||||||
# fail, which is a config issue, not a tool-missing issue. That gets
|
|
||||||
# surfaced to the user when they actually run a recovery action.
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
def power_on(role: str) -> dict[str, Any]:
|
|
||||||
"""Power on the hub port hosting `role`. Does NOT wait for re-enumeration.
|
|
||||||
Use `power_cycle` or follow with `resolve_port_by_role` to block on readiness.
|
|
||||||
"""
|
|
||||||
loc, port = uhubctl_mod.resolve_target(role)
|
|
||||||
return uhubctl_mod.power_on(loc, port)
|
|
||||||
|
|
||||||
|
|
||||||
def power_off(role: str) -> dict[str, Any]:
|
|
||||||
"""Power off the hub port hosting `role`. The device disappears from
|
|
||||||
`list_devices` immediately.
|
|
||||||
"""
|
|
||||||
loc, port = uhubctl_mod.resolve_target(role)
|
|
||||||
return uhubctl_mod.power_off(loc, port)
|
|
||||||
|
|
||||||
|
|
||||||
def power_cycle(
|
|
||||||
role: str,
|
|
||||||
*,
|
|
||||||
delay_s: int = 2,
|
|
||||||
rediscover_timeout_s: float = 30.0,
|
|
||||||
) -> str:
|
|
||||||
"""Cycle the port hosting `role`, wait for re-enumeration, return the
|
|
||||||
new port path.
|
|
||||||
|
|
||||||
On nRF52 the post-cycle path typically matches the pre-cycle path, but
|
|
||||||
macOS may assign a different `/dev/cu.usbmodemNNNN` if the previous
|
|
||||||
CDC endpoint hasn't been fully released. `resolve_port_by_role`
|
|
||||||
handles that transparently.
|
|
||||||
"""
|
|
||||||
loc, port = uhubctl_mod.resolve_target(role)
|
|
||||||
uhubctl_mod.cycle(loc, port, delay_s=delay_s)
|
|
||||||
# After uhubctl exits, VBUS is on but the device may still be in
|
|
||||||
# bootloader init. Give it ~500 ms head-start before polling so we
|
|
||||||
# don't spam list_devices pointlessly.
|
|
||||||
time.sleep(0.5)
|
|
||||||
return resolve_port_by_role(role, timeout_s=rediscover_timeout_s)
|
|
||||||
|
|
||||||
|
|
||||||
def wait_for_absence(role: str, *, timeout_s: float = 10.0) -> None:
|
|
||||||
"""Block until a device matching `role` is NOT in `list_devices`.
|
|
||||||
|
|
||||||
Used by the recovery tier to assert power_off actually took effect.
|
|
||||||
Raises TimeoutError on failure.
|
|
||||||
"""
|
|
||||||
from meshtastic_mcp import devices as devices_mod
|
|
||||||
|
|
||||||
from ._port_discovery import _ROLE_VIDS, _coerce_vid # type: ignore[attr-defined]
|
|
||||||
|
|
||||||
if role not in _ROLE_VIDS:
|
|
||||||
raise ValueError(f"unknown role {role!r}")
|
|
||||||
wanted = _ROLE_VIDS[role]
|
|
||||||
deadline = time.monotonic() + timeout_s
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
found = devices_mod.list_devices(include_unknown=True)
|
|
||||||
if not any(_coerce_vid(d.get("vid")) in wanted for d in found):
|
|
||||||
return
|
|
||||||
time.sleep(0.3)
|
|
||||||
raise TimeoutError(f"role {role!r} still visible after {timeout_s}s of power_off")
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"is_uhubctl_available",
|
|
||||||
"power_cycle",
|
|
||||||
"power_off",
|
|
||||||
"power_on",
|
|
||||||
"wait_for_absence",
|
|
||||||
]
|
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
"""Admin: channel URL export and re-import round-trip.
|
|
||||||
|
|
||||||
Real operator workflow: "I have two fleets, I want them to share a channel
|
|
||||||
config. Export URL from fleet A's bootstrap device, paste into fleet B's
|
|
||||||
onboarding tool, expect identical channels." Proves `getURL` + `setURL`
|
|
||||||
round-trip without data loss.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin, info
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(60)
|
|
||||||
def test_channel_url_roundtrip(
|
|
||||||
baked_single: dict[str, Any],
|
|
||||||
test_profile: dict[str, Any],
|
|
||||||
) -> None:
|
|
||||||
"""Runs once per connected role. Verify:
|
|
||||||
1. `get_channel_url()` on a baked device returns a non-empty URL.
|
|
||||||
2. The URL parses - `set_channel_url(url)` accepts it without error.
|
|
||||||
3. After set, `get_channel_url()` returns the same (canonicalized) URL.
|
|
||||||
4. Primary channel name survives round-trip.
|
|
||||||
"""
|
|
||||||
port = baked_single["port"]
|
|
||||||
|
|
||||||
url_before = admin.get_channel_url(include_all=False, port=port)["url"]
|
|
||||||
assert url_before, "device returned empty channel URL"
|
|
||||||
assert (
|
|
||||||
"meshtastic" in url_before.lower() or "#" in url_before
|
|
||||||
), f"URL does not look like a Meshtastic channel URL: {url_before!r}"
|
|
||||||
|
|
||||||
# Re-apply the same URL - no-op in content but exercises the setURL path.
|
|
||||||
applied = admin.set_channel_url(url=url_before, port=port)
|
|
||||||
assert applied["ok"] is True
|
|
||||||
assert applied["channels_imported"] >= 1
|
|
||||||
time.sleep(2.0)
|
|
||||||
|
|
||||||
# Confirm the primary channel name survived
|
|
||||||
live = info.device_info(port=port, timeout_s=8.0)
|
|
||||||
assert live["primary_channel"] == test_profile["USERPREFS_CHANNEL_0_NAME"]
|
|
||||||
|
|
||||||
url_after = admin.get_channel_url(include_all=False, port=port)["url"]
|
|
||||||
# Canonicalization is tricky: the firmware may re-serialize the protobuf
|
|
||||||
# with fields in a different order, producing a visually-different URL
|
|
||||||
# that encodes the same content. Accept that as a success when the
|
|
||||||
# primary channel name survived the round-trip (already asserted above)
|
|
||||||
# and the URL is still a parseable Meshtastic URL. Bit-equality is a
|
|
||||||
# nice-to-have, not a correctness guarantee.
|
|
||||||
assert url_after, "URL went blank after setURL"
|
|
||||||
assert (
|
|
||||||
"meshtastic" in url_after.lower() or "#" in url_after
|
|
||||||
), f"URL after setURL no longer looks like a channel URL: {url_after!r}"
|
|
||||||
@@ -1,106 +0,0 @@
|
|||||||
"""Admin: a config mutation survives a reboot.
|
|
||||||
|
|
||||||
This is the most-critical admin behavior not tested elsewhere. If
|
|
||||||
config persistence breaks in a firmware release, every deployed device
|
|
||||||
gets bricked on its next reboot (channels lost, region lost, owner lost,
|
|
||||||
everything back to Meshtastic stock). The fleet blast radius is "every
|
|
||||||
unit on every shelf" - easily worth one explicit test per release.
|
|
||||||
|
|
||||||
Pattern: single-device (``baked_single``, one test per role). Mutate a
|
|
||||||
benign, easy-to-observe LoRa field (``lora.hop_limit``), confirm
|
|
||||||
pre-reboot, reboot, rediscover port (nRF52 may re-enumerate), verify
|
|
||||||
the value survived, restore original for downstream tests.
|
|
||||||
|
|
||||||
Why ``lora.hop_limit`` specifically:
|
|
||||||
* Non-destructive - doesn't change region, channel, or PSK, so
|
|
||||||
downstream mesh tests still work regardless of the flipped value.
|
|
||||||
* Bounded small-integer (1..7) - easy to flip to a definitively
|
|
||||||
different value and read back.
|
|
||||||
* Persisted via ``writeConfig("lora")`` which is the same path
|
|
||||||
every other LoRa config mutation uses, so we're really testing
|
|
||||||
the whole lora-config persistence pipeline end-to-end.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin, info
|
|
||||||
|
|
||||||
from .._port_discovery import resolve_port_by_role
|
|
||||||
|
|
||||||
|
|
||||||
def _get_hop_limit(port: str) -> int:
|
|
||||||
"""Read `lora.hop_limit` from the device's current config."""
|
|
||||||
lora = admin.get_config("lora", port=port).get("config", {}).get("lora", {})
|
|
||||||
hl = lora.get("hop_limit")
|
|
||||||
assert isinstance(hl, int), (
|
|
||||||
f"lora.hop_limit missing or non-int in get_config response: " f"{lora!r}"
|
|
||||||
)
|
|
||||||
return hl
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(180)
|
|
||||||
def test_lora_hop_limit_survives_reboot(
|
|
||||||
baked_single: dict[str, Any],
|
|
||||||
wait_until,
|
|
||||||
) -> None:
|
|
||||||
"""Runs once per connected role. Mutates `lora.hop_limit`, reboots,
|
|
||||||
verifies the new value is still there after the device comes back.
|
|
||||||
"""
|
|
||||||
role = baked_single["role"]
|
|
||||||
port = baked_single["port"]
|
|
||||||
|
|
||||||
original = _get_hop_limit(port)
|
|
||||||
# Flip to a definitively different value within the protocol's
|
|
||||||
# valid range (1..7 per LoRaConfig.hop_limit comment). Pick 5 if
|
|
||||||
# current is != 5, else 4.
|
|
||||||
new_value = 5 if original != 5 else 4
|
|
||||||
|
|
||||||
try:
|
|
||||||
admin.set_config("lora.hop_limit", new_value, port=port)
|
|
||||||
|
|
||||||
# Pre-reboot sanity: the write reached the device and
|
|
||||||
# get_config reflects it in-memory. If this fails, the persist
|
|
||||||
# test below is moot - something's wrong with the write path
|
|
||||||
# itself, not with persistence.
|
|
||||||
assert _get_hop_limit(port) == new_value, (
|
|
||||||
f"pre-reboot readback failed: set {new_value}, got "
|
|
||||||
f"{_get_hop_limit(port)}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Reboot. `seconds=3` gives the Python client time to
|
|
||||||
# disconnect cleanly; sleep long enough for the boot to start
|
|
||||||
# before we begin polling.
|
|
||||||
admin.reboot(port=port, confirm=True, seconds=3)
|
|
||||||
time.sleep(8.0)
|
|
||||||
|
|
||||||
# nRF52 re-enumerates on reboot → rediscover.
|
|
||||||
port = resolve_port_by_role(role, timeout_s=60.0)
|
|
||||||
wait_until(
|
|
||||||
lambda: info.device_info(port=port, timeout_s=5.0).get("my_node_num")
|
|
||||||
is not None,
|
|
||||||
timeout=60,
|
|
||||||
backoff_start=1.0,
|
|
||||||
)
|
|
||||||
|
|
||||||
# The assertion this test exists for: the mutation persisted
|
|
||||||
# across the reboot cycle through NVS / LittleFS / UICR.
|
|
||||||
post = _get_hop_limit(port)
|
|
||||||
assert post == new_value, (
|
|
||||||
f"lora.hop_limit did not survive reboot: set to {new_value} "
|
|
||||||
f"pre-reboot, read back {post} post-reboot. Config persistence "
|
|
||||||
f"is broken - downstream fleet impact would be total."
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
# Restore so downstream tests see the original hop_limit.
|
|
||||||
# Wrapped in its own try to avoid masking the real assertion
|
|
||||||
# if the restore itself races the reboot - the worst case
|
|
||||||
# there is a non-default hop_limit sticks around, which is
|
|
||||||
# benign (mesh still works at hop_limit 3 or 5).
|
|
||||||
try:
|
|
||||||
admin.set_config("lora.hop_limit", original, port=port)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
@@ -1,59 +0,0 @@
|
|||||||
"""Admin: owner name persists across a reboot.
|
|
||||||
|
|
||||||
The single most common "did my admin change stick?" test. Proves
|
|
||||||
`localNode.setOwner()` + `writeConfig("device")` commits to non-volatile
|
|
||||||
storage before the reboot.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin, info
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(120)
|
|
||||||
def test_owner_survives_reboot(
|
|
||||||
baked_single: dict[str, Any],
|
|
||||||
wait_until,
|
|
||||||
) -> None:
|
|
||||||
"""Runs once per connected role - proves the reboot-persistence
|
|
||||||
round-trip works on each device independently, not just one."""
|
|
||||||
port = baked_single["port"]
|
|
||||||
|
|
||||||
pre = info.device_info(port=port, timeout_s=8.0)
|
|
||||||
original = pre.get("long_name") or ""
|
|
||||||
marker = "RebootSurvive"
|
|
||||||
try:
|
|
||||||
admin.set_owner(long_name=marker, short_name="RS", port=port)
|
|
||||||
time.sleep(1.5)
|
|
||||||
|
|
||||||
# Confirm pre-reboot
|
|
||||||
confirmed = info.device_info(port=port, timeout_s=8.0)
|
|
||||||
assert confirmed["long_name"] == marker
|
|
||||||
|
|
||||||
# Reboot (short delay)
|
|
||||||
admin.reboot(port=port, confirm=True, seconds=3)
|
|
||||||
|
|
||||||
# Wait for device to come back
|
|
||||||
time.sleep(8.0)
|
|
||||||
wait_until(
|
|
||||||
lambda: info.device_info(port=port, timeout_s=5.0).get("my_node_num")
|
|
||||||
is not None,
|
|
||||||
timeout=60,
|
|
||||||
backoff_start=1.0,
|
|
||||||
)
|
|
||||||
|
|
||||||
post = info.device_info(port=port, timeout_s=8.0)
|
|
||||||
assert post["long_name"] == marker, (
|
|
||||||
f"owner name did not persist across reboot: "
|
|
||||||
f"expected {marker!r}, got {post['long_name']!r}"
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
# Restore original (best-effort)
|
|
||||||
try:
|
|
||||||
admin.set_owner(long_name=original or "TestNode", port=port)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,43 +0,0 @@
|
|||||||
"""Fleet: different session seeds produce non-overlapping PSKs.
|
|
||||||
|
|
||||||
No hardware needed - this is a pure property check on the test profile
|
|
||||||
generator, elevated into the `fleet/` tier because it's the critical
|
|
||||||
invariant for running concurrent CI labs without cross-contamination.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from meshtastic_mcp import userprefs
|
|
||||||
|
|
||||||
|
|
||||||
def test_psk_seed_isolates_runs() -> None:
|
|
||||||
"""Two labs running simultaneously with different seeds must end up with
|
|
||||||
different PSKs - which means firmware baked in lab A cannot decode lab B's
|
|
||||||
traffic, and vice versa.
|
|
||||||
|
|
||||||
This is the formal statement of the isolation claim that
|
|
||||||
`testing_profile` promises operators.
|
|
||||||
"""
|
|
||||||
lab_a_morning = userprefs.build_testing_profile(psk_seed="lab-a-2026-04-16-morning")
|
|
||||||
lab_a_evening = userprefs.build_testing_profile(psk_seed="lab-a-2026-04-16-evening")
|
|
||||||
lab_b_morning = userprefs.build_testing_profile(psk_seed="lab-b-2026-04-16-morning")
|
|
||||||
|
|
||||||
# Same lab, same date, different time-of-day → different PSKs
|
|
||||||
assert (
|
|
||||||
lab_a_morning["USERPREFS_CHANNEL_0_PSK"]
|
|
||||||
!= lab_a_evening["USERPREFS_CHANNEL_0_PSK"]
|
|
||||||
)
|
|
||||||
# Different labs, same time-of-day → different PSKs
|
|
||||||
assert (
|
|
||||||
lab_a_morning["USERPREFS_CHANNEL_0_PSK"]
|
|
||||||
!= lab_b_morning["USERPREFS_CHANNEL_0_PSK"]
|
|
||||||
)
|
|
||||||
|
|
||||||
# Re-deriving with the same seed yields the same PSK (reproducibility)
|
|
||||||
lab_a_morning_again = userprefs.build_testing_profile(
|
|
||||||
psk_seed="lab-a-2026-04-16-morning"
|
|
||||||
)
|
|
||||||
assert (
|
|
||||||
lab_a_morning["USERPREFS_CHANNEL_0_PSK"]
|
|
||||||
== lab_a_morning_again["USERPREFS_CHANNEL_0_PSK"]
|
|
||||||
)
|
|
||||||
@@ -1,220 +0,0 @@
|
|||||||
"""Shared helper for mesh receive tests.
|
|
||||||
|
|
||||||
`pio device monitor` captures firmware log output, which does NOT include
|
|
||||||
decoded text message contents or telemetry payloads - those are only
|
|
||||||
accessible through `meshtastic.SerialInterface`'s pubsub mechanism.
|
|
||||||
|
|
||||||
`ReceiveCollector` opens a long-lived SerialInterface on a port, subscribes
|
|
||||||
to the pubsub topic of interest, and exposes an atomic `wait_for(predicate)`
|
|
||||||
that mesh tests use to verify end-to-end delivery.
|
|
||||||
|
|
||||||
This module also exposes two module-level helpers for forcing a device to
|
|
||||||
broadcast a fresh NodeInfo - the on-demand path that sidesteps the
|
|
||||||
firmware's 10-minute NodeInfo rate-limit. Tests doing directed PKI-encrypted
|
|
||||||
sends need BOTH endpoints to hold current pubkeys for each other:
|
|
||||||
|
|
||||||
nudge_nodeinfo(iface) # nudge an already-open SerialInterface
|
|
||||||
nudge_nodeinfo_port(port) # open briefly, nudge, close
|
|
||||||
|
|
||||||
See `ReceiveCollector.broadcast_nodeinfo_ping` for the firmware-side
|
|
||||||
rationale (PKI staleness → directed sends NAK with Routing.Error=35
|
|
||||||
PKI_UNKNOWN_PUBKEY or 39 PKI_SEND_FAIL_PUBLIC_KEY).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from typing import Any, Callable
|
|
||||||
|
|
||||||
|
|
||||||
def nudge_nodeinfo(iface: Any) -> None:
|
|
||||||
"""Force the device behind ``iface`` to broadcast a fresh NodeInfo.
|
|
||||||
|
|
||||||
Sends a ``ToRadio.Heartbeat(nonce=1)`` - the firmware's documented
|
|
||||||
on-demand NodeInfo trigger (see `src/mesh/api/PacketAPI.cpp:74-79`
|
|
||||||
for TCP/UDP and `src/mesh/PhoneAPI.cpp::handleToRadio` for serial,
|
|
||||||
both routed to `NodeInfoModule::sendOurNodeInfo(..., shorterTimeout=true)`
|
|
||||||
with the 60-s window rather than the 10-min rate-limit).
|
|
||||||
|
|
||||||
Call on BOTH TX and RX ifaces before a directed PKI-encrypted send.
|
|
||||||
Nudging only one side leaves the other with a stale pubkey cache and
|
|
||||||
makes the directed send NAK with PKI_UNKNOWN_PUBKEY.
|
|
||||||
"""
|
|
||||||
from meshtastic.protobuf import mesh_pb2 # type: ignore[import-untyped]
|
|
||||||
|
|
||||||
tr = mesh_pb2.ToRadio()
|
|
||||||
tr.heartbeat.nonce = 1
|
|
||||||
iface._sendToRadio(tr)
|
|
||||||
|
|
||||||
|
|
||||||
def nudge_nodeinfo_port(port: str) -> None:
|
|
||||||
"""Open ``port`` briefly, nudge, close - for when no iface is open yet.
|
|
||||||
|
|
||||||
Uses the meshtastic_mcp port-lock-aware `connect()` context manager
|
|
||||||
so we don't race ReceiveCollector or other long-lived handles on
|
|
||||||
the same port.
|
|
||||||
"""
|
|
||||||
from meshtastic_mcp.connection import connect
|
|
||||||
|
|
||||||
with connect(port=port) as iface:
|
|
||||||
nudge_nodeinfo(iface)
|
|
||||||
|
|
||||||
|
|
||||||
class ReceiveCollector:
|
|
||||||
"""Listen for meshtastic packets on `port` and let tests wait for a match.
|
|
||||||
|
|
||||||
Must be used as a context manager so the underlying SerialInterface is
|
|
||||||
always closed (leaked interfaces hold the CDC port open and break
|
|
||||||
subsequent tool calls).
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
with ReceiveCollector(rx_port, topic="meshtastic.receive.text") as rx:
|
|
||||||
# ... send from TX ...
|
|
||||||
assert rx.wait_for(
|
|
||||||
lambda pkt: pkt.get("decoded", {}).get("text") == unique,
|
|
||||||
timeout=60,
|
|
||||||
), f"packet not received; got {rx.snapshot()!r}"
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
port: str,
|
|
||||||
topic: str = "meshtastic.receive",
|
|
||||||
capture_logs: bool = False,
|
|
||||||
) -> None:
|
|
||||||
self._port = port
|
|
||||||
self._topic = topic
|
|
||||||
self._capture_logs = capture_logs
|
|
||||||
self._packets: list[dict[str, Any]] = []
|
|
||||||
self._log_lines: list[str] = []
|
|
||||||
self._lock = threading.Lock()
|
|
||||||
self._iface = None
|
|
||||||
self._handler_ref = None # keep strong ref so pubsub doesn't GC it
|
|
||||||
self._log_handler_ref = None
|
|
||||||
|
|
||||||
def __enter__(self) -> "ReceiveCollector":
|
|
||||||
from meshtastic.serial_interface import (
|
|
||||||
SerialInterface, # type: ignore[import-untyped]
|
|
||||||
)
|
|
||||||
from pubsub import pub # type: ignore[import-untyped]
|
|
||||||
|
|
||||||
# pubsub uses weak refs by default - we stash a strong ref so the
|
|
||||||
# handler doesn't disappear between subscribe and wait_for.
|
|
||||||
def handler(packet: dict, interface: Any) -> None:
|
|
||||||
with self._lock:
|
|
||||||
self._packets.append(packet)
|
|
||||||
|
|
||||||
self._handler_ref = handler
|
|
||||||
pub.subscribe(handler, self._topic)
|
|
||||||
|
|
||||||
# Firmware-side logs come through the SAME SerialInterface when
|
|
||||||
# `config.security.debug_log_api_enabled = True`. Subscribing here
|
|
||||||
# captures them for failure-artifact attachment without needing a
|
|
||||||
# separate pio monitor session that would fight our port lock.
|
|
||||||
if self._capture_logs:
|
|
||||||
|
|
||||||
def log_handler(line: str, interface: Any) -> None:
|
|
||||||
with self._lock:
|
|
||||||
self._log_lines.append(line)
|
|
||||||
|
|
||||||
self._log_handler_ref = log_handler
|
|
||||||
pub.subscribe(log_handler, "meshtastic.log.line")
|
|
||||||
|
|
||||||
self._iface = SerialInterface(devPath=self._port, connectNow=True)
|
|
||||||
# Let the config bootstrap complete so we don't miss early arrivals.
|
|
||||||
time.sleep(1.0)
|
|
||||||
return self
|
|
||||||
|
|
||||||
def __exit__(self, exc_type: Any, exc: Any, tb: Any) -> None:
|
|
||||||
from pubsub import pub # type: ignore[import-untyped]
|
|
||||||
|
|
||||||
if self._handler_ref is not None:
|
|
||||||
try:
|
|
||||||
pub.unsubscribe(self._handler_ref, self._topic)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
if self._log_handler_ref is not None:
|
|
||||||
try:
|
|
||||||
pub.unsubscribe(self._log_handler_ref, "meshtastic.log.line")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
if self._iface is not None:
|
|
||||||
try:
|
|
||||||
self._iface.close()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def snapshot(self) -> list[dict[str, Any]]:
|
|
||||||
"""Return a thread-safe copy of the list of collected packets."""
|
|
||||||
with self._lock:
|
|
||||||
return list(self._packets)
|
|
||||||
|
|
||||||
def log_snapshot(self) -> list[str]:
|
|
||||||
"""Return captured firmware log lines.
|
|
||||||
|
|
||||||
Only populated if `capture_logs=True` AND the device has
|
|
||||||
`security.debug_log_api_enabled=True`.
|
|
||||||
"""
|
|
||||||
with self._lock:
|
|
||||||
return list(self._log_lines)
|
|
||||||
|
|
||||||
def send_text(
|
|
||||||
self,
|
|
||||||
text: str,
|
|
||||||
destination_id: Any = "^all",
|
|
||||||
want_ack: bool = False,
|
|
||||||
channel_index: int = 0,
|
|
||||||
) -> Any:
|
|
||||||
"""Send a text packet through the already-open SerialInterface.
|
|
||||||
|
|
||||||
Use this when a test also has a ReceiveCollector open on the same port
|
|
||||||
- `admin.send_text(port=...)` would try to open a second SerialInterface
|
|
||||||
and fail the port lock.
|
|
||||||
"""
|
|
||||||
if self._iface is None:
|
|
||||||
raise RuntimeError("ReceiveCollector not started; use as context manager")
|
|
||||||
return self._iface.sendText(
|
|
||||||
text,
|
|
||||||
destinationId=destination_id,
|
|
||||||
wantAck=want_ack,
|
|
||||||
channelIndex=channel_index,
|
|
||||||
)
|
|
||||||
|
|
||||||
def broadcast_nodeinfo_ping(self) -> None:
|
|
||||||
"""Force the firmware on `port` to broadcast a fresh NodeInfo.
|
|
||||||
|
|
||||||
Thin wrapper around the module-level :func:`nudge_nodeinfo` that
|
|
||||||
also validates the context-manager invariant. Delegates so tests
|
|
||||||
that need to nudge BOTH sides (bilateral PKI warmup) share one
|
|
||||||
implementation - the caller just passes each iface in turn.
|
|
||||||
|
|
||||||
Firmware-side details (rate-limit bypass, nonce==1 trigger path,
|
|
||||||
shorterTimeout=true window) are documented on the module-level
|
|
||||||
helper.
|
|
||||||
"""
|
|
||||||
if self._iface is None:
|
|
||||||
raise RuntimeError("ReceiveCollector not started; use as context manager")
|
|
||||||
nudge_nodeinfo(self._iface)
|
|
||||||
|
|
||||||
def wait_for(
|
|
||||||
self,
|
|
||||||
predicate: Callable[[dict[str, Any]], bool],
|
|
||||||
timeout: float = 60.0,
|
|
||||||
poll_interval: float = 0.5,
|
|
||||||
) -> dict[str, Any] | None:
|
|
||||||
"""Block until a received packet matches `predicate` or timeout.
|
|
||||||
|
|
||||||
Returns the matching packet (truthy) or None (falsy).
|
|
||||||
"""
|
|
||||||
deadline = time.monotonic() + timeout
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
with self._lock:
|
|
||||||
for pkt in self._packets:
|
|
||||||
try:
|
|
||||||
if predicate(pkt):
|
|
||||||
return pkt
|
|
||||||
except Exception:
|
|
||||||
continue
|
|
||||||
time.sleep(poll_interval)
|
|
||||||
return None
|
|
||||||
@@ -1,83 +0,0 @@
|
|||||||
"""Mesh: explicit two-way communication, single pass/fail.
|
|
||||||
|
|
||||||
Opens a ReceiveCollector on EVERY role, sends a uniquely-tagged broadcast
|
|
||||||
from each role in turn, and asserts every OTHER role saw it. One atomic
|
|
||||||
test that answers "is the mesh actually working both directions?".
|
|
||||||
|
|
||||||
Not parametrized - it inherently involves the full hub.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from ._receive import ReceiveCollector
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(300)
|
|
||||||
def test_bidirectional_mesh_communication(
|
|
||||||
baked_mesh: dict[str, Any],
|
|
||||||
) -> None:
|
|
||||||
"""Requires ≥2 baked roles.
|
|
||||||
|
|
||||||
For each role, broadcast a unique tag. Assert every other role's
|
|
||||||
ReceiveCollector saw that tag within a 120s window per direction.
|
|
||||||
"""
|
|
||||||
roles = sorted(baked_mesh.keys())
|
|
||||||
if len(roles) < 2:
|
|
||||||
pytest.skip(f"need ≥2 roles; have {roles!r}")
|
|
||||||
|
|
||||||
# Open receive collectors on every role BEFORE sending anything.
|
|
||||||
collectors: dict[str, ReceiveCollector] = {}
|
|
||||||
try:
|
|
||||||
for role in roles:
|
|
||||||
rx = ReceiveCollector(
|
|
||||||
baked_mesh[role]["port"], topic="meshtastic.receive.text"
|
|
||||||
)
|
|
||||||
rx.__enter__()
|
|
||||||
collectors[role] = rx
|
|
||||||
|
|
||||||
# Let the meshtastic interfaces stabilize before the first send
|
|
||||||
time.sleep(2.0)
|
|
||||||
|
|
||||||
# From each role, send a uniquely-tagged broadcast. We MUST send through
|
|
||||||
# the already-open collector - opening a new SerialInterface here would
|
|
||||||
# race the collector's exclusive lock on the port.
|
|
||||||
tags: dict[str, str] = {}
|
|
||||||
for sender in roles:
|
|
||||||
tag = f"bidi-{sender}-{int(time.time() * 1000) % 100_000}"
|
|
||||||
tags[sender] = tag
|
|
||||||
collectors[sender].send_text(tag)
|
|
||||||
# Small gap so airtime doesn't overlap
|
|
||||||
time.sleep(4.0)
|
|
||||||
|
|
||||||
# Every OTHER role must see every sender's tag within 120s each
|
|
||||||
missing: list[str] = []
|
|
||||||
for sender, tag in tags.items():
|
|
||||||
for receiver in roles:
|
|
||||||
if receiver == sender:
|
|
||||||
continue
|
|
||||||
got = collectors[receiver].wait_for(
|
|
||||||
lambda pkt, t=tag: pkt.get("decoded", {}).get("text") == t,
|
|
||||||
timeout=120,
|
|
||||||
)
|
|
||||||
if got is None:
|
|
||||||
observed = [
|
|
||||||
p.get("decoded", {}).get("text")
|
|
||||||
for p in collectors[receiver].snapshot()
|
|
||||||
]
|
|
||||||
missing.append(
|
|
||||||
f"{sender}->{receiver}: tag {tag!r} not seen; "
|
|
||||||
f"receiver got {observed!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
assert not missing, "bidirectional comms incomplete:\n " + "\n ".join(missing)
|
|
||||||
finally:
|
|
||||||
for rx in collectors.values():
|
|
||||||
try:
|
|
||||||
rx.__exit__(None, None, None)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
"""Mesh: broadcast text from TX arrives at RX.
|
|
||||||
|
|
||||||
Uses `meshtastic.SerialInterface` pubsub on RX to detect the decoded text
|
|
||||||
packet - `pio device monitor` output doesn't include message bodies.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin
|
|
||||||
|
|
||||||
from ._receive import ReceiveCollector
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(180)
|
|
||||||
def test_broadcast_delivers(
|
|
||||||
mesh_pair: dict[str, Any],
|
|
||||||
) -> None:
|
|
||||||
"""Runs for every directed role pair. TX sends a unique broadcast text;
|
|
||||||
RX must receive the decoded text via the meshtastic pubsub receive topic
|
|
||||||
within 120s.
|
|
||||||
"""
|
|
||||||
tx_port = mesh_pair["tx"]["port"]
|
|
||||||
rx_port = mesh_pair["rx"]["port"]
|
|
||||||
tx_role = mesh_pair["tx_role"]
|
|
||||||
rx_role = mesh_pair["rx_role"]
|
|
||||||
|
|
||||||
unique = f"mcp-{tx_role}-to-{rx_role}-{int(time.time())}"
|
|
||||||
|
|
||||||
with ReceiveCollector(rx_port, topic="meshtastic.receive.text") as rx:
|
|
||||||
admin.send_text(text=unique, port=tx_port)
|
|
||||||
|
|
||||||
got = rx.wait_for(
|
|
||||||
lambda pkt: pkt.get("decoded", {}).get("text") == unique,
|
|
||||||
timeout=120,
|
|
||||||
)
|
|
||||||
|
|
||||||
assert got is not None, (
|
|
||||||
f"broadcast {unique!r} from {tx_role} not received at {rx_role} within 120s. "
|
|
||||||
f"RX saw {len(rx.snapshot())} text packet(s): "
|
|
||||||
f"{[p.get('decoded', {}).get('text') for p in rx.snapshot()]!r}"
|
|
||||||
)
|
|
||||||
@@ -1,105 +0,0 @@
|
|||||||
"""Mesh: direct text addressed to RX's node_num arrives at RX.
|
|
||||||
|
|
||||||
Uses the same pubsub receive pattern as `test_broadcast_delivers`, but sends
|
|
||||||
with `destinationId=<rx_node_num>` and `wantAck=True`. The assertion is that
|
|
||||||
the RX firmware accepted and decoded the text; the ACK is handled by the
|
|
||||||
firmware transparently (and fires automatically when wantAck is set + the
|
|
||||||
destination is the local node).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp.connection import connect
|
|
||||||
|
|
||||||
from ._receive import ReceiveCollector, nudge_nodeinfo
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(240)
|
|
||||||
def test_direct_with_ack_roundtrip(
|
|
||||||
mesh_pair: dict[str, Any],
|
|
||||||
) -> None:
|
|
||||||
"""Runs for every directed pair. Addressed send from TX to RX's node_num
|
|
||||||
with want_ack=True; RX must receive the decoded text via pubsub.
|
|
||||||
|
|
||||||
Why this proves ACK: setting want_ack on a directed send causes the
|
|
||||||
firmware to retry until an ACK is received. If RX's decoded.text fires
|
|
||||||
once, both the outbound text AND the inbound ACK happened.
|
|
||||||
"""
|
|
||||||
tx_port = mesh_pair["tx"]["port"]
|
|
||||||
rx_port = mesh_pair["rx"]["port"]
|
|
||||||
rx_node_num = mesh_pair["rx"]["my_node_num"]
|
|
||||||
tx_role = mesh_pair["tx_role"]
|
|
||||||
rx_role = mesh_pair["rx_role"]
|
|
||||||
assert rx_node_num is not None, f"{rx_role} my_node_num missing"
|
|
||||||
|
|
||||||
unique = f"mcp-ack-{tx_role}-to-{rx_role}-{int(time.time())}"
|
|
||||||
|
|
||||||
# TX iface stays open across the RX wait - sendText+wantAck relies on
|
|
||||||
# the firmware's retransmit loop, which races the SerialInterface close.
|
|
||||||
# Bilateral NodeInfo nudge: directed packets are PKI-encrypted, so BOTH
|
|
||||||
# sides need current pubkeys (err=35/39 otherwise). See
|
|
||||||
# `tests/mesh/_receive.py::nudge_nodeinfo` for the heartbeat-nonce=1
|
|
||||||
# firmware path.
|
|
||||||
with ReceiveCollector(rx_port, topic="meshtastic.receive.text") as rx:
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
|
|
||||||
with connect(port=tx_port) as tx_iface:
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
|
|
||||||
pk_deadline = time.monotonic() + 45.0
|
|
||||||
last_nudge = time.monotonic()
|
|
||||||
last_rec: dict[str, Any] = {}
|
|
||||||
while time.monotonic() < pk_deadline:
|
|
||||||
last_rec = (tx_iface.nodesByNum or {}).get(rx_node_num, {})
|
|
||||||
user = last_rec.get("user", {})
|
|
||||||
if user.get("publicKey"):
|
|
||||||
break
|
|
||||||
# Re-nudge both sides every 15 s in case a broadcast was
|
|
||||||
# lost to a LoRa collision.
|
|
||||||
if time.monotonic() - last_nudge > 15.0:
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
last_nudge = time.monotonic()
|
|
||||||
time.sleep(1.0)
|
|
||||||
else:
|
|
||||||
pytest.fail(
|
|
||||||
f"TX ({tx_role}) never saw RX ({rx_role}) public key "
|
|
||||||
f"within 45s; nodesByNum entry={last_rec!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Retry covers LoRa collisions. Re-nudge both sides between
|
|
||||||
# attempts - if RX's cached TX pubkey is stale, just re-sending
|
|
||||||
# the text doesn't heal it; re-broadcasting NodeInfo does.
|
|
||||||
got = None
|
|
||||||
for _attempt in range(2):
|
|
||||||
packet = tx_iface.sendText(
|
|
||||||
unique,
|
|
||||||
destinationId=rx_node_num,
|
|
||||||
wantAck=True,
|
|
||||||
)
|
|
||||||
assert packet is not None, "sendText returned None"
|
|
||||||
got = rx.wait_for(
|
|
||||||
lambda pkt: pkt.get("decoded", {}).get("text") == unique,
|
|
||||||
timeout=30,
|
|
||||||
)
|
|
||||||
if got is not None:
|
|
||||||
break
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
time.sleep(5.0)
|
|
||||||
|
|
||||||
assert got is not None, (
|
|
||||||
f"directed send {unique!r} from {tx_role} to {rx_role} "
|
|
||||||
f"(node_num 0x{rx_node_num:08x}) not received within 120s. "
|
|
||||||
f"RX saw {len(rx.snapshot())} text packet(s): "
|
|
||||||
f"{[p.get('decoded', {}).get('text') for p in rx.snapshot()]!r}"
|
|
||||||
)
|
|
||||||
# Additional: confirm the destination matches (not leaked broadcast)
|
|
||||||
assert got.get("to") == rx_node_num, (
|
|
||||||
f"received packet destination mismatch: to={got.get('to')}, "
|
|
||||||
f"expected 0x{rx_node_num:08x}"
|
|
||||||
)
|
|
||||||
@@ -1,39 +0,0 @@
|
|||||||
"""Mesh: two devices baked with the same session profile discover each other.
|
|
||||||
|
|
||||||
The fundamental "does my mesh work" test. If both devices share a PSK, LoRa
|
|
||||||
region, modem preset, and channel slot, they should hear each other's
|
|
||||||
NodeInfo packets within ~60s of boot and appear in each other's `nodesByNum`
|
|
||||||
DB.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp.connection import connect
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(180)
|
|
||||||
def test_mesh_formation_within_60s(mesh_pair: dict[str, Any], wait_until) -> None:
|
|
||||||
"""Runs for every directed role pair - so we prove `A sees B in its node
|
|
||||||
DB` AND `B sees A in its node DB` independently. A one-sided pass can
|
|
||||||
mask a real problem (e.g. device A's RX works but its TX is dead).
|
|
||||||
"""
|
|
||||||
observer_port = mesh_pair["tx"]["port"]
|
|
||||||
target_node_num = mesh_pair["rx"]["my_node_num"]
|
|
||||||
assert (
|
|
||||||
target_node_num is not None
|
|
||||||
), f"{mesh_pair['rx']['role']} my_node_num not populated"
|
|
||||||
|
|
||||||
def target_visible_from_observer() -> bool:
|
|
||||||
with connect(port=observer_port) as iface:
|
|
||||||
nodes = iface.nodesByNum or {}
|
|
||||||
return target_node_num in nodes
|
|
||||||
|
|
||||||
wait_until(
|
|
||||||
target_visible_from_observer,
|
|
||||||
timeout=120,
|
|
||||||
backoff_start=2.0,
|
|
||||||
backoff_max=10.0,
|
|
||||||
)
|
|
||||||
@@ -1,347 +0,0 @@
|
|||||||
"""Multi-hop NextHop directed-message delivery + relay-recovery (bench test).
|
|
||||||
|
|
||||||
This is the hardware/tier-3 validator for the NextHop DM reliability work
|
|
||||||
(see `docs/nexthop-routing-reliability.md`). The unit suite
|
|
||||||
`test/test_nexthop_routing` covers the routing *logic* exhaustively; this test
|
|
||||||
covers the *end-to-end* multi-hop behavior that only a real (or RF-separated)
|
|
||||||
mesh exercises:
|
|
||||||
|
|
||||||
* a directed DM that must traverse a relay is delivered (next_hop routing +
|
|
||||||
the M1/M2 ambiguity gate + M3 route learning all engage), and
|
|
||||||
* when the established relay drops and returns, delivery recovers rather than
|
|
||||||
black-holing (the M3 stale-route decay / re-learn path).
|
|
||||||
|
|
||||||
TOPOLOGY REQUIREMENT - why this usually SKIPS:
|
|
||||||
A NextHop relay only happens when the two endpoints are NOT direct neighbors.
|
|
||||||
Three co-located radios all hear each other, so A→C is a single direct hop and
|
|
||||||
next_hop never engages. To run this test the bench must be a *line* - A - B - C
|
|
||||||
- with the endpoints out of each other's direct RF range (physical distance or
|
|
||||||
attenuators). The `multihop_topology` fixture detects this automatically: it
|
|
||||||
warms the mesh, looks for a pair that is ≥1 hop apart, confirms the relay via
|
|
||||||
traceroute, and `pytest.skip`s cleanly when the bench is all-direct. So this
|
|
||||||
file is safe to commit and run anywhere - it only *asserts* when the topology
|
|
||||||
genuinely requires a relay.
|
|
||||||
|
|
||||||
REQUIREMENTS:
|
|
||||||
* ≥3 baked devices. The default hub profile is 2 roles (nrf52, esp32s3); add a
|
|
||||||
third via `--hub-profile=path/to/hub.yaml` (see conftest `hub_profile`).
|
|
||||||
* The relay-recovery test additionally needs uhubctl + a power-controllable
|
|
||||||
relay port (same gate the other power tests use).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp.connection import connect
|
|
||||||
from tests import _power
|
|
||||||
from tests._port_discovery import resolve_port_by_role
|
|
||||||
|
|
||||||
from ._receive import ReceiveCollector, nudge_nodeinfo, nudge_nodeinfo_port
|
|
||||||
|
|
||||||
|
|
||||||
def _hops_away(rec: dict[str, Any]) -> int | None:
|
|
||||||
"""Read a node's hop distance from a `nodesByNum` entry, tolerating either
|
|
||||||
the camelCase (`hopsAway`) or snake_case (`hops_away`) spelling depending on
|
|
||||||
the meshtastic-python version."""
|
|
||||||
for key in ("hopsAway", "hops_away"):
|
|
||||||
val = rec.get(key)
|
|
||||||
if isinstance(val, int):
|
|
||||||
return val
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _warm_mesh(ports: list[str], rounds: int = 2, settle: float = 6.0) -> None:
|
|
||||||
"""Flood a fresh NodeInfo from every node so the whole mesh (including
|
|
||||||
multi-hop pairs, reached via relayed broadcasts) populates pubkeys and hop
|
|
||||||
distances. Best-effort - a single node failing to nudge shouldn't abort."""
|
|
||||||
for _ in range(rounds):
|
|
||||||
for port in ports:
|
|
||||||
try:
|
|
||||||
nudge_nodeinfo_port(port)
|
|
||||||
except Exception: # noqa: BLE001 - warmup is best-effort
|
|
||||||
pass
|
|
||||||
time.sleep(0.5)
|
|
||||||
time.sleep(settle)
|
|
||||||
|
|
||||||
|
|
||||||
def _wait_for_pubkey(
|
|
||||||
tx_iface: Any, rx_num: int, rx_port: str, deadline_s: float = 90.0
|
|
||||||
) -> bool:
|
|
||||||
"""Block until `tx_iface` holds `rx_num`'s public key (directed PKI sends
|
|
||||||
NAK without it). Re-nudges both sides periodically; multi-hop warmup is
|
|
||||||
slower than the 2-device case because NodeInfo must be relayed, hence the
|
|
||||||
longer default deadline."""
|
|
||||||
deadline = time.monotonic() + deadline_s
|
|
||||||
last_nudge = time.monotonic()
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
rec = (tx_iface.nodesByNum or {}).get(rx_num, {})
|
|
||||||
if rec.get("user", {}).get("publicKey"):
|
|
||||||
return True
|
|
||||||
if time.monotonic() - last_nudge > 20.0:
|
|
||||||
nudge_nodeinfo_port(rx_port)
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
last_nudge = time.monotonic()
|
|
||||||
time.sleep(1.0)
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def _traceroute_route(tx_port: str, rx_num: int, rx_port: str) -> list[int] | None:
|
|
||||||
"""Run a traceroute TX→RX and return the forward `route` (list of relay node
|
|
||||||
numbers), or None if it couldn't be obtained. Mirrors test_traceroute's
|
|
||||||
request/PKI/retry pattern."""
|
|
||||||
from meshtastic.mesh_interface import MeshInterface
|
|
||||||
|
|
||||||
with ReceiveCollector(tx_port, topic="meshtastic.receive.traceroute") as tx:
|
|
||||||
nudge_nodeinfo_port(rx_port)
|
|
||||||
tx.broadcast_nodeinfo_ping()
|
|
||||||
if not _wait_for_pubkey(tx._iface, rx_num, rx_port, 60.0):
|
|
||||||
return None
|
|
||||||
for _attempt in range(2):
|
|
||||||
try:
|
|
||||||
tx._iface.sendTraceRoute(dest=rx_num, hopLimit=5)
|
|
||||||
break
|
|
||||||
except MeshInterface.MeshInterfaceError:
|
|
||||||
time.sleep(5.0)
|
|
||||||
else:
|
|
||||||
return None
|
|
||||||
pkt = tx.wait_for(lambda p: p.get("from") == rx_num, timeout=8.0)
|
|
||||||
if pkt is None:
|
|
||||||
return None
|
|
||||||
tr = (pkt.get("decoded", {}) or {}).get("traceroute") or {}
|
|
||||||
return [int(n) for n in (tr.get("route") or [])]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="session")
|
|
||||||
def multihop_topology(baked_mesh: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
"""Discover a real multi-hop pier (tx → relay → rx) on the bench, or skip.
|
|
||||||
|
|
||||||
Returns {tx_role, tx_port, rx_role, rx_port, rx_num, relay_role, relay_num}.
|
|
||||||
"""
|
|
||||||
roles = sorted(baked_mesh)
|
|
||||||
if len(roles) < 3:
|
|
||||||
pytest.skip(
|
|
||||||
"multi-hop NextHop test needs ≥3 baked devices arranged as a line "
|
|
||||||
"(endpoints out of direct RF range). Add a third role via "
|
|
||||||
f"--hub-profile. Detected roles: {roles}"
|
|
||||||
)
|
|
||||||
|
|
||||||
by_role = {r: (baked_mesh[r]["port"], baked_mesh[r]["my_node_num"]) for r in roles}
|
|
||||||
if any(num is None for _, num in by_role.values()):
|
|
||||||
pytest.skip("a baked device is missing my_node_num; can't map the topology")
|
|
||||||
|
|
||||||
_warm_mesh([port for port, _ in by_role.values()])
|
|
||||||
|
|
||||||
# Find an ordered pair that is ≥1 hop apart, using each node's own nodeDB
|
|
||||||
# (cheap - no traceroute yet). On an all-direct bench nothing qualifies.
|
|
||||||
multihop_pair: tuple[str, str] | None = None
|
|
||||||
for a_role in roles:
|
|
||||||
a_port, _ = by_role[a_role]
|
|
||||||
try:
|
|
||||||
with connect(port=a_port) as a_iface:
|
|
||||||
nodes = a_iface.nodesByNum or {}
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
continue
|
|
||||||
for c_role in roles:
|
|
||||||
if c_role == a_role:
|
|
||||||
continue
|
|
||||||
_, c_num = by_role[c_role]
|
|
||||||
hops = _hops_away(nodes.get(c_num, {}))
|
|
||||||
if hops is not None and hops >= 1:
|
|
||||||
multihop_pair = (a_role, c_role)
|
|
||||||
break
|
|
||||||
if multihop_pair:
|
|
||||||
break
|
|
||||||
|
|
||||||
if not multihop_pair:
|
|
||||||
pytest.skip(
|
|
||||||
"no multi-hop pair found - every device appears to be a direct "
|
|
||||||
"neighbor. Arrange the bench as a line (A - B - C) with the "
|
|
||||||
"endpoints out of direct RF range (distance or attenuators) so a "
|
|
||||||
"relay is actually required, then re-run."
|
|
||||||
)
|
|
||||||
|
|
||||||
a_role, c_role = multihop_pair
|
|
||||||
a_port, _ = by_role[a_role]
|
|
||||||
c_port, c_num = by_role[c_role]
|
|
||||||
|
|
||||||
route = _traceroute_route(a_port, c_num, c_port)
|
|
||||||
if not route:
|
|
||||||
pytest.skip(
|
|
||||||
f"{a_role}→{c_role} looked multi-hop but traceroute returned no "
|
|
||||||
"intermediate relay; can't identify the relay node to drive the "
|
|
||||||
"recovery test"
|
|
||||||
)
|
|
||||||
|
|
||||||
relay_num = route[0]
|
|
||||||
relay_role = next((r for r in roles if by_role[r][1] == relay_num), None)
|
|
||||||
return {
|
|
||||||
"tx_role": a_role,
|
|
||||||
"tx_port": a_port,
|
|
||||||
"rx_role": c_role,
|
|
||||||
"rx_port": c_port,
|
|
||||||
"rx_num": c_num,
|
|
||||||
"relay_role": relay_role,
|
|
||||||
"relay_num": relay_num,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(300)
|
|
||||||
def test_multihop_dm_delivers(multihop_topology: dict[str, Any]) -> None:
|
|
||||||
"""A directed wantAck DM that must traverse the relay is delivered.
|
|
||||||
|
|
||||||
Exercises the NextHop routing path end-to-end: TX picks a next hop toward
|
|
||||||
RX (M2 gate), the relay resolves the next_hop byte and forwards (M1), and
|
|
||||||
the route is learned from the returning ACK (M3). Retries absorb transient
|
|
||||||
LoRa loss; the assertion is on eventual delivery.
|
|
||||||
"""
|
|
||||||
tx_port = multihop_topology["tx_port"]
|
|
||||||
rx_port = multihop_topology["rx_port"]
|
|
||||||
rx_num = multihop_topology["rx_num"]
|
|
||||||
tx_role = multihop_topology["tx_role"]
|
|
||||||
rx_role = multihop_topology["rx_role"]
|
|
||||||
relay_role = multihop_topology["relay_role"]
|
|
||||||
|
|
||||||
unique = f"nexthop-mh-{tx_role}-to-{rx_role}-{int(time.time())}"
|
|
||||||
|
|
||||||
with ReceiveCollector(rx_port, topic="meshtastic.receive.text") as rx:
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
with connect(port=tx_port) as tx_iface:
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
if not _wait_for_pubkey(tx_iface, rx_num, rx_port, 90.0):
|
|
||||||
pytest.skip(
|
|
||||||
f"{tx_role} never learned {rx_role}'s pubkey over the relay; "
|
|
||||||
"multi-hop PKI warmup didn't complete"
|
|
||||||
)
|
|
||||||
got = None
|
|
||||||
for _attempt in range(3):
|
|
||||||
pkt = tx_iface.sendText(unique, destinationId=rx_num, wantAck=True)
|
|
||||||
assert pkt is not None
|
|
||||||
got = rx.wait_for(
|
|
||||||
lambda p: p.get("decoded", {}).get("text") == unique,
|
|
||||||
timeout=45,
|
|
||||||
)
|
|
||||||
if got is not None:
|
|
||||||
break
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
time.sleep(5.0)
|
|
||||||
|
|
||||||
assert got is not None, (
|
|
||||||
f"multi-hop directed DM {tx_role}→{rx_role} via relay "
|
|
||||||
f"{relay_role!r} never landed - NextHop multi-hop delivery is broken"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(600)
|
|
||||||
def test_multihop_relay_recovery(
|
|
||||||
multihop_topology: dict[str, Any],
|
|
||||||
power_cycle, # noqa: ARG001 - forces the uhubctl-availability skip
|
|
||||||
) -> None:
|
|
||||||
"""Delivery recovers after the established relay drops and returns.
|
|
||||||
|
|
||||||
Establishes a baseline DM (route via relay learned), powers the relay OFF
|
|
||||||
(confirming TX survives sending across a downed relay), then powers it back
|
|
||||||
ON and asserts directed delivery resumes - the M3 stale-route decay /
|
|
||||||
re-learn path. With a strict A - B - C line there is no path while B is down,
|
|
||||||
so we only assert TX doesn't crash during the outage; the delivery assertion
|
|
||||||
is after B returns.
|
|
||||||
"""
|
|
||||||
relay_role = multihop_topology["relay_role"]
|
|
||||||
if not relay_role:
|
|
||||||
pytest.skip(
|
|
||||||
"relay node isn't one of the baked hub roles, so it can't be "
|
|
||||||
"power-cycled; recovery test needs a controllable relay"
|
|
||||||
)
|
|
||||||
|
|
||||||
tx_port = multihop_topology["tx_port"]
|
|
||||||
rx_port = multihop_topology["rx_port"]
|
|
||||||
rx_num = multihop_topology["rx_num"]
|
|
||||||
tx_role = multihop_topology["tx_role"]
|
|
||||||
rx_role = multihop_topology["rx_role"]
|
|
||||||
|
|
||||||
base = f"mh-recover-base-{int(time.time())}"
|
|
||||||
post = f"mh-recover-post-{int(time.time())}"
|
|
||||||
|
|
||||||
# Baseline: confirm delivery works (so the route via the relay is learned)
|
|
||||||
# before we perturb anything - otherwise a later failure is ambiguous.
|
|
||||||
with ReceiveCollector(rx_port, topic="meshtastic.receive.text") as rx:
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
with connect(port=tx_port) as tx_iface:
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
if not _wait_for_pubkey(tx_iface, rx_num, rx_port, 90.0):
|
|
||||||
pytest.skip("multi-hop PKI warmup failed; can't run recovery test")
|
|
||||||
tx_iface.sendText(base, destinationId=rx_num, wantAck=True)
|
|
||||||
assert (
|
|
||||||
rx.wait_for(
|
|
||||||
lambda p: p.get("decoded", {}).get("text") == base, timeout=45
|
|
||||||
)
|
|
||||||
is not None
|
|
||||||
), "baseline multi-hop delivery failed - skipping recovery to avoid a false result"
|
|
||||||
|
|
||||||
# Power the relay OFF.
|
|
||||||
try:
|
|
||||||
_power.power_off(relay_role)
|
|
||||||
_power.wait_for_absence(relay_role, timeout_s=15.0)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
try:
|
|
||||||
_power.power_on(relay_role)
|
|
||||||
resolve_port_by_role(relay_role, timeout_s=30.0)
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
pass
|
|
||||||
pytest.skip(f"can't power-control relay {relay_role!r}: {exc}")
|
|
||||||
|
|
||||||
# With the only relay down there's no path; we just confirm TX accepts the
|
|
||||||
# send and survives its internal retries (it must not crash / wedge).
|
|
||||||
try:
|
|
||||||
with connect(port=tx_port) as tx_iface:
|
|
||||||
pkt = tx_iface.sendText(
|
|
||||||
f"mh-while-down-{int(time.time())}",
|
|
||||||
destinationId=rx_num,
|
|
||||||
wantAck=True,
|
|
||||||
)
|
|
||||||
assert pkt is not None
|
|
||||||
time.sleep(8.0) # let retransmissions + route decay run
|
|
||||||
except Exception as exc: # noqa: BLE001 - restore bench state before failing
|
|
||||||
_power.power_on(relay_role)
|
|
||||||
resolve_port_by_role(relay_role, timeout_s=30.0)
|
|
||||||
raise AssertionError(
|
|
||||||
f"TX crashed sending across a downed relay: {exc}"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
# Power the relay back ON and let it re-enumerate + boot.
|
|
||||||
_power.power_on(relay_role)
|
|
||||||
time.sleep(0.5)
|
|
||||||
try:
|
|
||||||
resolve_port_by_role(relay_role, timeout_s=30.0)
|
|
||||||
except Exception: # noqa: BLE001 - relay port isn't one we connect to directly
|
|
||||||
pass
|
|
||||||
time.sleep(8.0)
|
|
||||||
_warm_mesh([tx_port, rx_port], rounds=1) # re-flood so the relay re-learns
|
|
||||||
|
|
||||||
# Delivery should resume once the relay is back (M3 re-learn / decay path).
|
|
||||||
got = None
|
|
||||||
with ReceiveCollector(rx_port, topic="meshtastic.receive.text") as rx:
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
with connect(port=tx_port) as tx_iface:
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
_wait_for_pubkey(tx_iface, rx_num, rx_port, 90.0)
|
|
||||||
for _attempt in range(4):
|
|
||||||
pkt = tx_iface.sendText(post, destinationId=rx_num, wantAck=True)
|
|
||||||
assert pkt is not None
|
|
||||||
got = rx.wait_for(
|
|
||||||
lambda p: p.get("decoded", {}).get("text") == post,
|
|
||||||
timeout=45,
|
|
||||||
)
|
|
||||||
if got is not None:
|
|
||||||
break
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
time.sleep(6.0)
|
|
||||||
|
|
||||||
assert got is not None, (
|
|
||||||
f"after relay {relay_role!r} returned, multi-hop DM {tx_role}→{rx_role} "
|
|
||||||
"never resumed - stale-route recovery (M3) may be broken"
|
|
||||||
)
|
|
||||||
@@ -1,155 +0,0 @@
|
|||||||
"""Isolation test for peer-offline-then-back mid-conversation.
|
|
||||||
|
|
||||||
Verifies the mesh stack's behavior when a peer is physically powered
|
|
||||||
off mid-send via uhubctl, then powered back on.
|
|
||||||
|
|
||||||
Flow (parametrized over every directed mesh_pair):
|
|
||||||
1. Bilateral PKI warmup (same pattern as test_direct_with_ack).
|
|
||||||
2. TX sends a broadcast text "msg-1" - RX confirms receipt via pubsub.
|
|
||||||
3. Power OFF RX via uhubctl. The RX device disappears from the OS.
|
|
||||||
4. TX sends a directed text "msg-2" with wantAck=True. Firmware retries
|
|
||||||
internally for ~30s before giving up. Assertion: the packet object
|
|
||||||
was accepted by the TX stack (non-None) - we don't assert an ACK
|
|
||||||
since there's no peer to send one.
|
|
||||||
5. Power ON RX. Wait for re-enumeration + boot.
|
|
||||||
6. Bilateral PKI re-nudge - RX's in-RAM PKI cache was wiped on reboot,
|
|
||||||
so the first directed send may err=35 without a fresh NodeInfo ping.
|
|
||||||
7. TX sends a directed "msg-3" - RX receives it via pubsub, confirming
|
|
||||||
the mesh recovered.
|
|
||||||
|
|
||||||
Skips cleanly if uhubctl isn't installed (via the `power_cycle` fixture's
|
|
||||||
auto-skip). Skips for pair directions where RX isn't power-controllable
|
|
||||||
(e.g. a USB-IF hub that doesn't support PPPS for its port).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp.connection import connect
|
|
||||||
from tests import _power
|
|
||||||
from tests._port_discovery import resolve_port_by_role
|
|
||||||
|
|
||||||
from ._receive import ReceiveCollector, nudge_nodeinfo
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(360)
|
|
||||||
def test_peer_offline_then_recovers(
|
|
||||||
mesh_pair: dict[str, Any],
|
|
||||||
power_cycle, # noqa: ARG001 - forces uhubctl-availability skip
|
|
||||||
hub_devices: dict[str, str],
|
|
||||||
) -> None:
|
|
||||||
tx_port = mesh_pair["tx"]["port"]
|
|
||||||
rx_node_num = mesh_pair["rx"]["my_node_num"]
|
|
||||||
tx_role = mesh_pair["tx_role"]
|
|
||||||
rx_role = mesh_pair["rx_role"]
|
|
||||||
|
|
||||||
unique_pre = f"peer-offline-pre-{tx_role}-to-{rx_role}-{int(time.time())}"
|
|
||||||
unique_post = f"peer-offline-post-{tx_role}-to-{rx_role}-{int(time.time())}"
|
|
||||||
|
|
||||||
# Step 1 + 2: warm up + confirm baseline delivery works before the test.
|
|
||||||
with ReceiveCollector(
|
|
||||||
mesh_pair["rx"]["port"], topic="meshtastic.receive.text"
|
|
||||||
) as rx:
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
with connect(port=tx_port) as tx_iface:
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
# Wait for bilateral PKI (RX pubkey in TX's nodesByNum).
|
|
||||||
deadline = time.monotonic() + 45.0
|
|
||||||
last_nudge = time.monotonic()
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
rec = (tx_iface.nodesByNum or {}).get(rx_node_num, {})
|
|
||||||
if rec.get("user", {}).get("publicKey"):
|
|
||||||
break
|
|
||||||
if time.monotonic() - last_nudge > 15.0:
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
last_nudge = time.monotonic()
|
|
||||||
time.sleep(1.0)
|
|
||||||
else:
|
|
||||||
pytest.skip(
|
|
||||||
f"bilateral PKI never completed ({tx_role}→{rx_role}); "
|
|
||||||
"can't run the offline test without a warm baseline"
|
|
||||||
)
|
|
||||||
|
|
||||||
tx_iface.sendText(unique_pre, destinationId=rx_node_num, wantAck=True)
|
|
||||||
got = rx.wait_for(
|
|
||||||
lambda pkt: pkt.get("decoded", {}).get("text") == unique_pre,
|
|
||||||
timeout=30,
|
|
||||||
)
|
|
||||||
assert got is not None, (
|
|
||||||
f"baseline directed send ({tx_role}→{rx_role}) didn't land - "
|
|
||||||
"skipping offline test to avoid false positive"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Step 3: power off RX. uhubctl skips the test with a clear message if
|
|
||||||
# the RX role isn't on a controllable hub.
|
|
||||||
try:
|
|
||||||
_power.power_off(rx_role)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
pytest.skip(f"can't power-control {rx_role!r}: {exc}")
|
|
||||||
|
|
||||||
try:
|
|
||||||
_power.wait_for_absence(rx_role, timeout_s=10.0)
|
|
||||||
except TimeoutError:
|
|
||||||
_power.power_on(rx_role) # restore hub state before failing
|
|
||||||
resolve_port_by_role(rx_role, timeout_s=30.0)
|
|
||||||
pytest.fail(f"{rx_role!r} didn't disappear after power_off")
|
|
||||||
|
|
||||||
# Step 4: send to a peer that isn't there. Firmware will retry
|
|
||||||
# internally. We don't wait for an ACK (there won't be one); we just
|
|
||||||
# confirm TX's stack accepts the packet without crashing.
|
|
||||||
try:
|
|
||||||
with connect(port=tx_port) as tx_iface:
|
|
||||||
packet = tx_iface.sendText(
|
|
||||||
f"while-offline-{rx_role}",
|
|
||||||
destinationId=rx_node_num,
|
|
||||||
wantAck=True,
|
|
||||||
)
|
|
||||||
assert packet is not None
|
|
||||||
# Give firmware a moment to do a retry or two while RX is down.
|
|
||||||
time.sleep(5.0)
|
|
||||||
except Exception as exc: # noqa: BLE001 - TX should survive the peer being gone
|
|
||||||
# Restore RX before reraising so the bench state is sane.
|
|
||||||
_power.power_on(rx_role)
|
|
||||||
resolve_port_by_role(rx_role, timeout_s=30.0)
|
|
||||||
raise AssertionError(f"TX crashed when sending to offline peer: {exc}") from exc
|
|
||||||
|
|
||||||
# Step 5: power RX back on + rediscover.
|
|
||||||
_power.power_on(rx_role)
|
|
||||||
time.sleep(0.5)
|
|
||||||
new_rx_port = resolve_port_by_role(rx_role, timeout_s=30.0)
|
|
||||||
hub_devices[rx_role] = new_rx_port
|
|
||||||
|
|
||||||
# Step 6 + 7: bilateral re-warmup + directed send that should now work.
|
|
||||||
with ReceiveCollector(new_rx_port, topic="meshtastic.receive.text") as rx:
|
|
||||||
# RX rebooted → its PKI cache is gone. Re-warm.
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
with connect(port=tx_port) as tx_iface:
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
time.sleep(3.0)
|
|
||||||
|
|
||||||
got = None
|
|
||||||
for _attempt in range(3):
|
|
||||||
packet = tx_iface.sendText(
|
|
||||||
unique_post,
|
|
||||||
destinationId=rx_node_num,
|
|
||||||
wantAck=True,
|
|
||||||
)
|
|
||||||
assert packet is not None
|
|
||||||
got = rx.wait_for(
|
|
||||||
lambda pkt: pkt.get("decoded", {}).get("text") == unique_post,
|
|
||||||
timeout=30,
|
|
||||||
)
|
|
||||||
if got is not None:
|
|
||||||
break
|
|
||||||
rx.broadcast_nodeinfo_ping()
|
|
||||||
nudge_nodeinfo(tx_iface)
|
|
||||||
time.sleep(5.0)
|
|
||||||
|
|
||||||
assert got is not None, (
|
|
||||||
f"post-recovery directed send {unique_post!r} ({tx_role}→{rx_role}) "
|
|
||||||
"never landed - recovery path may be broken"
|
|
||||||
)
|
|
||||||
@@ -1,147 +0,0 @@
|
|||||||
"""Mesh: traceroute from TX to RX round-trips with no intermediate hops.
|
|
||||||
|
|
||||||
TX sends a `TRACEROUTE_APP` request (RouteDiscovery with `want_response=True`)
|
|
||||||
addressed to RX's node_num. RX's firmware (`modules/TraceRouteModule.cpp`)
|
|
||||||
replies with a RouteDiscovery payload whose `route` / `route_back` lists
|
|
||||||
contain any intermediate relays and `snr_towards` / `snr_back` carry per-hop
|
|
||||||
SNRs. In a 2-device direct mesh there are no relays between TX and RX, so
|
|
||||||
both route lists must be empty and each SNR list carries exactly one entry
|
|
||||||
for the direct TX↔RX link.
|
|
||||||
|
|
||||||
Validates the full TRACEROUTE_APP portnum round-trip: request encoding, RX
|
|
||||||
firmware dispatch, RouteDiscovery payload construction, wire response, and
|
|
||||||
client-side decode through `meshtastic.__init__.py::protocols[TRACEROUTE_APP]`
|
|
||||||
(which is what publishes the `meshtastic.receive.traceroute` pubsub topic).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic.mesh_interface import MeshInterface
|
|
||||||
|
|
||||||
from ._receive import ReceiveCollector, nudge_nodeinfo_port
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(240)
|
|
||||||
def test_traceroute_one_hop(mesh_pair: dict[str, Any]) -> None:
|
|
||||||
"""Runs for every directed pair. Asserts TX sends + RX responds, then
|
|
||||||
inspects the captured RouteDiscovery to confirm the path is direct.
|
|
||||||
|
|
||||||
Why the listener is on TX (not RX):
|
|
||||||
The traceroute RESPONSE is addressed to TX (the original requester).
|
|
||||||
The meshtastic Python client publishes `meshtastic.receive.traceroute`
|
|
||||||
on the interface that received that response - which is TX's iface.
|
|
||||||
A listener on RX would only see the inbound REQUEST, which lacks
|
|
||||||
the SNR-towards / SNR-back fields the firmware only fills on reply.
|
|
||||||
|
|
||||||
Why we ping RX's NodeInfo before sending:
|
|
||||||
Traceroute requests are directed sends (wantResponse=True, specific
|
|
||||||
destinationId) - subject to the same PKI_SEND_FAIL_PUBLIC_KEY trap
|
|
||||||
as `test_direct_with_ack`. We open RX briefly to trigger the
|
|
||||||
on-demand NodeInfo broadcast, then wait for TX's nodesByNum to
|
|
||||||
populate RX's publicKey before calling sendTraceRoute.
|
|
||||||
"""
|
|
||||||
tx_port = mesh_pair["tx"]["port"]
|
|
||||||
rx_port = mesh_pair["rx"]["port"]
|
|
||||||
rx_node_num = mesh_pair["rx"]["my_node_num"]
|
|
||||||
tx_role = mesh_pair["tx_role"]
|
|
||||||
rx_role = mesh_pair["rx_role"]
|
|
||||||
assert rx_node_num is not None, f"{rx_role} my_node_num missing"
|
|
||||||
|
|
||||||
with ReceiveCollector(
|
|
||||||
tx_port, topic="meshtastic.receive.traceroute"
|
|
||||||
) as tx_listener:
|
|
||||||
# Bilateral PKI warmup - traceroute requests are directed and
|
|
||||||
# PKI-encrypted, so both sides need current pubkeys. See
|
|
||||||
# `_receive.py::nudge_nodeinfo` and the test_direct_with_ack
|
|
||||||
# comment for the full rationale (one-sided nudge lets err=35
|
|
||||||
# PKI_UNKNOWN_PUBKEY slip through in whichever direction had
|
|
||||||
# stale RX-side cache).
|
|
||||||
nudge_nodeinfo_port(rx_port) # RX via brief side-connection
|
|
||||||
tx_listener.broadcast_nodeinfo_ping() # TX via already-open iface
|
|
||||||
|
|
||||||
# Poll TX's view of RX until the publicKey propagates. 45 s matches
|
|
||||||
# the cap used in `test_direct_with_ack`; the re-nudge at 15 s
|
|
||||||
# covers a LoRa collision on the first NodeInfo broadcast.
|
|
||||||
pk_deadline = time.monotonic() + 45.0
|
|
||||||
last_nudge = time.monotonic()
|
|
||||||
last_rec: dict[str, Any] = {}
|
|
||||||
while time.monotonic() < pk_deadline:
|
|
||||||
last_rec = (tx_listener._iface.nodesByNum or {}).get(rx_node_num, {})
|
|
||||||
if last_rec.get("user", {}).get("publicKey"):
|
|
||||||
break
|
|
||||||
if time.monotonic() - last_nudge > 15.0:
|
|
||||||
nudge_nodeinfo_port(rx_port)
|
|
||||||
tx_listener.broadcast_nodeinfo_ping()
|
|
||||||
last_nudge = time.monotonic()
|
|
||||||
time.sleep(1.0)
|
|
||||||
else:
|
|
||||||
pytest.fail(
|
|
||||||
f"TX ({tx_role}) never saw RX ({rx_role}) public key within "
|
|
||||||
f"45s; nodesByNum entry={last_rec!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# sendTraceRoute blocks internally on `waitForTraceRoute` and raises
|
|
||||||
# `MeshInterface.MeshInterfaceError` on timeout. One retry covers a
|
|
||||||
# transient LoRa collision on either the request or the reply.
|
|
||||||
ok = False
|
|
||||||
for _attempt in range(2):
|
|
||||||
try:
|
|
||||||
tx_listener._iface.sendTraceRoute(
|
|
||||||
dest=rx_node_num,
|
|
||||||
hopLimit=3,
|
|
||||||
)
|
|
||||||
ok = True
|
|
||||||
break
|
|
||||||
except MeshInterface.MeshInterfaceError:
|
|
||||||
time.sleep(5.0)
|
|
||||||
assert ok, (
|
|
||||||
f"sendTraceRoute {tx_role}→{rx_role} timed out twice; the mesh "
|
|
||||||
f"may be saturated or RX's TraceRouteModule is misrouting the "
|
|
||||||
f"reply"
|
|
||||||
)
|
|
||||||
|
|
||||||
# sendTraceRoute already waited for the response internally, but
|
|
||||||
# pubsub dispatch runs on the meshtastic-python reader thread -
|
|
||||||
# give it a short grace window to queue the packet.
|
|
||||||
packet = tx_listener.wait_for(
|
|
||||||
lambda p: p.get("from") == rx_node_num,
|
|
||||||
timeout=5.0,
|
|
||||||
)
|
|
||||||
assert packet is not None, (
|
|
||||||
f"sendTraceRoute returned OK but no `receive.traceroute` packet "
|
|
||||||
f"from RX (0x{rx_node_num:08x}) arrived via pubsub. Captured: "
|
|
||||||
f"{tx_listener.snapshot()!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Inspect the decoded RouteDiscovery. The meshtastic client stores
|
|
||||||
# the parsed protobuf (as a plain dict via MessageToDict) under
|
|
||||||
# `decoded.traceroute` for this portnum; keys are camelCase because
|
|
||||||
# protobuf JSON conversion uses `preserving_proto_field_name=False`
|
|
||||||
# by default.
|
|
||||||
decoded = packet.get("decoded", {})
|
|
||||||
route_info = decoded.get("traceroute") or {}
|
|
||||||
|
|
||||||
forward_hops = route_info.get("route") or []
|
|
||||||
back_hops = route_info.get("routeBack") or []
|
|
||||||
snr_towards = route_info.get("snrTowards") or []
|
|
||||||
|
|
||||||
assert forward_hops == [], (
|
|
||||||
f"traceroute forward `route` should be empty on a 2-device direct "
|
|
||||||
f"mesh (no intermediaries between {tx_role} and {rx_role}); got "
|
|
||||||
f"{forward_hops!r}"
|
|
||||||
)
|
|
||||||
assert back_hops == [], (
|
|
||||||
f"traceroute `routeBack` should be empty on a 2-device direct "
|
|
||||||
f"mesh; got {back_hops!r}"
|
|
||||||
)
|
|
||||||
# `snr_towards` has len(route) + 1 entries - one per hop plus a final
|
|
||||||
# entry for the destination's receive SNR. Direct mesh → len(route)
|
|
||||||
# is 0 → exactly 1 SNR entry.
|
|
||||||
assert len(snr_towards) == 1, (
|
|
||||||
f"traceroute `snrTowards` should carry exactly 1 entry (direct "
|
|
||||||
f"link SNR) on a 2-device mesh; got {snr_towards!r}"
|
|
||||||
)
|
|
||||||
@@ -1,63 +0,0 @@
|
|||||||
"""Monitor: boot log is clean - no panic markers in the first 60 seconds.
|
|
||||||
|
|
||||||
This is the single highest-signal test for catching firmware regressions.
|
|
||||||
If a commit broke something critical at boot (stack overflow, NULL deref, HAL
|
|
||||||
misconfig), this test fails within a minute of reboot.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin
|
|
||||||
|
|
||||||
# Substrings that indicate a panic/assert/crash. Case-insensitive.
|
|
||||||
_PANIC_MARKERS = [
|
|
||||||
"guru meditation",
|
|
||||||
"corrupt heap",
|
|
||||||
"abort()",
|
|
||||||
"assertion failed",
|
|
||||||
"***", # ESP-IDF "*** something" panic prefix
|
|
||||||
"panic",
|
|
||||||
"stack overflow",
|
|
||||||
"load prohibited",
|
|
||||||
"store prohibited",
|
|
||||||
"illegalinstr",
|
|
||||||
"watchdog got triggered",
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(180)
|
|
||||||
def test_boot_log_no_panic(
|
|
||||||
baked_single: dict[str, Any],
|
|
||||||
serial_capture,
|
|
||||||
role_env,
|
|
||||||
wait_until,
|
|
||||||
) -> None:
|
|
||||||
"""Runs once per connected role - each device must boot cleanly,
|
|
||||||
independently. A panic on one role shouldn't mask another."""
|
|
||||||
role = baked_single["role"]
|
|
||||||
port = baked_single["port"]
|
|
||||||
env = role_env(role)
|
|
||||||
|
|
||||||
# Start monitor BEFORE reboot so we catch the reset banner + early boot
|
|
||||||
cap = serial_capture(role, env=env)
|
|
||||||
time.sleep(1.0)
|
|
||||||
|
|
||||||
# Trigger reboot
|
|
||||||
admin.reboot(port=port, confirm=True, seconds=3)
|
|
||||||
# Wait through the reboot+boot window
|
|
||||||
time.sleep(60.0)
|
|
||||||
|
|
||||||
lines = cap.snapshot(max_lines=4000)
|
|
||||||
assert lines, "serial capture returned no log lines - monitor may have failed"
|
|
||||||
blob = "\n".join(lines).lower()
|
|
||||||
|
|
||||||
hits = [marker for marker in _PANIC_MARKERS if marker in blob]
|
|
||||||
assert (
|
|
||||||
not hits
|
|
||||||
), f"panic markers in boot log: {hits!r}\n\n" f"last 60 lines:\n" + "\n".join(
|
|
||||||
lines[-60:]
|
|
||||||
)
|
|
||||||
@@ -1,83 +0,0 @@
|
|||||||
"""Provisioning: baked admin keys end up in the device's security config.
|
|
||||||
|
|
||||||
Fleet operators pre-bake an `USERPREFS_USE_ADMIN_KEY_0` into firmware so that
|
|
||||||
remote-admin messages from a central controller are accepted. This test
|
|
||||||
verifies the key bytes make the round-trip: USERPREFS → build-time `-D` flag
|
|
||||||
→ firmware → `localConfig.security.admin_key`.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin, flash
|
|
||||||
|
|
||||||
# Deterministic 32-byte "admin key" - just the byte values 0..31 for easy
|
|
||||||
# recognition in the output, formatted as a C brace-init.
|
|
||||||
_ADMIN_KEY_BYTES = list(range(32))
|
|
||||||
_ADMIN_KEY_BRACE = "{ " + ", ".join(f"0x{b:02x}" for b in _ADMIN_KEY_BYTES) + " }"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.skip(
|
|
||||||
reason="test uses flash.erase_and_flash which shells to bin/device-install.sh "
|
|
||||||
"which needs mt-esp32s3-ota.bin (not in repo). TODO: switch to "
|
|
||||||
"esptool_erase_flash + flash.flash() like test_00_bake."
|
|
||||||
)
|
|
||||||
@pytest.mark.timeout(600)
|
|
||||||
def test_admin_key_baked(
|
|
||||||
hub_devices: dict[str, str],
|
|
||||||
test_profile: dict[str, Any],
|
|
||||||
) -> None:
|
|
||||||
"""Bake test_profile + admin key 0; verify `security.admin_key` contains
|
|
||||||
the baked bytes after boot. Re-bakes session profile (without admin key)
|
|
||||||
on teardown so downstream tests see baseline state.
|
|
||||||
"""
|
|
||||||
target = "esp32s3"
|
|
||||||
if target not in hub_devices:
|
|
||||||
pytest.skip(f"role {target!r} not on hub")
|
|
||||||
port = hub_devices[target]
|
|
||||||
env = os.environ.get("MESHTASTIC_MCP_ENV_ESP32S3", "t-beam-1w")
|
|
||||||
|
|
||||||
augmented = dict(test_profile)
|
|
||||||
augmented["USERPREFS_USE_ADMIN_KEY_0"] = _ADMIN_KEY_BRACE
|
|
||||||
|
|
||||||
try:
|
|
||||||
result = flash.erase_and_flash(
|
|
||||||
env=env,
|
|
||||||
port=port,
|
|
||||||
confirm=True,
|
|
||||||
userprefs_overrides=augmented,
|
|
||||||
)
|
|
||||||
assert result["exit_code"] == 0
|
|
||||||
|
|
||||||
security = admin.get_config(section="security", port=port)["config"]["security"]
|
|
||||||
# `admin_key` may be a list of byte-sequences under newer protobuf, or
|
|
||||||
# a single bytes field under older. We accept either as long as the
|
|
||||||
# baked bytes appear somewhere in the serialization.
|
|
||||||
key_field = security.get("admin_key")
|
|
||||||
import base64
|
|
||||||
import json
|
|
||||||
|
|
||||||
serialized = json.dumps(security)
|
|
||||||
|
|
||||||
# Protobuf→JSON typically base64-encodes bytes fields. Encode our
|
|
||||||
# expected bytes and look for them (or a substring) in the serialized
|
|
||||||
# security config.
|
|
||||||
b64 = base64.b64encode(bytes(_ADMIN_KEY_BYTES)).decode("ascii").rstrip("=")
|
|
||||||
assert (
|
|
||||||
b64[:40] in serialized or "admin_key" in serialized
|
|
||||||
), f"admin_key bytes not visible in security config: {security!r}"
|
|
||||||
assert (
|
|
||||||
key_field is not None
|
|
||||||
), "security.admin_key field absent - baking key 0 didn't stick"
|
|
||||||
finally:
|
|
||||||
# Restore session profile (no admin key)
|
|
||||||
restore = flash.erase_and_flash(
|
|
||||||
env=env,
|
|
||||||
port=port,
|
|
||||||
confirm=True,
|
|
||||||
userprefs_overrides=test_profile,
|
|
||||||
)
|
|
||||||
assert restore["exit_code"] == 0
|
|
||||||
@@ -1,60 +0,0 @@
|
|||||||
"""Provisioning: the pre-bake recipe (US/LONG_FAST/slot 88/private channel)
|
|
||||||
lands on the device exactly as specified.
|
|
||||||
|
|
||||||
This is THE test that proves the MCP's core value prop - flashing firmware
|
|
||||||
with a preset USERPREFS produces a device in the expected radio config without
|
|
||||||
any post-flash admin steps.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin, info
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(60)
|
|
||||||
def test_bake_sets_region_preset_and_slot(
|
|
||||||
baked_mesh: dict[str, Any],
|
|
||||||
test_profile: dict[str, Any],
|
|
||||||
) -> None:
|
|
||||||
"""After test_00_bake, both devices must report the exact region, modem
|
|
||||||
preset, slot, and channel name that the profile specified."""
|
|
||||||
for role, state in baked_mesh.items():
|
|
||||||
port = state["port"]
|
|
||||||
live = info.device_info(port=port, timeout_s=8.0)
|
|
||||||
lora = admin.get_config(section="lora", port=port)["config"]["lora"]
|
|
||||||
|
|
||||||
expected_region = test_profile["USERPREFS_CONFIG_LORA_REGION"].rsplit("_", 1)[
|
|
||||||
-1
|
|
||||||
]
|
|
||||||
expected_preset = test_profile["USERPREFS_LORACONFIG_MODEM_PRESET"].rsplit(
|
|
||||||
"_", 2
|
|
||||||
)[-2:]
|
|
||||||
expected_preset_str = "_".join(expected_preset)
|
|
||||||
|
|
||||||
assert (
|
|
||||||
live["region"] == expected_region
|
|
||||||
), f"{role}: region={live['region']!r}, expected {expected_region!r}"
|
|
||||||
|
|
||||||
# `modem_preset` is omitted from the protobuf→JSON dump when the
|
|
||||||
# device is using the default enum value (LONG_FAST). If the key is
|
|
||||||
# missing AND we expected LONG_FAST, that's a match. Otherwise compare.
|
|
||||||
live_preset = lora.get("modem_preset")
|
|
||||||
if live_preset is None:
|
|
||||||
assert expected_preset_str == "LONG_FAST", (
|
|
||||||
f"{role}: modem_preset omitted (means default LONG_FAST), "
|
|
||||||
f"but expected {expected_preset_str!r}"
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
assert live_preset in (
|
|
||||||
expected_preset_str,
|
|
||||||
expected_preset_str.upper(),
|
|
||||||
), f"{role}: modem_preset={live_preset!r}, expected {expected_preset_str!r}"
|
|
||||||
|
|
||||||
assert (
|
|
||||||
int(lora.get("channel_num", 0))
|
|
||||||
== test_profile["USERPREFS_LORACONFIG_CHANNEL_NUM"]
|
|
||||||
), f"{role}: channel_num={lora.get('channel_num')!r}"
|
|
||||||
assert live["primary_channel"] == test_profile["USERPREFS_CHANNEL_0_NAME"]
|
|
||||||
@@ -1,108 +0,0 @@
|
|||||||
"""Provisioning (negative): firmware baked WITHOUT
|
|
||||||
`USERPREFS_CONFIG_LORA_REGION` must refuse to transmit.
|
|
||||||
|
|
||||||
Real operator concern: FCC compliance. A device shipped without an explicit
|
|
||||||
region setting must not emit RF until the operator sets a region - this test
|
|
||||||
proves the firmware honors that invariant when the USERPREFS bake deliberately
|
|
||||||
omits the region key.
|
|
||||||
|
|
||||||
Teardown re-bakes the session `test_profile` so downstream shared-state
|
|
||||||
tiers (admin/mesh/telemetry) still see a correctly configured mesh.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin, flash, info
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.skip(
|
|
||||||
reason="test uses flash.erase_and_flash which shells to bin/device-install.sh "
|
|
||||||
"which needs mt-esp32s3-ota.bin (not in repo). TODO: switch to "
|
|
||||||
"esptool_erase_flash + flash.flash() like test_00_bake."
|
|
||||||
)
|
|
||||||
@pytest.mark.timeout(600)
|
|
||||||
def test_unset_region_blocks_tx(
|
|
||||||
hub_devices: dict[str, str],
|
|
||||||
no_region_profile: dict[str, Any],
|
|
||||||
test_profile: dict[str, Any],
|
|
||||||
serial_capture,
|
|
||||||
) -> None:
|
|
||||||
"""Bake a device with no LoRa region, then assert:
|
|
||||||
1. `config.lora.region` reads as "UNSET" (or 0).
|
|
||||||
2. An attempt to `send_text` surfaces a refusal - either the meshtastic
|
|
||||||
SDK raises, or the serial log contains a clear "region unset" marker.
|
|
||||||
|
|
||||||
Always re-bakes the session test_profile in the finalizer so downstream
|
|
||||||
categories are not left with a broken device.
|
|
||||||
"""
|
|
||||||
target = "esp32s3"
|
|
||||||
if target not in hub_devices:
|
|
||||||
pytest.skip(f"role {target!r} not on hub")
|
|
||||||
port = hub_devices[target]
|
|
||||||
|
|
||||||
# Pick the right env for this role - must match what test_00_bake used.
|
|
||||||
import os
|
|
||||||
|
|
||||||
env = os.environ.get("MESHTASTIC_MCP_ENV_ESP32S3", "t-beam-1w")
|
|
||||||
|
|
||||||
# Capture serial before the bake to see the "region unset" log line on boot
|
|
||||||
cap = serial_capture(target, env=env)
|
|
||||||
|
|
||||||
# Bake without region
|
|
||||||
result = flash.erase_and_flash(
|
|
||||||
env=env,
|
|
||||||
port=port,
|
|
||||||
confirm=True,
|
|
||||||
userprefs_overrides=no_region_profile,
|
|
||||||
)
|
|
||||||
assert (
|
|
||||||
result["exit_code"] == 0
|
|
||||||
), f"bake of no_region_profile failed:\n{result.get('stderr_tail', '')}"
|
|
||||||
|
|
||||||
try:
|
|
||||||
# After bake, device should boot with region=UNSET
|
|
||||||
live = info.device_info(port=port, timeout_s=12.0)
|
|
||||||
assert live.get("region") in (None, "UNSET", "UNSET_0", ""), (
|
|
||||||
f"expected region UNSET after baking without region pref; "
|
|
||||||
f"got {live.get('region')!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Attempting to send a message should either raise or be logged as
|
|
||||||
# refused. The meshtastic SDK's sendText may raise in this condition,
|
|
||||||
# or it may accept the call but the firmware rejects on air.
|
|
||||||
send_failed = False
|
|
||||||
try:
|
|
||||||
admin.send_text(text="should not transmit", port=port)
|
|
||||||
except Exception:
|
|
||||||
send_failed = True
|
|
||||||
|
|
||||||
# Give the firmware a moment to log anything
|
|
||||||
import time as _time
|
|
||||||
|
|
||||||
_time.sleep(3.0)
|
|
||||||
log = "\n".join(cap.snapshot(max_lines=2000))
|
|
||||||
# We expect EITHER the send raised at the Python layer, OR the serial
|
|
||||||
# log explicitly says region is unset.
|
|
||||||
log_says_unset = any(
|
|
||||||
marker in log.lower()
|
|
||||||
for marker in ("region unset", "region is unset", "no region set")
|
|
||||||
)
|
|
||||||
assert send_failed or log_says_unset, (
|
|
||||||
"expected send to fail or log 'region unset'; neither happened.\n"
|
|
||||||
f"log tail:\n{log[-2000:]}"
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
# Re-bake the session profile so downstream tests work.
|
|
||||||
restore = flash.erase_and_flash(
|
|
||||||
env=env,
|
|
||||||
port=port,
|
|
||||||
confirm=True,
|
|
||||||
userprefs_overrides=test_profile,
|
|
||||||
)
|
|
||||||
assert restore["exit_code"] == 0, (
|
|
||||||
"CRITICAL: failed to re-bake session profile after "
|
|
||||||
"no-region test; downstream tests will fail."
|
|
||||||
)
|
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
"""Provisioning: after a non-full factory_reset, USERPREFS defaults come back.
|
|
||||||
|
|
||||||
Real operator concern: "if someone resets my fleet device, will it come back
|
|
||||||
on my private mesh or on Meshtastic defaults?" A baked USERPREFS recipe
|
|
||||||
should be the factory floor for the device - reset goes back to THAT state,
|
|
||||||
not to stock Meshtastic.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin, info
|
|
||||||
|
|
||||||
from .._port_discovery import resolve_port_by_role
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(180)
|
|
||||||
def test_baked_prefs_survive_factory_reset(
|
|
||||||
baked_single: dict[str, Any],
|
|
||||||
test_profile: dict[str, Any],
|
|
||||||
wait_until,
|
|
||||||
) -> None:
|
|
||||||
"""Runs once per connected role. Flow:
|
|
||||||
1. Change owner name to a known-non-default value.
|
|
||||||
2. Trigger factory_reset(full=False).
|
|
||||||
3. Rediscover the port (macOS re-enumerates the CDC endpoint on nRF52
|
|
||||||
factory_reset; the path can change e.g. `/dev/cu.usbmodem101` →
|
|
||||||
`/dev/cu.usbmodem1101`).
|
|
||||||
4. Wait for device to come back.
|
|
||||||
5. Confirm owner is back to USERPREFS-baked default (or blank default if
|
|
||||||
not baked), and primary channel/region/slot are still the baked values.
|
|
||||||
"""
|
|
||||||
role = baked_single["role"]
|
|
||||||
port = baked_single["port"]
|
|
||||||
|
|
||||||
# Snapshot pre-reset config
|
|
||||||
pre_reset = info.device_info(port=port, timeout_s=8.0)
|
|
||||||
original_long_name = pre_reset.get("long_name")
|
|
||||||
|
|
||||||
# Poison the owner name with a non-default marker
|
|
||||||
admin.set_owner(long_name="PoisonMarker", short_name="POIZ", port=port)
|
|
||||||
time.sleep(2.0)
|
|
||||||
|
|
||||||
# Confirm poison stuck before reset
|
|
||||||
poisoned = info.device_info(port=port, timeout_s=8.0)
|
|
||||||
assert poisoned.get("long_name") == "PoisonMarker"
|
|
||||||
|
|
||||||
# Trigger non-full factory reset
|
|
||||||
admin.factory_reset(port=port, confirm=True, full=False)
|
|
||||||
|
|
||||||
# Device re-enumerates - rediscover its port before probing. nRF52's
|
|
||||||
# CDC endpoint drops and comes back with a new `/dev/cu.usbmodem*`
|
|
||||||
# path on macOS; ESP32-S3 usually keeps the same path but the helper
|
|
||||||
# works either way (it just returns the current path for this role).
|
|
||||||
# Early sleep lets the USB kernel driver settle before we start
|
|
||||||
# polling - list_devices during a transient re-enumeration can return
|
|
||||||
# an empty list and the helper's poll-with-backoff handles that too,
|
|
||||||
# so the sleep is optimization not correctness.
|
|
||||||
time.sleep(10.0)
|
|
||||||
port = resolve_port_by_role(role, timeout_s=60.0)
|
|
||||||
wait_until(
|
|
||||||
lambda: info.device_info(port=port, timeout_s=5.0).get("my_node_num")
|
|
||||||
is not None,
|
|
||||||
timeout=60,
|
|
||||||
backoff_start=1.0,
|
|
||||||
)
|
|
||||||
|
|
||||||
post = info.device_info(port=port, timeout_s=8.0)
|
|
||||||
# The key assertion: channel + region are STILL the USERPREFS-baked values,
|
|
||||||
# NOT Meshtastic stock defaults (which would be "LongFast" and the region
|
|
||||||
# the device shipped with).
|
|
||||||
assert post["primary_channel"] == test_profile["USERPREFS_CHANNEL_0_NAME"], (
|
|
||||||
f"after factory_reset, primary_channel reverted to "
|
|
||||||
f"{post['primary_channel']!r}, not baked {test_profile['USERPREFS_CHANNEL_0_NAME']!r}"
|
|
||||||
)
|
|
||||||
expected_region = test_profile["USERPREFS_CONFIG_LORA_REGION"].rsplit("_", 1)[-1]
|
|
||||||
assert post.get("region") == expected_region
|
|
||||||
|
|
||||||
# Owner name should NOT be "PoisonMarker" anymore
|
|
||||||
assert (
|
|
||||||
post.get("long_name") != "PoisonMarker"
|
|
||||||
), "factory_reset did not wipe the poisoned owner name"
|
|
||||||
|
|
||||||
# If we had an original_long_name, restore it so downstream tests see
|
|
||||||
# the same baseline.
|
|
||||||
if original_long_name and post.get("long_name") != original_long_name:
|
|
||||||
admin.set_owner(long_name=original_long_name, port=port)
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
"""Recovery tier - exercises `uhubctl` power control end-to-end.
|
|
||||||
|
|
||||||
Requires `uhubctl` installed AND at least one connected device on a
|
|
||||||
PPPS-capable hub. The whole tier skips cleanly via
|
|
||||||
`tests/recovery/conftest.py::_recovery_tier_guard` when either is missing.
|
|
||||||
"""
|
|
||||||
@@ -1,44 +0,0 @@
|
|||||||
"""Recovery-tier gating + shared helpers.
|
|
||||||
|
|
||||||
Session-scoped guard skips the whole tier when uhubctl isn't installed.
|
|
||||||
Tests under this directory assume uhubctl is callable AND that at least
|
|
||||||
one hub role is detected on a PPPS-capable port.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="session", autouse=True)
|
|
||||||
def _recovery_tier_guard() -> None:
|
|
||||||
"""Skip the tier when uhubctl is unavailable OR no device is on a
|
|
||||||
PPPS-capable hub. Prints the specific reason so operators know what
|
|
||||||
to fix."""
|
|
||||||
from tests import _power
|
|
||||||
|
|
||||||
if not _power.is_uhubctl_available():
|
|
||||||
pytest.skip(
|
|
||||||
"uhubctl not installed; recovery tier needs it. "
|
|
||||||
"Install via `brew install uhubctl` or `apt install uhubctl`.",
|
|
||||||
allow_module_level=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
# Probe: can we even list hubs? (A macOS user without sudo gets a
|
|
||||||
# permission error here - we'd rather find out once at tier-start than
|
|
||||||
# 6 tests later.)
|
|
||||||
from meshtastic_mcp import uhubctl
|
|
||||||
|
|
||||||
try:
|
|
||||||
hubs = uhubctl.list_hubs()
|
|
||||||
except uhubctl.UhubctlError as exc:
|
|
||||||
pytest.skip(
|
|
||||||
f"uhubctl list failed: {exc}. Try the udev rules or `sudo` as a fallback.",
|
|
||||||
allow_module_level=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
if not any(h["ppps"] for h in hubs):
|
|
||||||
pytest.skip(
|
|
||||||
"no PPPS-capable hubs detected - recovery tier has nothing to exercise.",
|
|
||||||
allow_module_level=True,
|
|
||||||
)
|
|
||||||
@@ -1,43 +0,0 @@
|
|||||||
"""Smoke test: `uhubctl_list` returns a well-formed structure.
|
|
||||||
|
|
||||||
No destructive action. Runs first in the tier as a sanity check that the
|
|
||||||
tier's dependencies (uhubctl binary + permissions) are actually satisfied.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import uhubctl
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(30)
|
|
||||||
def test_list_hubs_returns_at_least_one_ppps_hub() -> None:
|
|
||||||
hubs = uhubctl.list_hubs()
|
|
||||||
assert hubs, "uhubctl found no hubs at all - is a USB hub connected?"
|
|
||||||
assert any(h["ppps"] for h in hubs), (
|
|
||||||
"no PPPS-capable hubs detected; power control won't work. "
|
|
||||||
"Check that the hub supports Per-Port Power Switching."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(30)
|
|
||||||
def test_list_hubs_structure(hub_devices: dict[str, str]) -> None:
|
|
||||||
hubs = uhubctl.list_hubs()
|
|
||||||
for hub in hubs:
|
|
||||||
assert "location" in hub and hub["location"]
|
|
||||||
assert "ports" in hub and isinstance(hub["ports"], list)
|
|
||||||
for port in hub["ports"]:
|
|
||||||
assert "port" in port and isinstance(port["port"], int)
|
|
||||||
assert "status" in port
|
|
||||||
|
|
||||||
# At least one of the detected Meshtastic roles should show up in some
|
|
||||||
# port's device_vid - otherwise the recovery tier can't drive them.
|
|
||||||
seen_vids = {
|
|
||||||
p["device_vid"] for h in hubs for p in h["ports"] if p["device_vid"] is not None
|
|
||||||
}
|
|
||||||
expected_any = {0x239A, 0x303A, 0x10C4} & seen_vids
|
|
||||||
assert expected_any or not hub_devices, (
|
|
||||||
f"hub_devices detected roles {list(hub_devices)} but uhubctl sees "
|
|
||||||
f"VIDs {sorted(hex(v) for v in seen_vids)} - the devices may be on "
|
|
||||||
"a hub that uhubctl can't see (e.g. built-in laptop ports)."
|
|
||||||
)
|
|
||||||
@@ -1,60 +0,0 @@
|
|||||||
"""Hard reset via uhubctl must NOT wipe NVS. Verify the test profile's
|
|
||||||
region + channel survive a power-cycle.
|
|
||||||
|
|
||||||
Guards against a regression where a firmware change treats unexpected
|
|
||||||
power loss as a factory-reset trigger (e.g. bad EEPROM wear-leveling,
|
|
||||||
erase-on-boot-for-safety). Such a regression would be catastrophic for
|
|
||||||
field deployments.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin, info
|
|
||||||
from tests import _power
|
|
||||||
from tests._port_discovery import resolve_port_by_role
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(180)
|
|
||||||
def test_lora_config_survives_power_cycle(
|
|
||||||
baked_single: dict[str, object],
|
|
||||||
test_profile: dict[str, object],
|
|
||||||
) -> None:
|
|
||||||
role = baked_single["role"]
|
|
||||||
pre_port = baked_single["port"]
|
|
||||||
|
|
||||||
pre_config = admin.get_config(section="lora", port=pre_port)["config"]["lora"]
|
|
||||||
pre_region = pre_config.get("region")
|
|
||||||
pre_preset = pre_config.get("modem_preset")
|
|
||||||
assert pre_region, f"lora.region not set pre-cycle on {role}"
|
|
||||||
|
|
||||||
# Hard power-cycle.
|
|
||||||
_power.power_cycle(role, delay_s=2)
|
|
||||||
time.sleep(0.5)
|
|
||||||
new_port = resolve_port_by_role(role, timeout_s=30.0)
|
|
||||||
# Let the firmware complete boot before admin reads.
|
|
||||||
time.sleep(2.0)
|
|
||||||
# Quick readiness probe.
|
|
||||||
probe = info.device_info(port=new_port, timeout_s=10.0)
|
|
||||||
assert (
|
|
||||||
probe.get("my_node_num") is not None
|
|
||||||
), f"device {role!r} didn't respond after power-cycle"
|
|
||||||
|
|
||||||
post_config = admin.get_config(section="lora", port=new_port)["config"]["lora"]
|
|
||||||
post_region = post_config.get("region")
|
|
||||||
post_preset = post_config.get("modem_preset")
|
|
||||||
|
|
||||||
assert post_region == pre_region, (
|
|
||||||
f"lora.region wiped by power-cycle on {role}: "
|
|
||||||
f"pre={pre_region!r} post={post_region!r}"
|
|
||||||
)
|
|
||||||
assert post_preset == pre_preset, (
|
|
||||||
f"lora.modem_preset wiped by power-cycle on {role}: "
|
|
||||||
f"pre={pre_preset!r} post={post_preset!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Channel-0 name should also match the test profile.
|
|
||||||
pri_ch = admin.get_channel_url(port=new_port)
|
|
||||||
assert pri_ch.get("url"), f"channel URL empty after power-cycle on {role}"
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
"""Full power-cycle round-trip: off → verify gone → on → verify identity
|
|
||||||
preserved.
|
|
||||||
|
|
||||||
Parametrized over every connected role. Validates both the uhubctl
|
|
||||||
plumbing AND that the device survives a hard reset with the same
|
|
||||||
`my_node_num` (no firmware-level identity regeneration).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import info
|
|
||||||
from tests import _power
|
|
||||||
from tests._port_discovery import resolve_port_by_role
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(180)
|
|
||||||
def test_power_cycle_preserves_node_identity(
|
|
||||||
baked_single: dict[str, object],
|
|
||||||
) -> None:
|
|
||||||
role = baked_single["role"]
|
|
||||||
pre_port = baked_single["port"]
|
|
||||||
pre_node_num = baked_single["my_node_num"]
|
|
||||||
pre_fw = baked_single.get("firmware_version")
|
|
||||||
|
|
||||||
# Record pre-cycle state.
|
|
||||||
pre_info = info.device_info(port=pre_port, timeout_s=5.0)
|
|
||||||
assert pre_info.get("my_node_num") == pre_node_num
|
|
||||||
|
|
||||||
# Power off; confirm the device actually disappears from list_devices.
|
|
||||||
_power.power_off(role)
|
|
||||||
try:
|
|
||||||
_power.wait_for_absence(role, timeout_s=10.0)
|
|
||||||
except TimeoutError:
|
|
||||||
# If it didn't disappear, power it back on so we don't leave the
|
|
||||||
# hub in a weird state for the next test.
|
|
||||||
_power.power_on(role)
|
|
||||||
resolve_port_by_role(role, timeout_s=30.0)
|
|
||||||
pytest.fail(f"device {role!r} stayed visible after power_off")
|
|
||||||
|
|
||||||
# Power back on + re-discover port.
|
|
||||||
_power.power_on(role)
|
|
||||||
time.sleep(0.5) # head-start before polling
|
|
||||||
new_port = resolve_port_by_role(role, timeout_s=30.0)
|
|
||||||
|
|
||||||
# Give the firmware a moment to finish boot before we hit it with admin.
|
|
||||||
time.sleep(2.0)
|
|
||||||
|
|
||||||
post_info = info.device_info(port=new_port, timeout_s=10.0)
|
|
||||||
assert post_info.get("my_node_num") == pre_node_num, (
|
|
||||||
f"my_node_num changed across power-cycle: pre={pre_node_num:#x} "
|
|
||||||
f"post={post_info.get('my_node_num'):#x}"
|
|
||||||
)
|
|
||||||
# Firmware version must match (same bake, not a re-flash).
|
|
||||||
if pre_fw:
|
|
||||||
assert post_info.get("firmware_version") == pre_fw, (
|
|
||||||
f"firmware changed across cycle: pre={pre_fw} "
|
|
||||||
f"post={post_info.get('firmware_version')}"
|
|
||||||
)
|
|
||||||
@@ -1,77 +0,0 @@
|
|||||||
"""Telemetry: device-metrics packets arrive at the peer.
|
|
||||||
|
|
||||||
Two-path verification:
|
|
||||||
1. Listen on TX's pubsub for inbound telemetry packets originating from
|
|
||||||
RX's node_num - if one arrives within the window, telemetry works.
|
|
||||||
2. Fall back to checking TX's node DB for a populated `deviceMetrics`
|
|
||||||
block on the RX record (which the firmware writes on receipt).
|
|
||||||
|
|
||||||
Both paths prove the same invariant; path 1 gives faster failure signal,
|
|
||||||
path 2 handles the case where the packet arrived before we subscribed.
|
|
||||||
|
|
||||||
Warmup note: when this test runs after `test_baked_prefs_survive_factory_reset`,
|
|
||||||
both devices have empty node-DBs. We kick a broadcast text from RX through
|
|
||||||
its own ReceiveCollector so TX learns RX exists and starts accepting its
|
|
||||||
telemetry; without it, a fresh-boot pair can take 10+ min to swap NODEINFO
|
|
||||||
before the first telemetry arrives.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp.connection import connect
|
|
||||||
|
|
||||||
from ..mesh._receive import ReceiveCollector
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(600)
|
|
||||||
def test_device_telemetry_broadcast(mesh_pair: dict[str, Any]) -> None:
|
|
||||||
"""Runs for every directed pair. Waits up to ~8 minutes for TX to see
|
|
||||||
RX's device telemetry - either as a live inbound pubsub packet or as
|
|
||||||
a populated deviceMetrics on RX's node-DB record.
|
|
||||||
|
|
||||||
Firmware default telemetry interval is 900s; after a fresh boot the
|
|
||||||
first device-metrics broadcast happens within ~30-120s. We warm up
|
|
||||||
the mesh first with a cross-broadcast so NODEINFO is exchanged, then
|
|
||||||
wait up to 7 min for a telemetry packet.
|
|
||||||
"""
|
|
||||||
tx_port = mesh_pair["tx"]["port"]
|
|
||||||
rx_port = mesh_pair["rx"]["port"]
|
|
||||||
rx_node_num = mesh_pair["rx"]["my_node_num"]
|
|
||||||
|
|
||||||
# Open both sides' pubsub listeners up front so we capture anything that
|
|
||||||
# arrives during the warmup exchange.
|
|
||||||
with ReceiveCollector(tx_port, topic="meshtastic.receive.telemetry") as tx_rx:
|
|
||||||
with ReceiveCollector(rx_port, topic="meshtastic.receive.text") as rx_tx:
|
|
||||||
# Warmup: send a broadcast from RX through its own collector so
|
|
||||||
# TX learns about RX (NODEINFO rides along with TEXT_MESSAGE_APP).
|
|
||||||
# Skipping this turns a 5-min wait into a 15-min wait on a fresh
|
|
||||||
# factory-reset pair.
|
|
||||||
rx_tx.send_text(f"warmup-{int(time.time())}")
|
|
||||||
time.sleep(5.0)
|
|
||||||
|
|
||||||
# Path 1: wait for a telemetry packet from RX on TX's pubsub.
|
|
||||||
got = tx_rx.wait_for(
|
|
||||||
lambda pkt: pkt.get("from") == rx_node_num,
|
|
||||||
timeout=420, # 7 min - well above the 30-120s typical first broadcast
|
|
||||||
)
|
|
||||||
if got is not None:
|
|
||||||
return # Path 1 confirmed delivery.
|
|
||||||
|
|
||||||
# Path 2: re-query TX's node DB for a populated deviceMetrics on RX.
|
|
||||||
# Device may have reported telemetry before we subscribed, or the
|
|
||||||
# pubsub delivery might race with our window - re-check nodesByNum.
|
|
||||||
with connect(port=tx_port) as iface:
|
|
||||||
rec = (iface.nodesByNum or {}).get(rx_node_num, {})
|
|
||||||
metrics = rec.get("deviceMetrics") or {}
|
|
||||||
has_any = any(
|
|
||||||
metrics.get(k) is not None
|
|
||||||
for k in ("batteryLevel", "voltage", "channelUtilization", "airUtilTx")
|
|
||||||
)
|
|
||||||
assert has_any, (
|
|
||||||
f"no telemetry from node 0x{rx_node_num:08x} within 7 min; "
|
|
||||||
f"deviceMetrics={metrics!r}"
|
|
||||||
)
|
|
||||||
@@ -1,187 +0,0 @@
|
|||||||
"""Telemetry: on-demand device-metrics request gets a prompt reply.
|
|
||||||
|
|
||||||
Complementary to ``test_device_telemetry_broadcast`` - that one witnesses the
|
|
||||||
firmware's *periodic* broadcast (900 s default interval, up to ~7 min worst
|
|
||||||
case). This one exercises the *request/reply* path: TX sends a
|
|
||||||
``meshtastic_Telemetry`` with the ``device_metrics`` variant-tag set and
|
|
||||||
``want_response=True`` on ``TELEMETRY_APP`` to RX, and RX's
|
|
||||||
``modules/Telemetry/DeviceTelemetry.cpp::allocReply`` fires immediately with
|
|
||||||
populated ``DeviceMetrics``. On a direct 2-device mesh the whole round-trip
|
|
||||||
finishes in under a minute even from a cold boot.
|
|
||||||
|
|
||||||
Validates:
|
|
||||||
* ``sendData(portNum=TELEMETRY_APP, want_response=True)`` encodes + routes
|
|
||||||
to RX (directed, PKI-encrypted to RX's pubkey)
|
|
||||||
* RX's ``DeviceTelemetryModule::handleReceivedProtobuf`` dispatches to
|
|
||||||
``allocReply`` - which is only invoked by the framework when
|
|
||||||
``want_response`` is set on the incoming packet
|
|
||||||
* The reply carries a ``DeviceMetrics`` sub-message with at least one
|
|
||||||
non-zero field (uptime_seconds is guaranteed non-zero a few seconds
|
|
||||||
after boot, so it reliably survives protobuf's default-value
|
|
||||||
serialization stripping)
|
|
||||||
* The reply routes back to TX and gets matched against the original
|
|
||||||
request via ``request_id`` - using the library's ``onResponse``
|
|
||||||
callback mechanism, which stores the handler at
|
|
||||||
``responseHandlers[sent_packet.id]`` and dispatches when a packet
|
|
||||||
arrives with ``decoded.request_id == sent_packet.id``. This is more
|
|
||||||
precise than a pubsub ``from==rx_node_num`` filter, which can
|
|
||||||
accidentally match RX's periodic broadcast or a stale reply to a
|
|
||||||
different prior request.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import threading
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic.protobuf import ( # type: ignore[import-untyped]
|
|
||||||
portnums_pb2,
|
|
||||||
telemetry_pb2,
|
|
||||||
)
|
|
||||||
|
|
||||||
from ..mesh._receive import ReceiveCollector, nudge_nodeinfo_port
|
|
||||||
|
|
||||||
# Fields on the DeviceMetrics sub-message. The camelCase versions are what
|
|
||||||
# `google.protobuf.json_format.MessageToDict` emits (preserving_proto_field_name
|
|
||||||
# defaults to False); the snake_case names are the proto-source spellings.
|
|
||||||
_DEVICE_METRICS_FIELDS = (
|
|
||||||
"batteryLevel",
|
|
||||||
"voltage",
|
|
||||||
"channelUtilization",
|
|
||||||
"airUtilTx",
|
|
||||||
"uptimeSeconds",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(240)
|
|
||||||
def test_telemetry_request_reply(mesh_pair: dict[str, Any]) -> None:
|
|
||||||
"""Runs for every directed pair. TX requests RX's telemetry via
|
|
||||||
``want_response=True`` and asserts the reply arrives with populated
|
|
||||||
DeviceMetrics.
|
|
||||||
"""
|
|
||||||
tx_port = mesh_pair["tx"]["port"]
|
|
||||||
rx_port = mesh_pair["rx"]["port"]
|
|
||||||
rx_node_num = mesh_pair["rx"]["my_node_num"]
|
|
||||||
tx_role = mesh_pair["tx_role"]
|
|
||||||
rx_role = mesh_pair["rx_role"]
|
|
||||||
assert rx_node_num is not None, f"{rx_role} my_node_num missing"
|
|
||||||
|
|
||||||
# ReceiveCollector is still used to hold TX's SerialInterface open and
|
|
||||||
# give us `tx_listener._iface` for sendData / nodesByNum polling. The
|
|
||||||
# subscribed topic is irrelevant for this test (we match via
|
|
||||||
# onResponse, not pubsub), but keeping a concrete topic avoids the
|
|
||||||
# surprise of a pubsub wildcard receiving every packet type.
|
|
||||||
with ReceiveCollector(tx_port, topic="meshtastic.receive.telemetry") as tx_listener:
|
|
||||||
# Bilateral PKI warmup - nudge BOTH sides to rebroadcast their
|
|
||||||
# NodeInfo (with current pubkey) before the directed send.
|
|
||||||
# * Nudging only RX gets RX's key → TX, but leaves RX with a
|
|
||||||
# potentially stale TX pubkey → RX NAKs our request with
|
|
||||||
# err=35 (PKI_UNKNOWN_PUBKEY) and we see no reply.
|
|
||||||
# * Nudging only TX is the mirror failure.
|
|
||||||
# See `tests/mesh/_receive.py::nudge_nodeinfo` for firmware path.
|
|
||||||
nudge_nodeinfo_port(rx_port) # briefly opens RX to send heartbeat
|
|
||||||
tx_listener.broadcast_nodeinfo_ping() # TX via the already-open iface
|
|
||||||
|
|
||||||
pk_deadline = time.monotonic() + 45.0
|
|
||||||
last_nudge = time.monotonic()
|
|
||||||
last_rec: dict[str, Any] = {}
|
|
||||||
while time.monotonic() < pk_deadline:
|
|
||||||
last_rec = (tx_listener._iface.nodesByNum or {}).get(rx_node_num, {})
|
|
||||||
if last_rec.get("user", {}).get("publicKey"):
|
|
||||||
break
|
|
||||||
if time.monotonic() - last_nudge > 15.0:
|
|
||||||
# Re-nudge both sides - LoRa collisions can drop either
|
|
||||||
# direction's NodeInfo broadcast independently.
|
|
||||||
nudge_nodeinfo_port(rx_port)
|
|
||||||
tx_listener.broadcast_nodeinfo_ping()
|
|
||||||
last_nudge = time.monotonic()
|
|
||||||
time.sleep(1.0)
|
|
||||||
else:
|
|
||||||
pytest.fail(
|
|
||||||
f"TX ({tx_role}) never saw RX ({rx_role}) public key within "
|
|
||||||
f"45s; nodesByNum entry={last_rec!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Send the request. The Telemetry protobuf has a `which_variant`
|
|
||||||
# oneof tag that the firmware uses to decide which reply to build
|
|
||||||
# (see `src/modules/Telemetry/DeviceTelemetry.cpp::allocReply`):
|
|
||||||
# device_metrics_tag → getDeviceTelemetry()
|
|
||||||
# local_stats_tag → getLocalStatsTelemetry()
|
|
||||||
# anything else → return NULL (request silently dropped)
|
|
||||||
# An empty `Telemetry()` has `which_variant = UNSET (0)`, so we MUST
|
|
||||||
# explicitly set the variant. `CopyFrom(DeviceMetrics())` with a
|
|
||||||
# default-constructed sub-message is the canonical Python-protobuf
|
|
||||||
# idiom for "set the oneof tag without populating fields" - matching
|
|
||||||
# how `MeshInterface.sendTelemetry()` constructs requests for the
|
|
||||||
# other variants.
|
|
||||||
#
|
|
||||||
# Matching the reply: the meshtastic client's `onResponse` callback
|
|
||||||
# mechanism fires ONLY for packets whose `decoded.request_id` equals
|
|
||||||
# the original outgoing packet's `id`. That's exactly the semantic
|
|
||||||
# we want - rejects periodic broadcasts (no request_id), rejects
|
|
||||||
# stale replies to prior requests (different request_id), and
|
|
||||||
# tolerates the firmware's reply_id/request_id naming quirk
|
|
||||||
# (firmware's `setReplyTo` writes the original packet's id into
|
|
||||||
# `decoded.request_id`, not `decoded.reply_id`).
|
|
||||||
#
|
|
||||||
# One retry covers transient LoRa collisions on request or reply.
|
|
||||||
reply_holder: list[dict[str, Any]] = []
|
|
||||||
got_reply = threading.Event()
|
|
||||||
|
|
||||||
def _on_reply(packet: dict[str, Any]) -> None:
|
|
||||||
reply_holder.append(packet)
|
|
||||||
got_reply.set()
|
|
||||||
|
|
||||||
got = None
|
|
||||||
for _attempt in range(2):
|
|
||||||
got_reply.clear()
|
|
||||||
del reply_holder[:]
|
|
||||||
req = telemetry_pb2.Telemetry()
|
|
||||||
req.device_metrics.CopyFrom(telemetry_pb2.DeviceMetrics())
|
|
||||||
tx_listener._iface.sendData(
|
|
||||||
req,
|
|
||||||
destinationId=rx_node_num,
|
|
||||||
portNum=portnums_pb2.PortNum.TELEMETRY_APP,
|
|
||||||
wantResponse=True,
|
|
||||||
onResponse=_on_reply,
|
|
||||||
hopLimit=3,
|
|
||||||
)
|
|
||||||
if got_reply.wait(timeout=45.0):
|
|
||||||
got = reply_holder[0]
|
|
||||||
break
|
|
||||||
time.sleep(5.0)
|
|
||||||
|
|
||||||
assert got is not None, (
|
|
||||||
f"no telemetry reply from {rx_role} (0x{rx_node_num:08x}) within "
|
|
||||||
f"90s of 2 requests; onResponse callback never fired. Captured "
|
|
||||||
f"{len(tx_listener.snapshot())} unrelated telemetry packet(s): "
|
|
||||||
f"{[hex(p.get('from') or 0) for p in tx_listener.snapshot()]!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Sanity: the reply's origin matches - a firmware bug that routed
|
|
||||||
# the response to the wrong sender would make onResponse fire on
|
|
||||||
# the wrong packet.
|
|
||||||
assert got.get("from") == rx_node_num, (
|
|
||||||
f"telemetry reply origin mismatch: from=0x{got.get('from'):08x}, "
|
|
||||||
f"expected 0x{rx_node_num:08x}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Inspect the decoded Telemetry payload. The meshtastic client stores
|
|
||||||
# it under `decoded.telemetry`; DeviceMetrics under `.deviceMetrics`.
|
|
||||||
decoded = got.get("decoded", {})
|
|
||||||
telem = decoded.get("telemetry") or {}
|
|
||||||
dm = telem.get("deviceMetrics") or {}
|
|
||||||
|
|
||||||
# A populated reply must contain at least one DeviceMetrics field.
|
|
||||||
# Protobuf's JSON serializer strips default-valued (zero) fields,
|
|
||||||
# so a bare `deviceMetrics: {}` would mean the firmware wrote the
|
|
||||||
# sub-message but every field was zero - plausible right at boot
|
|
||||||
# but not for a device that's been running long enough for a test
|
|
||||||
# session's warmup + NodeInfo exchange (~10-30 s uptime minimum).
|
|
||||||
populated = [k for k in _DEVICE_METRICS_FIELDS if k in dm]
|
|
||||||
assert populated, (
|
|
||||||
f"telemetry reply from {rx_role} carried no DeviceMetrics fields; "
|
|
||||||
f"decoded.telemetry={telem!r}"
|
|
||||||
)
|
|
||||||
@@ -1,291 +0,0 @@
|
|||||||
"""Session-bake module - runs first in the tier order to flash both hub roles
|
|
||||||
with the session `test_profile`.
|
|
||||||
|
|
||||||
Ordered first by `pytest_collection_modifyitems` in `conftest.py` (bucket
|
|
||||||
-1) because `baked_mesh` only *verifies* state - it does not reflash. Without
|
|
||||||
the explicit order pin, the top-level path `tests/test_00_bake.py` falls
|
|
||||||
into the fallback bucket and sorts AFTER every tier, silently turning
|
|
||||||
`--force-bake` into a no-op for the tier tests.
|
|
||||||
|
|
||||||
Skipped entirely when `--assume-baked` is passed. All downstream hardware
|
|
||||||
tests either depend on `baked_mesh` (which verifies state) or do their own
|
|
||||||
per-test bake (provisioning/fleet tiers), so failing here gives one clear
|
|
||||||
actionable failure instead of a cascade of mismatches.
|
|
||||||
|
|
||||||
Hardware-specific env names live in a small role→env map at the top of this
|
|
||||||
file; override by setting `MESHTASTIC_MCP_ENV_<ROLE>` env vars (e.g.
|
|
||||||
`MESHTASTIC_MCP_ENV_NRF52=heltec-mesh-node-t114`).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
import serial # type: ignore[import-untyped]
|
|
||||||
from meshtastic_mcp import admin, boards, flash, hw_tools, info
|
|
||||||
|
|
||||||
# Default envs for a common lab setup. Override per-role via env var.
|
|
||||||
_DEFAULT_ENVS = {
|
|
||||||
"nrf52": "rak4631",
|
|
||||||
"esp32s3": "heltec-v3",
|
|
||||||
}
|
|
||||||
|
|
||||||
_ESP32_ARCHES = {
|
|
||||||
"esp32",
|
|
||||||
"esp32-s2",
|
|
||||||
"esp32s2",
|
|
||||||
"esp32-s3",
|
|
||||||
"esp32s3",
|
|
||||||
"esp32-c3",
|
|
||||||
"esp32c3",
|
|
||||||
"esp32-c6",
|
|
||||||
"esp32c6",
|
|
||||||
}
|
|
||||||
_NRF52_ARCHES = {"nrf52", "nrf52840"}
|
|
||||||
|
|
||||||
|
|
||||||
def _wait_port_free(port: str, *, timeout_s: float = 15.0, role: str = "") -> None:
|
|
||||||
"""Block until `port` can be exclusively opened, or raise after `timeout_s`.
|
|
||||||
|
|
||||||
Root cause for the retry loop: esptool / nrfutil / pio all take an
|
|
||||||
*exclusive* serial port lock (fcntl LOCK_EX on macOS, EAGAIN otherwise).
|
|
||||||
Anything that held the port recently - the TUI's startup `DevicePollerWorker._poll_once()`,
|
|
||||||
a prior `device_info` call, a lingering `meshtastic-mcp` subprocess
|
|
||||||
spawned by the operator's MCP host, or a stale `pio device monitor` -
|
|
||||||
can still be holding it when `test_00_bake` reaches the flash step. The
|
|
||||||
result is esptool exiting 2 in ~0.1s with `[Errno 35] Resource
|
|
||||||
temporarily unavailable`.
|
|
||||||
|
|
||||||
`pyserial.Serial(exclusive=True)` probes the same lock esptool takes;
|
|
||||||
a brief open/close cycle is the cleanest way to verify the port is
|
|
||||||
genuinely free before handing it to a subprocess we can't easily
|
|
||||||
retry. 200 ms poll interval keeps the failure fast while giving the
|
|
||||||
kernel time to release a just-closed descriptor.
|
|
||||||
|
|
||||||
Raises AssertionError (rather than a generic TimeoutError) so the
|
|
||||||
pytest summary shows the role + port + a hint at `lsof`.
|
|
||||||
"""
|
|
||||||
role_prefix = f"{role}: " if role else ""
|
|
||||||
deadline = time.monotonic() + timeout_s
|
|
||||||
last_exc: BaseException | None = None
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
try:
|
|
||||||
s = serial.Serial(port=port, exclusive=True, timeout=0.5)
|
|
||||||
except Exception as exc:
|
|
||||||
last_exc = exc
|
|
||||||
time.sleep(0.2)
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
s.close()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
return
|
|
||||||
raise AssertionError(
|
|
||||||
f"{role_prefix}port {port} still busy after {timeout_s:.0f}s - "
|
|
||||||
f"something else holds an exclusive lock. Last error: {last_exc!r}. "
|
|
||||||
f"Identify the holder with `lsof {port}` and kill it; common "
|
|
||||||
f"culprits are a lingering `meshtastic-mcp` subprocess from the "
|
|
||||||
f"MCP host (.mcp.json) or a stale `pio device monitor`."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _prepare_nrf52_for_upload(port: str) -> str:
|
|
||||||
"""Kick the RAK4631 (or similar nRF52 USB-DFU board) into bootloader mode
|
|
||||||
via 1200bps touch, then return the port where pio should upload.
|
|
||||||
|
|
||||||
Adafruit bootloader on RAK4631 interprets 1200bps-open-close as 'enter
|
|
||||||
DFU'. The device re-enumerates with a distinct USB VID/PID
|
|
||||||
(0x239A/0x0029) at a different `/dev/cu.usbmodem*` path.
|
|
||||||
|
|
||||||
`touch_1200bps` does the heavy lifting: bounded open/close, polls for the
|
|
||||||
Adafruit-bootloader PID specifically, retries the touch up to twice.
|
|
||||||
Fails loudly if the device doesn't enter DFU - no point trying pio
|
|
||||||
upload against an app-mode device, it'll just hang.
|
|
||||||
"""
|
|
||||||
result = flash.touch_1200bps(port=port, settle_ms=500, retries=2)
|
|
||||||
if not result.get("ok"):
|
|
||||||
raise AssertionError(
|
|
||||||
f"nRF52 at {port} did not enter DFU bootloader after "
|
|
||||||
f"{result.get('attempts')} 1200bps touches. Manual recovery: "
|
|
||||||
f"double-tap the reset button on the board, then re-run. "
|
|
||||||
f"Detected port set before/after touch was unchanged."
|
|
||||||
)
|
|
||||||
new_port = result["new_port"]
|
|
||||||
# Small settle so pio/nrfutil sees a fully-ready CDC endpoint.
|
|
||||||
time.sleep(1.0)
|
|
||||||
return new_port
|
|
||||||
|
|
||||||
|
|
||||||
def _env_for(role: str) -> str:
|
|
||||||
override = os.environ.get(f"MESHTASTIC_MCP_ENV_{role.upper()}")
|
|
||||||
if override:
|
|
||||||
return override
|
|
||||||
if role not in _DEFAULT_ENVS:
|
|
||||||
pytest.fail(
|
|
||||||
f"no default PlatformIO env for role {role!r}. "
|
|
||||||
f"Set MESHTASTIC_MCP_ENV_{role.upper()} to the env name."
|
|
||||||
)
|
|
||||||
return _DEFAULT_ENVS[role]
|
|
||||||
|
|
||||||
|
|
||||||
def _bake_role(
|
|
||||||
role: str,
|
|
||||||
port: str,
|
|
||||||
test_profile: dict[str, Any],
|
|
||||||
force_bake: bool,
|
|
||||||
) -> None:
|
|
||||||
"""Bake + boot + verify for a single role. Skips if already baked unless
|
|
||||||
`--force-bake` was passed."""
|
|
||||||
env = _env_for(role)
|
|
||||||
|
|
||||||
# If not forcing, check if already baked with session profile.
|
|
||||||
if not force_bake:
|
|
||||||
try:
|
|
||||||
live = info.device_info(port=port, timeout_s=8.0)
|
|
||||||
# Quick heuristic: region matches and primary channel matches.
|
|
||||||
expected_region_short = test_profile["USERPREFS_CONFIG_LORA_REGION"].rsplit(
|
|
||||||
"_", 1
|
|
||||||
)[-1]
|
|
||||||
if (
|
|
||||||
live.get("region") == expected_region_short
|
|
||||||
and live.get("primary_channel")
|
|
||||||
== test_profile["USERPREFS_CHANNEL_0_NAME"]
|
|
||||||
):
|
|
||||||
pytest.skip(
|
|
||||||
f"{role} at {port} already baked with session profile "
|
|
||||||
f"(pass --force-bake to reflash)"
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
# If we can't query, fall through and bake anyway.
|
|
||||||
pass
|
|
||||||
|
|
||||||
# All architectures go through `pio run -t upload` - pio knows the right
|
|
||||||
# protocol per variant (esptool for ESP32, adafruit-nrfutil for nRF52,
|
|
||||||
# picotool for RP2040). We don't use `bin/device-install.sh` for ESP32
|
|
||||||
# because it requires the external `mt-esp32s3-ota.bin` helper that's
|
|
||||||
# downloaded from releases, not generated by the build.
|
|
||||||
#
|
|
||||||
# IMPORTANT: `pio run -t upload` on ESP32 only overwrites the APP
|
|
||||||
# partition - the LittleFS partition (config + NodeDB) survives. That
|
|
||||||
# means USERPREFS-baked defaults never take effect on a device that was
|
|
||||||
# already provisioned, because NodeDB init prefers the saved config. To
|
|
||||||
# force USERPREFS to apply cleanly, we erase the full chip first on
|
|
||||||
# ESP32 boards. nRF52 DFU naturally wipes the user partition, so no
|
|
||||||
# erase needed there.
|
|
||||||
rec = boards.get_board(env)
|
|
||||||
arch = rec.get("architecture") or ""
|
|
||||||
# Make sure nothing else (TUI startup poll, MCP-host zombie, pio monitor)
|
|
||||||
# is holding the port before we hand it to a subprocess. Self-heals the
|
|
||||||
# [Errno 35] port-busy flake that otherwise fails the bake in ~0.1s.
|
|
||||||
_wait_port_free(port, role=role)
|
|
||||||
if arch in _NRF52_ARCHES:
|
|
||||||
upload_port = _prepare_nrf52_for_upload(port)
|
|
||||||
elif arch in _ESP32_ARCHES:
|
|
||||||
# Full chip erase - wipes NVS + LittleFS so USERPREFS defaults apply.
|
|
||||||
erase_result = hw_tools.esptool_erase_flash(port=port, confirm=True)
|
|
||||||
assert erase_result["exit_code"] == 0, (
|
|
||||||
f"{role}: esptool erase_flash failed:\n"
|
|
||||||
f"{erase_result.get('stderr_tail', '')}"
|
|
||||||
)
|
|
||||||
upload_port = port
|
|
||||||
else:
|
|
||||||
upload_port = port
|
|
||||||
|
|
||||||
# Post-erase, pre-upload: full chip erase on ESP32 drops the CDC
|
|
||||||
# endpoint for a moment while the bootloader re-enters download mode.
|
|
||||||
# Wait for the port to settle before pio reopens it for upload -
|
|
||||||
# otherwise a fast machine can race and hit the same errno 35.
|
|
||||||
if arch in _ESP32_ARCHES:
|
|
||||||
_wait_port_free(upload_port, role=role, timeout_s=10.0)
|
|
||||||
|
|
||||||
# NOTE: no `userprefs_overrides=` here. The session-scoped
|
|
||||||
# `_session_userprefs` autouse fixture in conftest.py has already baked
|
|
||||||
# the test profile into userPrefs.jsonc for the duration of the session
|
|
||||||
# and will restore the original file at session end. A local
|
|
||||||
# `temporary_overrides` here would be a no-op (file is already baked)
|
|
||||||
# AND would cause the session fixture's teardown to see different
|
|
||||||
# stat / mtime than it snapshotted - keep the mutation in one place.
|
|
||||||
result = flash.flash(
|
|
||||||
env=env,
|
|
||||||
port=upload_port,
|
|
||||||
confirm=True,
|
|
||||||
)
|
|
||||||
assert result["exit_code"] == 0, (
|
|
||||||
f"{role} bake failed: exit={result['exit_code']}\n"
|
|
||||||
f"stdout tail:\n{result.get('stdout_tail', '')}\n"
|
|
||||||
f"stderr tail:\n{result.get('stderr_tail', '')}"
|
|
||||||
)
|
|
||||||
|
|
||||||
# Post-flash: for nRF52, the DFU process only overwrites the app
|
|
||||||
# partition - the NVS region holding the existing NodeDB/config is
|
|
||||||
# untouched, so the firmware will prefer the saved config over the
|
|
||||||
# baked USERPREFS defaults. Trigger a full factory reset to wipe NVS
|
|
||||||
# so USERPREFS takes effect on the next boot.
|
|
||||||
#
|
|
||||||
# ESP32 devices had their full flash erased BEFORE upload via
|
|
||||||
# esptool_erase_flash, so they don't need this post-flash reset.
|
|
||||||
if arch in _NRF52_ARCHES:
|
|
||||||
# Give the device time to come up from DFU.
|
|
||||||
time.sleep(8.0)
|
|
||||||
# Wait for meshtastic to be responsive; `device_info` may take a
|
|
||||||
# few seconds on the first post-flash boot.
|
|
||||||
for _ in range(20):
|
|
||||||
try:
|
|
||||||
info.device_info(port=port, timeout_s=6.0)
|
|
||||||
break
|
|
||||||
except Exception:
|
|
||||||
time.sleep(1.5)
|
|
||||||
else:
|
|
||||||
raise AssertionError(f"{role}: device didn't respond after DFU flash")
|
|
||||||
# Trigger full factory reset (wipes NVS + identity)
|
|
||||||
admin.factory_reset(port=port, confirm=True, full=True)
|
|
||||||
# Wait for the device to reboot and come back with fresh config
|
|
||||||
# populated from USERPREFS defaults.
|
|
||||||
time.sleep(10.0)
|
|
||||||
for _ in range(30):
|
|
||||||
try:
|
|
||||||
live = info.device_info(port=port, timeout_s=6.0)
|
|
||||||
if live.get("my_node_num"):
|
|
||||||
break
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
time.sleep(2.0)
|
|
||||||
else:
|
|
||||||
raise AssertionError(f"{role}: device didn't return after factory_reset")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(600)
|
|
||||||
def test_bake_nrf52(
|
|
||||||
hub_devices: dict[str, str],
|
|
||||||
test_profile: dict[str, Any],
|
|
||||||
request: pytest.FixtureRequest,
|
|
||||||
) -> None:
|
|
||||||
"""Flash the nRF52840 role with the session test profile."""
|
|
||||||
if "nrf52" not in hub_devices:
|
|
||||||
pytest.skip("nRF52 not detected on hub")
|
|
||||||
_bake_role(
|
|
||||||
role="nrf52",
|
|
||||||
port=hub_devices["nrf52"],
|
|
||||||
test_profile=test_profile,
|
|
||||||
force_bake=request.config.getoption("--force-bake"),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(600)
|
|
||||||
def test_bake_esp32s3(
|
|
||||||
hub_devices: dict[str, str],
|
|
||||||
test_profile: dict[str, Any],
|
|
||||||
request: pytest.FixtureRequest,
|
|
||||||
) -> None:
|
|
||||||
"""Flash the ESP32-S3 role with the session test profile."""
|
|
||||||
if "esp32s3" not in hub_devices:
|
|
||||||
pytest.skip("ESP32-S3 not detected on hub")
|
|
||||||
_bake_role(
|
|
||||||
role="esp32s3",
|
|
||||||
port=hub_devices["esp32s3"],
|
|
||||||
test_profile=test_profile,
|
|
||||||
force_bake=request.config.getoption("--force-bake"),
|
|
||||||
)
|
|
||||||
@@ -1,152 +0,0 @@
|
|||||||
"""Tool-surface coverage: track which MCP tools the test suite actually exercises.
|
|
||||||
|
|
||||||
This is NOT line coverage (that's `coverage.py`). This measures which of the
|
|
||||||
38 public MCP tools in `meshtastic_mcp.server` got invoked during a pytest
|
|
||||||
run - a quick signal for "where are the test-coverage gaps".
|
|
||||||
|
|
||||||
Approach: introspect `meshtastic_mcp.server.app` for registered tools, find
|
|
||||||
the underlying handler functions in their source modules, and wrap each with
|
|
||||||
a counting shim. At session end, emit `tool_coverage.json` mapping each tool
|
|
||||||
name to its call count. Tools never called show `count=0`.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import pathlib
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
_counts: dict[str, int] = {}
|
|
||||||
_installed = False
|
|
||||||
|
|
||||||
|
|
||||||
def _bump(name: str) -> None:
|
|
||||||
_counts[name] = _counts.get(name, 0) + 1
|
|
||||||
|
|
||||||
|
|
||||||
def _wrap(module: Any, attr: str, tool_name: str) -> None:
|
|
||||||
original = getattr(module, attr, None)
|
|
||||||
if original is None or not callable(original):
|
|
||||||
return
|
|
||||||
|
|
||||||
def wrapper(*args: Any, **kwargs: Any) -> Any:
|
|
||||||
_bump(tool_name)
|
|
||||||
return original(*args, **kwargs)
|
|
||||||
|
|
||||||
wrapper.__wrapped__ = original # type: ignore[attr-defined]
|
|
||||||
wrapper.__name__ = attr
|
|
||||||
wrapper.__doc__ = original.__doc__
|
|
||||||
setattr(module, attr, wrapper)
|
|
||||||
|
|
||||||
|
|
||||||
# Mapping: MCP tool name → (module, function name). Mirrors the wiring in
|
|
||||||
# `meshtastic_mcp.server`. Keep synchronized manually - adding a tool without
|
|
||||||
# updating this map means it shows as count=0 in reports even if exercised.
|
|
||||||
_TOOL_MAP: dict[str, tuple[str, str]] = {
|
|
||||||
# Discovery & metadata
|
|
||||||
"list_devices": ("meshtastic_mcp.devices", "list_devices"),
|
|
||||||
"list_boards": ("meshtastic_mcp.boards", "list_boards"),
|
|
||||||
"get_board": ("meshtastic_mcp.boards", "get_board"),
|
|
||||||
# Build & flash
|
|
||||||
"build": ("meshtastic_mcp.flash", "build"),
|
|
||||||
"clean": ("meshtastic_mcp.flash", "clean"),
|
|
||||||
"pio_flash": ("meshtastic_mcp.flash", "flash"),
|
|
||||||
"erase_and_flash": ("meshtastic_mcp.flash", "erase_and_flash"),
|
|
||||||
"update_flash": ("meshtastic_mcp.flash", "update_flash"),
|
|
||||||
"touch_1200bps": ("meshtastic_mcp.flash", "touch_1200bps"),
|
|
||||||
# Serial log sessions - module-level functions on serial_session
|
|
||||||
"serial_open": ("meshtastic_mcp.serial_session", "open_session"),
|
|
||||||
"serial_read": ("meshtastic_mcp.serial_session", "read_session"),
|
|
||||||
"serial_list": ("meshtastic_mcp.registry", "all_sessions"),
|
|
||||||
"serial_close": ("meshtastic_mcp.serial_session", "close_session"),
|
|
||||||
# Device reads
|
|
||||||
"device_info": ("meshtastic_mcp.info", "device_info"),
|
|
||||||
"list_nodes": ("meshtastic_mcp.info", "list_nodes"),
|
|
||||||
# Device writes
|
|
||||||
"set_owner": ("meshtastic_mcp.admin", "set_owner"),
|
|
||||||
"get_config": ("meshtastic_mcp.admin", "get_config"),
|
|
||||||
"set_config": ("meshtastic_mcp.admin", "set_config"),
|
|
||||||
"get_channel_url": ("meshtastic_mcp.admin", "get_channel_url"),
|
|
||||||
"set_channel_url": ("meshtastic_mcp.admin", "set_channel_url"),
|
|
||||||
"set_debug_log_api": ("meshtastic_mcp.admin", "set_debug_log_api"),
|
|
||||||
"send_text": ("meshtastic_mcp.admin", "send_text"),
|
|
||||||
"reboot": ("meshtastic_mcp.admin", "reboot"),
|
|
||||||
"shutdown": ("meshtastic_mcp.admin", "shutdown"),
|
|
||||||
"factory_reset": ("meshtastic_mcp.admin", "factory_reset"),
|
|
||||||
"send_input_event": ("meshtastic_mcp.admin", "send_input_event"),
|
|
||||||
# `capture_screen` in server.py calls camera.get_camera - instrument that.
|
|
||||||
"capture_screen": ("meshtastic_mcp.camera", "get_camera"),
|
|
||||||
# USB power control via uhubctl.
|
|
||||||
"uhubctl_list": ("meshtastic_mcp.uhubctl", "list_hubs"),
|
|
||||||
"uhubctl_power": ("meshtastic_mcp.uhubctl", "power_on"),
|
|
||||||
"uhubctl_cycle": ("meshtastic_mcp.uhubctl", "cycle"),
|
|
||||||
# USERPREFS
|
|
||||||
"userprefs_manifest": ("meshtastic_mcp.userprefs", "build_manifest"),
|
|
||||||
"userprefs_get": ("meshtastic_mcp.userprefs", "read_state"),
|
|
||||||
"userprefs_set": ("meshtastic_mcp.userprefs", "merge_active"),
|
|
||||||
"userprefs_reset": ("meshtastic_mcp.userprefs", "reset"),
|
|
||||||
"userprefs_testing_profile": ("meshtastic_mcp.userprefs", "build_testing_profile"),
|
|
||||||
# Vendor hardware tools
|
|
||||||
"esptool_chip_info": ("meshtastic_mcp.hw_tools", "esptool_chip_info"),
|
|
||||||
"esptool_erase_flash": ("meshtastic_mcp.hw_tools", "esptool_erase_flash"),
|
|
||||||
"esptool_raw": ("meshtastic_mcp.hw_tools", "esptool_raw"),
|
|
||||||
"nrfutil_dfu": ("meshtastic_mcp.hw_tools", "nrfutil_dfu"),
|
|
||||||
"nrfutil_raw": ("meshtastic_mcp.hw_tools", "nrfutil_raw"),
|
|
||||||
"picotool_info": ("meshtastic_mcp.hw_tools", "picotool_info"),
|
|
||||||
"picotool_load": ("meshtastic_mcp.hw_tools", "picotool_load"),
|
|
||||||
"picotool_raw": ("meshtastic_mcp.hw_tools", "picotool_raw"),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def install() -> None:
|
|
||||||
"""Wrap every mapped tool function with the counting shim. Idempotent."""
|
|
||||||
global _installed
|
|
||||||
if _installed:
|
|
||||||
return
|
|
||||||
import importlib
|
|
||||||
|
|
||||||
# Whitelist the exact module paths this function is ever allowed to
|
|
||||||
# import. `module_path` below is iterated from `_TOOL_MAP` - a file-
|
|
||||||
# local, hardcoded dict literal - but a static whitelist makes the
|
|
||||||
# "no untrusted input here" invariant legible to reviewers and to
|
|
||||||
# the Semgrep `non-literal-import` audit rule.
|
|
||||||
_allowed_modules = frozenset(path for path, _attr in _TOOL_MAP.values())
|
|
||||||
|
|
||||||
for tool_name, (module_path, attr) in _TOOL_MAP.items():
|
|
||||||
# Defense in depth: if someone mutates `_TOOL_MAP` at runtime
|
|
||||||
# (shouldn't happen; it's module-level) the whitelist catches it.
|
|
||||||
# `module_path` is a key from the hardcoded `_TOOL_MAP` dict and
|
|
||||||
# is gated above by membership in `_allowed_modules` (itself
|
|
||||||
# derived from the same literal values). There is no path for
|
|
||||||
# untrusted input to reach the `import_module` call below; the
|
|
||||||
# Semgrep suppression must sit on the line immediately preceding
|
|
||||||
# the call (multi-line comment blocks between comment and call
|
|
||||||
# break the rule's scope detection).
|
|
||||||
if module_path not in _allowed_modules:
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
# nosemgrep: python.lang.security.audit.non-literal-import.non-literal-import
|
|
||||||
mod = importlib.import_module(module_path)
|
|
||||||
except ImportError:
|
|
||||||
continue
|
|
||||||
_wrap(mod, attr, tool_name)
|
|
||||||
_counts.setdefault(tool_name, 0)
|
|
||||||
_installed = True
|
|
||||||
|
|
||||||
|
|
||||||
def write_report(path: pathlib.Path) -> None:
|
|
||||||
"""Emit `tool_coverage.json` with call counts for every mapped tool."""
|
|
||||||
exercised = sum(1 for c in _counts.values() if c > 0)
|
|
||||||
total = len(_counts)
|
|
||||||
payload = {
|
|
||||||
"total_tools": total,
|
|
||||||
"exercised": exercised,
|
|
||||||
"coverage_pct": round(100.0 * exercised / total, 1) if total else 0.0,
|
|
||||||
"counts": dict(sorted(_counts.items())),
|
|
||||||
"unexercised": sorted(k for k, v in _counts.items() if v == 0),
|
|
||||||
}
|
|
||||||
path.write_text(json.dumps(payload, indent=2), encoding="utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def snapshot() -> dict[str, int]:
|
|
||||||
return dict(_counts)
|
|
||||||
@@ -1,7 +0,0 @@
|
|||||||
"""UI tier - input-broker-driven screen navigation tests.
|
|
||||||
|
|
||||||
Only runs when a screen-bearing role (esp32s3/heltec-v3) is present on the
|
|
||||||
hub AND the firmware was baked with `enable_ui_log=True` (so the
|
|
||||||
`Screen: frame N/M name=... reason=...` log lines are emitted). The
|
|
||||||
`tests/ui/conftest.py` fixture forces that bake stamp.
|
|
||||||
"""
|
|
||||||
@@ -1,176 +0,0 @@
|
|||||||
"""Parse `Screen: frame N/M name=X reason=Y` log lines from `_debug_log_buffer`.
|
|
||||||
|
|
||||||
The firmware emits one line per frame transition when
|
|
||||||
`USERPREFS_UI_TEST_LOG` is defined (see src/graphics/Screen.cpp). Tests use
|
|
||||||
these helpers to assert which frame is shown / to wait for a transition to
|
|
||||||
settle before taking a camera capture.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
import time
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Iterable, Iterator
|
|
||||||
|
|
||||||
FRAME_RE = re.compile(
|
|
||||||
r"Screen: frame (?P<idx>\d+)/(?P<count>\d+) name=(?P<name>\S+) reason=(?P<reason>\S+)"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class FrameEvent:
|
|
||||||
idx: int
|
|
||||||
count: int
|
|
||||||
name: str
|
|
||||||
reason: str
|
|
||||||
raw: str
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def parse(cls, line: str) -> "FrameEvent | None":
|
|
||||||
m = FRAME_RE.search(line)
|
|
||||||
if not m:
|
|
||||||
return None
|
|
||||||
return cls(
|
|
||||||
idx=int(m["idx"]),
|
|
||||||
count=int(m["count"]),
|
|
||||||
name=m["name"],
|
|
||||||
reason=m["reason"],
|
|
||||||
raw=line,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def iter_frame_events(lines: Iterable[str]) -> Iterator[FrameEvent]:
|
|
||||||
for line in lines:
|
|
||||||
evt = FrameEvent.parse(line)
|
|
||||||
if evt is not None:
|
|
||||||
yield evt
|
|
||||||
|
|
||||||
|
|
||||||
def get_current_frame(lines: list[str]) -> FrameEvent | None:
|
|
||||||
"""Return the most recent FrameEvent in `lines`, or None if none found."""
|
|
||||||
for line in reversed(lines):
|
|
||||||
evt = FrameEvent.parse(line)
|
|
||||||
if evt is not None:
|
|
||||||
return evt
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def wait_for_frame(
|
|
||||||
lines: list[str],
|
|
||||||
expected_name: str,
|
|
||||||
*,
|
|
||||||
timeout_s: float = 5.0,
|
|
||||||
poll_interval_s: float = 0.1,
|
|
||||||
reason: str | None = None,
|
|
||||||
) -> FrameEvent:
|
|
||||||
"""Poll `lines` (the `_debug_log_buffer`) until a FrameEvent with
|
|
||||||
`name=expected_name` appears after the call started. Raises TimeoutError
|
|
||||||
with context if it doesn't arrive in `timeout_s`.
|
|
||||||
|
|
||||||
`reason` optionally filters to events matching a specific cause
|
|
||||||
(e.g. `"fn_f1"`, `"next"`, `"rebuild"`).
|
|
||||||
"""
|
|
||||||
start_idx = len(lines)
|
|
||||||
deadline = time.monotonic() + timeout_s
|
|
||||||
last: FrameEvent | None = None
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
# Scan only lines appended since we started waiting.
|
|
||||||
for line in lines[start_idx:]:
|
|
||||||
evt = FrameEvent.parse(line)
|
|
||||||
if evt is None:
|
|
||||||
continue
|
|
||||||
last = evt
|
|
||||||
if evt.name == expected_name and (reason is None or evt.reason == reason):
|
|
||||||
return evt
|
|
||||||
time.sleep(poll_interval_s)
|
|
||||||
|
|
||||||
seen = [e.name for e in iter_frame_events(lines[start_idx:])]
|
|
||||||
raise TimeoutError(
|
|
||||||
f"frame name={expected_name!r} reason={reason!r} not seen in {timeout_s}s; "
|
|
||||||
f"saw {len(seen)} transition(s): {seen!r}; last={last!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def wait_for_any_frame(
|
|
||||||
lines: list[str],
|
|
||||||
*,
|
|
||||||
timeout_s: float = 5.0,
|
|
||||||
poll_interval_s: float = 0.1,
|
|
||||||
) -> FrameEvent:
|
|
||||||
"""Wait for ANY frame transition to appear after call-start. Useful for
|
|
||||||
`no-op` tests that want to confirm a transition did NOT happen (via
|
|
||||||
TimeoutError) vs. one that did.
|
|
||||||
"""
|
|
||||||
start_idx = len(lines)
|
|
||||||
deadline = time.monotonic() + timeout_s
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
for line in lines[start_idx:]:
|
|
||||||
evt = FrameEvent.parse(line)
|
|
||||||
if evt is not None:
|
|
||||||
return evt
|
|
||||||
time.sleep(poll_interval_s)
|
|
||||||
raise TimeoutError(f"no frame transition in {timeout_s}s")
|
|
||||||
|
|
||||||
|
|
||||||
def wait_for_reason(
|
|
||||||
lines: list[str],
|
|
||||||
reason: str,
|
|
||||||
*,
|
|
||||||
timeout_s: float = 5.0,
|
|
||||||
poll_interval_s: float = 0.1,
|
|
||||||
) -> FrameEvent:
|
|
||||||
"""Wait for a frame event with `reason=<reason>` after call-start.
|
|
||||||
|
|
||||||
Matches only on `reason` - useful when the caller knows *why* a
|
|
||||||
transition should happen (e.g. `fn_f1`, `rebuild`) but not which named
|
|
||||||
frame the firmware will land on for this particular board.
|
|
||||||
"""
|
|
||||||
start_idx = len(lines)
|
|
||||||
deadline = time.monotonic() + timeout_s
|
|
||||||
last: FrameEvent | None = None
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
for line in lines[start_idx:]:
|
|
||||||
evt = FrameEvent.parse(line)
|
|
||||||
if evt is None:
|
|
||||||
continue
|
|
||||||
last = evt
|
|
||||||
if evt.reason == reason:
|
|
||||||
return evt
|
|
||||||
time.sleep(poll_interval_s)
|
|
||||||
raise TimeoutError(
|
|
||||||
f"no frame with reason={reason!r} in {timeout_s}s; last={last!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def assert_no_frame_change(
|
|
||||||
lines: list[str],
|
|
||||||
*,
|
|
||||||
wait_s: float = 2.0,
|
|
||||||
) -> None:
|
|
||||||
"""Assert that NO new FrameEvent lines arrive within `wait_s`.
|
|
||||||
|
|
||||||
Used by idempotency / no-op tests (e.g. BACK on home frame).
|
|
||||||
"""
|
|
||||||
start_idx = len(lines)
|
|
||||||
time.sleep(wait_s)
|
|
||||||
new = [
|
|
||||||
e for e in (FrameEvent.parse(ln) for ln in lines[start_idx:]) if e is not None
|
|
||||||
]
|
|
||||||
if new:
|
|
||||||
raise AssertionError(
|
|
||||||
f"expected no frame change in {wait_s}s, but saw {len(new)} event(s): "
|
|
||||||
f"{[(e.reason, e.name) for e in new]!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"FRAME_RE",
|
|
||||||
"FrameEvent",
|
|
||||||
"assert_no_frame_change",
|
|
||||||
"get_current_frame",
|
|
||||||
"iter_frame_events",
|
|
||||||
"wait_for_any_frame",
|
|
||||||
"wait_for_frame",
|
|
||||||
"wait_for_reason",
|
|
||||||
]
|
|
||||||
@@ -1,381 +0,0 @@
|
|||||||
"""UI-tier fixtures: camera lifecycle, OCR warmup, per-test frame capture,
|
|
||||||
and a `ui_home_state` autouse guard that resets to the home frame before
|
|
||||||
every test (prevents state bleed if a prior test exited inside a menu).
|
|
||||||
|
|
||||||
The camera + OCR modules live in `meshtastic_mcp/{camera,ocr}.py` (production
|
|
||||||
code, so the `capture_screen` MCP tool can share them). These fixtures wire
|
|
||||||
them into pytest + write per-test captures to `tests/ui_captures/…`.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
import shutil
|
|
||||||
import time
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any, Iterator
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp import admin as admin_mod
|
|
||||||
from meshtastic_mcp import camera as camera_mod
|
|
||||||
from meshtastic_mcp import ocr as ocr_mod
|
|
||||||
from meshtastic_mcp.input_events import InputEventCode
|
|
||||||
|
|
||||||
from ._screen_log import FrameEvent, get_current_frame, wait_for_frame
|
|
||||||
|
|
||||||
# Roles that carry a screen the UI tier can drive. Only esp32s3 (heltec-v3
|
|
||||||
# SSD1306) qualifies today - nrf52 (rak4631) has no display.
|
|
||||||
UI_CAPABLE_ROLES = ("esp32s3",)
|
|
||||||
|
|
||||||
# Where per-test captures land. One subdirectory per session seed, then per
|
|
||||||
# sanitized test nodeid - identical pattern to other pytest artifacts.
|
|
||||||
CAPTURES_ROOT = Path(__file__).resolve().parent.parent / "ui_captures"
|
|
||||||
|
|
||||||
|
|
||||||
def _sanitize_nodeid(nodeid: str) -> str:
|
|
||||||
return re.sub(r"[^a-zA-Z0-9_.-]+", "_", nodeid)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Role gating ----------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def ui_capable_role(request: pytest.FixtureRequest, hub_devices: dict[str, Any]) -> str:
|
|
||||||
"""Resolve the single role the UI tier drives.
|
|
||||||
|
|
||||||
Today that's `esp32s3`. Skips if the hub doesn't have one. A future
|
|
||||||
multi-screen hub could pick a role per parametrization.
|
|
||||||
"""
|
|
||||||
for role in UI_CAPABLE_ROLES:
|
|
||||||
if role in hub_devices:
|
|
||||||
return role
|
|
||||||
pytest.skip(
|
|
||||||
f"no UI-capable role on hub; need one of {UI_CAPABLE_ROLES} in {sorted(hub_devices)}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def ui_port(ui_capable_role: str, hub_devices: dict[str, Any]) -> str:
|
|
||||||
port = (
|
|
||||||
hub_devices[ui_capable_role].get("port")
|
|
||||||
if isinstance(hub_devices[ui_capable_role], dict)
|
|
||||||
else hub_devices[ui_capable_role]
|
|
||||||
)
|
|
||||||
if not port:
|
|
||||||
pytest.skip(f"{ui_capable_role!r} has no usable port")
|
|
||||||
return port
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Camera + OCR session fixtures ---------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="session")
|
|
||||||
def camera(ui_capable_role_session: str | None) -> Iterator[camera_mod.CameraBackend]:
|
|
||||||
"""Session-scoped camera backend. Closed at teardown.
|
|
||||||
|
|
||||||
Backend + device selected by env vars (see `meshtastic_mcp.camera`).
|
|
||||||
Falls through to `NullBackend` when no camera is configured, so the
|
|
||||||
tests run end-to-end on machines without hardware; they just won't
|
|
||||||
have useful images.
|
|
||||||
"""
|
|
||||||
role = ui_capable_role_session or "esp32s3"
|
|
||||||
cam = camera_mod.get_camera(role)
|
|
||||||
try:
|
|
||||||
yield cam
|
|
||||||
finally:
|
|
||||||
cam.close()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="session")
|
|
||||||
def ui_capable_role_session(hub_devices: dict[str, Any]) -> str | None:
|
|
||||||
"""Session-scoped lookup mirroring `ui_capable_role` but non-skipping.
|
|
||||||
|
|
||||||
Used by the `camera` session fixture so it doesn't depend on a
|
|
||||||
test-scoped skip.
|
|
||||||
"""
|
|
||||||
for role in UI_CAPABLE_ROLES:
|
|
||||||
if role in hub_devices:
|
|
||||||
return role
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="session", autouse=True)
|
|
||||||
def _ocr_warm() -> None:
|
|
||||||
"""Pay easyocr's ~100 MB / cold-start cost ONCE per session.
|
|
||||||
|
|
||||||
Subsequent `ocr_text()` calls hit the cached reader and return quickly.
|
|
||||||
Swallows errors - if OCR isn't installed, warm is a no-op.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
ocr_mod.warm()
|
|
||||||
except Exception: # noqa: BLE001 - belt: never block the suite on OCR init
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="session")
|
|
||||||
def _ui_screen_kept_on(
|
|
||||||
ui_capable_role_session: str | None, hub_devices: dict[str, Any]
|
|
||||||
) -> Iterator[None]:
|
|
||||||
"""Keep the OLED on throughout the UI tier so input events aren't dropped.
|
|
||||||
|
|
||||||
Why: `InputBroker::handleInputEvent` (src/input/InputBroker.cpp:118-122)
|
|
||||||
silently DROPS any event that arrives while the screen is off - it just
|
|
||||||
wakes the screen and returns. Every first event in each test would
|
|
||||||
disappear. We set `display.screen_on_secs = 86400` at session start
|
|
||||||
(effectively "always on" for the test window) and restore the prior
|
|
||||||
value at teardown.
|
|
||||||
"""
|
|
||||||
if ui_capable_role_session is None:
|
|
||||||
yield
|
|
||||||
return
|
|
||||||
|
|
||||||
hub_entry = hub_devices[ui_capable_role_session]
|
|
||||||
port = hub_entry.get("port") if isinstance(hub_entry, dict) else hub_entry
|
|
||||||
if not port:
|
|
||||||
yield
|
|
||||||
return
|
|
||||||
|
|
||||||
original: int | None = None
|
|
||||||
try:
|
|
||||||
current = admin_mod.get_config(section="display", port=port)
|
|
||||||
original = int(
|
|
||||||
current.get("config", {}).get("display", {}).get("screen_on_secs") or 0
|
|
||||||
)
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
pass
|
|
||||||
|
|
||||||
try:
|
|
||||||
admin_mod.set_config("display.screen_on_secs", 86400, port=port)
|
|
||||||
# Send one wake event so the screen is actually ON going into the
|
|
||||||
# first test. The event itself gets dropped (screenWasOff), but the
|
|
||||||
# wake side-effect sticks.
|
|
||||||
try:
|
|
||||||
admin_mod.send_input_event(event_code=int(InputEventCode.FN_F1), port=port)
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
pass
|
|
||||||
time.sleep(1.5) # Let the screen finish its wake transition.
|
|
||||||
except (
|
|
||||||
Exception
|
|
||||||
): # noqa: BLE001 - best-effort; ui_home_state surfaces the real error
|
|
||||||
pass
|
|
||||||
|
|
||||||
try:
|
|
||||||
yield
|
|
||||||
finally:
|
|
||||||
if original is not None:
|
|
||||||
try:
|
|
||||||
admin_mod.set_config("display.screen_on_secs", original, port=port)
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Per-test capture + transcript ----------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class FrameCapture:
|
|
||||||
"""Per-test capture recorder. Created once per test via the
|
|
||||||
`frame_capture` fixture; call with a label to snapshot the screen.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
|
||||||
self,
|
|
||||||
cam: camera_mod.CameraBackend,
|
|
||||||
dir_path: Path,
|
|
||||||
lines: list[str],
|
|
||||||
nodeid: str,
|
|
||||||
) -> None:
|
|
||||||
self._cam = cam
|
|
||||||
self._dir = dir_path
|
|
||||||
self._lines = lines
|
|
||||||
self._nodeid = nodeid
|
|
||||||
self._step = 0
|
|
||||||
self.captures: list[dict[str, Any]] = []
|
|
||||||
self._transcript_path = dir_path / "transcript.md"
|
|
||||||
self._dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
self._transcript_path.write_text(
|
|
||||||
f"# {nodeid} - {time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())}\n\n",
|
|
||||||
encoding="utf-8",
|
|
||||||
)
|
|
||||||
|
|
||||||
def __call__(self, label: str) -> dict[str, Any]:
|
|
||||||
self._step += 1
|
|
||||||
stem = f"{self._step:03d}-{re.sub(r'[^a-zA-Z0-9_-]+', '-', label)}"
|
|
||||||
png_path = self._dir / f"{stem}.png"
|
|
||||||
ocr_path = self._dir / f"{stem}.ocr.txt"
|
|
||||||
|
|
||||||
try:
|
|
||||||
png = self._cam.capture()
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
png = b""
|
|
||||||
ocr_str = f"[capture error: {exc}]"
|
|
||||||
else:
|
|
||||||
camera_mod.save_capture(png, png_path)
|
|
||||||
try:
|
|
||||||
ocr_str = ocr_mod.ocr_text(png)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
ocr_str = f"[ocr error: {exc}]"
|
|
||||||
ocr_path.write_text(ocr_str or "", encoding="utf-8")
|
|
||||||
|
|
||||||
frame = get_current_frame(self._lines)
|
|
||||||
entry: dict[str, Any] = {
|
|
||||||
"step": self._step,
|
|
||||||
"label": label,
|
|
||||||
"png_path": str(png_path) if png else None,
|
|
||||||
"ocr_text": ocr_str,
|
|
||||||
"frame": (
|
|
||||||
{
|
|
||||||
"idx": frame.idx,
|
|
||||||
"name": frame.name,
|
|
||||||
"reason": frame.reason,
|
|
||||||
}
|
|
||||||
if frame is not None
|
|
||||||
else None
|
|
||||||
),
|
|
||||||
}
|
|
||||||
self.captures.append(entry)
|
|
||||||
|
|
||||||
with self._transcript_path.open("a", encoding="utf-8") as fh:
|
|
||||||
frame_str = (
|
|
||||||
f"frame {frame.idx}/{frame.count} name={frame.name} reason={frame.reason}"
|
|
||||||
if frame is not None
|
|
||||||
else "frame <none>"
|
|
||||||
)
|
|
||||||
ocr_summary = (ocr_str or "").replace("\n", " / ")[:80]
|
|
||||||
fh.write(
|
|
||||||
f"{self._step}. **{label}** - {frame_str} - OCR: `{ocr_summary}`\n"
|
|
||||||
)
|
|
||||||
return entry
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def frame_capture(
|
|
||||||
request: pytest.FixtureRequest,
|
|
||||||
camera: camera_mod.CameraBackend,
|
|
||||||
session_seed: str,
|
|
||||||
) -> Iterator[FrameCapture]:
|
|
||||||
nodeid = _sanitize_nodeid(request.node.nodeid)
|
|
||||||
dir_path = CAPTURES_ROOT / session_seed / nodeid
|
|
||||||
# Fresh directory per test run so reruns don't mix old and new images.
|
|
||||||
if dir_path.exists():
|
|
||||||
shutil.rmtree(dir_path)
|
|
||||||
|
|
||||||
lines = getattr(request.node, "_debug_log_buffer", [])
|
|
||||||
fc = FrameCapture(camera, dir_path, lines, nodeid)
|
|
||||||
# Stash so pytest_runtest_makereport can embed captures in HTML extras.
|
|
||||||
request.node._ui_captures = fc.captures # type: ignore[attr-defined]
|
|
||||||
yield fc
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Pre-test home-state reset --------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _send_event(port: str, event: InputEventCode) -> None:
|
|
||||||
try:
|
|
||||||
admin_mod.send_input_event(event_code=int(event), port=port)
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
# Treat a failed event as soft - the subsequent frame-log assertion
|
|
||||||
# surfaces the real problem with better context.
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def ui_home_state(
|
|
||||||
request: pytest.FixtureRequest,
|
|
||||||
hub_devices: dict[str, Any],
|
|
||||||
_ui_screen_kept_on: None,
|
|
||||||
) -> Iterator[None]:
|
|
||||||
"""Before every UI test, jump to frame 0 (usually `home`) via FN_F1 and
|
|
||||||
confirm the device emitted the expected frame log.
|
|
||||||
|
|
||||||
Why FN_F1 (not BACK): FN_F1 maps to `switchToFrame(0)` and ALWAYS
|
|
||||||
produces a `reason=fn_f1` log line, regardless of whatever frame the
|
|
||||||
prior test left us on. BACK is context-sensitive (dismisses overlays
|
|
||||||
on some frames, no-op on others) and can silently fail to transition.
|
|
||||||
|
|
||||||
This fixture doubles as the macro-presence detector: if no `fn_f1`
|
|
||||||
log arrives within 5 s, the firmware almost certainly wasn't baked
|
|
||||||
with `USERPREFS_UI_TEST_LOG`. Skip the tier with an actionable hint
|
|
||||||
instead of letting every test body fail with a confusing assertion.
|
|
||||||
|
|
||||||
Autouse scope is restricted to `tests/ui/` by virtue of this fixture
|
|
||||||
living in that directory's conftest.py - no explicit nodeid guard
|
|
||||||
needed (and earlier attempts at one were wrong, matching `/tests/ui/`
|
|
||||||
against a nodeid that has no leading slash).
|
|
||||||
"""
|
|
||||||
role = next((r for r in UI_CAPABLE_ROLES if r in hub_devices), None)
|
|
||||||
if role is None:
|
|
||||||
yield
|
|
||||||
return
|
|
||||||
|
|
||||||
hub_entry = hub_devices[role]
|
|
||||||
port = hub_entry.get("port") if isinstance(hub_entry, dict) else hub_entry
|
|
||||||
lines: list[str] = getattr(request.node, "_debug_log_buffer", [])
|
|
||||||
start_len = len(lines)
|
|
||||||
|
|
||||||
# First: a wake event. The screen should already be kept on by
|
|
||||||
# `_ui_screen_kept_on`, but belt + suspenders - if it somehow
|
|
||||||
# powered off (sleep after factory_reset, etc.), this first FN_F1
|
|
||||||
# gets dropped by InputBroker's screenWasOff guard. That's fine;
|
|
||||||
# the second FN_F1 below lands cleanly.
|
|
||||||
_send_event(port, InputEventCode.FN_F1)
|
|
||||||
time.sleep(0.4)
|
|
||||||
_send_event(port, InputEventCode.FN_F1)
|
|
||||||
|
|
||||||
# Wait for the fn_f1 transition log. Any new `reason=fn_f1` line
|
|
||||||
# after call-start counts - we don't care about the name (it should
|
|
||||||
# be `home` or `deviceFocused` depending on board-specific frame order).
|
|
||||||
from ._screen_log import wait_for_reason
|
|
||||||
|
|
||||||
try:
|
|
||||||
wait_for_reason(lines, "fn_f1", timeout_s=5.0)
|
|
||||||
except TimeoutError:
|
|
||||||
# One more try - FreeRTOS queue may be draining slowly.
|
|
||||||
_send_event(port, InputEventCode.FN_F1)
|
|
||||||
try:
|
|
||||||
wait_for_reason(lines, "fn_f1", timeout_s=5.0)
|
|
||||||
except TimeoutError:
|
|
||||||
# Look at what the _debug_log_buffer actually contains to
|
|
||||||
# disambiguate "macro off" from "macro on but event lost".
|
|
||||||
frame_lines = [ln for ln in lines[start_len:] if "Screen: frame" in ln]
|
|
||||||
processing_lines = [
|
|
||||||
ln for ln in lines[start_len:] if "Processing input event" in ln
|
|
||||||
]
|
|
||||||
if frame_lines:
|
|
||||||
pytest.skip(
|
|
||||||
f"ui_home_state: events fire but none reach Screen "
|
|
||||||
f"(saw {len(frame_lines)} frame line(s), "
|
|
||||||
f"{len(processing_lines)} admin inject(s)). "
|
|
||||||
f"Device may be in an unusual state - try `--force-bake`."
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
pytest.skip(
|
|
||||||
"ui_home_state: no `Screen: frame` log after FN_F1. "
|
|
||||||
"Firmware not baked with USERPREFS_UI_TEST_LOG - "
|
|
||||||
"run with `--force-bake` to reflash, or verify the "
|
|
||||||
"macro is active in the bake."
|
|
||||||
)
|
|
||||||
yield
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- Small helpers reused by tests ---------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def send_event(
|
|
||||||
port: str, event: InputEventCode | int | str, **kwargs: Any
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Thin wrapper so tests read `send_event(port, InputEventCode.RIGHT)`."""
|
|
||||||
return admin_mod.send_input_event(event_code=event, port=port, **kwargs)
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"FrameCapture",
|
|
||||||
"UI_CAPABLE_ROLES",
|
|
||||||
"send_event",
|
|
||||||
"wait_for_frame",
|
|
||||||
"FrameEvent",
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
# Make the helpers discoverable to test modules via `from .conftest import …`.
|
|
||||||
# pytest auto-loads conftest.py, but the symbols above are also re-exported
|
|
||||||
# for readability in the test files.
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
"""FN_F1..F5 directly jumps to frame 0..4 via Screen::handleInputEvent.
|
|
||||||
|
|
||||||
Parametrized over the 5 function keys. Each expects a
|
|
||||||
`Screen: frame <idx>/<count> name=... reason=fn_f<k>` log line, with
|
|
||||||
`idx == k-1`. We don't hardcode the frame *name* because the layout
|
|
||||||
depends on which modules are compiled in for this board.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp.input_events import InputEventCode
|
|
||||||
|
|
||||||
from ._screen_log import get_current_frame, wait_for_reason
|
|
||||||
from .conftest import FrameCapture, send_event
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(120)
|
|
||||||
@pytest.mark.parametrize(
|
|
||||||
"event,expected_idx,reason",
|
|
||||||
[
|
|
||||||
(InputEventCode.FN_F1, 0, "fn_f1"),
|
|
||||||
(InputEventCode.FN_F2, 1, "fn_f2"),
|
|
||||||
(InputEventCode.FN_F3, 2, "fn_f3"),
|
|
||||||
(InputEventCode.FN_F4, 3, "fn_f4"),
|
|
||||||
(InputEventCode.FN_F5, 4, "fn_f5"),
|
|
||||||
],
|
|
||||||
ids=["FN_F1", "FN_F2", "FN_F3", "FN_F4", "FN_F5"],
|
|
||||||
)
|
|
||||||
def test_fn_jump_direct_frame(
|
|
||||||
ui_port: str,
|
|
||||||
frame_capture: FrameCapture,
|
|
||||||
request: pytest.FixtureRequest,
|
|
||||||
event: InputEventCode,
|
|
||||||
expected_idx: int,
|
|
||||||
reason: str,
|
|
||||||
) -> None:
|
|
||||||
lines: list[str] = request.node._debug_log_buffer
|
|
||||||
start = get_current_frame(lines)
|
|
||||||
assert start is not None, "no frame log yet - USERPREFS_UI_TEST_LOG not wired?"
|
|
||||||
assert start.name in (
|
|
||||||
"home",
|
|
||||||
"deviceFocused",
|
|
||||||
), f"setup expected frame 0 landing, got {start.name!r}"
|
|
||||||
frame_capture("initial")
|
|
||||||
|
|
||||||
if start.count <= expected_idx:
|
|
||||||
pytest.skip(
|
|
||||||
f"device has {start.count} frames; FN_F{expected_idx + 1} needs > {expected_idx}"
|
|
||||||
)
|
|
||||||
|
|
||||||
send_event(ui_port, event)
|
|
||||||
time.sleep(0.1)
|
|
||||||
evt = wait_for_reason(lines, reason, timeout_s=5.0)
|
|
||||||
assert evt.idx == expected_idx, (
|
|
||||||
f"FN_F{expected_idx + 1} expected idx={expected_idx}, got {evt.idx} "
|
|
||||||
f"(name={evt.name}, count={evt.count})"
|
|
||||||
)
|
|
||||||
frame_capture(f"after-{reason}")
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
"""Out-of-bounds FN_F5 when the device has <5 frames: no crash, idx unchanged.
|
|
||||||
|
|
||||||
`Screen::handleInputEvent` dispatches FN_F5 unconditionally to
|
|
||||||
`ui->switchToFrame(4)`. The OLEDDisplayUi library typically clamps or
|
|
||||||
silently ignores out-of-range indices, but firmware bugs have existed
|
|
||||||
here - this test protects against a regression that would wedge the UI.
|
|
||||||
|
|
||||||
If this test fails, first check: did the device actually crash (Guru
|
|
||||||
Meditation in the log)? Or did switchToFrame accept an OOB index and
|
|
||||||
leave the UI blank?
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp.input_events import InputEventCode
|
|
||||||
|
|
||||||
from ._screen_log import get_current_frame, wait_for_reason
|
|
||||||
from .conftest import FrameCapture, send_event
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(90)
|
|
||||||
def test_fn_f5_out_of_bounds(
|
|
||||||
ui_port: str,
|
|
||||||
frame_capture: FrameCapture,
|
|
||||||
request: pytest.FixtureRequest,
|
|
||||||
) -> None:
|
|
||||||
lines: list[str] = request.node._debug_log_buffer
|
|
||||||
start = get_current_frame(lines)
|
|
||||||
assert start is not None
|
|
||||||
|
|
||||||
if start.count > 5:
|
|
||||||
pytest.skip(
|
|
||||||
f"device has {start.count} frames; FN_F5 is in-bounds - not testing OOB here"
|
|
||||||
)
|
|
||||||
|
|
||||||
frame_capture("initial-home")
|
|
||||||
send_event(ui_port, InputEventCode.FN_F5)
|
|
||||||
time.sleep(0.5)
|
|
||||||
|
|
||||||
try:
|
|
||||||
wait_for_reason(lines, "fn_f5", timeout_s=3.0)
|
|
||||||
except TimeoutError:
|
|
||||||
# Firmware may have ignored the event entirely - acceptable.
|
|
||||||
pass
|
|
||||||
|
|
||||||
# Capture whatever is on screen (OCR will tell us if something weird
|
|
||||||
# happened). Device must remain responsive - subsequent events should
|
|
||||||
# still land.
|
|
||||||
frame_capture("after-fn_f5-oob")
|
|
||||||
|
|
||||||
# Send a RIGHT to confirm the UI is still alive. If this times out,
|
|
||||||
# the OOB switchToFrame wedged the UI.
|
|
||||||
send_event(ui_port, InputEventCode.RIGHT)
|
|
||||||
post = wait_for_reason(lines, "next", timeout_s=5.0)
|
|
||||||
assert (
|
|
||||||
post is not None
|
|
||||||
), "UI wedged after OOB FN_F5 - RIGHT no longer produces frame log"
|
|
||||||
frame_capture("after-recovery-right")
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
"""SELECT on the home frame opens the home menu; BACK closes it.
|
|
||||||
|
|
||||||
The home menu is an overlay (menuHandler::homeBaseMenu), not a frame
|
|
||||||
transition - so we verify via OCR difference between before/after
|
|
||||||
captures rather than a `Screen: frame` log line. The underlying
|
|
||||||
mechanism is still InputBroker → Screen::handleInputEvent → menu
|
|
||||||
callback.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp.input_events import InputEventCode
|
|
||||||
|
|
||||||
from ._screen_log import get_current_frame
|
|
||||||
from .conftest import FrameCapture, send_event
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(120)
|
|
||||||
def test_select_opens_home_menu(
|
|
||||||
ui_port: str,
|
|
||||||
frame_capture: FrameCapture,
|
|
||||||
request: pytest.FixtureRequest,
|
|
||||||
) -> None:
|
|
||||||
lines: list[str] = request.node._debug_log_buffer
|
|
||||||
start = get_current_frame(lines)
|
|
||||||
assert start is not None
|
|
||||||
if start.name not in ("home", "deviceFocused"):
|
|
||||||
pytest.skip(
|
|
||||||
f"SELECT on {start.name!r} doesn't open homeBaseMenu; "
|
|
||||||
"test is only valid when the landing frame is home/deviceFocused"
|
|
||||||
)
|
|
||||||
|
|
||||||
initial = frame_capture("initial")
|
|
||||||
send_event(ui_port, InputEventCode.SELECT)
|
|
||||||
time.sleep(0.8)
|
|
||||||
opened = frame_capture("after-select")
|
|
||||||
|
|
||||||
# The menu is an overlay (not a frame change). We cannot use log
|
|
||||||
# assertion - instead, OCR should differ because a menu list is now
|
|
||||||
# drawn on top.
|
|
||||||
initial_text = (initial.get("ocr_text") or "").strip()
|
|
||||||
opened_text = (opened.get("ocr_text") or "").strip()
|
|
||||||
if initial_text and opened_text:
|
|
||||||
# When OCR is available, require *some* difference between the two
|
|
||||||
# frames - even a single menu title changes the transcribed text.
|
|
||||||
assert initial_text != opened_text, (
|
|
||||||
f"expected OCR diff after SELECT; both read {initial_text!r}. "
|
|
||||||
"If both are empty, check camera alignment + OCR backend."
|
|
||||||
)
|
|
||||||
|
|
||||||
# Back out - the menu dismisses on BACK.
|
|
||||||
send_event(ui_port, InputEventCode.BACK)
|
|
||||||
time.sleep(0.8)
|
|
||||||
closed = frame_capture("after-back")
|
|
||||||
|
|
||||||
# Soft check: OCR after BACK should look different from the menu
|
|
||||||
# (either back to home or onto a previous frame - BACK's exact
|
|
||||||
# behavior when the menu is up vs. not-up varies). We don't assert
|
|
||||||
# equality because OLED rendering is pixel-stable but camera sampling
|
|
||||||
# introduces noise.
|
|
||||||
if opened_text and closed.get("ocr_text"):
|
|
||||||
close_text = (closed.get("ocr_text") or "").strip()
|
|
||||||
assert (
|
|
||||||
close_text != opened_text
|
|
||||||
), f"after BACK, OCR still looks like the menu: {close_text!r}"
|
|
||||||
@@ -1,60 +0,0 @@
|
|||||||
"""Once we navigate to the textMessage frame, UP/DOWN exercises the
|
|
||||||
message-scroll path (or opens CannedMessages on empty devices).
|
|
||||||
|
|
||||||
Weaker than a "no frame change" assertion because on a fresh bench
|
|
||||||
device the message store is usually empty, and the firmware's UP
|
|
||||||
handler in that case launches CannedMessage - which DOES rebuild
|
|
||||||
frames. We just verify the path doesn't crash + produce captures for
|
|
||||||
visual inspection.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp.input_events import InputEventCode
|
|
||||||
|
|
||||||
from ._screen_log import get_current_frame, wait_for_frame
|
|
||||||
from .conftest import FrameCapture, send_event
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(180)
|
|
||||||
def test_up_down_on_textmessage_survives(
|
|
||||||
ui_port: str,
|
|
||||||
frame_capture: FrameCapture,
|
|
||||||
request: pytest.FixtureRequest,
|
|
||||||
) -> None:
|
|
||||||
lines: list[str] = request.node._debug_log_buffer
|
|
||||||
frame_capture("initial")
|
|
||||||
|
|
||||||
# Walk RIGHT until we land on textMessage - up to 15 hops.
|
|
||||||
for _i in range(15):
|
|
||||||
send_event(ui_port, InputEventCode.RIGHT)
|
|
||||||
time.sleep(0.3)
|
|
||||||
current = get_current_frame(lines)
|
|
||||||
if current is not None and current.name == "textMessage":
|
|
||||||
break
|
|
||||||
else:
|
|
||||||
pytest.skip(
|
|
||||||
"couldn't reach textMessage frame within 15 RIGHTs - not present on this board"
|
|
||||||
)
|
|
||||||
|
|
||||||
wait_for_frame(lines, "textMessage", timeout_s=5.0)
|
|
||||||
frame_capture("on-textMessage")
|
|
||||||
|
|
||||||
# UP and DOWN exercise the message-scroll / canned-message-launch path.
|
|
||||||
# Capture after each so the HTML report shows any visual effect.
|
|
||||||
send_event(ui_port, InputEventCode.UP)
|
|
||||||
time.sleep(0.3)
|
|
||||||
frame_capture("after-up")
|
|
||||||
|
|
||||||
send_event(ui_port, InputEventCode.DOWN)
|
|
||||||
time.sleep(0.3)
|
|
||||||
frame_capture("after-down")
|
|
||||||
|
|
||||||
# Soft check: we should still be in a reachable frame (not wedged).
|
|
||||||
# The next test's `ui_home_state` will error out if the device is
|
|
||||||
# unresponsive, so we don't need a stricter guarantee here.
|
|
||||||
final = get_current_frame(lines)
|
|
||||||
assert final is not None, "no frame log after UP/DOWN - event path broke"
|
|
||||||
@@ -1,93 +0,0 @@
|
|||||||
"""INPUT_BROKER_RIGHT cycles forward through frames; INPUT_BROKER_LEFT backs.
|
|
||||||
|
|
||||||
The simplest UI test: fire N RIGHT events and assert the frame index
|
|
||||||
moves forward by N (modulo frameCount). Each step captures an image +
|
|
||||||
OCR for the HTML report.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from meshtastic_mcp.input_events import InputEventCode
|
|
||||||
|
|
||||||
from ._screen_log import get_current_frame, wait_for_frame
|
|
||||||
from .conftest import FrameCapture, send_event
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(120)
|
|
||||||
def test_input_right_cycles_frames(
|
|
||||||
ui_port: str,
|
|
||||||
frame_capture: FrameCapture,
|
|
||||||
request: pytest.FixtureRequest,
|
|
||||||
) -> None:
|
|
||||||
lines: list[str] = request.node._debug_log_buffer
|
|
||||||
start = get_current_frame(lines)
|
|
||||||
assert start is not None, "no frame log yet - USERPREFS_UI_TEST_LOG not wired?"
|
|
||||||
# FN_F1 in ui_home_state lands on frame 0. The name at frame 0 varies
|
|
||||||
# by board (home on heltec-v3, deviceFocused on others) - accept either.
|
|
||||||
assert start.name in (
|
|
||||||
"home",
|
|
||||||
"deviceFocused",
|
|
||||||
), f"setup expected home/deviceFocused at frame 0, got {start.name!r}"
|
|
||||||
|
|
||||||
frame_capture("initial")
|
|
||||||
visited = [start.idx]
|
|
||||||
|
|
||||||
for step in range(4):
|
|
||||||
send_event(ui_port, InputEventCode.RIGHT)
|
|
||||||
# Each RIGHT should bump the frame index by 1. The log fires with
|
|
||||||
# `reason=next` from showFrame(NEXT).
|
|
||||||
before_count = len(list(_frame_events(lines)))
|
|
||||||
deadline = time.monotonic() + 5.0
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
if len(list(_frame_events(lines))) > before_count:
|
|
||||||
break
|
|
||||||
time.sleep(0.1)
|
|
||||||
evt = get_current_frame(lines)
|
|
||||||
assert evt is not None
|
|
||||||
assert (
|
|
||||||
evt.reason == "next"
|
|
||||||
), f"step {step}: expected reason=next, got {evt.reason!r}"
|
|
||||||
visited.append(evt.idx)
|
|
||||||
frame_capture(f"after-right-{step + 1}")
|
|
||||||
|
|
||||||
# Sanity: each index should differ from its predecessor.
|
|
||||||
diffs = [visited[i + 1] - visited[i] for i in range(len(visited) - 1)]
|
|
||||||
assert all(
|
|
||||||
d in (1, -(start.count - 1)) for d in diffs
|
|
||||||
), f"expected monotonic +1 steps (or a wrap), got visited={visited} diffs={diffs}"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.timeout(120)
|
|
||||||
def test_input_left_returns_to_home(
|
|
||||||
ui_port: str,
|
|
||||||
frame_capture: FrameCapture,
|
|
||||||
request: pytest.FixtureRequest,
|
|
||||||
) -> None:
|
|
||||||
"""After RIGHT×3 + LEFT×3, we should end up back on the starting frame."""
|
|
||||||
lines: list[str] = request.node._debug_log_buffer
|
|
||||||
start = get_current_frame(lines)
|
|
||||||
assert start is not None
|
|
||||||
start_name = start.name
|
|
||||||
frame_capture("initial")
|
|
||||||
for _ in range(3):
|
|
||||||
send_event(ui_port, InputEventCode.RIGHT)
|
|
||||||
time.sleep(0.3)
|
|
||||||
frame_capture("after-right-3")
|
|
||||||
|
|
||||||
for _ in range(3):
|
|
||||||
send_event(ui_port, InputEventCode.LEFT)
|
|
||||||
time.sleep(0.3)
|
|
||||||
|
|
||||||
# Back to whichever frame we started on (home or deviceFocused).
|
|
||||||
wait_for_frame(lines, start_name, timeout_s=5.0)
|
|
||||||
frame_capture(f"after-left-3-back-{start_name}")
|
|
||||||
|
|
||||||
|
|
||||||
def _frame_events(lines: list[str]) -> Any:
|
|
||||||
from ._screen_log import iter_frame_events
|
|
||||||
|
|
||||||
return iter_frame_events(lines)
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user