Merge remote-tracking branch 'origin/develop' into pr10715-fix
# Conflicts: # src/graphics/Screen.cpp # src/modules/Telemetry/Sensor/PMSA003ISensor.h # src/modules/Telemetry/Sensor/SCD30Sensor.h # src/modules/Telemetry/Sensor/SCD4XSensor.h # src/modules/Telemetry/Sensor/SEN5XSensor.cpp # src/modules/Telemetry/Sensor/SEN5XSensor.h # src/modules/Telemetry/Sensor/SFA30Sensor.h
This commit is contained in:
Thomas Göttgens
@@ -1,49 +0,0 @@
|
||||
# Claude Code slash commands for the mcp-server test suite
|
||||
|
||||
Three AI-assisted workflows wrapping `mcp-server/run-tests.sh` and the meshtastic MCP tools. Each one has a twin in `.github/prompts/` for Copilot users.
|
||||
|
||||
| Slash command | What it does | Copilot equivalent |
|
||||
| --------------------- | ------------------------------------------------------------------------- | ---------------------------------------- |
|
||||
| `/test [args]` | Runs the test suite (auto-detects hardware) and interprets failures | `.github/prompts/mcp-test.prompt.md` |
|
||||
| `/diagnose [role]` | Read-only device health report via the meshtastic MCP tools | `.github/prompts/mcp-diagnose.prompt.md` |
|
||||
| `/repro <test> [n=5]` | Re-runs one test N times, diffs firmware logs between passes and failures | `.github/prompts/mcp-repro.prompt.md` |
|
||||
|
||||
## Why two surfaces
|
||||
|
||||
The Claude Code commands and Copilot prompts cover the same three workflows but each speaks its host's idiom:
|
||||
|
||||
- **Claude Code** (`/test`) uses `$ARGUMENTS` for pass-through, has direct access to Bash + all MCP tools registered in the user's settings, and runs in the terminal context.
|
||||
- **Copilot** (`/mcp-test`) runs in VS Code's agent mode; it has terminal + MCP access too but typically asks the operator to confirm inputs interactively.
|
||||
|
||||
A contributor using either IDE gets equivalent assistance. Keep the two in sync when behavior changes — the diff of intent should be minimal.
|
||||
|
||||
## House rules
|
||||
|
||||
- **No destructive writes without explicit operator approval.** Skills that could reflash, factory-reset, or reboot a device must describe the action and stop — the operator authorizes.
|
||||
- **Interpret failures, don't just echo them.** The skill body should pull firmware log lines from `mcp-server/tests/report.html` (the `Meshtastic debug` section, attached by `tests/conftest.py::pytest_runtest_makereport`) and classify the failure.
|
||||
- **Keep MCP tool calls sequential per port.** SerialInterface holds an exclusive port lock; two parallel tool calls on the same port deadlock.
|
||||
- **Never speculate about root cause.** If the evidence doesn't support a classification, say "unknown" and list what you'd need to disambiguate.
|
||||
|
||||
## Adding a new command
|
||||
|
||||
1. Write the Claude Code version at `.claude/commands/<name>.md` with YAML frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
description: one-line purpose (used for auto-invocation by the model)
|
||||
argument-hint: [optional-hint]
|
||||
---
|
||||
```
|
||||
|
||||
2. Write the Copilot equivalent at `.github/prompts/mcp-<name>.prompt.md` with:
|
||||
|
||||
```yaml
|
||||
---
|
||||
mode: agent
|
||||
description: ...
|
||||
---
|
||||
```
|
||||
|
||||
3. Add the row to the table above. Cross-link in both bodies.
|
||||
|
||||
4. Smoke-test on Claude Code first (`/<name>` should appear in autocomplete), then in VS Code Copilot (`/mcp-<name>` in Chat).
|
||||
@@ -1,68 +0,0 @@
|
||||
---
|
||||
description: Produce a device health report using the meshtastic MCP tools (device_info, list_nodes, get_config, short serial log capture)
|
||||
argument-hint: [role=all|nrf52|esp32s3|<port>]
|
||||
---
|
||||
|
||||
# `/diagnose` — device health report
|
||||
|
||||
Call the meshtastic MCP tool bundle and format a structured health report for one or all detected devices. Zero guesswork for the operator.
|
||||
|
||||
## What to do
|
||||
|
||||
1. **Enumerate hardware.** Call `mcp__meshtastic__list_devices(include_unknown=True)`. For each entry where `likely_meshtastic=True`, capture `port`, `vid`, `pid`, `description`.
|
||||
|
||||
2. **Filter by `$ARGUMENTS`**:
|
||||
- No args, `all` → every likely-meshtastic device.
|
||||
- `nrf52` → only devices with `vid == 0x239a`.
|
||||
- `esp32s3` → only devices with `vid == 0x303a` or `vid == 0x10c4`.
|
||||
- A `/dev/cu.*` path → only that one port.
|
||||
- Anything else → treat as a substring match against the `port` string.
|
||||
|
||||
3. **For each selected device, in sequence (NOT parallel — SerialInterface holds an exclusive port lock):**
|
||||
- `mcp__meshtastic__device_info(port=<p>)` — captures `my_node_num`, `long_name`, `short_name`, `firmware_version`, `hw_model`, `region`, `num_nodes`, `primary_channel`.
|
||||
- `mcp__meshtastic__list_nodes(port=<p>)` — count of peers, which ones have `publicKey` set, SNR/RSSI distribution.
|
||||
- `mcp__meshtastic__get_config(section="lora", port=<p>)` — region, preset, channel_num, tx_power, hop_limit.
|
||||
- Optionally, if the device seems unhappy (fails to connect, `num_nodes==1` when ≥2 are plugged in, missing firmware*version), open a short firmware log window: `mcp__meshtastic__serial_open(port=<p>, env=<inferred-env>)`, wait 3s, `serial_read(session_id=<s>, max_lines=100)`, `serial_close(session_id=<s>)`. The env should be inferred from the VID map in `mcp-server/run-tests.sh` (nrf52 → rak4631, esp32s3 → heltec-v3) unless `MESHTASTIC_MCP_ENV*<ROLE>` is set.
|
||||
|
||||
4. **Hub health** (call once, not per-device): `mcp__meshtastic__uhubctl_list()` — enumerates every USB hub the host can see. Note which hubs advertise `ppps=true` and which hub hosts each Meshtastic device (cross-reference by VID). Flag it in the report if:
|
||||
- No hub advertises PPPS → `tests/recovery/` can't run on this setup; hard-recovery via `uhubctl_cycle` isn't available.
|
||||
- A Meshtastic device is on a non-PPPS hub → note it; operator may want to move the device to a PPPS hub to unlock auto-recovery.
|
||||
- `uhubctl_list` raises `ConfigError: uhubctl not found` → just say `uhubctl not installed` in the report; don't treat as a fault.
|
||||
|
||||
5. **Render per-device report** as:
|
||||
|
||||
```text
|
||||
[nrf52 @ /dev/cu.usbmodem1101] fw=2.7.23.bce2825, hw=RAK4631
|
||||
owner : Meshtastic 40eb / 40eb
|
||||
region/band : US, channel 88, LONG_FAST
|
||||
tx_power : 30 dBm, hop_limit=3
|
||||
peers : 1 (esp32s3 0x433c2428, pubkey ✓, SNR 6.0 / RSSI -24 dBm)
|
||||
primary ch : McpTest
|
||||
hub : 1-1.3 port 2 (PPPS, uhubctl-controllable)
|
||||
firmware : no panics in last 3s; NodeInfoModule emitted 2 broadcasts
|
||||
```
|
||||
|
||||
Keep it scannable. If a field is missing or abnormal (no pubkey for a known peer, region=UNSET, num_nodes inconsistent with the hub, device on non-PPPS hub), flag it inline with a short `⚠︎ <one-line reason>`.
|
||||
|
||||
6. **Cross-device correlation** (only when >1 device is inspected):
|
||||
- Do both sides see each other in `nodesByNum`? If one does and the other doesn't, that's asymmetric NodeInfo — flag it.
|
||||
- Do the LoRa configs match? (region, channel_num, modem_preset should all agree; mismatch = no mesh)
|
||||
- Do the primary channel NAMES match? Mismatch = different PSK = no decode.
|
||||
|
||||
7. **Recorder slice (cheap, always available).** The mcp-server runs an autouse log recorder that's been collecting from every connected device. Pull two short slices to surface anything weird that's already happened:
|
||||
- `mcp__meshtastic__logs_window(start="-2m", level="WARN|ERROR|CRIT", max_lines=20)` — recent firmware errors. If empty, say "no recent errors"; don't manufacture concern.
|
||||
- `mcp__meshtastic__telemetry_timeline(window="1h", field="free_heap", max_points=60)` — heap trend. If `slope_per_min < -50`, flag it and recommend `/leakhunt window=6h` for a deeper read; otherwise just note the current free heap.
|
||||
- If `recorder_status` shows `running:false` or `files.telemetry.last_ts` is null, note "recorder has no telemetry yet — enable `set_debug_log_api(True)` to populate" and skip this step gracefully.
|
||||
|
||||
8. **Suggest next actions only for specific, recognisable failure modes**:
|
||||
- Stale PKI pubkey one-way → "run `/test tests/mesh/test_direct_with_ack.py` — the retry + nodeinfo-ping heals this in the test path."
|
||||
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
|
||||
- Device unreachable, reachable via DFU → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds AND the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
|
||||
- CP2102-wedged-driver on macOS → see the note in `run-tests.sh`.
|
||||
- Heap slope strongly negative → "run `/leakhunt window=6h` for a full timeline + classification."
|
||||
|
||||
## What NOT to do
|
||||
|
||||
- No writes. No `set_config`, no `reboot`, no `factory_reset`. This is a read-only diagnostic skill — if the operator wants to change state, they'll ask explicitly.
|
||||
- No `flash` / `erase_and_flash`. Those are separate escalations.
|
||||
- No holding SerialInterface across tool calls — open, query, close; next device. The port lock is exclusive.
|
||||
@@ -1,103 +0,0 @@
|
||||
---
|
||||
description: Hunt for memory leaks (and other slow degradations) by reading the persistent recorder's heap timeline + log slice over a window
|
||||
argument-hint: [window=1h] [field=free_heap] [variant=local]
|
||||
---
|
||||
|
||||
<!-- markdownlint-disable MD029 -->
|
||||
|
||||
# `/leakhunt` — read the recorder, classify a memory leak
|
||||
|
||||
Use the always-on recorder (`mcp-server/.mtlog/`) to read a heap timeline plus the matching log slice and produce a one-page verdict: **steady / slow leak / fragmentation / OOM-imminent**. No firmware changes, no special build flags — the LocalStats telemetry packet that the firmware already broadcasts every ~60 s carries `heap_free_bytes` and `heap_total_bytes`.
|
||||
|
||||
## Two signal paths — pick the right one
|
||||
|
||||
| Path | Build flag | Cadence | Per-thread attribution | Cost |
|
||||
| --------------------- | ---------------- | -------------- | ---------------------- | ------------------------- |
|
||||
| LocalStats packet | (default) | ~60 s | No | Free — always on |
|
||||
| `[heap N]` log prefix | `-DDEBUG_HEAP=1` | every log line | Yes (Thread X leaked) | Bigger flash + log volume |
|
||||
|
||||
Both feed the same `telemetry_timeline(field="free_heap")` query — when DEBUG_HEAP is on, the recorder synthesizes telemetry rows from log prefixes (tagged `source: debug_heap`), so a single timeline call gets whichever signal is available. **For a slow leak diagnosis, the default path is plenty** (60 s cadence over 6 h = 360 points; linear regression over that nails sub-100-byte/min slopes). **DEBUG_HEAP is for attribution** — when the slope is real and you need to know which thread is leaking.
|
||||
|
||||
## What to do
|
||||
|
||||
1. **Parse `$ARGUMENTS`**: optional `window` (default `1h`, accepts `30m`/`6h`/`-3d`/etc.), optional `field` (default `free_heap`; alternates: `total_heap`, `battery_level`, anything in the LocalStats variant), optional `variant` (default `local`; alternates: `device`, `environment`, `power`, `airQuality`, `health`).
|
||||
|
||||
2. **Verify the recorder is alive** — call `mcp__meshtastic__recorder_status`. Check:
|
||||
- `running == True`
|
||||
- `files.telemetry.lines > 0` (at least one telemetry packet recorded — if zero, the device hasn't broadcast LocalStats yet OR `set_debug_log_api` has never been on; tell the operator to run `mcp__meshtastic__set_debug_log_api(enabled=True)` and wait one device-update interval)
|
||||
- `files.telemetry.last_ts` within the last 5 minutes (if older, the device is silent — log that, not "leak detected")
|
||||
|
||||
3. **Detect whether DEBUG_HEAP is active** — `mcp__meshtastic__logs_window(start="-2m", grep=r"\\[heap \\d+\\]", max_lines=3)`. If any line matches, the firmware has the prefix → DEBUG_HEAP is on, expect higher-cadence data and `heap_event` rows. If zero matches over the last 2 minutes, you're on the LocalStats-only path.
|
||||
|
||||
4. **Pull the timeline** — `mcp__meshtastic__telemetry_timeline(window=$window, variant=$variant, field=$field, max_points=200)`. Read:
|
||||
- `samples` — how many raw points contributed
|
||||
- `min`, `max` — total swing
|
||||
- `slope_per_min` — units per minute (linear regression over the whole window)
|
||||
|
||||
5. **Pull the log context for the same window** — `mcp__meshtastic__logs_window(start="-${window}", grep="Heap status|leaked heap|freed heap|out of memory|Alloc an err|panic|abort", max_lines=200)`. These are the strings the firmware emits when something memory-related happens (`DEBUG_HEAP` builds emit `"Heap status:"` and `"leaked heap"` lines; production builds emit `"Alloc an err"` on failure and `"out of memory"` on OOM).
|
||||
|
||||
6. **Pull marker events** so we know if the operator labeled phases — `mcp__meshtastic__events_window(start="-${window}", kind="mark|connection_lost|connection_established")`. If a `connection_lost` overlaps a sharp drop, that's not a leak; that's a reboot.
|
||||
|
||||
6a. **(DEBUG_HEAP only) Per-thread attribution** — `mcp__meshtastic__logs_window(start="-${window}", grep="leaked heap", max_lines=200)`. Each row has a structured `heap_event` field with `{kind, thread, before, after, delta}`. Aggregate by thread: sum the `delta` over the window per thread name. The thread with the largest cumulative negative delta is your suspect. Note the count too — a thread with 50× small leaks is different from 1× big leak.
|
||||
|
||||
7. **Classify** based on what the data says, NOT on what you wish it said. Use these rules in order:
|
||||
- **Insufficient data** (< 5 samples): say so. Suggest a longer window or longer wait. Stop.
|
||||
- **Reboot mid-window**: if any `connection_lost` event is present AND `free_heap` jumped UP at that timestamp, the device rebooted. Note it; pre-reboot trend may be a leak but you only have part of the curve.
|
||||
- **OOM-imminent**: any `Alloc an err=` or `out of memory` line in the log slice. This trumps everything; flag urgently.
|
||||
- **Slow leak**: `slope_per_min < -50` AND `max - min > 1000` AND no reboot. The heap is monotonically (or near-monotonically) declining. Estimate time-to-zero: `min / -slope_per_min` minutes. Surface it.
|
||||
- **Fragmentation suspect**: `slope_per_min` close to zero (|x| < 50) BUT min trends down across the window AND the log slice shows `Alloc an err` warnings WITHOUT total OOM. Means free total is OK but largest contiguous block is shrinking. Recommend a `DEBUG_HEAP` build to confirm.
|
||||
- **Steady**: |slope_per_min| < 50, no error lines. Heap is fine.
|
||||
- **Recovery curve**: slope is POSITIVE — heap recovered. Either a workload completed or GC fired. Note it; not a leak.
|
||||
|
||||
8. **Report**:
|
||||
|
||||
```text
|
||||
/leakhunt window=6h field=free_heap variant=local
|
||||
────────────────────────────────────────────────────
|
||||
recorder : running, telem last_ts 8s ago
|
||||
build : DEBUG_HEAP=ON (per-line prefix detected)
|
||||
samples : 14,200 over 6h (cadence ~1.5s, log-line synth)
|
||||
free_heap : min 92,344 / max 124,008 / range 31,664
|
||||
slope : -82 bytes/min (negative — heap declining)
|
||||
reboots : none in window
|
||||
OOM events : none
|
||||
error lines : 3× "Alloc an err=ESP_ERR_NO_MEM" at +4h12m, +5h08m, +5h44m
|
||||
thread leaks : (DEBUG_HEAP) MeshPacket -3,124 B over 18 events
|
||||
Router -1,408 B over 4 events
|
||||
others -240 B
|
||||
verdict : SLOW LEAK — primary suspect MeshPacket thread
|
||||
est. time-to-OOM: ~1,127 min (~18.8 h) at current slope
|
||||
evidence : (3 log line citations with uptimes)
|
||||
```
|
||||
|
||||
Then: **what to do next.**
|
||||
- SLOW LEAK, **DEBUG_HEAP off** → recommend rebuilding with the flag and re-running this skill. Concrete one-liner the operator can copy:
|
||||
```text
|
||||
mcp__meshtastic__build(env="<env>", build_flags={"DEBUG_HEAP": 1})
|
||||
mcp__meshtastic__pio_flash(env="<env>", port="<port>", confirm=True)
|
||||
```
|
||||
After flash, set debug_log_api back on and wait one window; re-run `/leakhunt`.
|
||||
- SLOW LEAK, **DEBUG_HEAP on** → cite the top-leaking thread name from step 6a. Point at the corresponding source file (`grep -rn "ThreadName(\"<name>\")" src/`); the operator decides what to fix.
|
||||
- FRAGMENTATION SUSPECT → propose pre-allocating any per-packet buffers; or rebuilding with `CONFIG_HEAP_TASK_TRACKING=y` on ESP32 to see who's holding the largest blocks.
|
||||
- OOM-IMMINENT → flag for immediate attention; don't wait for the next telemetry interval.
|
||||
- STEADY → say so; stop. Don't invent problems.
|
||||
|
||||
## What NOT to do
|
||||
|
||||
- Don't assume a leak from a single dip. LocalStats fires every ~60 s and the firmware naturally allocates+frees on each broadcast cycle; one packet sees the trough. Look at the slope, not the deltas.
|
||||
- Don't recommend code changes. This skill diagnoses; the operator decides what to fix.
|
||||
- Don't enable `set_debug_log_api` automatically — if it's off, telemetry isn't reaching pubsub anyway, and the recorder will be empty. Tell the operator to flip it on and wait, then re-run.
|
||||
- Don't run heavy workloads to "trigger the leak." The recorder is passive; we read what's there.
|
||||
|
||||
## Companion: `mark_event` for stress runs
|
||||
|
||||
If the operator wants to test under stimulus (e.g. blast 50 broadcasts and see what the heap does), they can frame the experiment with markers:
|
||||
|
||||
```text
|
||||
mark_event("burst-start")
|
||||
… run the workload …
|
||||
mark_event("burst-end")
|
||||
/leakhunt window=15m
|
||||
```
|
||||
|
||||
The markers land in both `events.jsonl` and `logs.jsonl`, so the report can show "free_heap dipped 8 KB during the burst window, recovered to baseline within 2 LocalStats cycles" → not a leak.
|
||||
@@ -1,70 +0,0 @@
|
||||
---
|
||||
description: Re-run a specific test N times in isolation to triage flakes, diff firmware logs between passes and failures
|
||||
argument-hint: <test-node-id> [count=5]
|
||||
---
|
||||
|
||||
<!-- markdownlint-disable MD029 -->
|
||||
|
||||
# `/repro` — flakiness triage for one test
|
||||
|
||||
Re-run a single pytest node ID N times in isolation, track pass rate, and surface what's _different_ in the firmware logs between the passing attempts and the failing ones. Turns "it's flaky, I guess" into "it fails when X, passes when Y."
|
||||
|
||||
## What to do
|
||||
|
||||
1. **Parse `$ARGUMENTS`**: first token is the pytest node id (e.g. `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip[nrf52->esp32s3]`); second token is an integer count (default `5`, cap at `20`). If the first token doesn't look like a test path (no `::` and no `tests/` prefix), treat the whole `$ARGUMENTS` as a `-k` filter instead.
|
||||
|
||||
2. **Sanity-check the hub first** (so we're not measuring "nothing plugged in" N times): call `mcp__meshtastic__list_devices`. If the test name contains `nrf52` or `esp32s3` and the matching VID isn't present, stop and report — re-running won't help.
|
||||
|
||||
3. **Loop N times**. For each iteration:
|
||||
|
||||
```bash
|
||||
./mcp-server/run-tests.sh <test-id> --tb=short -p no:cacheprovider
|
||||
```
|
||||
|
||||
Capture: exit code, duration, and (on failure) the `Meshtastic debug` firmware log section from `mcp-server/tests/report.html`. `-p no:cacheprovider` suppresses pytest's `.pytest_cache` writes so iterations don't influence each other.
|
||||
|
||||
4. **Track a small structured tally**:
|
||||
|
||||
```text
|
||||
attempt 1: PASS (42s)
|
||||
attempt 2: FAIL (128s) ← firmware log 200-line tail captured
|
||||
attempt 3: PASS (39s)
|
||||
attempt 4: FAIL (121s)
|
||||
attempt 5: PASS (41s)
|
||||
--------------------------------------
|
||||
pass rate: 3/5 (60%) | mean duration: 74s
|
||||
```
|
||||
|
||||
5. **On mixed outcomes**: diff the firmware log tails between a representative passing attempt and a representative failing attempt. Focus on:
|
||||
- Error-level lines only present in failures (`PKI_UNKNOWN_PUBKEY`, `Alloc an err=`, `Skip send`, `No suitable channel`)
|
||||
- Timing around the assertion event — did a broadcast go out, was there an ACK, did NAK fire?
|
||||
- Device state fields that changed (nodesByNum entries, region/preset, channel_num)
|
||||
|
||||
Surface the top 3 differences as a "passes when / fails when" table. Don't dump full logs — pull specific lines with uptime timestamps.
|
||||
|
||||
5a. **Archive recorder slices per attempt** (no extra device interaction; the recorder runs autouse). Right after each attempt finishes, capture its `(start_ts, end_ts)` and call `mcp__meshtastic__recorder_export(start=<start>, end=<end>, dest_dir="mcp-server/tests/repro_artifacts/<safe-test-id>/attempt_<n>/")`. This drops a `logs.jsonl`, `telemetry.jsonl`, `packets.jsonl`, and `events.jsonl` snapshot scoped to the attempt window. Use these for cross-attempt diffs in step 5: `jq '.line' logs.jsonl` is faster than re-running the test, and the telemetry slice lets you compare heap behavior across attempts.
|
||||
|
||||
6. **Classify the flake** into one of:
|
||||
- **LoRa airtime collision** → pass rate improves with fewer concurrent transmitters; propose a `time.sleep` gap or retry bump in the test body.
|
||||
- **PKI key staleness** → fails on first attempt, passes after self-heal; existing retry loop in `test_direct_with_ack.py` handles this.
|
||||
- **NodeInfo cooldown** → `Skip send NodeInfo since we sent it <600s ago` in fail-only logs; needs `broadcast_nodeinfo_ping()` warmup.
|
||||
- **Hardware-specific** (one direction fails, other passes; one device's firmware is older; driver wedged) → specific recovery pointer. For a device that's wedged past `touch_1200bps`, the next escalation is `uhubctl_cycle(role=..., confirm=True)` to hard-power-cycle its hub port (requires `uhubctl` installed).
|
||||
- **Device went dark mid-run** → fails from some attempt onward, never recovers, firmware log stops arriving. Almost always hardware: a Guru crash + frozen CDC. Hard-power-cycle via `uhubctl_cycle(role=..., confirm=True)` before the next iteration; if that also fails, escalate to replug.
|
||||
- **Genuinely unknown** → say so; don't invent a root cause.
|
||||
|
||||
7. **Report back** with:
|
||||
- Pass rate and mean duration.
|
||||
- Classification + evidence (the specific log lines that support it).
|
||||
- A suggested next step (re-run with specific args, open `/diagnose`, edit a specific test file, nothing).
|
||||
|
||||
## Examples
|
||||
|
||||
- `/repro tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip[esp32s3->nrf52] 10` — runs 10 times, diffs firmware logs.
|
||||
- `/repro broadcast_delivers` — no `::`, no `tests/`, so interpreted as `-k broadcast_delivers`; runs every matching test the default 5 times.
|
||||
- `/repro tests/telemetry/test_device_telemetry_broadcast.py 3` — shorter run for a slow test.
|
||||
|
||||
## Constraints
|
||||
|
||||
- Don't exceed `count=20` per invocation — airtime and USB wear add up. If the user asks for 50, negotiate down.
|
||||
- Don't rebuild firmware as part of triage; flakes that only reproduce under different firmware belong in a separate session.
|
||||
- If the FIRST attempt fails AND the rest all pass, that's a classic "state leak from a prior test" → say so and suggest running with `--force-bake` or starting from a clean state rather than chasing the first failure.
|
||||
@@ -1,47 +0,0 @@
|
||||
---
|
||||
description: Run the mcp-server test suite (auto-detects devices) and interpret the results
|
||||
argument-hint: [pytest-args]
|
||||
---
|
||||
|
||||
# `/test` — mcp-server test runner with interpretation
|
||||
|
||||
Run `mcp-server/run-tests.sh` and make sense of the output so the operator doesn't have to.
|
||||
|
||||
## What to do
|
||||
|
||||
1. **Invoke the wrapper.** From the firmware repo root, run:
|
||||
|
||||
```bash
|
||||
./mcp-server/run-tests.sh $ARGUMENTS
|
||||
```
|
||||
|
||||
The wrapper auto-detects connected Meshtastic devices, maps each to its PlatformIO env, exports the required `MESHTASTIC_MCP_ENV_*` env vars, and invokes pytest. If the user passed no arguments, the wrapper supplies a sensible default set (`tests/ --html=tests/report.html --self-contained-html --junitxml=tests/junit.xml -v --tb=short`). A `--report-log=tests/reportlog.jsonl` arg is always appended (unless the operator passed their own). `--assume-baked` is deliberately NOT in the defaults — `test_00_bake.py` has its own skip-if-already-baked check and runs the ~8 s verification by default. Operators can opt into the fast path with `--assume-baked`, or force a reflash with `--force-bake`.
|
||||
|
||||
2. **Read the pre-flight header.** First ~6 lines print the detected hub (role → port → env). If that line reads `detected hub : (none)`, the wrapper will narrow to `tests/unit` only — say so explicitly in your summary so the operator knows hardware tiers were skipped.
|
||||
|
||||
3. **On pass**: one-line summary of the form `N passed, M skipped in <duration>`. Don't enumerate the test names — the user can read those. Do mention any SKIPPED tests and name the cause:
|
||||
- `"role not present on hub"` → device unplugged; operator knows to reconnect.
|
||||
- `"firmware not baked with USERPREFS_UI_TEST_LOG"` → tests/ui skipped because the macro isn't in firmware yet; suggest `--force-bake`.
|
||||
- `"uhubctl not installed"` → tests/recovery + peer-offline skipped; suggest `brew install uhubctl` / `apt install uhubctl`.
|
||||
- `"no PPPS-capable hubs detected"` → tests/recovery skipped because the hub doesn't support per-port power; the tier will never run on that setup.
|
||||
- `"opencv-python-headless is not installed"` → tests/ui auto-deselected by run-tests.sh; suggest `pip install -e 'mcp-server/.[ui]'`.
|
||||
|
||||
4. **On failure**: for every FAILED test, open `mcp-server/tests/report.html` and extract the `Meshtastic debug` section for that test. pytest-html embeds the firmware log stream + device state dump there; the 200-line firmware log tail is usually enough to explain the failure. Summarise: which test, one-line assertion message, the firmware log lines that matter (things like `PKI_UNKNOWN_PUBKEY`, `Skip send NodeInfo`, `Error=`, `Guru Meditation`, `assertion failed`). For UI-tier failures also glance at `mcp-server/tests/ui_captures/<session>/<test>/transcript.md` — it records each step's frame + OCR.
|
||||
|
||||
5. **Classify the failure** as one of:
|
||||
- **Transient/flake**: LoRa collision, timing-sensitive assertion, first-attempt NAK + successful retry pattern. Propose `/repro <test_node_id>` to confirm.
|
||||
- **Environmental**: device unreachable, port busy, CP2102 driver wedged. Suggest the specific recovery in escalation order: (a) replug USB, (b) `touch_1200bps(port=...)` + `pio_flash` for nRF52 DFU, (c) `uhubctl_cycle(role="nrf52", confirm=True)` when a device is fully wedged past DFU (needs `uhubctl` installed — `baked_single`'s auto-recovery hook does this once automatically). Also check `git status userPrefs.jsonc`.
|
||||
- **Regression**: same assertion fails repeatedly, firmware log shows a new/unusual error. Surface the diff between expected and observed, identify the module likely responsible.
|
||||
|
||||
6. **Never run destructive recovery automatically.** If a failure looks like it needs a reflash, factory*reset, `uhubctl_cycle`, or USB replug, \_describe what to do* — don't execute. The operator decides.
|
||||
|
||||
## Arguments handling
|
||||
|
||||
- No args → wrapper's defaults (full suite).
|
||||
- `$ARGUMENTS` passed verbatim to the wrapper, which passes them to pytest.
|
||||
- Common operator invocations: `/test tests/mesh`, `/test tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip`, `/test --force-bake`, `/test -k telemetry`.
|
||||
|
||||
## Side-effects to mention in summary
|
||||
|
||||
- The session fixture snapshots `userPrefs.jsonc` at session start and restores at teardown (plus on `atexit`). After a clean run, `git status userPrefs.jsonc` should be empty. If the wrapper's pre-flight printed a warning about a stale sidecar, call that out — means a prior session crashed.
|
||||
- `mcp-server/tests/report.html` and `junit.xml` are regenerated on every run; the HTML is self-contained (shareable).
|
||||
@@ -76,7 +76,7 @@ runs:
|
||||
done
|
||||
|
||||
- name: PlatformIO ${{ inputs.arch }} download cache
|
||||
uses: actions/cache@v5
|
||||
uses: actions/cache@v6
|
||||
with:
|
||||
path: ~/.platformio/.cache
|
||||
key: pio-cache-${{ inputs.arch }}-${{ hashFiles('.github/actions/**', '**.ini') }}
|
||||
|
||||
@@ -5,7 +5,7 @@ runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
|
||||
+187
-101
@@ -1,5 +1,23 @@
|
||||
# Meshtastic Firmware - Copilot Instructions
|
||||
|
||||
> **TL;DR**
|
||||
>
|
||||
> | | |
|
||||
> | -------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
|
||||
> | Hardware tests | [meshtastic/meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) (`MESHTASTIC_FIRMWARE_ROOT` → this checkout) |
|
||||
> | Format | `trunk fmt` |
|
||||
> | Mirror docs | `AGENTS.md` (short pointer for agents that don't read this file) · `CLAUDE.md` (Claude Code) |
|
||||
>
|
||||
> **Need this? It's here.**
|
||||
>
|
||||
> | | |
|
||||
> | ------------------------------------------- | ---------------------------------------------------------- |
|
||||
> | General helpers (clamp, UTF-8, string fmt…) | `src/meshUtils.h` |
|
||||
> | Logging macros (LOG_DEBUG / INFO / WARN…) | `src/DebugConfiguration.h` |
|
||||
> | New module skeleton | inherit `ProtobufModule<T>` in `src/mesh/ProtobufModule.h` |
|
||||
> | Observer / event wiring | `src/Observer.h` |
|
||||
|
||||
This document provides context and guidelines for AI assistants working with the Meshtastic firmware codebase.
|
||||
|
||||
## Project Overview
|
||||
@@ -37,7 +55,7 @@ MQTT provides a bridge between Meshtastic mesh networks and the internet, enabli
|
||||
|
||||
Messages are published/subscribed using a hierarchical topic format:
|
||||
|
||||
```
|
||||
```text
|
||||
{root}/{channel_id}/{gateway_id}
|
||||
```
|
||||
|
||||
@@ -73,15 +91,15 @@ PKI (Public Key Infrastructure) messages have special handling:
|
||||
|
||||
## Encryption & Key Management
|
||||
|
||||
Meshtastic packets on the air are typically encrypted one of two ways: the **per-channel symmetric** layer (AES-CTR with a shared PSK) for broadcasts and channel traffic, and the **per-peer PKI** layer (X25519 ECDH → AES-256-CCM) for direct messages and remote admin. A channel with a 0-byte PSK (or Ham mode, which wipes PSKs) transmits cleartext — see the size table below. Both are implemented in `src/mesh/CryptoEngine.cpp`; the send/receive dispatch lives in `src/mesh/Router.cpp`; admin authorization lives in `src/modules/AdminModule.cpp`.
|
||||
Meshtastic packets on the air are typically encrypted one of two ways: the **per-channel symmetric** layer (AES-CTR with a shared PSK) for broadcasts and channel traffic, and the **per-peer PKI** layer (X25519 ECDH → AES-256-CCM) for direct messages and remote admin. A channel with a 0-byte PSK (or Ham mode, which wipes PSKs) transmits cleartext - see the size table below. Both are implemented in `src/mesh/CryptoEngine.cpp`; the send/receive dispatch lives in `src/mesh/Router.cpp`; admin authorization lives in `src/modules/AdminModule.cpp`.
|
||||
|
||||
### High-level model
|
||||
|
||||
- **Channels** are symmetric rooms: anyone with the PSK can read any message on the channel. Channel 0 is the "primary" channel and ships with the short-form default PSK on factory devices, forming the public mesh most users join. (The LoRa modem preset `LONG_FAST` lives on `config.lora.modem_preset` and is an independent field — don't conflate "channel 0 default PSK" with the modem preset name.)
|
||||
- **Channels** are symmetric rooms: anyone with the PSK can read any message on the channel. Channel 0 is the "primary" channel and ships with the short-form default PSK on factory devices, forming the public mesh most users join. (The LoRa modem preset `LONG_FAST` lives on `config.lora.modem_preset` and is an independent field - don't conflate "channel 0 default PSK" with the modem preset name.)
|
||||
- **DMs** addressed to a single node require PKI so that other holders of the channel PSK can't read them. Outside Ham mode, Meshtastic does not fall back to channel-symmetric encryption when the destination public key is unknown.
|
||||
- **Remote admin** is a DM carrying an `AdminMessage`. The receiver only acts on it if the sender's public key is on its allowlist (`config.security.admin_key[0..2]`).
|
||||
- **Ham mode** (`owner.is_licensed=true`, where `owner` is the local `meshtastic_User` record) disables PKI entirely and sends cleartext — FCC Part 97 prohibits encryption on amateur bands.
|
||||
- **No ratchet, no session.** Every packet is encrypted from scratch — a stateless design that matches the high-loss, store-and-forward nature of LoRa.
|
||||
- **Ham mode** (`owner.is_licensed=true`, where `owner` is the local `meshtastic_User` record) disables PKI entirely and sends cleartext - FCC Part 97 prohibits encryption on amateur bands.
|
||||
- **No ratchet, no session.** Every packet is encrypted from scratch - a stateless design that matches the high-loss, store-and-forward nature of LoRa.
|
||||
|
||||
### Symmetric channel encryption (AES-CTR)
|
||||
|
||||
@@ -104,55 +122,55 @@ Meshtastic packets on the air are typically encrypted one of two ways: the **per
|
||||
|
||||
- **Keypair**: Curve25519 (aka X25519), 32-byte public + 32-byte private. Stored in `config.security.public_key` / `private_key`; the public half is mirrored into `owner.public_key` so it rides along in NodeInfo broadcasts and propagates through the mesh like any other identity field.
|
||||
- **Key generation** (`generateKeyPair`): stirs `HardwareRNG::fill()` (64 B from platform TRNG when available), the 16-byte `myNodeInfo.device_id`, and a call to `random()` into the rweather/Crypto library's software RNG, then `Curve25519::dh1`. `regeneratePublicKey` recomputes the public half from a known private (used when restoring from backup).
|
||||
- **Keygen entry points**: at boot, `NodeDB` calls `generateKeyPair` (or `regeneratePublicKey` when a stored private key is present and passes a low-entropy check) **directly** when `!owner.is_licensed` and `config.lora.region != UNSET`. `ensurePkiKeys` wraps the same logic for runtime/admin flows — it's the path `AdminModule::handleSetConfig` runs when first assigning a valid region or when security config is written; **do not assume it's the universal boot-time gate**, because the NodeDB path bypasses it.
|
||||
- **Keygen entry points**: at boot, `NodeDB` calls `generateKeyPair` (or `regeneratePublicKey` when a stored private key is present and passes a low-entropy check) **directly** when `!owner.is_licensed` and `config.lora.region != UNSET`. `ensurePkiKeys` wraps the same logic for runtime/admin flows - it's the path `AdminModule::handleSetConfig` runs when first assigning a valid region or when security config is written; **do not assume it's the universal boot-time gate**, because the NodeDB path bypasses it.
|
||||
- **Handshake**: `Curve25519::dh2(local_private, remote_public) → 32-byte shared secret → SHA-256 → 32-byte AES-256 key`. Recomputed per packet. The SHA-256 step is effectively a KDF over the raw ECDH output.
|
||||
- **Cipher**: AES-256-CCM via `aes_ccm_ae` / `aes_ccm_ad` (`src/mesh/aes-ccm.cpp`). MAC length (the `M` parameter) is **8 bytes**. No AAD — the MAC covers ciphertext only.
|
||||
- **Cipher**: AES-256-CCM via `aes_ccm_ae` / `aes_ccm_ad` (`src/mesh/aes-ccm.cpp`). MAC length (the `M` parameter) is **8 bytes**. No AAD - the MAC covers ciphertext only.
|
||||
- **Nonce (13 bytes / 104 bit)**: `aes_ccm_ae`/`aes_ccm_ad` use a 13-byte CCM nonce (`L = 2` is hardcoded in `src/mesh/aes-ccm.cpp`), not a 16-byte nonce. For PKI packets, `CryptoEngine::initNonce(fromNode, packetNum, extraNonce)` starts from the usual packet-derived nonce material, then overwrites nonce bytes `4..7` with a fresh 32-bit `extraNonce = random()`. The effective nonce bytes are therefore: bytes `0..3` = `packet_id`, bytes `4..7` = transmitted `extraNonce`, bytes `8..11` = `from_node`, byte `12` = `0x00`. The receiver reconstructs the same 13-byte nonce from the packet metadata plus the appended `extraNonce`.
|
||||
- **Wire overhead**: 12 bytes appended to the ciphertext = 8-byte MAC ‖ 4-byte extraNonce. Defined as `MESHTASTIC_PKC_OVERHEAD = 12` in `src/mesh/RadioInterface.h`. Only the 4-byte `extraNonce` is sent; the rest of the 13-byte CCM nonce is reconstructed from packet fields as described above. The Router's send path checks this overhead against `MAX_LORA_PAYLOAD_LEN` before committing to PKI.
|
||||
- **Send selection** (`Router::send`): the sender enters the PKI path when **all** hold — we're the originator AND not Ham mode AND not Portduino simradio AND not on the `serial`/`gpio` channels (unless the packet is already marked `pki_encrypted`) AND `config.security.private_key.size == 32` AND destination is a single node (not broadcast) AND the portnum isn't infrastructure. `TRACEROUTE_APP`, `NODEINFO_APP`, `ROUTING_APP`, and `POSITION_APP` are routed through channel encryption even when DMed (these need to be readable by relaying peers). Once on the PKI path, if the destination's public key isn't in our NodeDB the send **fails** with `PKI_SEND_FAIL_PUBLIC_KEY` — it does not silently fall back to channel encryption. If the client explicitly set `pki_encrypted=true` and any condition blocks PKI, the send fails with `PKI_FAILED`.
|
||||
- **Send selection** (`Router::send`): the sender enters the PKI path when **all** hold - we're the originator AND not Ham mode AND not Portduino simradio AND not on the `serial`/`gpio` channels (unless the packet is already marked `pki_encrypted`) AND `config.security.private_key.size == 32` AND destination is a single node (not broadcast) AND the portnum isn't infrastructure. `TRACEROUTE_APP`, `NODEINFO_APP`, `ROUTING_APP`, and `POSITION_APP` are routed through channel encryption even when DMed (these need to be readable by relaying peers). Once on the PKI path, if the destination's public key isn't in our NodeDB the send **fails** with `PKI_SEND_FAIL_PUBLIC_KEY` - it does not silently fall back to channel encryption. If the client explicitly set `pki_encrypted=true` and any condition blocks PKI, the send fails with `PKI_FAILED`.
|
||||
- **Receive selection** (`Router::perhapsDecode`): try PKI decrypt first when `channel == 0` AND `isToUs(p)` AND not broadcast AND both peers have public keys in NodeDB AND `rawSize > MESHTASTIC_PKC_OVERHEAD`. On success the packet gets `pki_encrypted=true` stamped and the sender's public key copied into `p->public_key` for downstream authorization.
|
||||
|
||||
### Remote admin authorization
|
||||
|
||||
Implemented in `src/modules/AdminModule.cpp` → `handleReceivedProtobuf`. The authorization check runs in this order:
|
||||
|
||||
1. **Response messages** — if `messageIsResponse(r)` is true (the payload is a response to one of our earlier admin requests), it's accepted without any further check. The in-file comment flags this as a known-untightened gap: a stricter implementation would remember which `public_key` we last queried and reject responses that don't match.
|
||||
2. **Local admin** — `mp.from == 0` (phone app over BLE, serial CLI, internal module); never travels over the air. **Rejected** if `config.security.is_managed` is true, because managed devices expect admin to arrive over the air through an authorized remote path.
|
||||
3. **Legacy admin channel (deprecated)** — the packet arrived on a channel named literally `"admin"`. Gated by `config.security.admin_channel_enabled`; returns `NOT_AUTHORIZED` if the flag is false. Kept for backward compatibility; new deployments should use PKI admin.
|
||||
4. **PKI admin (preferred for remote)** — `mp.pki_encrypted == true` AND `mp.public_key` matches one of `config.security.admin_key[0..2]` (up to three authorized 32-byte Curve25519 public keys, typically copied from the admin node's own `user.public_key`).
|
||||
1. **Response messages** - if `messageIsResponse(r)` is true (the payload is a response to one of our earlier admin requests), it's accepted without any further check. The in-file comment flags this as a known-untightened gap: a stricter implementation would remember which `public_key` we last queried and reject responses that don't match.
|
||||
2. **Local admin** - `mp.from == 0` (phone app over BLE, serial CLI, internal module); never travels over the air. **Rejected** if `config.security.is_managed` is true, because managed devices expect admin to arrive over the air through an authorized remote path.
|
||||
3. **Legacy admin channel (deprecated)** - the packet arrived on a channel named literally `"admin"`. Gated by `config.security.admin_channel_enabled`; returns `NOT_AUTHORIZED` if the flag is false. Kept for backward compatibility; new deployments should use PKI admin.
|
||||
4. **PKI admin (preferred for remote)** - `mp.pki_encrypted == true` AND `mp.public_key` matches one of `config.security.admin_key[0..2]` (up to three authorized 32-byte Curve25519 public keys, typically copied from the admin node's own `user.public_key`).
|
||||
5. **Fallthrough** → `NOT_AUTHORIZED`.
|
||||
|
||||
On top of authorization, any remote admin message that **mutates** state (not a request, not a response) also has to pass a session-key check (`checkPassKey`): the client must first pull a fresh 8-byte `session_passkey` via `get_admin_session_key_request`, then echo that passkey back in the mutating message. The device rotates the passkey after 150 s and rejects values older than 300 s — a narrow anti-replay window on top of the PKI layer.
|
||||
On top of authorization, any remote admin message that **mutates** state (not a request, not a response) also has to pass a session-key check (`checkPassKey`): the client must first pull a fresh 8-byte `session_passkey` via `get_admin_session_key_request`, then echo that passkey back in the mutating message. The device rotates the passkey after 150 s and rejects values older than 300 s - a narrow anti-replay window on top of the PKI layer.
|
||||
|
||||
`config.security.is_managed = true` disables **local** admin writes (`mp.from == 0` is rejected). It does not by itself force every admin action through PKI — the legacy `"admin"` channel still authorizes remote admin when `config.security.admin_channel_enabled == true`. The AdminModule refuses to persist `is_managed=true` unless at least one `admin_key` is populated — a deliberate guard against operators locking themselves out.
|
||||
`config.security.is_managed = true` disables **local** admin writes (`mp.from == 0` is rejected). It does not by itself force every admin action through PKI - the legacy `"admin"` channel still authorizes remote admin when `config.security.admin_channel_enabled == true`. The AdminModule refuses to persist `is_managed=true` unless at least one `admin_key` is populated - a deliberate guard against operators locking themselves out.
|
||||
|
||||
### Key-rotation hazards (actions that invalidate peers)
|
||||
|
||||
- **`factory_reset_device`** (the "full" variant, calls `NodeDB::factoryReset(eraseBleBonds=true)`) → **wipes** the X25519 private key; a fresh keypair is generated on the next region-set. Every existing peer holds the old public key, so DMs to this node silently fail PKI decrypt until every peer re-exchanges NodeInfo.
|
||||
- **`factory_reset_config`** (the "partial" variant, calls `NodeDB::factoryReset()` with `eraseBleBonds=false`) → **preserves** the X25519 private key in `installDefaultConfig(preserveKey=true)`; the public key is zeroed and gets rebuilt from the preserved private key on the next boot via the NodeDB path's `regeneratePublicKey` call. Identity is preserved and the mesh does not need to re-exchange keys.
|
||||
- **`region=UNSET → valid region`** → `ensurePkiKeys` runs inside the same `handleSetConfig` path; missing keys get generated at that moment.
|
||||
- **Ham mode transitions** — entering Ham mode (`user.is_licensed=true`) runs `Channels::ensureLicensedOperation`, which **wipes every channel PSK** (all traffic becomes cleartext) and disables the legacy admin channel. The X25519 private key is preserved on the device but not used because `Router::send` skips PKI when `owner.is_licensed` is true. Leaving Ham mode re-enables PKI with the preserved keypair but does not restore the wiped channel PSKs — the operator has to re-set them.
|
||||
- **Ham mode transitions** - entering Ham mode (`user.is_licensed=true`) runs `Channels::ensureLicensedOperation`, which **wipes every channel PSK** (all traffic becomes cleartext) and disables the legacy admin channel. The X25519 private key is preserved on the device but not used because `Router::send` skips PKI when `owner.is_licensed` is true. Leaving Ham mode re-enables PKI with the preserved keypair but does not restore the wiped channel PSKs - the operator has to re-set them.
|
||||
- **Channel 0 PSK change** → every peer must re-learn the channel hash; cached NodeInfo becomes temporarily unreachable until the next broadcast.
|
||||
- **`security.private_key` blanked via admin** → regenerates both halves (unless in Ham mode) and propagates the new public key via NodeInfo.
|
||||
|
||||
## NodeDB Layout (v25)
|
||||
|
||||
`DEVICESTATE_CUR_VER = 25`, `DEVICESTATE_MIN_VER = 24`. The on-device NodeDB was split in v25 into a slim header table plus four optional satellite stores. Older v24 saves auto-migrate at boot. Old training-data instincts (`node->user.long_name`, `node->position.latitude_i`, `node->is_favorite`, `node->device_metrics.battery_level`) are wrong now — the fields aren't there. Read this section before touching anything that walks `nodeDB->meshNodes`.
|
||||
`DEVICESTATE_CUR_VER = 25`, `DEVICESTATE_MIN_VER = 24`. The on-device NodeDB was split in v25 into a slim header table plus four optional satellite stores. Older v24 saves auto-migrate at boot. Old training-data instincts (`node->user.long_name`, `node->position.latitude_i`, `node->is_favorite`, `node->device_metrics.battery_level`) are wrong now - the fields aren't there. Read this section before touching anything that walks `nodeDB->meshNodes`.
|
||||
|
||||
### Slim `NodeInfoLite`
|
||||
|
||||
`UserLite` is flattened onto `NodeInfoLite` (no nested sub-message); `position` and `device_metrics` are removed entirely (tags reserved). MAC address is dropped. Long names are capped at 25 chars (`max_size:25` in `deviceonly.options`); `hw_model` and `role` are `int_size:8`. Encoded size dropped from ~166 B → ~105 B per node.
|
||||
|
||||
Booleans are bit-packed into `NodeInfoLite.bitfield`. **Do not read or write the bits directly** — use the inline helpers in `src/mesh/NodeDB.h`:
|
||||
Booleans are bit-packed into `NodeInfoLite.bitfield`. **Do not read or write the bits directly** - use the inline helpers in `src/mesh/NodeDB.h`:
|
||||
|
||||
```cpp
|
||||
nodeInfoLiteHasUser(n) // bit 5 — user fields populated
|
||||
nodeInfoLiteHasUser(n) // bit 5 - user fields populated
|
||||
nodeInfoLiteIsFavorite(n) // bit 3
|
||||
nodeInfoLiteIsIgnored(n) // bit 4
|
||||
nodeInfoLiteIsMuted(n) // bit 1
|
||||
nodeInfoLiteIsLicensed(n) // bit 6 — Ham mode peer
|
||||
nodeInfoLiteIsLicensed(n) // bit 6 - Ham mode peer
|
||||
nodeInfoLiteIsKeyManuallyVerified(n) // bit 0
|
||||
nodeInfoLiteHasIsUnmessagable(n) // bit 8 — "is_unmessagable was sent"
|
||||
nodeInfoLiteHasIsUnmessagable(n) // bit 8 - "is_unmessagable was sent"
|
||||
nodeInfoLiteIsUnmessagable(n) // bit 7
|
||||
// via_mqtt is bit 2 (mask exposed; predicate uses the mask directly)
|
||||
|
||||
@@ -170,9 +188,9 @@ Four `std::unordered_map<NodeNum, …>` members on `NodeDB`, each gated by its o
|
||||
| `nodeEnvironment` | `meshtastic_EnvironmentMetrics` | `MESHTASTIC_EXCLUDE_ENVIRONMENTDB` |
|
||||
| `nodeStatus` | `meshtastic_StatusMessage` | `MESHTASTIC_EXCLUDE_STATUSDB` |
|
||||
|
||||
Defaults are ON (i.e., maps **excluded**) for STM32WL only — see `src/mesh/mesh-pb-constants.h`. On every other arch all four maps are present. When excluded, the map member is absent and the corresponding accessors return `false`.
|
||||
Defaults are ON (i.e., maps **excluded**) for STM32WL only - see `src/mesh/mesh-pb-constants.h`. On every other arch all four maps are present. When excluded, the map member is absent and the corresponding accessors return `false`.
|
||||
|
||||
All four maps are guarded by **`mutable concurrency::Lock satelliteMutex`** — concurrent access from receive threads, the phone API state machine, and the renderer is the rule, not the exception.
|
||||
All four maps are guarded by **`mutable concurrency::Lock satelliteMutex`** - concurrent access from receive threads, the phone API state machine, and the renderer is the rule, not the exception.
|
||||
|
||||
### Accessor convention
|
||||
|
||||
@@ -185,32 +203,49 @@ bool copyNodeEnvironment(NodeNum, meshtastic_EnvironmentMetrics &out) const;
|
||||
bool copyNodeStatus(NodeNum, meshtastic_StatusMessage &out) const;
|
||||
```
|
||||
|
||||
Each takes the lock, copies the value if present, returns `false` if the entry is absent or the DB is excluded. Pass-by-out-param is deliberate — pointer-style accessors would invite UAF and lock-leak bugs across the renderer. The "has any X" convenience predicates (`hasValidPosition` etc.) are implemented in terms of these.
|
||||
Each takes the lock, copies the value if present, returns `false` if the entry is absent or the DB is excluded. Pass-by-out-param is deliberate - pointer-style accessors would invite UAF and lock-leak bugs across the renderer. The "has any X" convenience predicates (`hasValidPosition` etc.) are implemented in terms of these.
|
||||
|
||||
Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which dispatches on `which_variant` for device vs environment metrics) — these own the lock and the eviction hooks.
|
||||
Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which dispatches on `which_variant` for device vs environment metrics) - these own the lock and the eviction hooks.
|
||||
|
||||
### Eviction
|
||||
|
||||
Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly.
|
||||
Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `demoteOldestHotNodesToWarm` (the over-cap warm-tier migration), `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly. (Note: `enforceSatelliteCaps`/`evictSatelliteOverCap` call `.erase()` directly on purpose - that's a satellite-only cap trim where the node _stays_ in the header, a different operation from this chokepoint.)
|
||||
|
||||
### Warm tier (long-tail identity)
|
||||
|
||||
On every arch except STM32WL and bare nRF52832 (`WARM_NODE_COUNT > 0`), a node evicted from the header table is not forgotten outright: `WarmNodeStore` (`src/mesh/WarmNodeStore.{h,cpp}`) keeps a 40 B `{num, last_heard, public_key}` record per evicted node - primarily so PKI DMs to/from a long-tail node keep decrypting without re-running a NodeInfo exchange (the rest of `NodeInfoLite` rebuilds from traffic in seconds).
|
||||
|
||||
- **Write:** `getOrCreateMeshNode`'s eviction and `demoteOldestHotNodesToWarm` (the over-cap boot migration) call `warmStore.absorb(num, last_heard, key)` _before_ the node leaves the header.
|
||||
- **Read-back:** `getOrCreateMeshNode` calls `warmStore.take()` to rehydrate `last_heard` + key when a warm node is re-admitted; `copyPublicKey()` falls back to the warm tier so the PKI send path finds keys for evicted peers.
|
||||
- **Persistence:** nRF52840 uses a 12 KB raw-flash record-ring at `0xEA000` (below LittleFS; append + replay + compact-on-rotate, link-guarded by `nrf52840_s140_v7.ld` and `extra_scripts/nrf52_warm_region.py`). Everywhere else: a `/prefs/warm.dat` snapshot flushed by `saveIfDirty()` on the node-DB save cadence.
|
||||
- **Tunables** (`mesh-pb-constants.h`): `WARM_NODE_COUNT` (per-arch; `0` disables the tier) and `MAX_NUM_NODES` (hot cap - 120 on nRF52840/generic ESP32 to fit the 28 KB LittleFS; ESP32-S3 keeps its flash-scaled 100/200/250, portduino 250). Verbose migration/self-care tracing routes through `LOG_MIGRATION`, gated by `MESHTASTIC_NODEDB_MIGRATION_VERBOSE`.
|
||||
|
||||
### Satellite caps
|
||||
|
||||
Only the freshest `MAX_SATELLITE_NODES` nodes keep satellite payloads; the rest of the header table carries just the `NodeInfoLite`. The cap is **per-platform**: 40 on RAM-constrained parts (nRF52840, generic ESP32) since the four maps live in internal SRAM (not PSRAM, ~408 B/node across the four), and 250 on flash-rich hosts (ESP32-S3, portduino) so every hot node can carry rich data as before the cap existed. `enforceSatelliteCaps()` trims each map to the cap on load (returns whether it trimmed); `evictSatelliteOverCap()` trims before each insert. Eviction is by the owning node's hot `last_heard` (stalest first, demoted/absent nodes rank as `last_heard==0`); self is never trimmed.
|
||||
|
||||
### On-boot self-care
|
||||
|
||||
`NodeDB::nodeDBSelfCare()` runs once identity is established (the constructor after key (re)gen, and `reloadFromDisk()` - _not_ inside `loadFromDisk`, where `getNodeNum()` is still 0). It confirms self is present (warns if a non-empty DB is missing us - a foreign/over-cap file), pins self to index 0, demotes/trims only **non-self** overflow into the warm tier, then rewrites `nodes.proto` **once** and only if it healed something - and never while encrypted storage is locked (it would persist placeholder defaults). `loadFromDisk` deliberately leaves the loaded store untrimmed for this pass.
|
||||
|
||||
### Sync flow: thin NodeInfo + post-COMPLETE_ID replay (no opt-in)
|
||||
|
||||
There is no capability flag and no special "gradient" nonce. The **default** sync flow is:
|
||||
|
||||
1. Config / module-config / channel / metadata segments (same as before).
|
||||
2. `STATE_SEND_OWN_NODEINFO` — **our own** NodeInfo, still bundled with our position and device_metrics (because the replay snapshot excludes our own NodeNum). Emitted via `ConvertToNodeInfo(lite)`.
|
||||
3. `STATE_SEND_OTHER_NODEINFOS` — every other peer's NodeInfo, **always thin** (no `position`, no `device_metrics`). Emitted via `ConvertToNodeInfoThin(lite)`.
|
||||
4. `STATE_SEND_FILEMANIFEST` → `STATE_SEND_COMPLETE_ID` — the phone sees `config_complete_id` and treats sync as done.
|
||||
5. `STATE_SEND_PACKETS` — live mesh packets, with a trailing replay drain interleaved. The replay drain walks four cached satellite stores in order (positions → telemetry → environment → status) and emits each cached entry as an ordinary `MeshPacket` on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` device + environment variants, `NODE_STATUS_APP`). These are indistinguishable on the wire from live mesh traffic, so clients need no special handling — any code that already updates UI on `POSITION_APP` etc. works.
|
||||
2. `STATE_SEND_OWN_NODEINFO` - **our own** NodeInfo, still bundled with our position and device_metrics (because the replay snapshot excludes our own NodeNum). Emitted via `ConvertToNodeInfo(lite)`.
|
||||
3. `STATE_SEND_OTHER_NODEINFOS` - every other peer's NodeInfo, **always thin** (no `position`, no `device_metrics`). Emitted via `ConvertToNodeInfoThin(lite)`.
|
||||
4. `STATE_SEND_FILEMANIFEST` → `STATE_SEND_COMPLETE_ID` - the phone sees `config_complete_id` and treats sync as done.
|
||||
5. `STATE_SEND_PACKETS` - live mesh packets, with a trailing replay drain interleaved. The replay drain walks four cached satellite stores in order (positions → telemetry → environment → status) and emits each cached entry as an ordinary `MeshPacket` on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` device + environment variants, `NODE_STATUS_APP`). These are indistinguishable on the wire from live mesh traffic, so clients need no special handling - any code that already updates UI on `POSITION_APP` etc. works.
|
||||
|
||||
`PhoneAPI::sendConfigComplete()` arms `replayPhase = REPLAY_PHASE_POSITIONS` for default/full sync and `SPECIAL_NONCE_ONLY_NODES`, while `SPECIAL_NONCE_ONLY_CONFIG` skips replay. The drain runs inside `STATE_SEND_PACKETS` via `popReplayPacket()`, lower priority than live traffic. When all four phases drain, `replayPhase` flips back to `REPLAY_PHASE_IDLE` and the snapshot vectors get `shrink_to_fit`ed.
|
||||
|
||||
STM32WL and any other build with all four `MESHTASTIC_EXCLUDE_*DB` flags set produces zero replay packets — `popReplayPacket` advances through each phase in microseconds without emitting anything.
|
||||
STM32WL and any other build with all four `MESHTASTIC_EXCLUDE_*DB` flags set produces zero replay packets - `popReplayPacket` advances through each phase in microseconds without emitting anything.
|
||||
|
||||
Special nonces that still mean something:
|
||||
|
||||
- `SPECIAL_NONCE_ONLY_CONFIG` (69420) — skip node sync entirely, just config.
|
||||
- `SPECIAL_NONCE_ONLY_NODES` (69421) — skip config segments, jump straight to `STATE_SEND_OWN_NODEINFO`. Still gets the post-COMPLETE_ID replay drain.
|
||||
- `SPECIAL_NONCE_ONLY_CONFIG` (69420) - skip node sync entirely, just config.
|
||||
- `SPECIAL_NONCE_ONLY_NODES` (69421) - skip config segments, jump straight to `STATE_SEND_OWN_NODEINFO`. Still gets the post-COMPLETE_ID replay drain.
|
||||
|
||||
There are no other reserved nonces; everything else is a fresh random `want_config_id` from the client.
|
||||
|
||||
@@ -220,17 +255,17 @@ The legacy migration code lives in **`src/mesh/NodeDBLegacyMigration.cpp`**, not
|
||||
|
||||
### Read-site rules of thumb
|
||||
|
||||
- Never `node->position.X` / `node->device_metrics.X` — those fields no longer exist. Pull from the satellite map via `copyNodePosition` / `copyNodeTelemetry`.
|
||||
- Never `node->user.long_name` — `long_name`, `short_name`, `public_key`, `hw_model`, `role`, `macaddr` (gone), `is_licensed`, `is_unmessagable` are flat on `NodeInfoLite`.
|
||||
- Never `node->is_favorite` / `node->is_ignored` / `node->via_mqtt` / `node->is_key_manually_verified` — use the bitfield helpers.
|
||||
- Never assume `nodeDB->getMeshNode(num)->position.time` — call `copyNodePosition` and check the return.
|
||||
- Never `node->position.X` / `node->device_metrics.X` - those fields no longer exist. Pull from the satellite map via `copyNodePosition` / `copyNodeTelemetry`.
|
||||
- Never `node->user.long_name` - `long_name`, `short_name`, `public_key`, `hw_model`, `role`, `macaddr` (gone), `is_licensed`, `is_unmessagable` are flat on `NodeInfoLite`.
|
||||
- Never `node->is_favorite` / `node->is_ignored` / `node->via_mqtt` / `node->is_key_manually_verified` - use the bitfield helpers.
|
||||
- Never assume `nodeDB->getMeshNode(num)->position.time` - call `copyNodePosition` and check the return.
|
||||
- Don't lock `satelliteMutex` yourself in renderer code; the copy-out accessors already do.
|
||||
|
||||
Unit tests for the conversion layer live in `test/test_type_conversions/test_main.cpp` (Unity) — bitfield round-trips, `long_name` truncation, thin-vs-full conversions. Add cases there when extending the schema.
|
||||
Unit tests for the conversion layer live in `test/test_type_conversions/test_main.cpp` (Unity) - bitfield round-trips, `long_name` truncation, thin-vs-full conversions. Add cases there when extending the schema.
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
```text
|
||||
firmware/
|
||||
├── src/ # Main source code
|
||||
│ ├── main.cpp # Application entry point
|
||||
@@ -285,21 +320,23 @@ firmware/
|
||||
|
||||
### Formatting & the trunk toolchain
|
||||
|
||||
`trunk fmt` is the project formatter (`trunk_check` CI rejects unformatted code). For Claude Code users, `.claude/settings.json` ships a PostToolUse hook that runs `trunk fmt --force` on every file the agent writes or edits. The hook is pure sh/grep/sed — no python or jq required — but trunk itself must be able to run:
|
||||
`trunk fmt` is the project formatter (`trunk_check` CI rejects unformatted code). For Claude Code users, `.claude/settings.json` ships a PostToolUse hook that runs `trunk fmt --force` on every file the agent writes or edits. The hook is pure sh/grep/sed - no python or jq required - but trunk itself must be able to run:
|
||||
|
||||
- Trunk's launcher (`~/.cache/trunk/launcher/trunk`, or `trunk` on PATH) downloads the CLI version pinned in `.trunk/trunk.yaml` on first use and again whenever that pin is bumped. **The launcher needs `curl` or `wget`**; without one it fails with "Cannot download… please install curl or wget", and the hook surfaces that as a warning on every write.
|
||||
- No curl/wget available (e.g. a minimal WSL image)? Bootstrap by hand with any Python (PlatformIO bundles one at `~/.platformio/penv/bin/python`): download `https://trunk.io/releases/<ver>/trunk-<ver>-linux-x86_64.tar.gz` and place the `trunk` binary at `~/.cache/trunk/cli/<ver>-linux-x86_64/trunk` (chmod +x), where `<ver>` is the `cli.version` from `.trunk/trunk.yaml`.
|
||||
- The hook fails loudly by design (visible warning, non-blocking). Silent no-op formatting hooks hide real breakage — don't re-add `2>/dev/null || true` around the whole thing.
|
||||
- More generally: don't assume a stock Linux userland in hooks or helper scripts — minimal WSL/container images may lack `python3`, `curl`, `wget`, and `jq`. Prefer plain sh + coreutils, or PlatformIO's bundled Python for anything heavier.
|
||||
- The hook fails loudly by design (visible warning, non-blocking). Silent no-op formatting hooks hide real breakage - don't re-add `2>/dev/null || true` around the whole thing.
|
||||
- More generally: don't assume a stock Linux userland in hooks or helper scripts - minimal WSL/container images may lack `python3`, `curl`, `wget`, and `jq`. Prefer plain sh + coreutils, or PlatformIO's bundled Python for anything heavier.
|
||||
|
||||
### General Style
|
||||
|
||||
- Follow existing code style - run `trunk fmt` before commits
|
||||
- Prefer `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR` for logging
|
||||
- **Format node IDs and packet IDs as `0x%08x` in logs.** This covers `NodeNum`/`PacketId` and the `uint32_t` packet fields `from`, `to`, `id`, `dest`, `source`, `request_id`, and `node_id`. They are 32-bit, so 8 hex digits is exact - `%08x` never truncates or leaves a value ragged. Do **not** use `%x` (variable width) or `%0x` (a no-op typo for `%08x` - the `0` flag does nothing without a width). User-facing display uses `!%08x` (the `!xxxxxxxx` convention), e.g. `Applet::hexifyNodeNum`.
|
||||
- **Do not zero-pad one-byte values to 8.** `next_hop`, `relay_node`, and the next-hop hint are `uint8_t` last-byte route hints, and `channel` is a one-byte hash/index - log these as `0x%x` (or `%d`). Padding a byte to `0x000000ab` falsely implies a full node number. The same goes for I2C addresses, register values, flags/bitmasks, and error/reason codes: they are not IDs, so leave them `0x%x`.
|
||||
- Use `assert()` for invariants that should never fail
|
||||
- C++17 features are available (`std::optional`, structured bindings, `if constexpr`, etc.)
|
||||
- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
|
||||
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
|
||||
- **Keep code comments minimal - one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
|
||||
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check - raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
|
||||
|
||||
### Naming Conventions
|
||||
|
||||
@@ -389,7 +426,7 @@ Multiple display driver families in `src/graphics/`:
|
||||
|
||||
**InkHUD** (`src/graphics/niche/InkHUD/`) is an event-driven e-ink UI framework:
|
||||
|
||||
- Applet-based architecture — modular display tiles
|
||||
- Applet-based architecture - modular display tiles
|
||||
- Read-only, static display optimized for minimal refreshes and low power
|
||||
- Configured per-variant via `nicheGraphics.h`
|
||||
- Separate PlatformIO config: `src/graphics/niche/InkHUD/PlatformioConfig.ini`
|
||||
@@ -446,6 +483,7 @@ Key defines in variant.h:
|
||||
- Regenerate with `bin/regen-protos.sh`
|
||||
- Message types prefixed with `meshtastic_`
|
||||
- Nanopb `.options` files control field sizes and encoding
|
||||
- **Never edit or commit files under `src/mesh/generated/`.** They are regenerated from the [`meshtastic/protobufs`](https://github.com/meshtastic/protobufs) submodule by the `update_protobufs.yml` GitHub Action and any hand edits will be overwritten - guaranteed merge conflict on the next sync. To change a wire format, open a PR against the protobufs repo first; the workflow then re-runs `bin/regen-protos.sh` and opens a PR here with the regenerated sources.
|
||||
|
||||
### Conditional Compilation
|
||||
|
||||
@@ -465,7 +503,7 @@ Key defines in variant.h:
|
||||
|
||||
## Build System
|
||||
|
||||
## Agent Tooling Baseline
|
||||
### Agent Tooling Baseline
|
||||
|
||||
Mirror counterpart: `AGENTS.md` under **Agent Tooling Baseline**.
|
||||
|
||||
@@ -518,7 +556,7 @@ pio run -e native-macos # Build headless macOS meshtasticd (Homebrew prere
|
||||
|
||||
1. Create directory under `variants/<arch>/<name>/`
|
||||
2. Add `variant.h` with pin definitions and hardware capability defines
|
||||
3. Add `platformio.ini` with build config — use `extends` to reference common base (e.g., `esp32s3_base`)
|
||||
3. Add `platformio.ini` with build config - use `extends` to reference common base (e.g., `esp32s3_base`)
|
||||
4. Set `custom_meshtastic_support_level = 1` (PR builds) or `2` (merge builds)
|
||||
5. For e-ink displays, add `nicheGraphics.h` for InkHUD configuration
|
||||
|
||||
@@ -618,27 +656,74 @@ Most workflows can be triggered manually via `workflow_dispatch` for testing.
|
||||
|
||||
### Native unit tests (C++)
|
||||
|
||||
Unit tests in `test/` directory with 17 test suites:
|
||||
Unit tests in `test/` directory. The canonical suite count is in `test/native-suite-count` and is cross-checked on every full run. Current suites:
|
||||
|
||||
- `test_admin_radio/` - LoRa region/config validation and AdminModule dispatch
|
||||
- `test_atak/` - ATAK integration
|
||||
- `test_crypto/` - Cryptography
|
||||
- `test_default/` - Default configuration
|
||||
- `test_hop_scaling/` - Hop scaling histogram and required-hop logic
|
||||
- `test_http_content_handler/` - HTTP handling
|
||||
- `test_mac_from_string/` - MAC address parsing
|
||||
- `test_mesh_module/` - Module framework
|
||||
- `test_meshpacket_serializer/` - Packet serialization
|
||||
- `test_mqtt/` - MQTT integration
|
||||
- `test_nexthop_routing/` - Next-hop routing logic
|
||||
- `test_nodedb_blocked/` - NodeDB blocked-node handling
|
||||
- `test_packet_history/` - Packet history tracking
|
||||
- `test_packet_signing/` - Packet signing
|
||||
- `test_position_module/` - Position module behaviour
|
||||
- `test_position_precision/` - Position precision helpers
|
||||
- `test_radio/` - Radio interface
|
||||
- `test_rtc/` - RTC / time handling
|
||||
- `test_serial/` - Serial communication
|
||||
- `test_traffic_management/` - Traffic management
|
||||
- `test_traffic_management/` - Traffic management (dedup, rate-limit, hop-trim, role exceptions)
|
||||
- `test_transmit_history/` - Retransmission tracking
|
||||
- `test_type_conversions/` - NodeDB v25 type conversion (bitfield round-trips, NodeInfoLite)
|
||||
- `test_utf8/` - UTF-8 utilities
|
||||
- `test_warm_store/` - Warm-tier node store
|
||||
|
||||
Run command (preferred — avoids pipe-buffering and the Ubuntu externally-managed-environment error):
|
||||
**Preferred run command - `bin/run-tests.sh`** (uses the `coverage` env with ASan/LSan sanitizers; emits a machine-readable verdict on the final line; update `test/native-suite-count` when adding or removing suites):
|
||||
|
||||
```bash
|
||||
./bin/run-tests.sh # all suites
|
||||
./bin/run-tests.sh -f test_traffic_management # single suite (yields FILTERED, not GREEN)
|
||||
```
|
||||
|
||||
Exit codes and verdicts (exact counts will vary; examples below are illustrative):
|
||||
|
||||
| Exit | Verdict | Meaning |
|
||||
| ---- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 0 | `GREEN` | All canonical suites ran, all passed, no ignored test cases |
|
||||
| 1 | `RED` | At least one failure, build error, or sanitizer fault |
|
||||
| 2 | `AMBER` | All that ran passed, but something was lost: a suite silently went missing on a full run, individual test cases were skipped (`TEST_IGNORE`), or `test/native-suite-count` disagrees with the `test/` directory count |
|
||||
| 3 | `FILTERED` | A `-f` run completed cleanly; suites outside the filter were intentionally not run |
|
||||
|
||||
Examples - exact counts will vary by suite count and env:
|
||||
|
||||
```text
|
||||
# GREEN: all suites ran and passed
|
||||
RESULT: GREEN N/N suites passed [canonical: N/N]
|
||||
|
||||
# RED: real test failure
|
||||
RESULT: RED 1 failed
|
||||
|
||||
# RED: sanitizer exit-time abort (all tests passed but process aborted at exit)
|
||||
RESULT: RED exit-time abort (tests passed; likely sanitizer - see hint above)
|
||||
|
||||
# AMBER: native-suite-count disagrees with test/ directory count (too low)
|
||||
RESULT: AMBER test/ has 24 suite directories but native-suite-count says 5 - update test/native-suite-count after registering new suites
|
||||
|
||||
# AMBER: native-suite-count disagrees with test/ directory count (too high)
|
||||
RESULT: AMBER test/ has 24 suite directories but native-suite-count says 99 - update test/native-suite-count after removing suites
|
||||
|
||||
# FILTERED: single suite run completed cleanly
|
||||
RESULT: FILTERED 1/24 suites ran (not run: test_admin_radio test_atak …) - filtered: test_serial [canonical: 1/24]
|
||||
```
|
||||
|
||||
> **Copilot interface note:** When running tests via the Copilot chat interface, edits made through the chat may not be reflected in the on-disk files that the test binary reads. If tests pass in chat but fail locally (or vice versa), verify the files on disk match what you expect before trusting the result. Always confirm with a local terminal run.
|
||||
|
||||
Raw `pio test` (no sanitizers, no verdict logic) - use only when you need to override the env:
|
||||
|
||||
```bash
|
||||
~/.platformio/penv/bin/python -m platformio test -e native -f test_your_suite > /tmp/test_out.txt 2>&1
|
||||
@@ -646,21 +731,21 @@ grep -E 'error:|PASS|FAIL|succeeded|failed' /tmp/test_out.txt
|
||||
tail -15 /tmp/test_out.txt
|
||||
```
|
||||
|
||||
Do **not** use `pio test … | tail -N` — it discards build errors and shows stale cached results. Do **not** use `pio test … | grep` — line-buffering makes the terminal appear hung until the process exits. Redirect to a file first, then grep.
|
||||
Do **not** pipe `pio test` - line-buffering makes the terminal appear hung and hides build errors.
|
||||
|
||||
Simulation testing: `bin/test-simulator.sh`
|
||||
|
||||
Quick entry point for new test modules: `test/README.md` (native unit-test authoring guide, skeleton, pitfalls, and setup checklist).
|
||||
|
||||
### Hardware-in-the-loop tests (`mcp-server/tests/`)
|
||||
### Hardware-in-the-loop tests ([meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp))
|
||||
|
||||
Separate pytest suite that exercises real USB-connected Meshtastic devices. See the **MCP Server & Hardware Test Harness** section below for invocation, tier layout, and agent usage rules.
|
||||
Separate pytest suite that exercises real USB-connected Meshtastic devices. It now lives in the standalone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) repo, run against a firmware checkout via `MESHTASTIC_FIRMWARE_ROOT`. See the **MCP Server & Hardware Test Harness** section below for invocation, tier layout, and agent usage rules.
|
||||
|
||||
## MCP Server & Hardware Test Harness
|
||||
|
||||
The `mcp-server/` directory houses a firmware-aware [MCP](https://modelcontextprotocol.io/) server plus a pytest-based integration suite. AI agents that speak MCP get a well-defined tool surface for flashing, configuring, and inspecting physical Meshtastic devices — use it instead of hand-rolling `pio` or `meshtastic --port` calls where possible. `mcp-server/README.md` is the operator-facing setup doc; this section is the agent-facing usage contract.
|
||||
The firmware-aware [MCP](https://modelcontextprotocol.io/) server plus its pytest-based integration suite now live in the standalone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) repo. AI agents that speak MCP get a well-defined tool surface for flashing, configuring, and inspecting physical Meshtastic devices - use it instead of hand-rolling `pio` or `meshtastic --port` calls where possible. The meshtastic-mcp repo's README is the operator-facing setup doc; this section is the agent-facing usage contract.
|
||||
|
||||
The repo registers the server via `.mcp.json` at the repo root — Claude Code picks it up automatically once `mcp-server/.venv/` is built (`cd mcp-server && python3 -m venv .venv && .venv/bin/pip install -e '.[test]'`).
|
||||
The repo registers the server via `.mcp.json` at the repo root - Claude Code / Copilot pick it up automatically and run it through `uvx --from git+https://github.com/meshtastic/meshtastic-mcp meshtastic-mcp`, so the MCP tools work with no local build. To run the pytest hardware harness instead, clone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) and point `MESHTASTIC_FIRMWARE_ROOT` at this firmware checkout.
|
||||
|
||||
### When to use which surface
|
||||
|
||||
@@ -668,11 +753,11 @@ The repo registers the server via `.mcp.json` at the repo root — Claude Code p
|
||||
| ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| Find a connected device | `mcp__meshtastic__list_devices` |
|
||||
| Read a live node's config/state | `mcp__meshtastic__device_info`, `list_nodes`, `get_config` |
|
||||
| Mutate a device (owner, region, channels, reboot) | `set_owner`, `set_config`, `set_channel_url`, `reboot`, `shutdown`, `factory_reset` — all require `confirm=True` |
|
||||
| Mutate a device (owner, region, channels, reboot) | `set_owner`, `set_config`, `set_channel_url`, `reboot`, `shutdown`, `factory_reset` - all require `confirm=True` |
|
||||
| Flash firmware to a variant | `pio_flash` (any arch) or `erase_and_flash` (ESP32 factory install) |
|
||||
| Stream serial logs while debugging | `serial_open` → `serial_read` loop → `serial_close` |
|
||||
| Administer `userPrefs.jsonc` build-time constants | `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest` |
|
||||
| Run the regression suite | `./mcp-server/run-tests.sh` (or `/test` slash command) |
|
||||
| Run the regression suite | `./run-tests.sh` from a meshtastic-mcp checkout (or `/test` slash command) |
|
||||
| Diagnose a specific device | `/diagnose [role]` slash command (read-only) |
|
||||
| Triage a flaky test | `/repro <node-id> [count]` slash command |
|
||||
|
||||
@@ -680,7 +765,7 @@ The repo registers the server via `.mcp.json` at the repo root — Claude Code p
|
||||
|
||||
### MCP tool surface (43 tools)
|
||||
|
||||
Grouped by purpose. Full argument shapes in `mcp-server/README.md`; a few high-value signatures are called out here.
|
||||
Grouped by purpose. Full argument shapes in the meshtastic-mcp repo's README; a few high-value signatures are called out here.
|
||||
|
||||
- **Discovery & metadata**: `list_devices`, `list_boards`, `get_board`
|
||||
- **Build & flash**: `build`, `clean`, `pio_flash`, `erase_and_flash` (ESP32 only), `update_flash` (ESP32 OTA), `touch_1200bps`
|
||||
@@ -690,48 +775,49 @@ Grouped by purpose. Full argument shapes in `mcp-server/README.md`; a few high-v
|
||||
- **userPrefs admin** (build-time constants, not runtime config): `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest`, `userprefs_testing_profile`
|
||||
- **Vendor escape hatches**: `esptool_chip_info`, `esptool_erase_flash`, `esptool_raw`, `nrfutil_dfu`, `nrfutil_raw`, `picotool_info`, `picotool_load`, `picotool_raw`
|
||||
- **USB power control** (via `uhubctl`, per-port PPPS toggle): `uhubctl_list` (read-only), `uhubctl_power(action='on'|'off', confirm=True)`, `uhubctl_cycle(delay_s, confirm=True)`. Target by raw `(location, port)` or by `role` (`"nrf52"`, `"esp32s3"`); role lookup checks `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` + `_PORT_<ROLE>` env vars first, falls back to VID auto-detection.
|
||||
- **Observability** (UI tier + operator ad-hoc): `capture_screen(role, ocr=True)` — grabs a USB-webcam frame of the device OLED and optionally OCRs it. Requires `mcp-server[ui]` extras (`opencv-python-headless`, `easyocr`) and `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` env var; falls through to a 1×1 black PNG `NullBackend` when unconfigured.
|
||||
- **Observability** (UI tier + operator ad-hoc): `capture_screen(role, ocr=True)` - grabs a USB-webcam frame of the device OLED and optionally OCRs it. Requires `meshtastic-mcp[ui]` extras (`opencv-python-headless`, `easyocr`) and `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` env var; falls through to a 1×1 black PNG `NullBackend` when unconfigured.
|
||||
|
||||
`confirm=True` is a tool-level gate on top of whatever permission prompt your MCP host shows. **Don't bypass it** by asking the host to auto-approve — it exists specifically because MCP hosts sometimes remember "always allow this tool" and that's dangerous for `factory_reset`, `erase_and_flash`, `uhubctl_power(action='off')`, and `uhubctl_cycle`.
|
||||
`confirm=True` is a tool-level gate on top of whatever permission prompt your MCP host shows. **Don't bypass it** by asking the host to auto-approve - it exists specifically because MCP hosts sometimes remember "always allow this tool" and that's dangerous for `factory_reset`, `erase_and_flash`, `uhubctl_power(action='off')`, and `uhubctl_cycle`.
|
||||
|
||||
**TCP / native-host nodes.** Setting `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` makes `list_devices` surface a `meshtasticd` daemon (e.g. the `native-macos` build) as a synthetic `tcp://host:port` entry, and `connect()` routes through `meshtastic.tcp_interface.TCPInterface` instead of `SerialInterface`. Every read/write/admin tool that flows through `connect()` works against the daemon transparently. USB-only tools (`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`, `serial_open`, `esptool_*`, `nrfutil_*`, `picotool_*`) raise a clear `ConnectionError` when handed a `tcp://` port; `pio_flash` against a `native*` env raises a `FlashError` (no upload step — use `build` and run the binary directly). The pytest harness still assumes USB-attached devices per role; TCP-aware fixtures are deferred. See `mcp-server/README.md` § "TCP / native-host nodes".
|
||||
**TCP / native-host nodes.** Setting `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` makes `list_devices` surface a `meshtasticd` daemon (e.g. the `native-macos` build) as a synthetic `tcp://host:port` entry, and `connect()` routes through `meshtastic.tcp_interface.TCPInterface` instead of `SerialInterface`. Every read/write/admin tool that flows through `connect()` works against the daemon transparently. USB-only tools (`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`, `serial_open`, `esptool_*`, `nrfutil_*`, `picotool_*`) raise a clear `ConnectionError` when handed a `tcp://` port; `pio_flash` against a `native*` env raises a `FlashError` (no upload step - use `build` and run the binary directly). The pytest harness still assumes USB-attached devices per role; TCP-aware fixtures are deferred. See the meshtastic-mcp repo's README § "TCP / native-host nodes".
|
||||
|
||||
### Hardware test suite (`mcp-server/run-tests.sh`)
|
||||
### Hardware test suite (`run-tests.sh`, from a meshtastic-mcp checkout)
|
||||
|
||||
The wrapper auto-detects connected devices (VID → role map: `0x239A` → `nrf52`, `0x303A`/`0x10C4` → `esp32s3`), maps each role to a PlatformIO env (`nrf52` → `rak4631`, `esp32s3` → `heltec-v3`, overridable via `MESHTASTIC_MCP_ENV_<ROLE>`), then invokes pytest. Zero pre-flight config needed from the operator.
|
||||
|
||||
Suite tiers (collected + run in this order via `pytest_collection_modifyitems`):
|
||||
|
||||
1. `tests/unit/` — pure Python (boards parse, pio wrapper, userPrefs parse, testing profile, uhubctl parser). No hardware.
|
||||
2. `tests/test_00_bake.py` — flashes each detected device with current `userPrefs.jsonc` merged with the session's test profile. Has its own skip-if-already-baked check comparing region + primary channel to the session profile; skips cheaply on warm devices.
|
||||
3. `tests/mesh/` — multi-device mesh: bidirectional send, broadcast delivery, direct-with-ACK, mesh formation within 60s. Parametrized `[nrf52->esp32s3]` and `[esp32s3->nrf52]`. Includes `test_peer_offline_recovery` which uses uhubctl to physically power off one peer mid-conversation (requires uhubctl; skips without).
|
||||
4. `tests/telemetry/` — `DEVICE_METRICS_APP` broadcast timing.
|
||||
5. `tests/monitor/` — boot-log panic check.
|
||||
6. `tests/recovery/` — `uhubctl` power-cycle round-trip + NVS persistence across hard reset. Requires `uhubctl` installed and a PPPS-capable hub; entire tier auto-skips otherwise.
|
||||
7. `tests/ui/` — input-broker-driven screen navigation with camera + OCR evidence.
|
||||
8. `tests/fleet/` — PSK seed session isolation.
|
||||
9. `tests/admin/` — channel URL roundtrip, owner persistence across reboot.
|
||||
10. `tests/provisioning/` — region + modem + slot bake, admin key presence, `UNSET` region blocks TX, userPrefs survive factory reset.
|
||||
1. `tests/unit/` - pure Python (boards parse, pio wrapper, userPrefs parse, testing profile, uhubctl parser). No hardware.
|
||||
2. `tests/test_00_bake.py` - flashes each detected device with current `userPrefs.jsonc` merged with the session's test profile. Has its own skip-if-already-baked check comparing region + primary channel to the session profile; skips cheaply on warm devices.
|
||||
3. `tests/mesh/` - multi-device mesh: bidirectional send, broadcast delivery, direct-with-ACK, mesh formation within 60s. Parametrized `[nrf52->esp32s3]` and `[esp32s3->nrf52]`. Includes `test_peer_offline_recovery` which uses uhubctl to physically power off one peer mid-conversation (requires uhubctl; skips without).
|
||||
4. `tests/telemetry/` - `DEVICE_METRICS_APP` broadcast timing.
|
||||
5. `tests/monitor/` - boot-log panic check.
|
||||
6. `tests/recovery/` - `uhubctl` power-cycle round-trip + NVS persistence across hard reset. Requires `uhubctl` installed and a PPPS-capable hub; entire tier auto-skips otherwise.
|
||||
7. `tests/ui/` - input-broker-driven screen navigation with camera + OCR evidence.
|
||||
8. `tests/fleet/` - PSK seed session isolation.
|
||||
9. `tests/admin/` - channel URL roundtrip, owner persistence across reboot.
|
||||
10. `tests/provisioning/` - region + modem + slot bake, admin key presence, `UNSET` region blocks TX, userPrefs survive factory reset.
|
||||
|
||||
Invocation patterns:
|
||||
|
||||
```bash
|
||||
./mcp-server/run-tests.sh # full suite (auto-bake-if-needed)
|
||||
./mcp-server/run-tests.sh --force-bake # reflash before testing
|
||||
./mcp-server/run-tests.sh --assume-baked # skip bake (caller vouches for device state)
|
||||
./mcp-server/run-tests.sh tests/mesh # one tier
|
||||
./mcp-server/run-tests.sh tests/mesh/test_direct_with_ack.py # one file
|
||||
./mcp-server/run-tests.sh -k telemetry # name filter
|
||||
# run from a meshtastic-mcp checkout, with MESHTASTIC_FIRMWARE_ROOT=/path/to/firmware
|
||||
./run-tests.sh # full suite (auto-bake-if-needed)
|
||||
./run-tests.sh --force-bake # reflash before testing
|
||||
./run-tests.sh --assume-baked # skip bake (caller vouches for device state)
|
||||
./run-tests.sh tests/mesh # one tier
|
||||
./run-tests.sh tests/mesh/test_direct_with_ack.py # one file
|
||||
./run-tests.sh -k telemetry # name filter
|
||||
```
|
||||
|
||||
**No hardware detected?** The wrapper auto-narrows to `tests/unit/` only and prints `detected hub : (none)` in the pre-flight header. Agents interpreting the output should call this out explicitly — a 52-test green run without hardware is qualitatively different from a 12-unit-test green run.
|
||||
**No hardware detected?** The wrapper auto-narrows to `tests/unit/` only and prints `detected hub : (none)` in the pre-flight header. Agents interpreting the output should call this out explicitly - a 52-test green run without hardware is qualitatively different from a 12-unit-test green run.
|
||||
|
||||
**Artifacts every run produces:**
|
||||
|
||||
- `mcp-server/tests/report.html` — self-contained pytest-html. Each test gets a `Meshtastic debug` section with the tail of firmware log + device state dump. **Open this first** on failures; it's the canonical evidence source.
|
||||
- `mcp-server/tests/junit.xml` — CI-parseable.
|
||||
- `mcp-server/tests/reportlog.jsonl` — pytest-reportlog stream (`$report_type` keyed JSONL). Consumed by the live TUI.
|
||||
- `mcp-server/tests/fwlog.jsonl` — firmware log mirror from the `meshtastic.log.line` pubsub topic. Populated by the `_firmware_log_stream` autouse session fixture.
|
||||
- `tests/report.html` - self-contained pytest-html. Each test gets a `Meshtastic debug` section with the tail of firmware log + device state dump. **Open this first** on failures; it's the canonical evidence source.
|
||||
- `tests/junit.xml` - CI-parseable.
|
||||
- `tests/reportlog.jsonl` - pytest-reportlog stream (`$report_type` keyed JSONL). Consumed by the live TUI.
|
||||
- `tests/fwlog.jsonl` - firmware log mirror from the `meshtastic.log.line` pubsub topic. Populated by the `_firmware_log_stream` autouse session fixture.
|
||||
|
||||
### Live TUI (`meshtastic-mcp-test-tui`)
|
||||
|
||||
@@ -751,7 +837,7 @@ A Textual-based live view that wraps `run-tests.sh`. Tails reportlog for per-tes
|
||||
Launch:
|
||||
|
||||
```bash
|
||||
cd mcp-server
|
||||
# from a meshtastic-mcp checkout (MESHTASTIC_FIRMWARE_ROOT set)
|
||||
.venv/bin/meshtastic-mcp-test-tui # full suite
|
||||
.venv/bin/meshtastic-mcp-test-tui tests/mesh # args pass through to pytest
|
||||
```
|
||||
@@ -776,29 +862,29 @@ House rules for agents running these prompts:
|
||||
|
||||
### Key fixtures (test authors + agents debugging)
|
||||
|
||||
`mcp-server/tests/conftest.py` provides:
|
||||
`tests/conftest.py` (in the meshtastic-mcp checkout) provides:
|
||||
|
||||
- **`_session_userprefs`** (autouse session) — snapshots `userPrefs.jsonc` at session start, merges the session test profile via `userprefs.merge_active(test_profile)`, restores at teardown. Four layers of safety: pytest teardown + `atexit` + sidecar file (`userPrefs.jsonc.mcp-session-bak`) + startup self-heal in `run-tests.sh`. **Do not edit `userPrefs.jsonc` from inside a test.**
|
||||
- **`_firmware_log_stream`** (autouse session) — subscribes to `meshtastic.log.line` pubsub on every connected `SerialInterface` and mirrors lines to `tests/fwlog.jsonl`. Drives the TUI firmware-log pane.
|
||||
- **`_debug_log_buffer`** (autouse per-test) — captures last 200 firmware log lines + device state for attachment to the pytest-html `Meshtastic debug` section on failure.
|
||||
- **`hub_devices`** (session) — `dict[role, SerialInterface]` with session-long exclusive port locks. Reason the TUI's device poller is gated to startup + post-run only.
|
||||
- **`baked_mesh`** — parametrized mesh-pair fixture; depends on `test_00_bake`. `pytest_generate_tests` in `conftest.py` auto-generates `[nrf52->esp32s3]` and `[esp32s3->nrf52]` variants.
|
||||
- **`test_profile`** — session-scoped dict: region, primary channel, admin key, PSK seed. Derived from `MESHTASTIC_MCP_SEED` (defaults to `mcp-<user>-<host>`).
|
||||
- **`_session_userprefs`** (autouse session) - snapshots `userPrefs.jsonc` at session start, merges the session test profile via `userprefs.merge_active(test_profile)`, restores at teardown. Four layers of safety: pytest teardown + `atexit` + sidecar file (`userPrefs.jsonc.mcp-session-bak`) + startup self-heal in `run-tests.sh`. **Do not edit `userPrefs.jsonc` from inside a test.**
|
||||
- **`_firmware_log_stream`** (autouse session) - subscribes to `meshtastic.log.line` pubsub on every connected `SerialInterface` and mirrors lines to `tests/fwlog.jsonl`. Drives the TUI firmware-log pane.
|
||||
- **`_debug_log_buffer`** (autouse per-test) - captures last 200 firmware log lines + device state for attachment to the pytest-html `Meshtastic debug` section on failure.
|
||||
- **`hub_devices`** (session) - `dict[role, SerialInterface]` with session-long exclusive port locks. Reason the TUI's device poller is gated to startup + post-run only.
|
||||
- **`baked_mesh`** - parametrized mesh-pair fixture; depends on `test_00_bake`. `pytest_generate_tests` in `conftest.py` auto-generates `[nrf52->esp32s3]` and `[esp32s3->nrf52]` variants.
|
||||
- **`test_profile`** - session-scoped dict: region, primary channel, admin key, PSK seed. Derived from `MESHTASTIC_MCP_SEED` (defaults to `mcp-<user>-<host>`).
|
||||
|
||||
### Firmware integration points tied to the test harness
|
||||
|
||||
Two firmware changes exist specifically so the test harness works reliably. **Keep these in mind when touching related code.**
|
||||
|
||||
- **`src/mesh/StreamAPI.cpp` + `StreamAPI.h`** — `emitLogRecord` uses a dedicated `fromRadioScratchLog` + `txBufLog` pair and a `concurrency::Lock streamLock`. Before this fix, `debug_log_api_enabled=true` would tear `FromRadio` protobufs on the serial transport because `emitTxBuffer` and `emitLogRecord` shared a single scratch buffer. The conftest enables the log stream session-wide; without this fix the device would corrupt its own FromRadio replies mid-session.
|
||||
- **`src/mesh/PhoneAPI.cpp`** — `ToRadio` `Heartbeat(nonce=1)` triggers `nodeInfoModule->sendOurNodeInfo(NODENUM_BROADCAST, true, 0, true)` for serial clients, mirroring the pre-existing behavior for TCP/UDP clients in `PacketAPI.cpp`. The mesh tests rely on this to force a NodeInfo broadcast right after connect so the peer discovers them before the test's first assertion.
|
||||
- **`src/mesh/StreamAPI.cpp` + `StreamAPI.h`** - `emitLogRecord` uses a dedicated `fromRadioScratchLog` + `txBufLog` pair and a `concurrency::Lock streamLock`. Before this fix, `debug_log_api_enabled=true` would tear `FromRadio` protobufs on the serial transport because `emitTxBuffer` and `emitLogRecord` shared a single scratch buffer. The conftest enables the log stream session-wide; without this fix the device would corrupt its own FromRadio replies mid-session.
|
||||
- **`src/mesh/PhoneAPI.cpp`** - `ToRadio` `Heartbeat(nonce=1)` triggers `nodeInfoModule->sendOurNodeInfo(NODENUM_BROADCAST, true, 0, true)` for serial clients, mirroring the pre-existing behavior for TCP/UDP clients in `PacketAPI.cpp`. The mesh tests rely on this to force a NodeInfo broadcast right after connect so the peer discovers them before the test's first assertion.
|
||||
|
||||
If you're modifying `StreamAPI`, `PhoneAPI`, `NodeInfoModule`, or `userPrefs` flow, run `./mcp-server/run-tests.sh` at minimum before asking for review.
|
||||
If you're modifying `StreamAPI`, `PhoneAPI`, `NodeInfoModule`, or `userPrefs` flow, run `./run-tests.sh` (from a meshtastic-mcp checkout, with `MESHTASTIC_FIRMWARE_ROOT` pointed here) at minimum before asking for review.
|
||||
|
||||
### Recovery playbooks
|
||||
|
||||
| Symptom | First check | Fix |
|
||||
| --------------------------------------------------------------------------------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `userPrefs.jsonc` dirty after test run | `git status --porcelain userPrefs.jsonc` | If non-empty, re-run `./mcp-server/run-tests.sh` once — the pre-flight self-heal restores from sidecar. If still dirty, `git checkout userPrefs.jsonc`. |
|
||||
| `userPrefs.jsonc` dirty after test run | `git status --porcelain userPrefs.jsonc` | If non-empty, re-run `./run-tests.sh` (from a meshtastic-mcp checkout) once - the pre-flight self-heal restores from sidecar. If still dirty, `git checkout userPrefs.jsonc`. |
|
||||
| Port busy / wedged CP2102 on macOS | `lsof /dev/cu.usbserial-0001` | Kill the holder. USB replug if the kernel still reports busy. Often a stale `pio device monitor` or zombie `meshtastic_mcp` process. |
|
||||
| nRF52 appears unresponsive | `list_devices` shows VID `0x239A` but `device_info` times out | `touch_1200bps(port=...)` drops it into the DFU bootloader → `pio_flash` re-installs. |
|
||||
| Device fully wedged (Guru Meditation, frozen CDC, no DFU) | `list_devices` shows the VID but every admin call times out | `uhubctl_cycle(role="nrf52", confirm=True)` hard-power-cycles the port via USB hub PPPS. `baked_single`'s auto-recovery hook does this once automatically if uhubctl is installed. Falls back to physical replug if no PPPS hub. |
|
||||
@@ -808,17 +894,17 @@ If you're modifying `StreamAPI`, `PhoneAPI`, `NodeInfoModule`, or `userPrefs` fl
|
||||
| Entire `tests/recovery/` tier skipped | `command -v uhubctl` | Expected if `uhubctl` isn't on PATH. Install via `brew install uhubctl` (macOS) or `apt install uhubctl` (Debian/Ubuntu). Also skips if no hub advertises PPPS. |
|
||||
| Entire `tests/ui/` tier skipped ("firmware not baked with USERPREFS_UI_TEST_LOG") | reportlog.jsonl for the skip reason | Re-run with `--force-bake` so the UI-log macro gets compiled into the fresh firmware. First run after the Round-3 landing always re-bakes. |
|
||||
| `tests/ui/` runs but captures are all 1×1 black PNGs | `MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3` | Env var not set → `NullBackend`. Point a USB webcam at the heltec-v3 OLED and set the device index; `.venv/bin/python -c "import cv2; [print(i, cv2.VideoCapture(i).read()[0]) for i in range(5)]"` discovers it. |
|
||||
| Tests fail only on first attempt then pass on rerun | — | State leak from a prior session. Run with `--force-bake` to reset to a known state. |
|
||||
| Tests fail only on first attempt then pass on rerun | - | State leak from a prior session. Run with `--force-bake` to reset to a known state. |
|
||||
|
||||
### Never do these without asking
|
||||
|
||||
- `factory_reset` — wipes node identity; regenerates PKI keypair. Mesh peers will reject old DMs until re-exchange. Legitimate only when the operator explicitly wants it.
|
||||
- `erase_and_flash` — full chip erase; destroys all on-device state.
|
||||
- `esptool_erase_flash` / `esptool_raw` write/erase — bypasses pio's safety chain.
|
||||
- `set_config` on `lora.region` — changes regulatory domain; requires physical-location context the operator has and the agent doesn't.
|
||||
- `reboot` / `shutdown` mid-test — breaks fixture invariants.
|
||||
- `factory_reset` - wipes node identity; regenerates PKI keypair. Mesh peers will reject old DMs until re-exchange. Legitimate only when the operator explicitly wants it.
|
||||
- `erase_and_flash` - full chip erase; destroys all on-device state.
|
||||
- `esptool_erase_flash` / `esptool_raw` write/erase - bypasses pio's safety chain.
|
||||
- `set_config` on `lora.region` - changes regulatory domain; requires physical-location context the operator has and the agent doesn't.
|
||||
- `reboot` / `shutdown` mid-test - breaks fixture invariants.
|
||||
- `push -f`, `rebase -i`, `reset --hard`, or any history-rewriting git operation.
|
||||
- Clicking computer-use tools on web links in Mail/Messages/PDFs — open URLs via the claude-in-chrome MCP so the extension's link-safety checks apply.
|
||||
- Clicking computer-use tools on web links in Mail/Messages/PDFs - open URLs via the claude-in-chrome MCP so the extension's link-safety checks apply.
|
||||
|
||||
## Resources
|
||||
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
---
|
||||
mode: agent
|
||||
description: Device health report via the meshtastic MCP tools (Copilot equivalent of the Claude Code /diagnose slash command)
|
||||
---
|
||||
|
||||
# `/mcp-diagnose` — device health report
|
||||
|
||||
Equivalent of `.claude/commands/diagnose.md`. Use when the operator asks to "check the devices", "what's the mesh looking like", "is nrf52 alive", etc.
|
||||
|
||||
This prompt assumes the meshtastic MCP server is registered with your VS Code Copilot agent. If it isn't, fall back to running `./mcp-server/run-tests.sh tests/unit` plus a short `device_info` script via the terminal.
|
||||
|
||||
## What to do
|
||||
|
||||
1. **Enumerate hardware** via the `list_devices` MCP tool (with `include_unknown=True`). For each entry where `likely_meshtastic=True`, capture `port`, `vid`, `pid`, `description`.
|
||||
|
||||
2. **Apply the operator's filter** (if any):
|
||||
- No filter → every likely-meshtastic device.
|
||||
- `nrf52` → `vid == 0x239a`
|
||||
- `esp32s3` → `vid == 0x303a` or `vid == 0x10c4`
|
||||
- A `/dev/cu.*` path → only that port.
|
||||
- Anything else → substring match on port.
|
||||
|
||||
3. **For each selected device, in sequence (don't parallelize — SerialInterface holds an exclusive port lock):**
|
||||
- `device_info(port=<p>)` → `my_node_num`, `long_name`, `short_name`, `firmware_version`, `hw_model`, `region`, `num_nodes`, `primary_channel`
|
||||
- `list_nodes(port=<p>)` → peer count, which peers have `publicKey`, SNR/RSSI distribution
|
||||
- `get_config(section="lora", port=<p>)` → region, preset, channel_num, tx_power, hop_limit
|
||||
- If anything looks off (can't connect, `num_nodes` wrong, missing `firmware_version`), open a short firmware-log window: `serial_open(port=<p>, env=<inferred>)`, wait 3 seconds, `serial_read(session_id, max_lines=100)`, `serial_close(session_id)`. Infer env from VID (0x239a → `rak4631`, 0x303a/0x10c4 → `heltec-v3`) unless an `MESHTASTIC_MCP_ENV_<ROLE>` env var overrides it.
|
||||
|
||||
4. **Hub health** (call once, not per-device): `uhubctl_list()` — enumerates every USB hub the host sees. Cross-reference each Meshtastic device's VID to find which hub + port it's on. Flag in the report if:
|
||||
- No hub advertises `ppps=true` → `tests/recovery/` can't run; hard-recovery via `uhubctl_cycle` isn't available.
|
||||
- A Meshtastic device is on a non-PPPS hub → note it; moving to a PPPS hub unlocks auto-recovery.
|
||||
- `uhubctl_list` raises `ConfigError: uhubctl not found` → report as "uhubctl not installed"; don't treat as a device fault.
|
||||
|
||||
5. **Render per-device report** as a compact block:
|
||||
|
||||
```text
|
||||
[nrf52 @ /dev/cu.usbmodem1101] fw=2.7.23.bce2825, hw=RAK4631
|
||||
owner : Meshtastic 40eb / 40eb
|
||||
region/band : US, channel 88, LONG_FAST
|
||||
tx_power : 30 dBm, hop_limit=3
|
||||
peers : 1 (esp32s3 0x433c2428, pubkey ✓, SNR 6.0 / RSSI -24 dBm)
|
||||
primary ch : McpTest
|
||||
hub : 1-1.3 port 2 (PPPS, uhubctl-controllable)
|
||||
firmware : no panics in last 3s
|
||||
```
|
||||
|
||||
Flag abnormalities inline with `⚠︎ <short reason>` — missing pubkey on a known peer, region UNSET, mismatched channel name, device on non-PPPS hub, etc.
|
||||
|
||||
6. **Cross-device correlation** (when >1 device selected):
|
||||
- Do both see each other in `nodesByNum`?
|
||||
- Do `region`, `channel_num`, `modem_preset` match across devices?
|
||||
- Do the primary channel names match? (Different name → different PSK → no decode.)
|
||||
|
||||
7. **Suggest next steps only for recognizable failure modes**, never speculatively:
|
||||
- Stale PKI one-way → "`/mcp-test tests/mesh/test_direct_with_ack.py` — the test's retry+nodeinfo-ping heals this."
|
||||
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
|
||||
- Device unreachable, DFU reachable → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds and the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
|
||||
- CP2102-wedged-driver on macOS → see `run-tests.sh` notes.
|
||||
|
||||
## Hard constraints
|
||||
|
||||
- **Read-only.** No `set_config`, no `reboot`, no `factory_reset`, no `flash`. If the operator wants mutation, they'll escalate explicitly.
|
||||
- **Open/query/close per device.** Never hold multiple SerialInterfaces to the same port. The port lock is exclusive.
|
||||
- **Don't infer env beyond the VID map** — if the operator has an unusual board, ask them which env to use rather than guessing.
|
||||
@@ -1,68 +0,0 @@
|
||||
---
|
||||
mode: agent
|
||||
description: Re-run a specific test N times to triage flakes; diff firmware logs between passes and failures (Copilot equivalent of the Claude Code /repro slash command)
|
||||
---
|
||||
|
||||
# `/mcp-repro` — flakiness triage for one test
|
||||
|
||||
Equivalent of `.claude/commands/repro.md`. Use when the operator says "that one test is flaky — dig in", "repro the direct_with_ack failure", "why does X sometimes fail?".
|
||||
|
||||
## What to do
|
||||
|
||||
1. **Parse the operator's input** into two pieces:
|
||||
- **Test identifier** — either a pytest node id (has `::` or starts with `tests/`) or a `-k`-style filter (plain substring like `direct_with_ack`).
|
||||
- **Count** — integer, default `5`, cap at `20`. If the operator asks for 50, negotiate down and explain (airtime + USB wear).
|
||||
|
||||
2. **Sanity-check the hub** via the `list_devices` MCP tool. If the test name references `nrf52` or `esp32s3` and the matching VID isn't present, stop and report — re-running won't help.
|
||||
|
||||
3. **Loop** N times. Each iteration:
|
||||
|
||||
```bash
|
||||
./mcp-server/run-tests.sh <test-id> --tb=short -p no:cacheprovider
|
||||
```
|
||||
|
||||
`-p no:cacheprovider` keeps pytest from caching anything between iterations. Capture: exit code, duration, and (on failure) the `Meshtastic debug` firmware-log section from `mcp-server/tests/report.html`.
|
||||
|
||||
4. **Tally** results as you go:
|
||||
|
||||
```text
|
||||
attempt 1: PASS (42s)
|
||||
attempt 2: FAIL (128s) ← fw log captured
|
||||
attempt 3: PASS (39s)
|
||||
attempt 4: FAIL (121s)
|
||||
attempt 5: PASS (41s)
|
||||
--------------------------------------------------
|
||||
pass rate: 3/5 (60%) | mean duration: 74s
|
||||
```
|
||||
|
||||
5. **On mixed outcomes, diff the firmware logs** between one representative pass and one representative fail. Focus on:
|
||||
- Error-level lines present only in failures (`PKI_UNKNOWN_PUBKEY`, `Alloc an err=`, `Skip send`, `No suitable channel`, `NAK`)
|
||||
- Timing around the assertion point (broadcast sent? ACK received? retry fired?)
|
||||
- Device-state fields that changed between attempts
|
||||
|
||||
Surface the top 3 differences as a compact "passes when / fails when" table with uptime timestamps. Don't dump full logs.
|
||||
|
||||
6. **Classify** the flake into one of:
|
||||
- **LoRa airtime collision** — pass rate improves with fewer concurrent transmitters. Suggest a `time.sleep` gap or retry bump in the test body.
|
||||
- **PKI key staleness** — first attempt fails, subsequent ones pass; existing retry-loop pattern in `test_direct_with_ack.py` is the fix.
|
||||
- **NodeInfo cooldown** — `Skip send NodeInfo since we sent it <600s ago` in fail-only logs; needs a `broadcast_nodeinfo_ping()` warmup.
|
||||
- **Hardware-specific** — one direction consistently fails, firmware versions differ, CP2102 driver wedged, etc. For a device wedged past `touch_1200bps`, recommend `uhubctl_cycle(role=..., confirm=True)` to hard-power-cycle its hub port (requires `uhubctl` installed).
|
||||
- **Device went dark mid-run** — fails from some iteration onward and never recovers; firmware log stops arriving. Almost always a Guru crash with frozen CDC. Recommend `uhubctl_cycle` before the next iteration; escalate to replug if that also fails.
|
||||
- **Unknown** — say so. Don't invent a root cause.
|
||||
|
||||
7. **Report back** with:
|
||||
- Pass rate + mean duration.
|
||||
- Classification + the specific log evidence for it.
|
||||
- A concrete next step (tighter assertion, more retries, open `/mcp-diagnose`, file a bug, nothing).
|
||||
|
||||
## Examples
|
||||
|
||||
- `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip[esp32s3->nrf52] 10` — 10 runs of that parametrized case.
|
||||
- `broadcast_delivers` — no `::`, no `tests/`; treat as `-k broadcast_delivers`; runs every match 5 times.
|
||||
- `tests/telemetry/test_device_telemetry_broadcast.py 3` — shorter count for a slow test.
|
||||
|
||||
## Notes
|
||||
|
||||
- If the FIRST attempt fails and the rest pass, that's a state-leak signature — suggest starting from `--force-bake` or a clean device state rather than chasing the first-failure firmware logs.
|
||||
- If ALL N fail, this isn't a flake — it's a regression. Say so, stop iterating, escalate to `/mcp-test` for full-suite context.
|
||||
- Don't rebuild firmware during triage. Flakes that only reproduce under different firmware belong in a separate session with a plan.
|
||||
@@ -1,57 +0,0 @@
|
||||
---
|
||||
mode: agent
|
||||
description: Run the mcp-server test suite and interpret results (Copilot equivalent of the Claude Code /test slash command)
|
||||
---
|
||||
|
||||
# `/mcp-test` — mcp-server test runner with interpretation
|
||||
|
||||
Equivalent of the Claude Code `/test` slash command in `.claude/commands/test.md`. Use this when the operator asks you to "run the tests", "check the mcp test suite", "run the mesh tests", etc.
|
||||
|
||||
## What to do
|
||||
|
||||
1. **Invoke the wrapper** from the firmware repo root:
|
||||
|
||||
```bash
|
||||
./mcp-server/run-tests.sh [pytest-args]
|
||||
```
|
||||
|
||||
If the operator specified a subset (e.g. "just the mesh tests"), pass it through as `tests/mesh` or a pytest `-k filter`. If they said nothing, use the wrapper's defaults (full suite with pytest-html report).
|
||||
|
||||
The wrapper auto-detects connected Meshtastic devices, maps each to its PlatformIO env, exports the required env vars, and invokes pytest. Zero pre-flight config needed from the operator.
|
||||
|
||||
2. **Read the pre-flight header** (first few lines of wrapper output). The `detected hub :` line lists role → port → env mappings. If it reads `(none)`, the wrapper narrowed to `tests/unit` only — call that out explicitly so the operator knows hardware tiers were skipped.
|
||||
|
||||
3. **On pass**: one-line summary like `N passed, M skipped in <duration>`. Don't enumerate test names. DO mention any non-placeholder SKIPs and name the cause:
|
||||
- `"role not present on hub"` → device unplugged; operator should reconnect.
|
||||
- `"firmware not baked with USERPREFS_UI_TEST_LOG"` → tests/ui skipped; the UI-log compile macro isn't in the baked firmware. Suggest `--force-bake`.
|
||||
- `"uhubctl not installed"` → tests/recovery + `test_peer_offline_recovery` skipped. Suggest `brew install uhubctl` / `apt install uhubctl`.
|
||||
- `"no PPPS-capable hubs detected"` → tests/recovery skipped because the attached hub doesn't support per-port power switching; won't run on that setup.
|
||||
- `"opencv-python-headless is not installed"` → tests/ui auto-deselected by `run-tests.sh`. Suggest `pip install -e 'mcp-server/.[ui]'`.
|
||||
|
||||
4. **On failure**: open `mcp-server/tests/report.html` (pytest-html output, self-contained) and extract the `Meshtastic debug` section for each failed test. That section includes a firmware log stream (last 200 lines) and device state dump. For each failure, summarise:
|
||||
- test name
|
||||
- one-line assertion message
|
||||
- the specific firmware log lines that explain why (look for `PKI_UNKNOWN_PUBKEY`, `Skip send NodeInfo`, `Error=`, `Guru Meditation`, `assertion failed`, `No suitable channel`)
|
||||
- for UI-tier failures also check `mcp-server/tests/ui_captures/<session>/<test>/transcript.md` (per-step frame + OCR)
|
||||
|
||||
5. **Classify each failure** as one of:
|
||||
- **Transient flake** — LoRa collision, first-attempt NAK with self-heal pattern, timing-sensitive assertion. Suggest `/mcp-repro <test-id>` to confirm.
|
||||
- **Environmental** — device unreachable, port busy, CP2102 driver wedged on macOS. Suggest recovery in escalation order: (a) replug USB, (b) `touch_1200bps` + `pio_flash` for nRF52 DFU, (c) `uhubctl_cycle(role=..., confirm=True)` for a device wedged past DFU (needs `uhubctl` installed; `baked_single` does this once automatically when available). Also check `git status userPrefs.jsonc`.
|
||||
- **Regression** — same assertion fails repeatedly on re-runs, firmware log shows novel errors. Identify the firmware module likely responsible.
|
||||
|
||||
6. **Do NOT run destructive recovery automatically**. If a failure looks like it needs a reflash, factory*reset, `uhubctl_cycle`, or replug — \_describe the steps* and let the operator decide. Never burn airtime or flash cycles without approval.
|
||||
|
||||
## Arguments convention
|
||||
|
||||
Operators generally invoke this prompt either with no arguments (full suite) or with a specific subset. Examples:
|
||||
|
||||
- `tests/mesh` — one tier
|
||||
- `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip` — one test
|
||||
- `--force-bake` — reflash devices first
|
||||
- `-k telemetry` — name-filter
|
||||
|
||||
## Side-effects to confirm in your summary
|
||||
|
||||
- `userPrefs.jsonc` should be clean after a successful run. The session fixture in `mcp-server/tests/conftest.py` (`_session_userprefs`) snapshots and restores. Check `git status --porcelain userPrefs.jsonc` and report if it's non-empty.
|
||||
- `mcp-server/tests/report.html` and `junit.xml` regenerate on every run.
|
||||
- The wrapper prints a warning if a `.mcp-session-bak` sidecar was left over from a crashed prior session and auto-restores from it — mention that if it happened.
|
||||
@@ -6,9 +6,9 @@ Guide for developing a new Meshtastic firmware module.
|
||||
|
||||
Choose the appropriate base class:
|
||||
|
||||
1. **`MeshModule`** — Raw base class. Override `wantPacket()` and `handleReceived()`. Returns `ProcessMessage::STOP` or `ProcessMessage::CONTINUE`.
|
||||
2. **`SinglePortModule`** — Handles a single `meshtastic_PortNum`. Constructor takes `(name, portNum)`. Simplified `wantPacket()` checking `decoded.portnum`. Use `allocDataPacket()` to create outgoing packets.
|
||||
3. **`ProtobufModule<T>`** — Template for protobuf-encoded modules. Constructor takes `(name, portNum, fields)`. Override `handleReceivedProtobuf()`. Use `allocDataProtobuf(payload)` to create outgoing packets.
|
||||
1. **`MeshModule`** - Raw base class. Override `wantPacket()` and `handleReceived()`. Returns `ProcessMessage::STOP` or `ProcessMessage::CONTINUE`.
|
||||
2. **`SinglePortModule`** - Handles a single `meshtastic_PortNum`. Constructor takes `(name, portNum)`. Simplified `wantPacket()` checking `decoded.portnum`. Use `allocDataPacket()` to create outgoing packets.
|
||||
3. **`ProtobufModule<T>`** - Template for protobuf-encoded modules. Constructor takes `(name, portNum, fields)`. Override `handleReceivedProtobuf()`. Use `allocDataProtobuf(payload)` to create outgoing packets.
|
||||
|
||||
Most modules also mix in `concurrency::OSThread` for periodic background tasks.
|
||||
|
||||
@@ -32,7 +32,7 @@ class MyModule : public ProtobufModule<meshtastic_MyMessage>, private concurrenc
|
||||
// Generate response packet (optional)
|
||||
virtual meshtastic_MeshPacket *allocReply() override;
|
||||
|
||||
// Periodic task — return next run interval in ms, or disable()
|
||||
// Periodic task - return next run interval in ms, or disable()
|
||||
virtual int32_t runOnce() override;
|
||||
|
||||
// Modify packet in-flight before delivery (optional)
|
||||
|
||||
@@ -6,10 +6,10 @@ Guide for adding a new I2C telemetry sensor driver to Meshtastic firmware.
|
||||
|
||||
Telemetry sensors live in `src/modules/Telemetry/Sensor/`. There are 50+ existing drivers organized by measurement type. Each sensor integrates with one of the telemetry modules:
|
||||
|
||||
- **EnvironmentTelemetryModule** — Temperature, humidity, pressure, gas, light
|
||||
- **AirQualityTelemetryModule** — Particulate matter, VOCs
|
||||
- **PowerTelemetryModule** — Voltage, current, power monitoring
|
||||
- **HealthTelemetryModule** — Heart rate, SpO2, body temperature
|
||||
- **EnvironmentTelemetryModule** - Temperature, humidity, pressure, gas, light
|
||||
- **AirQualityTelemetryModule** - Particulate matter, VOCs
|
||||
- **PowerTelemetryModule** - Voltage, current, power monitoring
|
||||
- **HealthTelemetryModule** - Heart rate, SpO2, body temperature
|
||||
|
||||
## Sensor Driver Pattern
|
||||
|
||||
@@ -75,10 +75,10 @@ The scan runs at boot and populates a device map that telemetry modules use to d
|
||||
If the sensor provides data not covered by existing telemetry fields:
|
||||
|
||||
1. Add fields to the appropriate message in `protobufs/meshtastic/telemetry.proto`:
|
||||
- `EnvironmentMetrics` — Environmental measurements
|
||||
- `AirQualityMetrics` — Air quality data
|
||||
- `PowerMetrics` — Power/energy data
|
||||
- `HealthMetrics` — Health/biometric data
|
||||
- `EnvironmentMetrics` - Environmental measurements
|
||||
- `AirQualityMetrics` - Air quality data
|
||||
- `PowerMetrics` - Power/energy data
|
||||
- `HealthMetrics` - Health/biometric data
|
||||
2. Add a `.options` constraint if needed (field sizes for nanopb)
|
||||
3. Regenerate: `bin/regen-protos.sh`
|
||||
|
||||
|
||||
@@ -20,14 +20,14 @@ variants/
|
||||
|
||||
Each variant needs at minimum:
|
||||
|
||||
- `variant.h` — Pin definitions and hardware capabilities
|
||||
- `platformio.ini` — Build configuration
|
||||
- `variant.h` - Pin definitions and hardware capabilities
|
||||
- `platformio.ini` - Build configuration
|
||||
|
||||
Optional files:
|
||||
|
||||
- `pins_arduino.h` — Arduino pin mapping overrides
|
||||
- `rfswitch.h` — RF switch control for multi-band radios
|
||||
- `nicheGraphics.h` — InkHUD e-ink configuration
|
||||
- `pins_arduino.h` - Arduino pin mapping overrides
|
||||
- `rfswitch.h` - RF switch control for multi-band radios
|
||||
- `nicheGraphics.h` - InkHUD e-ink configuration
|
||||
|
||||
## variant.h Template
|
||||
|
||||
@@ -101,25 +101,25 @@ upload_speed = 921600
|
||||
|
||||
### Common Base Configs
|
||||
|
||||
- `esp32_base` / `esp32-common.ini` — ESP32
|
||||
- `esp32s3_base` — ESP32-S3
|
||||
- `esp32c3_base` — ESP32-C3
|
||||
- `esp32c6_base` — ESP32-C6
|
||||
- `nrf52840_base` / `nrf52.ini` — nRF52840
|
||||
- `rp2040_base` — RP2040/RP2350
|
||||
- `esp32_base` / `esp32-common.ini` - ESP32
|
||||
- `esp32s3_base` - ESP32-S3
|
||||
- `esp32c3_base` - ESP32-C3
|
||||
- `esp32c6_base` - ESP32-C6
|
||||
- `nrf52840_base` / `nrf52.ini` - nRF52840
|
||||
- `rp2040_base` - RP2040/RP2350
|
||||
|
||||
### Support Levels
|
||||
|
||||
- `custom_meshtastic_support_level = 1` — Built on every PR (actively supported)
|
||||
- `custom_meshtastic_support_level = 2` — Built only on merge to main branches
|
||||
- `board_level = extra` — Only built on full releases
|
||||
- `custom_meshtastic_support_level = 1` - Built on every PR (actively supported)
|
||||
- `custom_meshtastic_support_level = 2` - Built only on merge to main branches
|
||||
- `board_level = extra` - Only built on full releases
|
||||
|
||||
## Build Manifest Metadata
|
||||
|
||||
`bin/platformio-custom.py` emits UI capability flags in the build manifest:
|
||||
|
||||
- `custom_meshtastic_has_mui = true/false` — Override MUI detection
|
||||
- `custom_meshtastic_has_ink_hud = true/false` — Override InkHUD detection
|
||||
- `custom_meshtastic_has_mui = true/false` - Override MUI detection
|
||||
- `custom_meshtastic_has_ink_hud = true/false` - Override InkHUD detection
|
||||
- Architecture names are normalized (e.g., `esp32s3` → `esp32-s3`)
|
||||
|
||||
## InkHUD E-Ink Variants
|
||||
@@ -127,7 +127,7 @@ upload_speed = 921600
|
||||
For e-ink display variants using the InkHUD framework, add `nicheGraphics.h`:
|
||||
|
||||
```cpp
|
||||
// nicheGraphics.h — InkHUD configuration for this variant
|
||||
// nicheGraphics.h - InkHUD configuration for this variant
|
||||
#define INKHUD // Enable InkHUD
|
||||
// Configure display, applets, and refresh behavior per device
|
||||
```
|
||||
|
||||
@@ -23,7 +23,7 @@ jobs:
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
path: meshtasticd
|
||||
|
||||
@@ -23,7 +23,7 @@ jobs:
|
||||
outputs:
|
||||
artifact-id: ${{ steps.upload-firmware.outputs.artifact-id }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ jobs:
|
||||
runs-on: macos-${{ inputs.macos_ver }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ jobs:
|
||||
- all
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: 3.x
|
||||
@@ -78,7 +78,7 @@ jobs:
|
||||
if: ${{ inputs.target != '' }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
- name: Get release version string
|
||||
run: |
|
||||
echo "long=$(./bin/buildinfo.py long)" >> $GITHUB_OUTPUT
|
||||
@@ -107,7 +107,7 @@ jobs:
|
||||
needs: [version, build]
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
ref: ${{github.event.pull_request.head.ref}}
|
||||
repository: ${{github.event.pull_request.head.repo.full_name}}
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
name: Build PortDuino WASM
|
||||
|
||||
# Reusable workflow - called as the `build-wasm` job of the main CI workflow
|
||||
# (main_matrix.yml), so the WebAssembly portduino node is built like every other
|
||||
# platform. Builds the [env:native-wasm] target (src/platform/portduino/wasm/) with
|
||||
# `pio run -e native-wasm` so it can't silently bit-rot. Software/CI only - asserts the
|
||||
# full mesh stack + RadioLib + Crypto compile and link to meshnode.{mjs,wasm}
|
||||
# via the meshtastic/platform-wasm PlatformIO platform (emcc/Asyncify). It does
|
||||
# NOT exercise the radio (that's CH341/WebUSB, hardware).
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
build-wasm:
|
||||
name: Build PortDuino WASM
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
# Python + PlatformIO. (`pio run -e native-wasm` installs its own libdeps and the
|
||||
# platform-wasm/framework-portduino packages; it needs no native apt libs.)
|
||||
- name: Setup native build
|
||||
uses: ./.github/actions/setup-native
|
||||
|
||||
- name: Setup Emscripten SDK
|
||||
uses: emscripten-core/setup-emsdk@v16
|
||||
with:
|
||||
version: 6.0.1
|
||||
actions-cache-folder: emsdk-cache
|
||||
|
||||
- name: Build PortDuino WASM
|
||||
run: platformio run -e native-wasm
|
||||
|
||||
- name: Assert WASM artifacts
|
||||
run: |
|
||||
OUT=.pio/build/native-wasm
|
||||
if [ ! -s "$OUT/meshnode.mjs" ] || [ ! -s "$OUT/meshnode.wasm" ]; then
|
||||
echo "::error::wasm artifacts missing"; exit 1
|
||||
fi
|
||||
# The emcc link succeeds even if embedded EM_ASM JS is malformed, so
|
||||
# parse the generated ES-module loader to catch runtime-JS breakage
|
||||
# (e.g. a formatter splitting !== inside EM_ASM).
|
||||
node --check "$OUT/meshnode.mjs" || { echo "::error::meshnode.mjs failed JS parse"; exit 1; }
|
||||
ls -lah "$OUT"/meshnode.*
|
||||
|
||||
- name: Upload WASM artifacts
|
||||
if: success()
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: portduino-wasm
|
||||
overwrite: true
|
||||
path: |
|
||||
.pio/build/native-wasm/meshnode.mjs
|
||||
.pio/build/native-wasm/meshnode.wasm
|
||||
retention-days: 14
|
||||
@@ -47,7 +47,7 @@ jobs:
|
||||
runs-on: ${{ inputs.runs-on }}
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
|
||||
@@ -103,7 +103,7 @@ jobs:
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ jobs:
|
||||
|
||||
- name: Post or update PR comment
|
||||
if: steps.download.outcome == 'success'
|
||||
uses: actions/github-script@v8
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: |
|
||||
const fs = require('fs');
|
||||
|
||||
@@ -31,7 +31,7 @@ jobs:
|
||||
merge-multiple: true
|
||||
|
||||
- name: Post or update web flasher link comment
|
||||
uses: actions/github-script@v8
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: |
|
||||
const marker = '<!-- web-flasher-link -->';
|
||||
@@ -41,7 +41,7 @@ jobs:
|
||||
// Resolve the PR by matching the run's head SHA against the repo's open
|
||||
// PRs. workflow_run.pull_requests is empty for fork PRs, and
|
||||
// listPullRequestsAssociatedWithCommit won't return an open fork PR by
|
||||
// its head commit — but pulls.list includes fork PRs. Matching on head
|
||||
// its head commit - but pulls.list includes fork PRs. Matching on head
|
||||
// SHA also enforces that the run is for the PR's current commit, so stale
|
||||
// re-runs of an outdated commit won't match.
|
||||
const openPrs = await github.paginate(github.rest.pulls.list, {
|
||||
@@ -55,7 +55,7 @@ jobs:
|
||||
const prNumber = pr.number;
|
||||
|
||||
// Restrict to trusted authors. NOTE: author_association is computed for
|
||||
// the GITHUB_TOKEN, which cannot see *private/concealed* org memberships —
|
||||
// the GITHUB_TOKEN, which cannot see *private/concealed* org memberships -
|
||||
// those members come back as CONTRIBUTOR, not MEMBER. So gating on MEMBER
|
||||
// alone silently excludes most maintainers. We allow the trusted set the
|
||||
// token can actually identify (members, collaborators, and anyone with a
|
||||
@@ -86,7 +86,7 @@ jobs:
|
||||
// Read each built board's manifest (.mt.json). activelySupported,
|
||||
// displayName and architecture come straight from the board's
|
||||
// custom_meshtastic_* platformio config, so the list is in sync with
|
||||
// the firmware itself — no external device database needed.
|
||||
// the firmware itself - no external device database needed.
|
||||
const fs = require('fs');
|
||||
let boards = [];
|
||||
try {
|
||||
@@ -132,8 +132,8 @@ jobs:
|
||||
.join('\n');
|
||||
|
||||
// Shields.io badges. Only non-user-controlled, charset-constrained values
|
||||
// (version, commit sha, counts, dates) go into badge URLs — never board
|
||||
// names or the PR title — so the rendered comment cannot be spoofed.
|
||||
// (version, commit sha, counts, dates) go into badge URLs - never board
|
||||
// names or the PR title - so the rendered comment cannot be spoofed.
|
||||
const shieldText = (s) =>
|
||||
encodeURIComponent(String(s).replace(/-/g, '--').replace(/_/g, '__').replace(/ /g, '_'));
|
||||
const shield = (label, message, color) =>
|
||||
|
||||
@@ -6,7 +6,7 @@ name: Post Web Flasher Build Placeholder
|
||||
#
|
||||
# SECURITY: this uses pull_request_target (write token, runs for fork PRs) but is
|
||||
# safe because it never checks out or runs PR code and posts a fully static body
|
||||
# — no PR title, branch name, or other untrusted input is used anywhere.
|
||||
# - no PR title, branch name, or other untrusted input is used anywhere.
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
@@ -22,7 +22,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Post web flasher build-in-progress placeholder
|
||||
uses: actions/github-script@v8
|
||||
uses: actions/github-script@v9
|
||||
with:
|
||||
script: |
|
||||
const marker = '<!-- web-flasher-link -->';
|
||||
@@ -31,14 +31,14 @@ jobs:
|
||||
|
||||
// Trusted authors only (matches the real workflow). author_association
|
||||
// can't reflect private org membership for the token, so concealed
|
||||
// members appear as CONTRIBUTOR — include it, or maintainers are excluded.
|
||||
// members appear as CONTRIBUTOR - include it, or maintainers are excluded.
|
||||
const allowedAssociations = ['OWNER', 'MEMBER', 'COLLABORATOR', 'CONTRIBUTOR'];
|
||||
if (!allowedAssociations.includes(pr.author_association)) {
|
||||
core.info(`Author association ${pr.author_association} is not trusted; skipping.`);
|
||||
return;
|
||||
}
|
||||
|
||||
// Only seed a placeholder when no flasher comment exists yet — never
|
||||
// Only seed a placeholder when no flasher comment exists yet - never
|
||||
// overwrite a real (or existing placeholder) comment.
|
||||
const comments = await github.paginate(github.rest.issues.listComments, {
|
||||
owner, repo, issue_number: pr.number, per_page: 100,
|
||||
|
||||
@@ -18,7 +18,7 @@ jobs:
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
|
||||
@@ -40,7 +40,7 @@ jobs:
|
||||
- check
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
- uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: 3.x
|
||||
@@ -64,7 +64,7 @@ jobs:
|
||||
version:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
- name: Get release version string
|
||||
run: |
|
||||
echo "long=$(./bin/buildinfo.py long)" >> $GITHUB_OUTPUT
|
||||
@@ -87,7 +87,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
if: ${{ github.event_name != 'workflow_dispatch' && github.repository == 'meshtastic/firmware' }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
- name: Check ${{ matrix.check.board }}
|
||||
@@ -146,6 +146,14 @@ jobs:
|
||||
checks: write
|
||||
uses: ./.github/workflows/test_native.yml
|
||||
|
||||
build-wasm:
|
||||
# Build the WebAssembly portduino node ([env:native-wasm]) as part of normal CI,
|
||||
# like the other platforms. It's a dedicated job (not a row in the `build`
|
||||
# matrix) because its artifact is meshnode.{mjs,wasm} - not a flashable
|
||||
# .bin/.uf2/.hex - and it needs the Emscripten SDK; board_level=extra keeps
|
||||
# it out of generate_ci_matrix.py.
|
||||
uses: ./.github/workflows/build_portduino_wasm.yml
|
||||
|
||||
docker:
|
||||
permissions: # Needed for pushing to GHCR.
|
||||
contents: read
|
||||
@@ -190,7 +198,7 @@ jobs:
|
||||
needs: [version, build]
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
|
||||
- uses: actions/download-artifact@v8
|
||||
with:
|
||||
@@ -255,7 +263,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
needs: [build]
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Download current manifests
|
||||
uses: actions/download-artifact@v8
|
||||
@@ -383,7 +391,7 @@ jobs:
|
||||
# - MacOS
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
@@ -484,7 +492,7 @@ jobs:
|
||||
needs: [release-artifacts, version]
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v6
|
||||
@@ -539,7 +547,7 @@ jobs:
|
||||
esp32,esp32s3,esp32c3,esp32c6,nrf52840,rp2040,rp2350,stm32
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
|
||||
- name: Trunk Check
|
||||
uses: trunk-io/trunk-action@v1
|
||||
@@ -31,7 +31,7 @@ jobs:
|
||||
pull-requests: write # For trunk to create PRs
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
|
||||
- name: Trunk Upgrade
|
||||
uses: trunk-io/trunk-action/upgrade@v1
|
||||
|
||||
@@ -33,7 +33,7 @@ jobs:
|
||||
needs: build-debian-src
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
path: meshtasticd
|
||||
|
||||
@@ -23,7 +23,7 @@ jobs:
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ jobs:
|
||||
needs: build-debian-src
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
path: meshtasticd
|
||||
|
||||
@@ -40,7 +40,7 @@ jobs:
|
||||
checks: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
|
||||
@@ -91,7 +91,7 @@ jobs:
|
||||
shell: bash
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
# Always use master branch for version bumps
|
||||
ref: master
|
||||
|
||||
@@ -21,7 +21,7 @@ jobs:
|
||||
steps:
|
||||
# step 1
|
||||
- name: clone application source code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
|
||||
# step 2
|
||||
- name: full scan
|
||||
|
||||
@@ -13,7 +13,7 @@ jobs:
|
||||
steps:
|
||||
# step 1
|
||||
- name: clone application source code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Stale PR+Issues
|
||||
uses: actions/stale@v10.2.0
|
||||
uses: actions/stale@v10.3.0
|
||||
with:
|
||||
days-before-stale: 45
|
||||
stale-issue-message: This issue has not had any comment or update in the last month. If it is still relevant, please post update comments. If no comments are made, this issue will be closed automagically in 7 days.
|
||||
|
||||
@@ -14,7 +14,7 @@ jobs:
|
||||
name: Native Simulator Tests
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
@@ -50,7 +50,7 @@ jobs:
|
||||
python3 -c 'from meshtastic.test import testSimulator; testSimulator()'
|
||||
# The Python harness sends exit_simulator and exits; the simulator is
|
||||
# expected to terminate on its own. Give it a moment, then verify.
|
||||
# If it is still alive the exit handshake is broken — fail loudly and
|
||||
# If it is still alive the exit handshake is broken - fail loudly and
|
||||
# do NOT fall through to `wait`, which would otherwise block until the
|
||||
# job's hard timeout.
|
||||
for i in $(seq 1 10); do
|
||||
@@ -88,7 +88,7 @@ jobs:
|
||||
name: Native PlatformIO Tests
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
@@ -149,7 +149,7 @@ jobs:
|
||||
- platformio-tests
|
||||
if: always()
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v7
|
||||
|
||||
- name: Get release version string
|
||||
run: echo "long=$(./bin/buildinfo.py long)" >> $GITHUB_OUTPUT
|
||||
@@ -161,6 +161,14 @@ jobs:
|
||||
name: platformio-test-report-${{ steps.version.outputs.long }}
|
||||
merge-multiple: true
|
||||
|
||||
- name: Drop no-status testsuites from the report
|
||||
# PlatformIO emits a self-closing <testsuite tests="0"/> row for every test_* dir
|
||||
# crossed with every hardware variant it cannot run on the native host (~4900 rows).
|
||||
# They carry no pass/fail/skip status and bury the suites that actually ran. Strip
|
||||
# them so the Test Report lists only suites with a real status. Only the copy the
|
||||
# reporter renders is trimmed; the uploaded artifact keeps the full XML.
|
||||
run: sed -i -E 's#<testsuite [^>]*tests="0"[^>]*/>##g' testreport.xml
|
||||
|
||||
- name: Test Report
|
||||
uses: dorny/test-reporter@v3.0.0
|
||||
with:
|
||||
|
||||
@@ -20,7 +20,7 @@ jobs:
|
||||
runs-on: test-runner
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
|
||||
# - uses: actions/setup-python@v6
|
||||
# with:
|
||||
|
||||
@@ -18,7 +18,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
|
||||
- name: Trunk Check
|
||||
uses: trunk-io/trunk-action@v1
|
||||
|
||||
@@ -16,7 +16,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
|
||||
- name: Trunk Check
|
||||
uses: trunk-io/trunk-action@v1
|
||||
|
||||
@@ -11,7 +11,7 @@ jobs:
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v6
|
||||
uses: actions/checkout@v7
|
||||
with:
|
||||
submodules: true
|
||||
|
||||
@@ -23,7 +23,7 @@ jobs:
|
||||
GIT_BRANCH: ${{ github.ref_name }}
|
||||
run: |
|
||||
git fetch --prune origin $GIT_BRANCH
|
||||
git checkout origin/$GIT_BRANCH
|
||||
git checkout FETCH_HEAD
|
||||
|
||||
- name: Download nanopb
|
||||
run: |
|
||||
|
||||
@@ -66,3 +66,11 @@ userPrefs.jsonc.mcp-session-bak
|
||||
# compiled .proto outputs are ephemeral build artifacts.
|
||||
build/fixtures/
|
||||
bin/_generated/
|
||||
|
||||
# portduino WASM build (pio run -e native-wasm) + Emscripten SDK. The artifacts land in
|
||||
# .pio/build/native-wasm/ (already ignored via .pio/); these cover the retired
|
||||
# standalone emcc build's output dirs and any in-repo emsdk checkout.
|
||||
build/wasm/
|
||||
build/wasm-obj/
|
||||
build/wasm-*.log
|
||||
.emsdk/
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"meshtastic": {
|
||||
"command": "./mcp-server/.venv/bin/python",
|
||||
"args": ["-m", "meshtastic_mcp"],
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"--from",
|
||||
"git+https://github.com/meshtastic/meshtastic-mcp",
|
||||
"meshtastic-mcp"
|
||||
],
|
||||
"env": {
|
||||
"MESHTASTIC_FIRMWARE_ROOT": "."
|
||||
}
|
||||
|
||||
@@ -1,2 +1,6 @@
|
||||
.github/workflows/main_matrix.yml
|
||||
src/mesh/compression/unishox2.cpp
|
||||
# Emscripten/WebUSB browser glue for the wasm node — not part of the firmware
|
||||
# binary or its security surface. The format-string rule false-positives on its
|
||||
# benign retry/diagnostic console logs.
|
||||
src/platform/portduino/wasm/js/
|
||||
|
||||
@@ -12,10 +12,10 @@
|
||||
# defensive loops over flaky sources (pubsub handlers, device
|
||||
# re-enumeration polls). One failed iteration shouldn't abort the loop.
|
||||
# B404 import_subprocess
|
||||
# mcp-server wraps PlatformIO, esptool, nrfutil, picotool, and the
|
||||
# pytest test-runner — subprocess is a load-bearing import here, not
|
||||
# a smell. The "consider possible security implications" advisory is
|
||||
# redundant given the file-level review already applied.
|
||||
# the bin/ proto + fixture tooling (regen-py-protos, gen-fake-nodedb-seed,
|
||||
# seed-json-to-proto) shells out to protoc and helper scripts — subprocess
|
||||
# is a load-bearing import here, not a smell. The "consider possible security
|
||||
# implications" advisory is redundant given the file-level review already applied.
|
||||
# B603 subprocess_without_shell_equals_true
|
||||
# all subprocess calls use a static argv list; `shell=False` is the
|
||||
# default and we never string-interpolate user input into the command.
|
||||
|
||||
+68
-1
@@ -7,7 +7,68 @@ plugins:
|
||||
ref: v1.10.0
|
||||
uri: https://github.com/trunk-io/plugins
|
||||
lint:
|
||||
# Custom file set + formatter that rewrites Unicode em-dash (U+2014, UTF-8
|
||||
# bytes e2 80 94) and en-dash (U+2013, UTF-8 bytes e2 80 93) to an ASCII
|
||||
# hyphen. The sed pattern matches the byte sequences under LC_ALL=C rather
|
||||
# than literal characters on purpose, so the rule survives editors/hooks
|
||||
# that themselves strip Unicode dashes. These dashes only ever appear in
|
||||
# comments and log/error strings (LLM-authored prose), never in code, so
|
||||
# the rewrite is purely cosmetic. Runs under `trunk fmt`; enforced by
|
||||
# `trunk check` and the trunk-fmt-pre-commit action.
|
||||
files:
|
||||
- name: ascii-dash-targets
|
||||
extensions:
|
||||
- c
|
||||
- cc
|
||||
- cpp
|
||||
- h
|
||||
- hpp
|
||||
- ino
|
||||
- md
|
||||
- py
|
||||
- sh
|
||||
- yaml
|
||||
- yml
|
||||
- json
|
||||
- toml
|
||||
- ini
|
||||
- name: cpp-sources
|
||||
extensions:
|
||||
- c
|
||||
- cc
|
||||
- cpp
|
||||
- cxx
|
||||
- h
|
||||
- hh
|
||||
- hpp
|
||||
- hxx
|
||||
- ino
|
||||
definitions:
|
||||
- name: ascii-dash
|
||||
files: [ascii-dash-targets]
|
||||
commands:
|
||||
- name: format
|
||||
output: rewrite
|
||||
formatter: true
|
||||
run: env LC_ALL=C sed -e 's/\xe2\x80\x94/-/g' -e 's/\xe2\x80\x93/-/g' ${target}
|
||||
success_codes: [0]
|
||||
# Flags preprocessor conditionals that contain too many defined() terms
|
||||
# (e.g. the 13-driver display chains in main.cpp). The compiler has no
|
||||
# warning for this; the fix is an umbrella feature macro (HAS_TFT /
|
||||
# HAS_SCREEN, already in src/configuration.h). Tune the limit via
|
||||
# MAX_DEFINED in bin/lint-ifdef-complexity.sh (default 5 -> warns at 6+).
|
||||
- name: too-many-defined
|
||||
files: [cpp-sources]
|
||||
commands:
|
||||
- name: lint
|
||||
output: regex
|
||||
parse_regex: (?P<path>.+):(?P<line>\d+):(?P<col>\d+):(?P<severity>\w+):(?P<message>.+):(?P<code>[a-z-]+)
|
||||
run: ${workspace}/bin/lint-ifdef-complexity.sh ${target}
|
||||
success_codes: [0]
|
||||
read_output_from: stdout
|
||||
enabled:
|
||||
- ascii-dash@SYSTEM
|
||||
- too-many-defined@SYSTEM
|
||||
- checkov@3.2.529
|
||||
- renovate@43.150.0
|
||||
- prettier@3.8.3
|
||||
@@ -36,11 +97,17 @@ lint:
|
||||
- bin/**
|
||||
# Fake-NodeDB fixture JSONL files contain deterministic synthetic
|
||||
# public_key_hex (64-char hex) values that gitleaks misidentifies as
|
||||
# generic-api-key. These are not secrets — they're test fixtures
|
||||
# generic-api-key. These are not secrets - they're test fixtures
|
||||
# produced by bin/gen-fake-nodedb-seed.py with a fixed RNG seed.
|
||||
- linters: [gitleaks]
|
||||
paths:
|
||||
- test/fixtures/nodedb/seed_v25_*.jsonl
|
||||
# The UA font's em/en dashes live only in trailing comments documenting
|
||||
# each glyph's codepoint (e.g. "8212=:2549"); rewriting them would make
|
||||
# those docs inaccurate. Glyph byte data is unaffected either way.
|
||||
- linters: [ascii-dash]
|
||||
paths:
|
||||
- src/graphics/fonts/**
|
||||
runtimes:
|
||||
enabled:
|
||||
- python@3.14.4
|
||||
|
||||
@@ -1,31 +1,49 @@
|
||||
# Agent instructions
|
||||
|
||||
This repository is the [Meshtastic](https://meshtastic.org) firmware — a C++17 embedded codebase targeting ESP32 / nRF52 / RP2040 / STM32WL / Linux-Portduino LoRa mesh radios — plus a Python MCP server in `mcp-server/` that AI agents use to flash, configure, and test connected devices.
|
||||
> **TL;DR**
|
||||
>
|
||||
> | | |
|
||||
> | -------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
|
||||
> | Hardware tests | [meshtastic/meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) (`MESHTASTIC_FIRMWARE_ROOT` → this checkout) |
|
||||
> | Format | `trunk fmt` |
|
||||
> | Mirror docs | `.github/copilot-instructions.md` (canonical) · `CLAUDE.md` (Claude Code) |
|
||||
>
|
||||
> **Need this? It's here.**
|
||||
>
|
||||
> | | |
|
||||
> | ------------------------------------------- | ---------------------------------------------------------- |
|
||||
> | General helpers (clamp, UTF-8, string fmt…) | `src/meshUtils.h` |
|
||||
> | Logging macros (LOG_DEBUG / INFO / WARN…) | `src/DebugConfiguration.h` |
|
||||
> | New module skeleton | inherit `ProtobufModule<T>` in `src/mesh/ProtobufModule.h` |
|
||||
> | Observer / event wiring | `src/Observer.h` |
|
||||
|
||||
This repository is the [Meshtastic](https://meshtastic.org) firmware - a C++17 embedded codebase targeting ESP32 / nRF52 / RP2040 / STM32WL / Linux-Portduino LoRa mesh radios. The Python MCP server that AI agents use to flash, configure, and test connected devices now lives in its own repo, [meshtastic/meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp); this repo registers it via `.mcp.json` (run through `uvx`) so its tools are available automatically.
|
||||
|
||||
## Primary instruction file
|
||||
|
||||
**Read `.github/copilot-instructions.md` first.** That file is the canonical agent-facing document for this repo. It covers project layout, coding conventions (naming, module framework, Observer pattern, thread safety), the build system, CI/CD, the native C++ test suite, and — most importantly for automation work — the **MCP Server & Hardware Test Harness** section. Read it top-to-bottom before starting any non-trivial change.
|
||||
**Read `.github/copilot-instructions.md` first.** That file is the canonical agent-facing document for this repo. It covers project layout, coding conventions (naming, module framework, Observer pattern, thread safety), the build system, CI/CD, the native C++ test suite, and - most importantly for automation work - the **MCP Server & Hardware Test Harness** section. Read it top-to-bottom before starting any non-trivial change.
|
||||
|
||||
This file (`AGENTS.md`) is a short pointer + quick reference for agents that don't read `.github/copilot-instructions.md` by default.
|
||||
|
||||
## Quick command reference
|
||||
|
||||
| Action | Command |
|
||||
| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Build a firmware variant | `pio run -e <env>` (e.g. `pio run -e rak4631`, `pio run -e heltec-v3`) |
|
||||
| Build native macOS host binary | `pio run -e native-macos` (Homebrew prereqs + CH341 LoRa setup in `variants/native/portduino/platformio.ini`) |
|
||||
| Clean + rebuild | `pio run -e <env> -t clean && pio run -e <env>` |
|
||||
| Flash a device | `pio run -e <env> -t upload --upload-port <port>` (or use the `pio_flash` MCP tool) |
|
||||
| Run firmware unit tests (native) | `~/.platformio/penv/bin/python -m platformio test -e native > /tmp/test_out.txt 2>&1` then `grep -E 'error:\|PASS\|FAIL\|succeeded\|failed' /tmp/test_out.txt` (redirect first — piping causes line-buffering) |
|
||||
| Run MCP hardware tests | `./mcp-server/run-tests.sh` |
|
||||
| Live TUI test runner | `mcp-server/.venv/bin/meshtastic-mcp-test-tui` |
|
||||
| Format before commit | `trunk fmt` |
|
||||
| Regenerate protobuf bindings | `bin/regen-protos.sh` |
|
||||
| Generate CI matrix | `./bin/generate_ci_matrix.py all [--level pr]` |
|
||||
| Action | Command |
|
||||
| -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Build a firmware variant | `pio run -e <env>` (e.g. `pio run -e rak4631`, `pio run -e heltec-v3`) |
|
||||
| Build native macOS host binary | `pio run -e native-macos` (Homebrew prereqs + CH341 LoRa setup in `variants/native/portduino/platformio.ini`) |
|
||||
| Clean + rebuild | `pio run -e <env> -t clean && pio run -e <env>` |
|
||||
| Flash a device | `pio run -e <env> -t upload --upload-port <port>` (or use the `pio_flash` MCP tool) |
|
||||
| Run firmware unit tests (native) | `./bin/run-tests.sh` (preferred - ASan/LSan + RED/AMBER/GREEN verdict); or raw: `~/.platformio/penv/bin/python -m platformio test -e native > /tmp/test_out.txt 2>&1` |
|
||||
| Run MCP hardware tests | From a [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) checkout: `MESHTASTIC_FIRMWARE_ROOT=/path/to/firmware ./run-tests.sh` |
|
||||
| Live TUI test runner | `uvx --from git+https://github.com/meshtastic/meshtastic-mcp meshtastic-mcp-test-tui` |
|
||||
| Format before commit | `trunk fmt` |
|
||||
| Regenerate protobuf bindings | `bin/regen-protos.sh` |
|
||||
| Generate CI matrix | `./bin/generate_ci_matrix.py all [--level pr]` |
|
||||
|
||||
## MCP server (device + test automation)
|
||||
|
||||
The `mcp-server/` package exposes ~32 MCP tools for device discovery, building, flashing, serial monitoring, and live-node administration. Tools are grouped as:
|
||||
The [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) server exposes ~32 MCP tools for device discovery, building, flashing, serial monitoring, and live-node administration. Tools are grouped as:
|
||||
|
||||
- **Discovery**: `list_devices`, `list_boards`, `get_board`
|
||||
- **Build & flash**: `build`, `clean`, `pio_flash`, `erase_and_flash` (ESP32 factory), `update_flash` (ESP32 OTA), `touch_1200bps`
|
||||
@@ -35,39 +53,34 @@ The `mcp-server/` package exposes ~32 MCP tools for device discovery, building,
|
||||
- **userPrefs admin**: `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest`, `userprefs_testing_profile`
|
||||
- **Vendor escape hatches**: `esptool_*`, `nrfutil_*`, `picotool_*`
|
||||
|
||||
Setup: `cd mcp-server && python3 -m venv .venv && .venv/bin/pip install -e '.[test]'`. The repo registers the server via `.mcp.json` — Claude Code picks it up automatically.
|
||||
Setup: nothing to build - `.mcp.json` runs the server via `uvx --from git+https://github.com/meshtastic/meshtastic-mcp meshtastic-mcp`, so Claude Code picks it up automatically. To run the pytest hardware harness, clone [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) and set `MESHTASTIC_FIRMWARE_ROOT` to this firmware checkout.
|
||||
|
||||
See `mcp-server/README.md` for argument shapes and the **MCP Server & Hardware Test Harness** section of `.github/copilot-instructions.md` for agent usage rules (tool surface, fixture contract, firmware integration points, recovery playbooks).
|
||||
See the meshtastic-mcp repo's README for argument shapes and the **MCP Server & Hardware Test Harness** section of `.github/copilot-instructions.md` for agent usage rules (tool surface, fixture contract, firmware integration points, recovery playbooks).
|
||||
|
||||
## Slash commands (AI-assisted workflows)
|
||||
|
||||
Three test-and-diagnose workflows exist as slash commands:
|
||||
|
||||
- **`/test` (Claude Code) / `/mcp-test` (Copilot)** — run the hardware test suite and interpret failures
|
||||
- **`/diagnose` / `/mcp-diagnose`** — read-only device health report
|
||||
- **`/repro` / `/mcp-repro`** — flakiness triage: re-run one test N times, diff firmware logs between passes and failures
|
||||
|
||||
Bodies live in `.claude/commands/` and `.github/prompts/` respectively. `.claude/commands/README.md` is the index.
|
||||
The test-and-diagnose workflows (`/test`, `/diagnose`, `/repro`, `/leakhunt`) now ship with the [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) repo as skills - run them from a checkout of that repo (with `MESHTASTIC_FIRMWARE_ROOT` pointed here). The MCP tools they build on are still available in this repo via `.mcp.json`.
|
||||
|
||||
## Encryption at a glance
|
||||
|
||||
Two layers, both in `src/mesh/CryptoEngine.cpp`:
|
||||
|
||||
- **Channel (symmetric)** — **AES-CTR** with a channel-wide PSK (AES-128 or AES-256). Nonce = packet_id ‖ from_node ‖ block_counter. No AEAD; integrity is soft (channel-hash filter). The well-known default PSK lives in `src/mesh/Channels.h`; a 1-byte PSK is a short-form index into it.
|
||||
- **Per-peer PKI** — **X25519 ECDH** (Curve25519, 32-byte keys) → SHA-256 → **AES-256-CCM** with an 8-byte MAC. Fresh 32-bit `extraNonce` per packet, sent in the clear alongside the MAC. 12-byte wire overhead (`MESHTASTIC_PKC_OVERHEAD`). Used for DMs. Also used for remote admin (`src/modules/AdminModule.cpp`), where AdminMessage authorization is gated by `config.security.admin_key[0..2]`. Disabled entirely in Ham mode (`user.is_licensed=true`).
|
||||
- **Channel (symmetric)** - **AES-CTR** with a channel-wide PSK (AES-128 or AES-256). Nonce = packet_id ‖ from_node ‖ block_counter. No AEAD; integrity is soft (channel-hash filter). The well-known default PSK lives in `src/mesh/Channels.h`; a 1-byte PSK is a short-form index into it.
|
||||
- **Per-peer PKI** - **X25519 ECDH** (Curve25519, 32-byte keys) → SHA-256 → **AES-256-CCM** with an 8-byte MAC. Fresh 32-bit `extraNonce` per packet, sent in the clear alongside the MAC. 12-byte wire overhead (`MESHTASTIC_PKC_OVERHEAD`). Used for DMs. Also used for remote admin (`src/modules/AdminModule.cpp`), where AdminMessage authorization is gated by `config.security.admin_key[0..2]`. Disabled entirely in Ham mode (`user.is_licensed=true`).
|
||||
|
||||
Key rotation to never trigger casually: only the **full** factory reset (`factory_reset_device`, `eraseBleBonds=true`) wipes `security.private_key` and regenerates the keypair — every peer holds the old public key, so DMs silently fail PKI decrypt until NodeInfo re-exchanges. The **partial** config reset (`factory_reset_config`) preserves the private key and doesn't invalidate peer relationships. Explicitly blanking `security.private_key` via admin also triggers regen. See the **Encryption & Key Management** section of `.github/copilot-instructions.md` for the full spec (nonce layout, send/receive selection logic including infrastructure-portnum exceptions, admin-key + session-passkey authorization, `is_managed` scope, key-rotation hazards).
|
||||
Key rotation to never trigger casually: only the **full** factory reset (`factory_reset_device`, `eraseBleBonds=true`) wipes `security.private_key` and regenerates the keypair - every peer holds the old public key, so DMs silently fail PKI decrypt until NodeInfo re-exchanges. The **partial** config reset (`factory_reset_config`) preserves the private key and doesn't invalidate peer relationships. Explicitly blanking `security.private_key` via admin also triggers regen. See the **Encryption & Key Management** section of `.github/copilot-instructions.md` for the full spec (nonce layout, send/receive selection logic including infrastructure-portnum exceptions, admin-key + session-passkey authorization, `is_managed` scope, key-rotation hazards).
|
||||
|
||||
## House rules
|
||||
|
||||
- **No destructive device operations without operator approval.** `factory_reset`, `erase_and_flash`, `reboot`, `shutdown`, history-rewriting git ops — describe the action and stop. Operator authorizes.
|
||||
- **No destructive device operations without operator approval.** `factory_reset`, `erase_and_flash`, `reboot`, `shutdown`, history-rewriting git ops - describe the action and stop. Operator authorizes.
|
||||
- **One MCP call per serial port at a time.** The port lock is exclusive; concurrent calls deadlock. Sequence: open → read/mutate → close, then next device.
|
||||
- **`userPrefs.jsonc` is session state during tests.** The `_session_userprefs` fixture snapshots + restores it; never edit it from inside a test.
|
||||
- **Don't speculate about firmware root causes.** When evidence doesn't support a classification, say "unknown" and list what would disambiguate.
|
||||
- **Run `trunk fmt` before proposing a commit.** The `trunk_check` CI gate will reject unformatted code. Claude Code runs it automatically via the PostToolUse hook in `.claude/settings.json`; trunk's launcher needs `curl` or `wget` to bootstrap its pinned CLI — see **Formatting & the trunk toolchain** in `.github/copilot-instructions.md` for the no-curl bootstrap procedure.
|
||||
- **Run `trunk fmt` before proposing a commit.** The `trunk_check` CI gate will reject unformatted code. Claude Code runs it automatically via the PostToolUse hook in `.claude/settings.json`; trunk's launcher needs `curl` or `wget` to bootstrap its pinned CLI - see **Formatting & the trunk toolchain** in `.github/copilot-instructions.md` for the no-curl bootstrap procedure.
|
||||
- **Never edit or commit files under `src/mesh/generated/`.** They are regenerated from the [`meshtastic/protobufs`](https://github.com/meshtastic/protobufs) repo by the `update_protobufs.yml` workflow (entry point: `bin/regen-protos.sh`). Local edits will be overwritten and create merge conflicts. If a `.proto` change is needed, open a PR against the protobufs repo first, then let the workflow re-sync this repo.
|
||||
- **`confirm=True` on destructive MCP tools is a real gate, not a formality.** Don't bypass it via auto-approve settings.
|
||||
- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
|
||||
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
|
||||
- **Keep code comments minimal - one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
|
||||
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check - raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
|
||||
|
||||
## Typical agent workflows
|
||||
|
||||
@@ -79,9 +92,9 @@ Key rotation to never trigger casually: only the **full** factory reset (`factor
|
||||
|
||||
### Inspecting live node state
|
||||
|
||||
1. `device_info(port=...)` — short summary (node num, firmware version, region, peer count)
|
||||
2. `list_nodes(port=...)` — full peer table (SNR, RSSI, pubkey presence, last_heard)
|
||||
3. `get_config(section="lora", port=...)` — LoRa settings for cross-device comparison
|
||||
1. `device_info(port=...)` - short summary (node num, firmware version, region, peer count)
|
||||
2. `list_nodes(port=...)` - full peer table (SNR, RSSI, pubkey presence, last_heard)
|
||||
3. `get_config(section="lora", port=...)` - LoRa settings for cross-device comparison
|
||||
|
||||
Sequence these; don't parallelize on the same port.
|
||||
|
||||
@@ -89,42 +102,40 @@ Sequence these; don't parallelize on the same port.
|
||||
|
||||
1. Build locally: `pio run -e <env>`
|
||||
2. Flash the test device: `pio_flash(env=..., port=..., confirm=True)`
|
||||
3. Run the suite: `./mcp-server/run-tests.sh tests/<tier>` or `/test tests/<tier>`
|
||||
4. On failure, open `mcp-server/tests/report.html` → `Meshtastic debug` section for the firmware log tail + device state dump
|
||||
3. Run the suite from a meshtastic-mcp checkout: `MESHTASTIC_FIRMWARE_ROOT=/path/to/firmware ./run-tests.sh tests/<tier>` (or the `/test` skill)
|
||||
4. On failure, open the run's `tests/report.html` → `Meshtastic debug` section for the firmware log tail + device state dump
|
||||
5. Iterate
|
||||
|
||||
### Debugging a flaky test
|
||||
|
||||
1. `/repro <test-node-id> [count]` — re-runs the test N times, diffs firmware logs between passes and failures
|
||||
1. `/repro <test-node-id> [count]` - re-runs the test N times, diffs firmware logs between passes and failures
|
||||
2. If the first attempt always fails and the rest pass, that's a state-leak pattern → suggest `--force-bake` or a clean device state, don't chase the first failure
|
||||
3. If all N fail, this isn't a flake — it's a regression. Stop iterating and escalate to `/test` for full-suite context.
|
||||
3. If all N fail, this isn't a flake - it's a regression. Stop iterating and escalate to `/test` for full-suite context.
|
||||
|
||||
## Where to look
|
||||
|
||||
| Path | What's there |
|
||||
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `src/` | Firmware C++ source (`mesh/`, `modules/`, `platform/`, `graphics/`, `gps/`, `motion/`, `mqtt/`, …) |
|
||||
| `src/mesh/` | Core: NodeDB, Router, Channels, CryptoEngine, radio interfaces, StreamAPI, PhoneAPI |
|
||||
| `src/modules/` | Feature modules; `Telemetry/Sensor/` has 50+ I2C sensor drivers |
|
||||
| `variants/` | 200+ hardware variant definitions (`variant.h` + `platformio.ini` per board) |
|
||||
| `protobufs/` | `.proto` definitions; regenerate with `bin/regen-protos.sh` |
|
||||
| `test/` | Firmware unit tests (17 suites; `pio test -e native`) |
|
||||
| `mcp-server/` | Python MCP server + pytest hardware integration tests |
|
||||
| `mcp-server/tests/` | Tiered pytest suite: `unit/`, `mesh/`, `telemetry/`, `monitor/`, `recovery/`, `ui/`, `fleet/`, `admin/`, `provisioning/` |
|
||||
| `.claude/commands/` | Claude Code slash command bodies |
|
||||
| `.github/prompts/` | Copilot prompt bodies (mirrors of the Claude Code ones) |
|
||||
| `.github/copilot-instructions.md` | **Primary agent instructions — read this** |
|
||||
| `.github/workflows/` | CI pipelines |
|
||||
| `.mcp.json` | MCP server registration for Claude Code |
|
||||
| Path | What's there |
|
||||
| -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `src/` | Firmware C++ source (`mesh/`, `modules/`, `platform/`, `graphics/`, `gps/`, `motion/`, `mqtt/`, …) |
|
||||
| `src/mesh/` | Core: NodeDB, Router, Channels, CryptoEngine, radio interfaces, StreamAPI, PhoneAPI |
|
||||
| `src/modules/` | Feature modules; `Telemetry/Sensor/` has 50+ I2C sensor drivers |
|
||||
| `variants/` | 200+ hardware variant definitions (`variant.h` + `platformio.ini` per board) |
|
||||
| `protobufs/` | `.proto` definitions; regenerate with `bin/regen-protos.sh` |
|
||||
| `test/` | Firmware unit tests (19 suites; `./bin/run-tests.sh` preferred, falls back to `pio test -e native`) |
|
||||
| [meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) | Standalone MCP server + tiered pytest hardware harness (`unit/`, `mesh/`, `telemetry/`, `monitor/`, `recovery/`, `ui/`, `fleet/`, `admin/`, `provisioning/`) - registered here via `.mcp.json` |
|
||||
| `.github/prompts/` | Copilot prompt bodies (firmware scaffolding: new module / sensor / variant) |
|
||||
| `.github/copilot-instructions.md` | **Primary agent instructions - read this** |
|
||||
| `.github/workflows/` | CI pipelines |
|
||||
| `.mcp.json` | MCP server registration for Claude Code |
|
||||
|
||||
## Recovery one-liners
|
||||
|
||||
- **`userPrefs.jsonc` dirty after a test run?** Re-run `./mcp-server/run-tests.sh` once (pre-flight self-heals from the sidecar). If still dirty: `git checkout userPrefs.jsonc`.
|
||||
- **`userPrefs.jsonc` dirty after a test run?** Re-run the meshtastic-mcp harness once (pre-flight self-heals from the sidecar). If still dirty: `git checkout userPrefs.jsonc`.
|
||||
- **nRF52 not responding?** `mcp__meshtastic__touch_1200bps(port=...)` drops it into the DFU bootloader, then `pio_flash` re-installs.
|
||||
- **Device fully wedged (no DFU)?** `mcp__meshtastic__uhubctl_cycle(role="nrf52", confirm=True)` hard-power-cycles it via USB hub PPPS. Needs `uhubctl` installed (`brew install uhubctl` / `apt install uhubctl`); on Linux without udev rules, permission errors fail fast, so use `sudo uhubctl` yourself or configure udev access.
|
||||
- **Port busy?** `lsof <port>` to find the holder. Usually a stale `pio device monitor` or zombie `meshtastic_mcp` process. Kill it.
|
||||
- **Multiple MCP servers running?** `ps aux | grep meshtastic_mcp` — zombies hold ports. Kill all but the one your host spawned.
|
||||
- **macOS: `LIBUSB_ERROR_BUSY` on a CH341 LoRa adapter?** A third-party WCH `CH34xVCPDriver` is claiming interface 0. Find the bundle ID with `ioreg -p IOUSB -l -w 0 | grep -B2 -A30 0x5512`, then `sudo kmutil unload -b <bundleID>`. Apple's bundled CH34x kext targets the CH340 UART (PID 0x7523), not the SPI bridge — it's never the culprit.
|
||||
- **Multiple MCP servers running?** `ps aux | grep meshtastic_mcp` - zombies hold ports. Kill all but the one your host spawned.
|
||||
- **macOS: `LIBUSB_ERROR_BUSY` on a CH341 LoRa adapter?** A third-party WCH `CH34xVCPDriver` is claiming interface 0. Find the bundle ID with `ioreg -p IOUSB -l -w 0 | grep -B2 -A30 0x5512`, then `sudo kmutil unload -b <bundleID>`. Apple's bundled CH34x kext targets the CH340 UART (PID 0x7523), not the SPI bridge - it's never the culprit.
|
||||
|
||||
## Environment variables (test harness)
|
||||
|
||||
@@ -135,7 +146,7 @@ Sequence these; don't parallelize on the same port.
|
||||
| `MESHTASTIC_MCP_FLASH_LOG` | File path to tee pio/esptool/nrfutil/picotool output. `run-tests.sh` sets this to `tests/flash.log` so the TUI can stream live flash progress. |
|
||||
| `MESHTASTIC_MCP_TCP_HOST` | `host` or `host:port` of a `meshtasticd` daemon (e.g. the `native-macos` build). Surfaces it in `list_devices` as `tcp://host:port` so `connect()`-based tools target it transparently. Default port 4403. |
|
||||
| `MESHTASTIC_UHUBCTL_BIN` | Absolute path to `uhubctl` binary. Default: PATH lookup. |
|
||||
| `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` | Pin a role to a specific uhubctl hub location (e.g. `1-1.3`). Wins over VID auto-detection — use when multiple devices share a VID. |
|
||||
| `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` | Pin a role to a specific uhubctl hub location (e.g. `1-1.3`). Wins over VID auto-detection - use when multiple devices share a VID. |
|
||||
| `MESHTASTIC_UHUBCTL_PORT_<ROLE>` | Pin a role to a specific hub port number. Required alongside `LOCATION_<ROLE>`. |
|
||||
| `MESHTASTIC_UI_CAMERA_BACKEND` | Camera backend for UI tier + `capture_screen` tool: `opencv` / `ffmpeg` / `null` / `auto` (default). |
|
||||
| `MESHTASTIC_UI_CAMERA_DEVICE` | Generic camera device (index or path). Used by the UI tier when no per-role var is set. |
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
# Claude Code instructions
|
||||
|
||||
> **TL;DR**
|
||||
>
|
||||
> | | |
|
||||
> | -------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
> | Local tests | `./bin/run-tests.sh` (exit 0 GREEN · 1 RED · 2 AMBER · 3 FILTERED) |
|
||||
> | Hardware tests | [meshtastic/meshtastic-mcp](https://github.com/meshtastic/meshtastic-mcp) (`MESHTASTIC_FIRMWARE_ROOT` → this checkout) |
|
||||
> | Format | `trunk fmt` |
|
||||
> | Mirror docs | `.github/copilot-instructions.md` (canonical) · `AGENTS.md` |
|
||||
>
|
||||
> **Need this? It's here.**
|
||||
>
|
||||
> | | |
|
||||
> | ------------------------------------------- | ---------------------------------------------------------- |
|
||||
> | General helpers (clamp, UTF-8, string fmt…) | `src/meshUtils.h` |
|
||||
> | Logging macros (LOG_DEBUG / INFO / WARN…) | `src/DebugConfiguration.h` |
|
||||
> | New module skeleton | inherit `ProtobufModule<T>` in `src/mesh/ProtobufModule.h` |
|
||||
> | Observer / event wiring | `src/Observer.h` |
|
||||
|
||||
**Read `.github/copilot-instructions.md` first.** That file is the canonical agent-facing document for this repo. It covers project layout, coding conventions, the build system, CI/CD, the native C++ test suite, and the MCP Server & Hardware Test Harness. Read it top-to-bottom before starting any non-trivial change.
|
||||
|
||||
This file (`CLAUDE.md`) is a short pointer for Claude Code sessions. Slash commands live in `.claude/commands/`.
|
||||
@@ -29,8 +29,8 @@ Meshtastic enables text messaging, location sharing, and telemetry over a decent
|
||||
|
||||
### Get Started
|
||||
|
||||
- 🔧 **[Building Instructions](https://meshtastic.org/docs/development/firmware/build)** – Learn how to compile the firmware from source.
|
||||
- ⚡ **[Flashing Instructions](https://meshtastic.org/docs/getting-started/flashing-firmware/)** – Install or update the firmware on your device.
|
||||
- 🔧 **[Building Instructions](https://meshtastic.org/docs/development/firmware/build)** - Learn how to compile the firmware from source.
|
||||
- ⚡ **[Flashing Instructions](https://meshtastic.org/docs/getting-started/flashing-firmware/)** - Install or update the firmware on your device.
|
||||
|
||||
Join our community and help improve Meshtastic! 🚀
|
||||
|
||||
|
||||
+32
@@ -10,3 +10,35 @@
|
||||
## Reporting a Vulnerability
|
||||
|
||||
We support the private reporting of potential security vulnerabilities. Please go to the Security tab to file a report with a description of the potential vulnerability and reproduction scripts (preferred) or steps, and our developers will review.
|
||||
|
||||
Before filing, please read the Security Model below. Behavior whose only precondition is local API access to a node, or possession of a channel's pre-shared key, is intended by design and is not considered a vulnerability.
|
||||
|
||||
## Security Model
|
||||
|
||||
Meshtastic is an off-grid mesh protocol that runs on constrained microcontrollers within a 256 byte LoRa packet limit. These constraints shape its security design and rule out the heavier schemes used by IP-based protocols. This section summarizes what the firmware protects, the assumptions it rests on, and its known limits. Fuller write-ups are in the documentation:
|
||||
|
||||
- Encryption overview: https://meshtastic.org/docs/overview/encryption/
|
||||
- Technical reference: https://meshtastic.org/docs/development/reference/encryption-technical/
|
||||
- Known limitations and future work: https://meshtastic.org/docs/about/overview/encryption/limitations/
|
||||
|
||||
### Cryptographic mechanisms
|
||||
|
||||
- Channels are encrypted with a pre-shared key (PSK) using AES256-CTR. Channel traffic is encrypted but not authenticated, so anyone holding the PSK can read channel messages and can send messages as any node on that channel.
|
||||
- Direct messages and admin messages use public key cryptography (x25519 key exchange with AES-CCM), providing confidentiality, authentication, and integrity between nodes on 2.5.0 or newer that have exchanged keys.
|
||||
- Admin sessions use short-lived session IDs to limit replay of control messages.
|
||||
|
||||
### Local trust boundary
|
||||
|
||||
A client connected to a node over Bluetooth, USB serial, WiFi, or Ethernet has full local API access. From that connection it can read decrypted traffic, send messages as the node, change configuration (subject to managed mode), and read the node's private key for backup. This is intended behavior. The firmware trusts the local link the same way a phone or laptop trusts a directly attached device, and anything within reach of that connection (a shared LAN, a USB cable to an untrusted host, a paired phone) should be treated as part of the node itself.
|
||||
|
||||
### Node identity (Trust On First Use)
|
||||
|
||||
There is no central authority to sign node keys. The first public key a node hears for a given node number is the one it binds to that node number, a Trust On First Use (TOFU) model that is a hard requirement of a decentralized mesh. Clients and firmware reduce the impact of this by keeping favorited nodes from rolling out of the node database and by flagging public-key changes in the client UI.
|
||||
|
||||
Firmware 2.8.X adds XEdDSA packet signing to further secure node identity claims and the authenticity of subsequent messages. It reuses each node's existing x25519 key pair to produce signatures, so a receiver can verify that a packet came from the holder of the bound key. Once a node has been seen signing, unsigned packets claiming that identity can be rejected.
|
||||
|
||||
### Known limitations
|
||||
|
||||
- No perfect forward secrecy. Traffic captured today can be decrypted later if a key is compromised, for example through a lost node or a mishandled channel key.
|
||||
- Channel messages are not authenticated, as noted above. Although as of 2.8, channel messages will be xedDSA signed as a means of verification that is non-breaking.
|
||||
- Setting WiFi credentials, or performing any other local administration, on an ESP32 over an untrusted network exposes that traffic, including the credentials, to the network. Provision and administer nodes over a trusted channel instead: Bluetooth, USB serial, or remote admin over the mesh. There is no current roadmap item to secure local administration over untrusted WiFi, though it may be addressed in a future release.
|
||||
|
||||
+2
-2
@@ -4,7 +4,7 @@
|
||||
# trunk-ignore-all(hadolint/DL3013): Do not pin pip package versions
|
||||
|
||||
# Ensure the Alpine version is updated in both stages of the container!
|
||||
FROM alpine:3.23 AS builder
|
||||
FROM alpine:3.24 AS builder
|
||||
ARG PIO_ENV=native
|
||||
|
||||
# Enable Alpine community repository (for 'py3-grpcio-tools')
|
||||
@@ -35,7 +35,7 @@ RUN bash ./bin/build-native.sh "$PIO_ENV" && \
|
||||
|
||||
# ##### PRODUCTION BUILD #############
|
||||
|
||||
FROM alpine:3.23
|
||||
FROM alpine:3.24
|
||||
LABEL org.opencontainers.image.title="Meshtastic" \
|
||||
org.opencontainers.image.description="Alpine Meshtastic daemon" \
|
||||
org.opencontainers.image.url="https://meshtastic.org" \
|
||||
|
||||
@@ -7,7 +7,7 @@ attribute access) to use the new namespace (e.g., `meshtastic_v25`).
|
||||
|
||||
Why: the .proto files declare `package meshtastic;`, so protoc emits
|
||||
`from meshtastic import mesh_pb2 as ...` lines. That would shadow the PyPI
|
||||
`meshtastic` package which other parts of the mcp-server depend on. Renaming
|
||||
`meshtastic` package which the meshtastic-mcp tooling depends on. Renaming
|
||||
to a local namespace keeps both available.
|
||||
|
||||
Usage:
|
||||
|
||||
@@ -104,7 +104,9 @@ Lora:
|
||||
### Define GPS
|
||||
|
||||
GPS:
|
||||
# SerialPath: /dev/ttyS0
|
||||
# SerialPath: /dev/ttyS0 # Use a physical serial GPS device (mutually exclusive with GpsdHost)
|
||||
# GpsdHost: localhost # Use gpsd daemon as GPS source instead of a serial device
|
||||
# GpsdPort: 2947 # gpsd TCP port (default: 2947)
|
||||
# ExtraPins:
|
||||
# - 22
|
||||
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
# https://github.com/wehooper4/Meshtastic-Hardware/tree/main/NebraHat
|
||||
# Use for 1 watt hat
|
||||
Meta:
|
||||
name: NebraHat 1W
|
||||
support: community
|
||||
compatible:
|
||||
- raspberry-pi
|
||||
|
||||
Lora:
|
||||
Module: sx1262 # Nebra SX1262 Pi Hat - 1W
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
# CS: 8 # Newer version of MeshtasticD do not need this? If issues uncomment this line
|
||||
IRQ: 22
|
||||
Busy: 4
|
||||
Reset: 18
|
||||
RXen: 25
|
||||
I2C:
|
||||
I2CDevice: /dev/i2c-1
|
||||
@@ -0,0 +1,20 @@
|
||||
# https://github.com/wehooper4/Meshtastic-Hardware/tree/main/NebraHat
|
||||
# Use for 2 watt hat
|
||||
Meta:
|
||||
name: NebraHat 2W
|
||||
support: community
|
||||
compatible:
|
||||
- raspberry-pi
|
||||
|
||||
Lora:
|
||||
Module: sx1262 # Nebra SX1262 Pi Hat - 2W
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
SX126X_MAX_POWER: 8
|
||||
# CS: 8 # Newer version of MeshtasticD do not need this? If issues uncomment this line
|
||||
IRQ: 22
|
||||
Busy: 4
|
||||
Reset: 18
|
||||
RXen: 25
|
||||
I2C:
|
||||
I2CDevice: /dev/i2c-1
|
||||
@@ -18,4 +18,4 @@ Lora:
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
spidev: spidev0.0
|
||||
# CS: 8
|
||||
# CS: 8
|
||||
|
||||
@@ -18,4 +18,4 @@ Lora:
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
spidev: spidev0.1
|
||||
# CS: 7
|
||||
# CS: 7
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
# https://github.com/wehooper4/Meshtastic-Hardware/tree/main/ZebraHAT
|
||||
# Use for 1 watt hat
|
||||
Meta:
|
||||
name: ZebraHat 1W
|
||||
support: community
|
||||
compatible:
|
||||
- raspberry-pi
|
||||
|
||||
Lora:
|
||||
Module: sx1262 # Zebra SX1262 Pi Hat - 1W
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
SX126X_MAX_POWER: 18
|
||||
CS: 24
|
||||
IRQ: 22
|
||||
Busy: 27
|
||||
Reset: 17
|
||||
I2C:
|
||||
I2CDevice: /dev/i2c-1
|
||||
@@ -0,0 +1,20 @@
|
||||
# https://github.com/wehooper4/Meshtastic-Hardware/tree/main/ZebraHAT
|
||||
# Use for 2 watt hat
|
||||
Meta:
|
||||
name: ZebraHat 2W
|
||||
support: community
|
||||
compatible:
|
||||
- raspberry-pi
|
||||
|
||||
Lora:
|
||||
Module: sx1262 # Zebra SX1262 Pi Hat - 2W
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
SX126X_MAX_POWER: 8
|
||||
CS: 24
|
||||
IRQ: 22
|
||||
Busy: 27
|
||||
Reset: 17
|
||||
RXen: 25
|
||||
I2C:
|
||||
I2CDevice: /dev/i2c-1
|
||||
@@ -29,6 +29,8 @@ Lora:
|
||||
- pin: 50 # GPIO1_C2 (physical 16)
|
||||
gpiochip: 1
|
||||
line: 18
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
spidev: spidev0.1
|
||||
# CS: # GPIO0_A7 (SPI1_CSN1, physical 26)
|
||||
# pin: 7
|
||||
|
||||
@@ -29,6 +29,8 @@ Lora:
|
||||
- pin: 50 # GPIO1_C2 (physical 16)
|
||||
gpiochip: 1
|
||||
line: 18
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
spidev: spidev0.1
|
||||
# CS: # GPIO0_A7 (SPI1_CSN1, physical 26)
|
||||
# pin: 7
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
# Station G3 + BQ35LORA900V1M Primary Slot
|
||||
# Board Doc: https://wiki.bqvoy.com/en/devkits/station-g3
|
||||
Meta:
|
||||
name: B&Q Station G3 + BQ35LORA900V1M Primary Slot
|
||||
support: official
|
||||
compatible:
|
||||
- luckfox-lyra-zero-w # Armbian
|
||||
|
||||
Lora:
|
||||
Module: sx1262
|
||||
IRQ: # GPIO0_A5 (physical 15)
|
||||
pin: 5
|
||||
gpiochip: 0
|
||||
line: 5
|
||||
Reset: # GPIO1_D1 (physical 36)
|
||||
pin: 57
|
||||
gpiochip: 1
|
||||
line: 25
|
||||
Busy: # GPIO0_B4 (physical 18)
|
||||
pin: 12
|
||||
gpiochip: 0
|
||||
line: 12
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
spidev: spidev0.0
|
||||
# CS: # GPIO0_B2 (physical 24)
|
||||
# pin: 10
|
||||
# gpiochip: 0
|
||||
# line: 10
|
||||
@@ -29,6 +29,8 @@ Lora:
|
||||
- pin: 13 # GPIO0_B5
|
||||
gpiochip: 0
|
||||
line: 13
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
spidev: spidev0.1
|
||||
# CS: # GPIO0_B1
|
||||
# pin: 9
|
||||
|
||||
@@ -29,6 +29,8 @@ Lora:
|
||||
- pin: 13 # GPIO0_B5
|
||||
gpiochip: 0
|
||||
line: 13
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
spidev: spidev0.1
|
||||
# CS: # GPIO0_B1
|
||||
# pin: 9
|
||||
|
||||
@@ -31,6 +31,8 @@ Lora:
|
||||
- pin: 103 # GPIO3_A7 (physical 16)
|
||||
gpiochip: 3
|
||||
line: 7
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
spidev: spidev0.1
|
||||
# CS: # GPIO0_B7 (SPI0_CSN1, physical 26)
|
||||
# pin: 15
|
||||
|
||||
@@ -31,6 +31,8 @@ Lora:
|
||||
- pin: 103 # GPIO3_A7 (physical 16)
|
||||
gpiochip: 3
|
||||
line: 7
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
spidev: spidev0.1
|
||||
# CS: # GPIO0_B7 (SPI0_CSN1, physical 26)
|
||||
# pin: 15
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
Meta:
|
||||
name: Lora Meshstick SX1262
|
||||
name: PiggyStick LR1121
|
||||
support: community
|
||||
compatible:
|
||||
- usb
|
||||
@@ -12,6 +12,6 @@ Lora:
|
||||
Busy: 4
|
||||
spidev: ch341
|
||||
DIO3_TCXO_VOLTAGE: 1.8
|
||||
# USB_Serialnum: 12345678
|
||||
# USB_Serialnum: 12345678
|
||||
USB_PID: 0x5512
|
||||
USB_VID: 0x1A86
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
# Station G3 + BQ35LORA900V1M Primary Slot
|
||||
# Board Doc: https://wiki.bqvoy.com/en/devkits/station-g3
|
||||
Meta:
|
||||
name: B&Q Station G3 + BQ35LORA900V1M Primary Slot
|
||||
support: official
|
||||
compatible:
|
||||
- raspberry-pi
|
||||
|
||||
Lora:
|
||||
Module: sx1262
|
||||
IRQ: 22
|
||||
Reset: 16
|
||||
Busy: 24
|
||||
DIO3_TCXO_VOLTAGE: true
|
||||
DIO2_AS_RF_SWITCH: true
|
||||
spidev: spidev0.0
|
||||
#CS: 8
|
||||
@@ -1,3 +1,5 @@
|
||||
# This config works with all revisions of the Meshtoad USB radio.
|
||||
|
||||
Meta:
|
||||
name: meshtoad-e22
|
||||
support: official
|
||||
|
||||
@@ -0,0 +1,252 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Meshtastic Ethernet OTA Upload Tool
|
||||
|
||||
Uploads firmware to RP2350-based Meshtastic devices via Ethernet (W5500).
|
||||
Compresses firmware with GZIP and sends it over TCP using the MOTA protocol.
|
||||
Authenticates using SHA256 challenge-response with a pre-shared key (PSK).
|
||||
|
||||
Usage:
|
||||
python bin/eth-ota-upload.py --host 192.168.1.100 firmware.bin
|
||||
python bin/eth-ota-upload.py --host 192.168.1.100 --psk mySecretKey firmware.bin
|
||||
python bin/eth-ota-upload.py --host 192.168.1.100 --psk-hex 6d65736874... firmware.bin
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import gzip
|
||||
import hashlib
|
||||
import socket
|
||||
import struct
|
||||
import sys
|
||||
import time
|
||||
|
||||
# Default PSK matching the firmware default: "meshtastic_ota_default_psk_v1!!!"
|
||||
DEFAULT_PSK = b"meshtastic_ota_default_psk_v1!!!"
|
||||
|
||||
|
||||
def crc32(data: bytes) -> int:
|
||||
"""Compute CRC32 matching ErriezCRC32 (standard CRC32 with final XOR)."""
|
||||
import binascii
|
||||
|
||||
return binascii.crc32(data) & 0xFFFFFFFF
|
||||
|
||||
|
||||
def load_firmware(path: str) -> bytes:
|
||||
"""Load firmware file, compressing with GZIP if not already compressed."""
|
||||
# Reject UF2 files — OTA requires raw .bin firmware
|
||||
if path.lower().endswith(".uf2"):
|
||||
bin_path = path.rsplit(".", 1)[0] + ".bin"
|
||||
print(f"ERROR: UF2 files cannot be used for OTA updates.")
|
||||
print(f" The Updater/picoOTA expects raw .bin firmware.")
|
||||
print(f" Try: {bin_path}")
|
||||
sys.exit(1)
|
||||
|
||||
with open(path, "rb") as f:
|
||||
data = f.read()
|
||||
|
||||
# Check if already GZIP compressed (magic bytes 1f 8b)
|
||||
if data[:2] == b"\x1f\x8b":
|
||||
print(f"Firmware already GZIP compressed: {len(data):,} bytes")
|
||||
return data
|
||||
|
||||
print(f"Firmware raw size: {len(data):,} bytes")
|
||||
compressed = gzip.compress(data, compresslevel=9)
|
||||
ratio = len(compressed) / len(data) * 100
|
||||
print(f"GZIP compressed: {len(compressed):,} bytes ({ratio:.1f}%)")
|
||||
return compressed
|
||||
|
||||
|
||||
def authenticate(sock: socket.socket, psk: bytes) -> bool:
|
||||
"""Perform SHA256 challenge-response authentication with the device."""
|
||||
# Receive 32-byte nonce from server
|
||||
nonce = b""
|
||||
while len(nonce) < 32:
|
||||
chunk = sock.recv(32 - len(nonce))
|
||||
if not chunk:
|
||||
print("ERROR: Connection closed during authentication")
|
||||
return False
|
||||
nonce += chunk
|
||||
|
||||
# Compute SHA256(nonce || PSK)
|
||||
h = hashlib.sha256()
|
||||
h.update(nonce)
|
||||
h.update(psk)
|
||||
response = h.digest()
|
||||
|
||||
# Send 32-byte response
|
||||
sock.sendall(response)
|
||||
|
||||
# Wait for auth result (1 byte)
|
||||
result = sock.recv(1)
|
||||
if not result:
|
||||
print("ERROR: No authentication response")
|
||||
return False
|
||||
|
||||
if result[0] == 0x06: # ACK
|
||||
print("Authentication successful.")
|
||||
return True
|
||||
elif result[0] == 0x07: # OTA_ERR_AUTH
|
||||
print("ERROR: Authentication failed — wrong PSK")
|
||||
return False
|
||||
else:
|
||||
print(f"ERROR: Unexpected auth response 0x{result[0]:02X}")
|
||||
return False
|
||||
|
||||
|
||||
def upload_firmware(host: str, port: int, firmware: bytes, psk: bytes, timeout: float) -> bool:
|
||||
"""Upload firmware over TCP using the MOTA protocol with PSK authentication."""
|
||||
fw_crc = crc32(firmware)
|
||||
fw_size = len(firmware)
|
||||
|
||||
print(f"Connecting to {host}:{port}...")
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
sock.settimeout(timeout)
|
||||
|
||||
try:
|
||||
sock.connect((host, port))
|
||||
print("Connected.")
|
||||
|
||||
# Step 1: Authenticate
|
||||
print("Authenticating...")
|
||||
if not authenticate(sock, psk):
|
||||
return False
|
||||
|
||||
# Step 2: Send 12-byte MOTA header: magic(4) + size(4) + crc32(4)
|
||||
header = struct.pack("<4sII", b"MOTA", fw_size, fw_crc)
|
||||
sock.sendall(header)
|
||||
print(f"Header sent: size={fw_size:,}, CRC32=0x{fw_crc:08X}")
|
||||
|
||||
# Wait for ACK (1 byte)
|
||||
ack = sock.recv(1)
|
||||
if not ack or ack[0] != 0x06:
|
||||
error_codes = {
|
||||
0x02: "Size error",
|
||||
0x04: "Invalid magic",
|
||||
0x05: "Update.begin() failed",
|
||||
}
|
||||
code = ack[0] if ack else 0xFF
|
||||
msg = error_codes.get(code, f"Unknown error 0x{code:02X}")
|
||||
print(f"ERROR: Server rejected header: {msg}")
|
||||
return False
|
||||
|
||||
print("Header accepted. Uploading firmware...")
|
||||
|
||||
# Send firmware in 1KB chunks
|
||||
chunk_size = 1024
|
||||
sent = 0
|
||||
start_time = time.time()
|
||||
|
||||
while sent < fw_size:
|
||||
end = min(sent + chunk_size, fw_size)
|
||||
chunk = firmware[sent:end]
|
||||
sock.sendall(chunk)
|
||||
sent = end
|
||||
|
||||
# Progress bar
|
||||
pct = sent * 100 // fw_size
|
||||
bar_len = 40
|
||||
filled = bar_len * sent // fw_size
|
||||
bar = "█" * filled + "░" * (bar_len - filled)
|
||||
elapsed = time.time() - start_time
|
||||
speed = sent / elapsed if elapsed > 0 else 0
|
||||
sys.stdout.write(f"\r [{bar}] {pct:3d}% {sent:,}/{fw_size:,} ({speed/1024:.1f} KB/s)")
|
||||
sys.stdout.flush()
|
||||
|
||||
elapsed = time.time() - start_time
|
||||
print(f"\n Transfer complete in {elapsed:.1f}s")
|
||||
|
||||
# Wait for final result (1 byte)
|
||||
print("Waiting for verification...")
|
||||
result = sock.recv(1)
|
||||
if not result:
|
||||
print("ERROR: No response from device")
|
||||
return False
|
||||
|
||||
result_codes = {
|
||||
0x00: "OK — Update staged, device rebooting",
|
||||
0x01: "CRC mismatch",
|
||||
0x02: "Size error",
|
||||
0x03: "Write error",
|
||||
0x04: "Magic mismatch",
|
||||
0x05: "Updater.begin() failed",
|
||||
0x07: "Auth failed",
|
||||
0x08: "Timeout",
|
||||
}
|
||||
code = result[0]
|
||||
msg = result_codes.get(code, f"Unknown result 0x{code:02X}")
|
||||
|
||||
if code == 0x00:
|
||||
print(f"SUCCESS: {msg}")
|
||||
return True
|
||||
else:
|
||||
print(f"ERROR: {msg}")
|
||||
return False
|
||||
|
||||
except socket.timeout:
|
||||
print("ERROR: Connection timed out")
|
||||
return False
|
||||
except ConnectionRefusedError:
|
||||
print(f"ERROR: Connection refused by {host}:{port}")
|
||||
return False
|
||||
except OSError as e:
|
||||
print(f"ERROR: {e}")
|
||||
return False
|
||||
finally:
|
||||
sock.close()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Upload firmware to Meshtastic RP2350 devices via Ethernet OTA"
|
||||
)
|
||||
parser.add_argument("firmware", help="Path to firmware .bin or .bin.gz file")
|
||||
parser.add_argument("--host", required=True, help="Device IP address")
|
||||
parser.add_argument(
|
||||
"--port", type=int, default=4243, help="OTA port (default: 4243)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--timeout",
|
||||
type=float,
|
||||
default=60.0,
|
||||
help="Socket timeout in seconds (default: 60)",
|
||||
)
|
||||
psk_group = parser.add_mutually_exclusive_group()
|
||||
psk_group.add_argument(
|
||||
"--psk",
|
||||
type=str,
|
||||
help="Pre-shared key as UTF-8 string (default: meshtastic_ota_default_psk_v1!!!)",
|
||||
)
|
||||
psk_group.add_argument(
|
||||
"--psk-hex",
|
||||
type=str,
|
||||
help="Pre-shared key as hex string (e.g., 6d65736874...)",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
# Resolve PSK
|
||||
if args.psk:
|
||||
psk = args.psk.encode("utf-8")
|
||||
elif args.psk_hex:
|
||||
try:
|
||||
psk = bytes.fromhex(args.psk_hex)
|
||||
except ValueError:
|
||||
print("ERROR: Invalid hex string for --psk-hex")
|
||||
sys.exit(1)
|
||||
else:
|
||||
psk = DEFAULT_PSK
|
||||
|
||||
print("Meshtastic Ethernet OTA Upload")
|
||||
print("=" * 40)
|
||||
|
||||
firmware = load_firmware(args.firmware)
|
||||
|
||||
if upload_firmware(args.host, args.port, firmware, psk, args.timeout):
|
||||
print("\nDevice is rebooting with new firmware.")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print("\nUpload failed.")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+37
@@ -0,0 +1,37 @@
|
||||
#!/usr/bin/env bash
|
||||
# lint-ifdef-complexity.sh - flag preprocessor conditionals that OR together
|
||||
# too many defined() terms (e.g. long display-driver chains in main.cpp).
|
||||
#
|
||||
# Such chains should be replaced by a single umbrella feature macro computed
|
||||
# once in a central header (this repo already has HAS_TFT / HAS_SCREEN in
|
||||
# src/configuration.h). The compiler has no warning for this, so we lint it.
|
||||
#
|
||||
# Emits one line per offending logical #if/#elif in the format
|
||||
# <path>:<line>:<col>:<severity>:<message>:<code>
|
||||
# which trunk parses via parse_regex. Always exits 0; findings go to stdout.
|
||||
#
|
||||
# Usage: lint-ifdef-complexity.sh <file> [<file> ...]
|
||||
# Tune the limit with MAX_DEFINED (default 5 → warns at 6+ terms).
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
MAX_DEFINED="${MAX_DEFINED:-5}"
|
||||
|
||||
awk -v max="${MAX_DEFINED}" '
|
||||
# Start of a #if / #elif (but not #ifdef / #ifndef, which carry one term).
|
||||
/^[[:space:]]*#[[:space:]]*(if|elif)([[:space:](]|$)/ {
|
||||
startln = FNR
|
||||
logical = $0
|
||||
# Splice backslash line-continuations into one logical line.
|
||||
while (logical ~ /\\[[:space:]]*$/ && (getline nxt) > 0) {
|
||||
sub(/\\[[:space:]]*$/, "", logical)
|
||||
logical = logical " " nxt
|
||||
}
|
||||
tmp = logical
|
||||
n = gsub(/defined/, "", tmp) # gsub returns the match count
|
||||
if (n > max) {
|
||||
printf "%s:%d:1:warning:preprocessor conditional ORs %d defined() terms (limit %d); replace the driver list with an umbrella feature macro such as HAS_TFT or HAS_SCREEN:too-many-defined\n", \
|
||||
FILENAME, startln, n, max
|
||||
}
|
||||
}
|
||||
' "$@"
|
||||
@@ -22,9 +22,10 @@ if [[ ! -d bin/_generated/meshtastic ]]; then
|
||||
fi
|
||||
|
||||
# 2) Pick a Python interpreter that has the meshtastic deps installed.
|
||||
# Prefer the mcp-server venv (most likely to be set up by the operator).
|
||||
# Prefer a local venv; otherwise fall back to system python3 (install the
|
||||
# `meshtastic` package there, or run this under `uv run --with meshtastic`).
|
||||
PY="python3"
|
||||
for cand in mcp-server/.venv/bin/python3 .venv/bin/python3; do
|
||||
for cand in .venv/bin/python3; do
|
||||
if [[ -x "$cand" ]]; then
|
||||
PY="$cand"
|
||||
break
|
||||
@@ -70,4 +71,5 @@ echo "Done. To load on Portduino native:"
|
||||
echo " cp build/fixtures/nodedb/nodes_v25_1000.proto ~/.portduino/default/prefs/nodes.proto"
|
||||
echo ""
|
||||
echo "To push to a hardware device:"
|
||||
echo " Use the mcp-server tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"
|
||||
echo " Use the meshtastic-mcp tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"
|
||||
echo " (MCP server: https://github.com/meshtastic/meshtastic-mcp)"
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
# Namespace rewrite:
|
||||
# The .proto files declare `package meshtastic;`, which makes protoc emit
|
||||
# imports like `from meshtastic import mesh_pb2`. That conflicts with the
|
||||
# PyPI `meshtastic` package (which the mcp-server relies on for its
|
||||
# PyPI `meshtastic` package (which the meshtastic-mcp tooling relies on for its
|
||||
# SerialInterface/BLEInterface transport). We post-process the generated
|
||||
# files to live under `meshtastic_v25` instead — both the directory layout
|
||||
# and all internal imports — so they coexist cleanly with the PyPI package.
|
||||
|
||||
Executable
+311
@@ -0,0 +1,311 @@
|
||||
#!/usr/bin/env bash
|
||||
# Run native PlatformIO unit tests and emit a single, unambiguous verdict.
|
||||
#
|
||||
# Why this exists: PlatformIO reports failures three different ways ([FAILED], :FAIL:,
|
||||
# [ERRORED]) and an all-pass run prints "N succeeded" with NO "0 failed" clause — so naive
|
||||
# greps produce false greens (see .notes/test-passfail-filter.md). This script encodes the
|
||||
# correct logic once, and cross-checks the number of suites that actually ran against the
|
||||
# canonical set in test/ so a suite silently going missing shows up as AMBER, not green.
|
||||
#
|
||||
# Usage:
|
||||
# ./bin/run-tests.sh # run all suites, full verdict + count cross-check
|
||||
# ./bin/run-tests.sh -f test_utf8 # run one suite (yields FILTERED, not GREEN)
|
||||
# ./bin/run-tests.sh -e native # override env (default: coverage)
|
||||
# ./bin/run-tests.sh --quiet # only print the final RESULT line
|
||||
#
|
||||
# Exit codes: 0 = GREEN, 1 = RED, 2 = AMBER, 3 = FILTERED.
|
||||
#
|
||||
# Verdicts:
|
||||
# GREEN — all canonical suites ran, all passed, no ignored test cases.
|
||||
# AMBER — all that ran passed, but something was lost: a suite silently went missing on a
|
||||
# full run, or individual test cases were skipped (Unity TEST_IGNORE / :IGNORE:).
|
||||
# FILTERED — a -f run completed cleanly; suites not in the filter were intentionally skipped.
|
||||
# Use this when iterating on a single suite; it is not a quality signal.
|
||||
# RED — at least one failure, build error, or sanitizer fault.
|
||||
#
|
||||
# The final line is machine-readable, e.g.:
|
||||
# RESULT: GREEN N/N suites passed
|
||||
# RESULT: AMBER N/M suites ran (missing: test_radio test_serial) — all that ran passed
|
||||
# RESULT: AMBER 3 test case(s) ignored
|
||||
# RESULT: FILTERED 1/N suites ran (not run: …) — filtered: test_utf8
|
||||
# RESULT: RED test_traffic_management: 1 failed (or: build/crash error)
|
||||
# RESULT: RED sanitizer fault — SUMMARY: AddressSanitizer: 1272 byte(s) leaked (tests may have
|
||||
# all passed; the coverage build aborts at exit on an ASan/LSan fault — often shown only
|
||||
# as [ERRORED]/SIGHUP. The script names it and points at running the binary bare.)
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
cd "$ROOT_DIR"
|
||||
|
||||
ENV="coverage"
|
||||
FILTER=""
|
||||
QUIET=false
|
||||
PASSTHRU=()
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
-f)
|
||||
FILTER="$2"
|
||||
PASSTHRU+=("-f" "$2")
|
||||
shift 2
|
||||
;;
|
||||
-e)
|
||||
ENV="$2"
|
||||
shift 2
|
||||
;;
|
||||
--quiet)
|
||||
QUIET=true
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
PASSTHRU+=("$1")
|
||||
shift
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Locate pio (PATH, then the standard PlatformIO venv).
|
||||
PIO="$(command -v pio || command -v platformio || echo "$HOME/.platformio/penv/bin/pio")"
|
||||
if [[ ! -x $PIO ]] && ! command -v "$PIO" >/dev/null 2>&1; then
|
||||
echo "RESULT: RED pio not found (looked in PATH and ~/.platformio/penv/bin)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
LOG="$(mktemp -t meshtest.XXXXXX.log)"
|
||||
MARKER=""
|
||||
PROGRESS_PID=""
|
||||
trap 'rm -f "$LOG" "${MARKER:-}"; [[ -n ${PROGRESS_PID:-} ]] && kill "$PROGRESS_PID" 2>/dev/null' EXIT
|
||||
|
||||
# Canonical suite set = the directories in test/. This is the source of truth for
|
||||
# "what should run"; a filtered run only expects its filtered suite.
|
||||
mapfile -t ALL_SUITES < <(find test -maxdepth 1 -type d -name 'test_*' -printf '%f\n' | sort)
|
||||
EXPECTED_COUNT=${#ALL_SUITES[@]}
|
||||
|
||||
# Canonical suite count — the registered total, maintained in test/native-suite-count.
|
||||
# Update that file whenever a test suite is added or removed.
|
||||
CANONICAL_COUNT_FILE="test/native-suite-count"
|
||||
if [[ -f $CANONICAL_COUNT_FILE ]]; then
|
||||
CANONICAL_COUNT=$(tr -d '[:space:]' <"$CANONICAL_COUNT_FILE")
|
||||
else
|
||||
CANONICAL_COUNT=""
|
||||
fi
|
||||
|
||||
# Cached object-count for this env, written after each completed build (in the gitignored build
|
||||
# dir). Used as the progress denominator: accurate for a full rebuild (every object recompiles),
|
||||
# only a rough upper bound for an incremental run.
|
||||
BASELINE_FILE=".pio/build/${ENV}/.runtests-objcount"
|
||||
|
||||
# Progress trail file (gitignored build dir). ALWAYS written so a backgrounded/piped run can be
|
||||
# checked mid-build with `tail -f` — that's the whole point: don't fly blind on a 20-min rebuild.
|
||||
PROGRESS_FILE=".pio/build/${ENV}/.runtests-progress"
|
||||
|
||||
# --- Progress heartbeat ------------------------------------------------------
|
||||
# Emit ONE status line every few seconds: build = objects (re)compiled this run / cached total +
|
||||
# best-effort ETA; test = suites finished / expected. Appends to $PROGRESS_FILE always (tail it to
|
||||
# check on a backgrounded run); also live-updates the tty when $5=1 (interactive --quiet). Never
|
||||
# touches $LOG, which is parsed for the verdict, so piped/CI captures stay clean.
|
||||
progress_monitor() {
|
||||
local marker="$1" objtotal="$2" testtotal="$3" pfile="$4" totty="$5" start now el done ran eta line
|
||||
start=$(date +%s)
|
||||
while :; do
|
||||
now=$(date +%s)
|
||||
el=$((now - start))
|
||||
if grep -q 'Testing\.\.\.' "$LOG" 2>/dev/null; then
|
||||
ran=$(grep -cE "${ENV}:test_[a-z0-9_]+ \[(PASSED|FAILED|ERRORED)\]" "$LOG" 2>/dev/null)
|
||||
line=$(printf '[test] %s/%s suites done — %dm%02ds' "$ran" "$testtotal" $((el / 60)) $((el % 60)))
|
||||
else
|
||||
done=$(find ".pio/build/${ENV}" -name '*.o' -newer "$marker" 2>/dev/null | wc -l)
|
||||
if ((objtotal > 0 && done > 0)); then
|
||||
eta=$((objtotal > done ? (objtotal - done) * el / done : 0))
|
||||
line=$(printf '[build] %d/%d objs — %dm%02ds — ETA ~%dm%02ds' \
|
||||
"$done" "$objtotal" $((el / 60)) $((el % 60)) $((eta / 60)) $((eta % 60)))
|
||||
else
|
||||
# done==0 (incremental: nothing to rebuild yet) or no cached baseline — no ETA yet.
|
||||
line=$(printf '[build] %d objs compiled — %dm%02ds' "$done" $((el / 60)) $((el % 60)))
|
||||
fi
|
||||
fi
|
||||
printf '%s\n' "$line" >>"$pfile" 2>/dev/null # file trail (always)
|
||||
[[ $totty == 1 ]] && printf '\r\033[K%s' "$line" >/dev/tty 2>/dev/null # live line (human)
|
||||
sleep 4
|
||||
done
|
||||
}
|
||||
|
||||
# Launch the heartbeat for every run. It writes the progress file unconditionally; the live tty
|
||||
# line only when interactive AND --quiet (where pio's own output is hidden — otherwise pio's
|
||||
# streamed compile lines already show progress and a \r line would just fight them).
|
||||
mkdir -p ".pio/build/${ENV}" 2>/dev/null || true
|
||||
: >"$PROGRESS_FILE" 2>/dev/null || true
|
||||
MARKER="$(mktemp -t meshtest-mark.XXXXXX)"
|
||||
TOTTY=0
|
||||
{ $QUIET && [[ -t 1 ]]; } && TOTTY=1
|
||||
progress_monitor "$MARKER" "$(cat "$BASELINE_FILE" 2>/dev/null || echo 0)" \
|
||||
"$([[ -n $FILTER ]] && echo 1 || echo "$EXPECTED_COUNT")" "$PROGRESS_FILE" "$TOTTY" &
|
||||
PROGRESS_PID=$!
|
||||
|
||||
if ! $QUIET; then
|
||||
echo "Running: $PIO test -e $ENV ${PASSTHRU[*]-} (expecting $EXPECTED_COUNT suites)"
|
||||
fi
|
||||
echo "progress: tail -f $PROGRESS_FILE" >&2
|
||||
if [[ ! -t 1 ]] && ! $QUIET; then
|
||||
echo "hint: stdout is a pipe — build errors appear at the top of output and may be lost; use --quiet to get just the RESULT line" >&2
|
||||
fi
|
||||
|
||||
# Run pio, tee to log. PIPESTATUS[0] is pio's real exit (NOT tee's).
|
||||
if $QUIET; then
|
||||
"$PIO" test -e "$ENV" "${PASSTHRU[@]}" >"$LOG" 2>&1
|
||||
else
|
||||
"$PIO" test -e "$ENV" "${PASSTHRU[@]}" 2>&1 | tee "$LOG"
|
||||
fi
|
||||
PIO_RC=${PIPESTATUS[0]}
|
||||
|
||||
# Stop the heartbeat, clear its line, and cache this build's object total for next time.
|
||||
if [[ -n $PROGRESS_PID ]]; then
|
||||
kill "$PROGRESS_PID" 2>/dev/null
|
||||
wait "$PROGRESS_PID" 2>/dev/null
|
||||
PROGRESS_PID=""
|
||||
# Clear the live line only if we were writing one — opening /dev/tty when there is none is
|
||||
# itself a redirect-open error the trailing 2>/dev/null cannot suppress.
|
||||
[[ $TOTTY == 1 ]] && printf '\r\033[K' >/dev/tty 2>/dev/null
|
||||
fi
|
||||
[[ -d ".pio/build/${ENV}" ]] && find ".pio/build/${ENV}" -name '*.o' 2>/dev/null | wc -l >"$BASELINE_FILE" 2>/dev/null || true
|
||||
|
||||
# --- Outcome detection -------------------------------------------------------
|
||||
# The SAME outcome is spelled differently depending on which layer emitted the line — this is
|
||||
# the trap that produces false greens (grepping ":PASS" misses pio's "[PASSED]", grepping
|
||||
# "[FAILED]" misses Unity's ":FAIL:"). So every regex below matches BOTH spellings:
|
||||
# pass: Unity per-assertion ":PASS" | pio per-suite "[PASSED]" | summary "N succeeded"
|
||||
# fail: Unity per-assertion ":FAIL:" | pio per-suite "[FAILED]" | summary "M failed"
|
||||
# error: pio build/crash "[ERRORED]" | Unity "M Failures" | compiler "error:"
|
||||
# Match \b after :PASS/:FAIL so ":PASSED"/":FAILED" forms are also caught either way.
|
||||
FAIL_RE=':FAIL\b|\[FAILED\]|\[ERRORED\]|[1-9][0-9]* failed|[0-9]+ Tests [1-9][0-9]* Failures|error:|undefined reference|Segmentation fault|terminate called|SIGHUP|SIGSEGV|SIGABRT'
|
||||
# Positive proof tests actually ran & passed (absence != success). Accept any pass spelling:
|
||||
# the per-test/per-suite tokens OR a success summary line.
|
||||
PASS_RE=':PASS\b|\[PASSED\]|test cases: *[0-9]+ succeeded|[0-9]+ Tests 0 Failures'
|
||||
# Sanitizer (ASan/LSan/UBSan/TSan) fault signatures. The coverage build is sanitizer-instrumented
|
||||
# and aborts NON-ZERO at exit on a fault — most often a LeakSanitizer leak — AFTER every test has
|
||||
# already printed [PASSED]. pio then reports [ERRORED]/SIGHUP with no :FAIL: anywhere, so it
|
||||
# masquerades as a phantom "N-1 of N succeeded". See .notes/test-passfail-filter.md.
|
||||
# Match only real FAULT lines, never the benign "AddressSanitizer: failed to intercept '...'"
|
||||
# startup noise that prints on every sanitizer run (it'd mislabel a normal [FAILED] as a leak).
|
||||
# Formats per LLVM/Google sanitizer docs: ASan/LSan emit "==PID==ERROR: <San>: ...", UBSan emits
|
||||
# "file:line:col: runtime error: ...", TSan emits "WARNING: ThreadSanitizer: ..."; all close with
|
||||
# a "SUMMARY: <San>: ..." line (LSan-under-ASan reports its SUMMARY as "AddressSanitizer").
|
||||
SAN_RE='(ERROR|WARNING): (Address|Leak|Thread|UndefinedBehavior)Sanitizer:|SUMMARY: (Address|Leak|Thread|UndefinedBehavior)Sanitizer:|Direct leak of|Indirect leak of|detected memory leaks|heap-use-after-free|heap-buffer-overflow|stack-buffer-overflow|attempting double-free|LeakSanitizer has encountered a fatal error|runtime error:'
|
||||
|
||||
# Suites that produced a per-suite verdict. pio emits "coverage:test_x [PASSED|FAILED|ERRORED]";
|
||||
# a SKIPPED suite (hardware-only on native) is "accounted for" too, so it doesn't read as missing.
|
||||
mapfile -t RAN_SUITES < <(grep -oE "${ENV}:test_[a-z0-9_]+ \[(PASSED|FAILED|ERRORED)\]" "$LOG" |
|
||||
sed -E "s/^${ENV}:(test_[a-z0-9_]+) .*/\1/" | sort -u)
|
||||
RAN_COUNT=${#RAN_SUITES[@]}
|
||||
# Suites pio explicitly skipped (don't count these as "missing" in the canonical cross-check).
|
||||
mapfile -t SKIPPED_SUITES < <(grep -oE "${ENV}:test_[a-z0-9_]+.*\bSKIPPED\b" "$LOG" |
|
||||
grep -oE "test_[a-z0-9_]+" | sort -u)
|
||||
|
||||
verdict_red() {
|
||||
local detail bin
|
||||
detail="$(grep -nE '\[FAILED\]|:FAIL:|\[ERRORED\]' "$LOG" | head -3 | sed 's/^/ /')"
|
||||
echo ""
|
||||
echo "RED — failures detected:"
|
||||
[[ -n $detail ]] && echo "$detail"
|
||||
grep -E 'test cases:' "$LOG" | tail -1 | sed 's/^/ /'
|
||||
|
||||
# Path to the test binary for the "run it bare" hint. For native/coverage the test program is
|
||||
# the env executable (e.g. .pio/build/coverage/meshtasticd), NOT a file named 'program'.
|
||||
bin="$(find ".pio/build/${ENV}" -maxdepth 1 -type f -executable ! -name '*.so' 2>/dev/null | head -1)"
|
||||
[[ -z $bin ]] && bin=".pio/build/${ENV}/<program> (build it first: $PIO test -e ${ENV} ${FILTER:+-f $FILTER} --without-testing)"
|
||||
|
||||
# Sanitizer fault (ASan/LSan/UBSan/TSan): name the real cause instead of "build/crash error".
|
||||
if grep -qE "$SAN_RE" "$LOG"; then
|
||||
grep -nE "$SAN_RE" "$LOG" | head -4 | sed 's/^/ /'
|
||||
echo " -> sanitizer fault: if every test above is PASS, this is an exit-time abort, not a failed assertion."
|
||||
echo " -> read the full report by running the binary BARE (gdb hides it via ptrace): ./$bin 2>&1 | tail -40"
|
||||
echo "RESULT: RED sanitizer fault — $(grep -ohE 'SUMMARY: [A-Za-z]+Sanitizer:.*' "$LOG" | tail -1 || echo 'see report above')"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# All tests passed but the process still aborted at EXIT (ERRORED/SIGHUP/SIGABRT) and the
|
||||
# sanitizer report was swallowed by the runner (often surfaced only as SIGHUP). Almost always a
|
||||
# sanitizer fault — point at how to surface it rather than calling it a generic crash.
|
||||
if grep -qE "$PASS_RE" "$LOG" && grep -qE '\[ERRORED\]|SIGHUP|SIGABRT' "$LOG" && ! grep -qE ':FAIL\b|\[FAILED\]' "$LOG"; then
|
||||
echo " -> all tests passed but the process aborted at EXIT — likely an ASan/LSan fault whose report"
|
||||
echo " the runner swallowed (commonly shown as SIGHUP). Run the binary BARE to see it: ./$bin 2>&1 | tail -40"
|
||||
echo "RESULT: RED exit-time abort (tests passed; likely sanitizer — see hint above)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "RESULT: RED $(grep -oE '[0-9]+ failed' "$LOG" | tail -1 || echo 'build/crash error')"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# RED: pio non-zero, any failure marker, or no positive summary at all (build died early).
|
||||
if [[ $PIO_RC -ne 0 ]] || grep -qE "$FAIL_RE" "$LOG"; then
|
||||
verdict_red
|
||||
fi
|
||||
if ! grep -qE "$PASS_RE" "$LOG"; then
|
||||
echo ""
|
||||
echo "RESULT: RED no success summary found (build error / no tests ran?) — see log"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Canonical-count rating suffix — appended to every verdict line so the result is always
|
||||
# rated against the registered total, not just the directory count.
|
||||
# If the two counts diverge (suite added/removed without updating native-suite-count), that
|
||||
# is itself surfaced as AMBER before we reach any verdict.
|
||||
canonical_rating() {
|
||||
if [[ -n $CANONICAL_COUNT ]]; then
|
||||
echo "[canonical: ${RAN_COUNT}/${CANONICAL_COUNT}]"
|
||||
fi
|
||||
}
|
||||
|
||||
# AMBER: directory count disagrees with native-suite-count — file needs updating.
|
||||
if [[ -n $CANONICAL_COUNT && $EXPECTED_COUNT -ne $CANONICAL_COUNT ]]; then
|
||||
echo ""
|
||||
if [[ $EXPECTED_COUNT -gt $CANONICAL_COUNT ]]; then
|
||||
echo "RESULT: AMBER test/ has $EXPECTED_COUNT suite directories but native-suite-count says $CANONICAL_COUNT — update test/native-suite-count after registering new suites"
|
||||
else
|
||||
echo "RESULT: AMBER test/ has $EXPECTED_COUNT suite directories but native-suite-count says $CANONICAL_COUNT — update test/native-suite-count after removing suites"
|
||||
fi
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# AMBER: individual test cases were skipped (Unity TEST_IGNORE → :IGNORE: in output).
|
||||
# Applies to both full and filtered runs — a skipped test case is a lost signal either way.
|
||||
mapfile -t IGNORED_TESTS < <(grep -oE '[^:]+:[0-9]+:[^:]+:IGNORE:.*' "$LOG" 2>/dev/null | sed 's/:IGNORE:.*//' | sort -u)
|
||||
IGNORED_COUNT=${#IGNORED_TESTS[@]}
|
||||
if [[ $IGNORED_COUNT -gt 0 ]]; then
|
||||
IGNORE_DETAIL="$(printf '%s\n' "${IGNORED_TESTS[@]}" | head -5 | sed 's/^/ /')"
|
||||
echo ""
|
||||
echo "$IGNORE_DETAIL"
|
||||
echo ""
|
||||
echo "RESULT: AMBER ${IGNORED_COUNT} test case(s) ignored $(canonical_rating)"
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# AMBER: full run only — a canonical suite neither ran NOR was explicitly skipped (silently missing).
|
||||
ACCOUNTED_COUNT=$((RAN_COUNT + ${#SKIPPED_SUITES[@]}))
|
||||
if [[ -z $FILTER && $ACCOUNTED_COUNT -lt $EXPECTED_COUNT ]]; then
|
||||
missing=()
|
||||
for s in "${ALL_SUITES[@]}"; do
|
||||
printf '%s\n' "${RAN_SUITES[@]}" "${SKIPPED_SUITES[@]}" | grep -qx "$s" || missing+=("$s")
|
||||
done
|
||||
echo ""
|
||||
echo "RESULT: AMBER ${RAN_COUNT}/${EXPECTED_COUNT} suites ran (missing: ${missing[*]}) — all that ran passed $(canonical_rating)"
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# FILTERED: a -f run completed cleanly. Suites outside the filter were intentionally not run;
|
||||
# this is not a quality signal and is distinct from suites that went missing unexpectedly.
|
||||
if [[ -n $FILTER ]]; then
|
||||
not_run=()
|
||||
for s in "${ALL_SUITES[@]}"; do
|
||||
printf '%s\n' "${RAN_SUITES[@]}" "${SKIPPED_SUITES[@]}" | grep -qx "$s" || not_run+=("$s")
|
||||
done
|
||||
echo "RESULT: FILTERED ${RAN_COUNT}/${EXPECTED_COUNT} suites ran (not run: ${not_run[*]}) — filtered: $FILTER $(canonical_rating)"
|
||||
exit 3
|
||||
fi
|
||||
|
||||
# GREEN: all canonical suites ran, all passed, no ignored test cases.
|
||||
echo "RESULT: GREEN ${RAN_COUNT}/${EXPECTED_COUNT} suites passed $(canonical_rating)"
|
||||
exit 0
|
||||
@@ -0,0 +1,54 @@
|
||||
{
|
||||
"build": {
|
||||
"arduino": {
|
||||
"ldscript": "nrf52840_s140_v6.ld"
|
||||
},
|
||||
"core": "nRF5",
|
||||
"cpu": "cortex-m4",
|
||||
"extra_flags": "-DNRF52840_XXAA",
|
||||
"f_cpu": "64000000L",
|
||||
"hwids": [
|
||||
["0x239A", "0x4405"],
|
||||
["0x239A", "0x0029"],
|
||||
["0x239A", "0x002A"],
|
||||
["0x239A", "0x0071"]
|
||||
],
|
||||
"usb_product": "HT-n5262",
|
||||
"mcu": "nrf52840",
|
||||
"variant": "heltec_mesh_tower_v2",
|
||||
"variants_dir": "variants",
|
||||
"bsp": {
|
||||
"name": "adafruit"
|
||||
},
|
||||
"softdevice": {
|
||||
"sd_flags": "-DS140",
|
||||
"sd_name": "s140",
|
||||
"sd_version": "6.1.1",
|
||||
"sd_fwid": "0x00B6"
|
||||
},
|
||||
"bootloader": {
|
||||
"settings_addr": "0xFF000"
|
||||
}
|
||||
},
|
||||
"connectivity": ["bluetooth"],
|
||||
"debug": {
|
||||
"jlink_device": "nRF52840_xxAA",
|
||||
"onboard_tools": ["jlink"],
|
||||
"svd_path": "nrf52840.svd",
|
||||
"openocd_target": "nrf52840-mdk-rs"
|
||||
},
|
||||
"frameworks": ["arduino"],
|
||||
"name": "Heltec MeshTower V2 (Adafruit BSP)",
|
||||
"upload": {
|
||||
"maximum_ram_size": 248832,
|
||||
"maximum_size": 815104,
|
||||
"speed": 115200,
|
||||
"protocol": "nrfutil",
|
||||
"protocols": ["jlink", "nrfjprog", "nrfutil", "stlink"],
|
||||
"use_1200bps_touch": true,
|
||||
"require_upload_port": true,
|
||||
"wait_for_upload_port": true
|
||||
},
|
||||
"url": "https://heltec.org",
|
||||
"vendor": "Heltec"
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
{
|
||||
"build": {
|
||||
"arduino": {
|
||||
"ldscript": "nrf52840_s140_v7.ld"
|
||||
},
|
||||
"core": "nRF5",
|
||||
"cpu": "cortex-m4",
|
||||
"extra_flags": "-DARDUINO_WIO_WM1110 -DNRF52840_XXAA",
|
||||
"f_cpu": "64000000L",
|
||||
"hwids": [
|
||||
["0x239A", "0x8029"],
|
||||
["0x239A", "0x0029"],
|
||||
["0x239A", "0x002A"],
|
||||
["0x239A", "0x802A"],
|
||||
["0x2886", "0x0057"]
|
||||
],
|
||||
"usb_product": "X1-BOOT",
|
||||
"mcu": "nrf52840",
|
||||
"variant": "Seeed_Mesh-Tracker-X1",
|
||||
"bsp": {
|
||||
"name": "adafruit"
|
||||
},
|
||||
"softdevice": {
|
||||
"sd_flags": "-DS140",
|
||||
"sd_name": "s140",
|
||||
"sd_version": "7.3.0",
|
||||
"sd_fwid": "0x0123"
|
||||
},
|
||||
"bootloader": {
|
||||
"settings_addr": "0xFF000"
|
||||
}
|
||||
},
|
||||
"connectivity": ["bluetooth"],
|
||||
"debug": {
|
||||
"jlink_device": "nRF52840_xxAA",
|
||||
"svd_path": "nrf52840.svd",
|
||||
"openocd_target": "nrf52840-mdk-rs"
|
||||
},
|
||||
"frameworks": ["arduino"],
|
||||
"name": "Seeed Mesh Tracker X1",
|
||||
"upload": {
|
||||
"maximum_ram_size": 248832,
|
||||
"maximum_size": 815104,
|
||||
"speed": 115200,
|
||||
"protocol": "nrfutil",
|
||||
"protocols": [
|
||||
"jlink",
|
||||
"nrfjprog",
|
||||
"nrfutil",
|
||||
"stlink",
|
||||
"cmsis-dap",
|
||||
"blackmagic"
|
||||
],
|
||||
"use_1200bps_touch": true,
|
||||
"require_upload_port": true,
|
||||
"wait_for_upload_port": true
|
||||
},
|
||||
"url": "https://www.seeedstudio.com/SenseCAP-MeshTracker-X1-for-Meshtastic-p-6793.html",
|
||||
"vendor": "Seeed Studio"
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
{
|
||||
"build": {
|
||||
"arduino": {
|
||||
"ldscript": "nrf52840_s140_v6.ld"
|
||||
},
|
||||
"core": "nRF5",
|
||||
"cpu": "cortex-m4",
|
||||
"extra_flags": "-DARDUINO_NRF52840_T_IMPULSE_PLUS -DNRF52840_XXAA",
|
||||
"f_cpu": "64000000L",
|
||||
"hwids": [["0x239A", "0x8029"]],
|
||||
"usb_product": "T-Impulse-Plus-nRF52840",
|
||||
"mcu": "nrf52840",
|
||||
"variant": "t-impulse-plus",
|
||||
"variants_dir": "variants",
|
||||
"bsp": {
|
||||
"name": "adafruit"
|
||||
},
|
||||
"softdevice": {
|
||||
"sd_flags": "-DS140",
|
||||
"sd_name": "s140",
|
||||
"sd_version": "6.1.1",
|
||||
"sd_fwid": "0x00B6"
|
||||
},
|
||||
"bootloader": {
|
||||
"settings_addr": "0xFF000"
|
||||
}
|
||||
},
|
||||
"connectivity": ["bluetooth"],
|
||||
"debug": {
|
||||
"jlink_device": "nRF52840_xxAA",
|
||||
"onboard_tools": ["jlink"],
|
||||
"svd_path": "nrf52840.svd"
|
||||
},
|
||||
"frameworks": ["arduino"],
|
||||
"name": "Lilygo T-Impulse-Plus-nRF52840",
|
||||
"upload": {
|
||||
"maximum_ram_size": 248832,
|
||||
"maximum_size": 815104,
|
||||
"require_upload_port": true,
|
||||
"speed": 115200,
|
||||
"protocol": "nrfutil",
|
||||
"protocols": [
|
||||
"jlink",
|
||||
"nrfjprog",
|
||||
"nrfutil",
|
||||
"stlink",
|
||||
"cmsis-dap",
|
||||
"blackmagic"
|
||||
]
|
||||
},
|
||||
"url": "https://www.lilygo.cc/",
|
||||
"vendor": "Lilygo"
|
||||
}
|
||||
Vendored
+1
-1
@@ -33,5 +33,5 @@ if [[ -n $GPG_KEY_ID ]]; then
|
||||
debuild -S -nc -k"$GPG_KEY_ID"
|
||||
else
|
||||
# Build the source deb without signing (forks)
|
||||
debuild -S -nc
|
||||
debuild -S -nc -us -uc
|
||||
fi
|
||||
|
||||
@@ -0,0 +1,277 @@
|
||||
# LoRa Region → Preset Compatibility - Client Implementation Spec
|
||||
|
||||
**Status:** Draft for 2.8 · **Audience:** Meshtastic client app developers (Android first,
|
||||
Apple second, then web/python) · **Firmware side:** implemented in `firmware`
|
||||
(`FromRadio.region_presets`, see below).
|
||||
|
||||
> This document lives in the firmware repo while the feature is developed. It is meant to
|
||||
> graduate to `meshtastic/protobufs` (and/or the docs site) alongside the upstream protobuf
|
||||
> PR that reserves `FromRadio` field **19**.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why this exists
|
||||
|
||||
For 2.8 the LoRa regions and modem presets were reworked. **Not every modem preset is legal
|
||||
in every region** - narrow EU SRD bands, the EU 868 "narrow" band, amateur/ham bands, and
|
||||
the 2.4 GHz band each accept only a specific subset of presets. The firmware already
|
||||
enforces this internally (it clamps or rejects illegal combinations), but until now a client
|
||||
had no way to _know_ the rules, so a user could pick an illegal region+preset pair in the UI
|
||||
and only discover the problem after the device silently corrected it.
|
||||
|
||||
This feature has the firmware **declare the legal region→preset combinations** to the client
|
||||
during the `want_config` handshake, so the client UI can constrain the preset picker to the
|
||||
valid set for the currently selected region (and warn about licensed-only bands). It is
|
||||
purely advisory metadata - the firmware remains the source of truth and still
|
||||
validates/clamps on its own.
|
||||
|
||||
---
|
||||
|
||||
## 2. Protocol additions
|
||||
|
||||
Three new messages in `meshtastic/mesh.proto`, plus one new `FromRadio` oneof variant.
|
||||
|
||||
### 2.1 `FromRadio.region_presets` (field 19)
|
||||
|
||||
```proto
|
||||
message FromRadio {
|
||||
uint32 id = 1;
|
||||
oneof payload_variant {
|
||||
// ... fields 2..18 unchanged ...
|
||||
LoRaRegionPresetMap region_presets = 19;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2.2 Messages
|
||||
|
||||
```proto
|
||||
// A distinct set of legal modem presets shared by one or more LoRa regions.
|
||||
message LoRaPresetGroup {
|
||||
repeated Config.LoRaConfig.ModemPreset presets = 1; // legal presets for this group
|
||||
Config.LoRaConfig.ModemPreset default_preset = 2; // always one of `presets`
|
||||
bool licensed_only = 3; // ham/amateur band → warn/gate
|
||||
}
|
||||
|
||||
// Associates a single LoRa region with its preset group (by index).
|
||||
message LoRaRegionPresets {
|
||||
Config.LoRaConfig.RegionCode region = 1;
|
||||
uint32 group_index = 2; // index into LoRaRegionPresetMap.groups
|
||||
}
|
||||
|
||||
// The full map, delivered grouped to fit one FromRadio packet.
|
||||
message LoRaRegionPresetMap {
|
||||
repeated LoRaPresetGroup groups = 1; // each distinct preset list
|
||||
repeated LoRaRegionPresets region_groups = 2; // every known region → a group index
|
||||
}
|
||||
```
|
||||
|
||||
### 2.3 Why grouped (and the size envelope clients should respect)
|
||||
|
||||
A `FromRadio` packet is capped at **512 bytes** (`MAX_TO_FROM_RADIO_SIZE`). Most regions
|
||||
share one identical preset list (the "standard" 9-preset list), so the map is delivered
|
||||
**grouped**: `groups` holds each _distinct_ preset list once, and `region_groups` maps every
|
||||
known region to one of those groups by index. This keeps the encoded size additive
|
||||
(`groups` + `region_groups`) rather than multiplicative, well under the cap.
|
||||
|
||||
nanopb (firmware) array bounds - clients do **not** need to enforce these, but they bound
|
||||
what you can receive:
|
||||
|
||||
| field | max_count |
|
||||
| ----------------------------------- | ------------------------------------ |
|
||||
| `LoRaRegionPresetMap.groups` | 8 |
|
||||
| `LoRaRegionPresetMap.region_groups` | 38 (= number of `RegionCode` values) |
|
||||
| `LoRaPresetGroup.presets` | 11 |
|
||||
|
||||
---
|
||||
|
||||
## 3. When it is delivered
|
||||
|
||||
`region_presets` is sent **once** during the `want_config` handshake, as a single
|
||||
`FromRadio` message, in this position:
|
||||
|
||||
```text
|
||||
my_info → (deviceuiConfig) → node_info(self) → metadata → region_presets → channel… → config… → moduleConfig… → node_info(others)… → fileInfo… → config_complete_id → (live packets)
|
||||
```
|
||||
|
||||
i.e. **immediately after `metadata` and before the first `channel`**.
|
||||
|
||||
- It is included for a normal full `want_config` and for the **config-only** nonce.
|
||||
- It is **omitted** for the **nodes-only** nonce (that path skips metadata/config entirely).
|
||||
- A client must **not** assume it always arrives (see §5).
|
||||
|
||||
---
|
||||
|
||||
## 4. Decoding into a usable lookup
|
||||
|
||||
Flatten the grouped wire form into `Map<RegionCode, RegionPresetInfo>`:
|
||||
|
||||
```text
|
||||
struct RegionPresetInfo { Set<ModemPreset> presets; ModemPreset default; bool licensedOnly }
|
||||
|
||||
fun decode(map: LoRaRegionPresetMap): Map<RegionCode, RegionPresetInfo> {
|
||||
result = {}
|
||||
for (rg in map.region_groups) {
|
||||
if (rg.group_index >= map.groups.size) continue // defensive: malformed/forward data
|
||||
g = map.groups[rg.group_index]
|
||||
result[rg.region] = RegionPresetInfo(
|
||||
presets = g.presets.toSet(),
|
||||
default = g.default_preset,
|
||||
licensedOnly = g.licensed_only)
|
||||
}
|
||||
return result
|
||||
}
|
||||
```
|
||||
|
||||
Persist this map alongside the rest of the downloaded config so the LoRa config screen can
|
||||
read it synchronously.
|
||||
|
||||
---
|
||||
|
||||
## 5. Semantics & rules (the load-bearing part)
|
||||
|
||||
These rules are what keep the UX correct across firmware versions. Implement all of them.
|
||||
|
||||
1. **Absent region ⇒ no constraint.** If a `RegionCode` does not appear in `region_groups`,
|
||||
the client has _no_ compatibility info for it and **must not restrict** its preset
|
||||
choices (fall back to allowing the full `ModemPreset` list). This happens for a handful
|
||||
of `RegionCode` enum values that have no firmware band table entry (today: `EU_874`,
|
||||
`EU_917`, `ITU1_70CM`, `ITU2_70CM`, `ITU3_70CM`).
|
||||
|
||||
2. **Absent message ⇒ no constraint.** Firmware older than 2.8 never sends `region_presets`.
|
||||
New clients **must** tolerate the message being absent entirely and keep their existing
|
||||
(unconstrained) behavior. Do not block the config screen waiting for it.
|
||||
|
||||
3. **`default_preset`** is always a member of that group's `presets`. Use it to pre-select a
|
||||
preset when the user switches to a region whose valid set does not include the currently
|
||||
selected preset (instead of leaving an illegal selection or guessing).
|
||||
|
||||
4. **`licensed_only`** marks ham/amateur bands. Surface a warning or gate (the firmware also
|
||||
requires the operator's `is_licensed` flag for these regions; coordinate the two so the
|
||||
user isn't allowed to pick a licensed band without acknowledging licensing).
|
||||
|
||||
5. **EU region auto-swap caveat.** The firmware treats the EU sibling regions
|
||||
(`EU_868` / `EU_866` / `EU_N_868`) specially: if the user is in one of them and selects a
|
||||
preset that belongs to a sibling's list, the firmware **swaps the region** rather than
|
||||
rejecting the preset. Consequence for clients: **do not assume the region is immutable
|
||||
across a preset change** - after an admin config write, re-read the resulting
|
||||
`LoRaConfig` and reflect the (possibly changed) region back into the UI.
|
||||
|
||||
6. **Use it as a UI guard, not a validator of truth.** The firmware still validates/clamps
|
||||
on its own. The map exists to prevent the user from _selecting_ an illegal combo; it is
|
||||
not a security or correctness boundary.
|
||||
|
||||
---
|
||||
|
||||
## 6. UI/UX recommendations
|
||||
|
||||
- In the LoRa config screen, when a region is selected, **filter/enable the modem-preset
|
||||
picker to that region's `presets`** (when `use_preset`/`use_modem_preset` is on).
|
||||
- If the current preset is not in the newly selected region's set, switch the selection to
|
||||
that region's `default_preset`.
|
||||
- Show a **licensed badge / confirmation** for regions where `licensed_only == true`.
|
||||
- If a region is absent from the map (rule §5.1) or the whole message is absent (§5.2),
|
||||
render the full preset list as before - never show an empty picker.
|
||||
|
||||
---
|
||||
|
||||
## 7. Forward / backward compatibility
|
||||
|
||||
- **Old clients, new firmware:** an unknown `FromRadio` oneof variant (field 19) is ignored
|
||||
by protobuf/nanopb decoders; the relative ordering of the known messages is unchanged, so
|
||||
existing apps are unaffected.
|
||||
- **New clients, old firmware:** message simply never arrives → treat as "no constraints"
|
||||
(§5.2).
|
||||
- **Enum growth:** new `RegionCode`/`ModemPreset` values may appear over time. Decoders
|
||||
should pass through unknown enum values rather than crashing; an unknown region in
|
||||
`region_groups` is harmless (the client just won't have a localized name for it).
|
||||
|
||||
---
|
||||
|
||||
## 8. Platform notes
|
||||
|
||||
> Verified against the `main` branch of each repo. Both have been refactored away from
|
||||
> older layouts; re-pin file paths against a specific commit if you need them durable.
|
||||
|
||||
### 8.1 Android - `meshtastic/Meshtastic-Android` (Kotlin / Compose, KMP)
|
||||
|
||||
- **Protobufs are a published Maven artifact, _not_ a submodule.** Declared in
|
||||
`gradle/libs.versions.toml` (`org.meshtastic:protobufs`, currently `2.7.25`); generated
|
||||
package is **`org.meshtastic.proto`**. **A `region_presets`-aware build requires a new
|
||||
published `org.meshtastic:protobufs` release**, then bumping that one version string.
|
||||
- **The protobufs are Wire-generated**, so the `FromRadio` oneof is **not** a
|
||||
`payloadVariantCase` enum - each arm is a **nullable field**. Handle the new variant in
|
||||
`FromRadioPacketHandlerImpl.handleFromRadio(...)`
|
||||
(`core/data/.../manager/FromRadioPacketHandlerImpl.kt`) by adding a
|
||||
`regionPresets != null -> …` arm to the existing `when { … }`, delegating to a handler
|
||||
(mirror `handleLocalMetadata` / `handleConfigComplete`).
|
||||
- **State holder:** expose the decoded map from `RadioConfigRepository` /
|
||||
`RadioConfigRepositoryImpl` as a `Flow` (mirroring `localConfigFlow`/`channelSetFlow`),
|
||||
consumed by `feature/settings/.../radio/RadioConfigViewModel.kt`.
|
||||
- **UI:** the region & preset dropdowns are `DropDownPreference`s in
|
||||
`feature/settings/.../radio/component/LoRaConfigItemList.kt` (public composable
|
||||
`LoRaConfigScreen`). Gate/filter the `ChannelOption` (preset) dropdown by the selected
|
||||
`RegionInfo`'s entry in the map.
|
||||
|
||||
### 8.2 Apple - `meshtastic/Meshtastic-Apple` (Swift / SwiftUI)
|
||||
|
||||
- **Protobufs are vendored** into a local Swift package `MeshtasticProtobufs`
|
||||
(`MeshtasticProtobufs/Sources/meshtastic/*.pb.swift`), generated from the `protobufs` git
|
||||
submodule via `scripts/gen_protos.sh`. **To get field 19:** advance the `protobufs`
|
||||
submodule, run `scripts/gen_protos.sh`, commit the regenerated `.pb.swift` + submodule
|
||||
pointer. (No published-artifact dependency - Apple can regenerate from any commit.)
|
||||
- **Dispatch:** `AccessoryManager.processFromRadio(_:)`
|
||||
(`Meshtastic/Accessory/Accessory Manager/AccessoryManager.swift`) is a real
|
||||
`switch decodedInfo.payloadVariant { … }` - add a `.regionPresets` case, with the handler
|
||||
in `AccessoryManager+FromRadio.swift` (mirror `handleConfig` / `handleMetadata`).
|
||||
- **Persistence:** config is **SwiftData** (`@Model` entities), upserted via
|
||||
`MeshPackets`/`UpdateSwiftData.swift`. Store the decoded map (e.g. on a settings/connection
|
||||
model) so the LoRa view can read it.
|
||||
- **UI:** `Meshtastic/Views/Settings/Config/LoRaConfig.swift` (`struct LoRaConfig: View`)
|
||||
has the `Picker("Region", …)` (`RegionCodes.userSelectable`) and `Picker("Presets", …)`
|
||||
(`ModemPresets.userSelectable`, gated on `usePreset`). Filter the presets picker by the
|
||||
selected region's entry. Enums live in `Meshtastic/Enums/LoraConfigEnums.swift`.
|
||||
|
||||
### 8.3 Other clients
|
||||
|
||||
- **python (`meshtastic` / Meshtastic-python)** and **web** consume the published protobufs;
|
||||
they will see `region_presets` once their protobuf dependency includes field 19, and can
|
||||
ignore it until then (it decodes as an unknown field).
|
||||
|
||||
---
|
||||
|
||||
## 9. Reference payload (current firmware table)
|
||||
|
||||
For decoder unit tests. With the 2.8 region table, the firmware emits **6 groups**. Group
|
||||
indices are assigned in region-table order (first region to use a profile creates its group),
|
||||
so they are stable as listed here:
|
||||
|
||||
| group_index | default_preset | licensed_only | presets |
|
||||
| ----------------------- | -------------- | ------------- | -------------------------------------------------------------------------------------------------------------- |
|
||||
| 0 (standard) | `LONG_FAST` | false | LONG_FAST, LONG_SLOW, MEDIUM_SLOW, MEDIUM_FAST, SHORT_SLOW, SHORT_FAST, LONG_MODERATE, SHORT_TURBO, LONG_TURBO |
|
||||
| 1 (EU 868) | `LONG_FAST` | false | LONG_FAST, LONG_SLOW, MEDIUM_SLOW, MEDIUM_FAST, SHORT_SLOW, SHORT_FAST, LONG_MODERATE |
|
||||
| 2 (EU 866 SRD / "lite") | `LITE_FAST` | false | LITE_FAST, LITE_SLOW |
|
||||
| 3 (EU 868 narrow) | `NARROW_SLOW` | false | NARROW_FAST, NARROW_SLOW |
|
||||
| 4 (ham 20 kHz) | `TINY_FAST` | **true** | TINY_FAST, TINY_SLOW |
|
||||
| 5 (ham 100 kHz) | `NARROW_SLOW` | **true** | NARROW_FAST, NARROW_SLOW |
|
||||
|
||||
`region_groups` (region → group_index):
|
||||
|
||||
| group | regions |
|
||||
| ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 0 | US, EU_433, CN, JP, ANZ, ANZ_433, RU, KR, TW, IN, NZ_865, TH, UA_433, UA_868, MY_433, MY_919, SG_923, PH_433, PH_868, PH_915, KZ_433, KZ_863, NP_865, BR_902, LORA_24 |
|
||||
| 1 | EU_868 |
|
||||
| 2 | EU_866 |
|
||||
| 3 | EU_N_868 |
|
||||
| 4 | ITU1_2M, ITU2_2M, ITU3_2M |
|
||||
| 5 | ITU2_125CM |
|
||||
|
||||
> Note groups **3** and **5** carry the same preset list (NARROW\_\*) but are distinct groups
|
||||
> because they differ in `licensed_only`. Decoders must key on the group, not on the preset
|
||||
> list, to preserve the licensing flag.
|
||||
>
|
||||
> Regions **absent** from the table (no constraint info; see §5.1): `EU_874`, `EU_917`,
|
||||
> `ITU1_70CM`, `ITU2_70CM`, `ITU3_70CM`.
|
||||
|
||||
This table is generated from the firmware's region table at runtime; treat the firmware as
|
||||
authoritative and these values as the expected snapshot for the 2.8 table.
|
||||
@@ -0,0 +1,454 @@
|
||||
# Mesh Beacon Module - Function, Settings, and Client Interface Spec
|
||||
|
||||
Status: draft, tracks firmware branch `feat/mesh-beacon`.
|
||||
Audience: firmware reviewers (Part 1) and client-app developers - Android / Apple / Web / Python (Part 2).
|
||||
|
||||
The Mesh Beacon module lets a node periodically **advertise the existence of a mesh** to
|
||||
nodes that are not yet on it - broadcasting a short human-readable message plus an optional
|
||||
"join offer" (a channel, region, and modem preset). It is the mechanism behind invitations
|
||||
like _"Join us on NarrowSlow"_: a node sitting on one preset/region can shout an invitation
|
||||
that listeners on other presets/regions can hear and surface to their user.
|
||||
|
||||
The module is deliberately **advisory**. The firmware never auto-joins an advertised
|
||||
channel or auto-switches preset/region in response to a received beacon - it delivers the
|
||||
information to the client app and stops there. All "should I act on this?" decisions belong
|
||||
to the client and, ultimately, the user.
|
||||
|
||||
---
|
||||
|
||||
## Part 1 - Function and settings choices
|
||||
|
||||
### 1.1 Two roles in one module
|
||||
|
||||
| Role | Class | Active when | What it does |
|
||||
| --------------- | --------------------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Broadcaster** | `MeshBeaconBroadcastModule` | `FLAG_BROADCAST_ENABLED` set | Periodically transmits `MESH_BEACON_APP` packets on the configured radio settings. |
|
||||
| **Listener** | `MeshBeaconListenerModule` | `FLAG_LISTEN_ENABLED` set | Receives `MESH_BEACON_APP` packets and caches the offer for the client (the packet itself flows to the client unchanged). |
|
||||
|
||||
The boolean toggles live in a single `flags` bitfield (see [§1.8](#18-settings-reference-moduleconfigmeshbeaconconfig-tag-17)) - broadcasting and
|
||||
listening can be enabled independently on the same node. The whole module compiles out under the
|
||||
`MESHTASTIC_EXCLUDE_BEACON` build flag.
|
||||
|
||||
### 1.2 Wire message
|
||||
|
||||
Beacons travel on a dedicated port number:
|
||||
|
||||
```protobuf
|
||||
MESH_BEACON_APP = 37 // meshtastic/portnums.proto
|
||||
ENCODING: protobuf (meshtastic.MeshBeacon)
|
||||
```
|
||||
|
||||
```protobuf
|
||||
message MeshBeacon {
|
||||
string message = 1; // human-readable text, max 100 bytes (buffer 101)
|
||||
ChannelSettings offer_channel = 2; // optional advertised channel (name + PSK + slot)
|
||||
Config.LoRaConfig.RegionCode offer_region = 3; // optional advertised region (UNSET = none)
|
||||
optional Config.LoRaConfig.ModemPreset offer_preset = 4; // optional advertised preset
|
||||
}
|
||||
```
|
||||
|
||||
`.options` size caps (enforced at generation and on send):
|
||||
`message ≤ 100`, `offer_channel.name ≤ 12`, `offer_channel.psk ≤ 32`.
|
||||
|
||||
The three `offer_*` fields together describe _"there is a reachable mesh on this
|
||||
region+preset, here is the channel to use."_ Any subset may be present; an empty message with
|
||||
a populated offer (or vice-versa) is valid.
|
||||
|
||||
### 1.3 Transmission behaviour
|
||||
|
||||
Every outgoing beacon packet is stamped uniformly (`sendBeacon` → `stampPacket`):
|
||||
|
||||
- `to = NODENUM_BROADCAST`
|
||||
- `from = local node` (see [§1.6](#16-broadcast_send_as_node-currently-disabled) for the disabled spoof path)
|
||||
- **`hop_limit = 0`** - beacons are **zero-hop**. They are never rebroadcast by the mesh; only
|
||||
direct RF neighbours hear them. This is the primary spam-control mechanism. (`hop_start` is
|
||||
normally `0` too, but `FLAG_LEGACY_SPLIT` raises it to `1` for old-firmware compatibility - see
|
||||
[§1.5](#15-legacy-split-flag_legacy_split).)
|
||||
- `priority = BACKGROUND`, `want_ack = false`.
|
||||
|
||||
Broadcasting is additionally gated at runtime by:
|
||||
|
||||
- airtime utilisation (`isTxAllowedAirUtil()`), and
|
||||
- device role - **`CLIENT_HIDDEN` never broadcasts**.
|
||||
|
||||
#### Interval
|
||||
|
||||
`broadcast_interval_secs` controls cadence. The floor is **3600 s (1 hour)**
|
||||
(`default_mesh_beacon_min_broadcast_interval_secs`); `0` means "use default". Values below the
|
||||
floor are silently raised, both at config-set time (AdminModule) and at runtime.
|
||||
|
||||
The cadence is **reboot-safe**. Each broadcast's time is persisted to flash via `TransmitHistory`
|
||||
(keyed by `MESH_BEACON_APP`), and the broadcaster reads it back on boot - so a node that reboots
|
||||
(or crash-loops) won't re-broadcast until a full interval has elapsed since its last real send,
|
||||
rather than firing ~30 s after every boot. The timestamp is written **before** the transmit, so a
|
||||
brown-out during the high-current LoRa TX still counts as "sent." This mirrors `NodeInfoModule` /
|
||||
`PositionModule`.
|
||||
|
||||
#### Radio switching for TX
|
||||
|
||||
A beacon's whole point is often to reach a mesh on a _different_ preset/region/channel than the
|
||||
broadcaster currently runs. Before transmitting a beacon tagged with target radio settings, the
|
||||
module temporarily reconfigures the radio (`reconfigureForBeaconTX`), sends, then restores the
|
||||
prior config. Per-packet target settings are held in an 8-entry **sidecar table** keyed by packet
|
||||
ID - chosen so the `MeshPacket` proto carries no extra per-packet radio fields, and normal
|
||||
(non-beacon) traffic is never touched.
|
||||
|
||||
Two safety guards run before any radio switch (`beaconTxConfigInvalid`):
|
||||
|
||||
1. **An unlicensed node never keys up on a licensed-only (ham) region.** (The reverse - a licensed
|
||||
node operating in a non-ham region - is allowed. The switch only touches preset/region/channel,
|
||||
never `owner.is_licensed`.)
|
||||
2. **The preset must be valid for the target region** (`validateConfigLora`).
|
||||
|
||||
If either fails, the radio is **not** switched and the radio driver **drops** the packet rather
|
||||
than letting it fall through onto the current config.
|
||||
|
||||
#### Channel encryption on an override channel
|
||||
|
||||
Encryption keys off the **primary** channel slot, and the radio-thread channel switch happens
|
||||
_after_ encryption. So when a beacon goes out on an override channel (different name/PSK), the
|
||||
module installs the beacon channel into the primary slot for the synchronous duration of
|
||||
`send()`, then restores it (`sendBeaconPacket`). This guarantees the packet is encrypted with the
|
||||
beacon channel's key and stamped with its hash - not the primary's. Meshtastic threading is
|
||||
cooperative, so there is no preemption between swap and restore.
|
||||
|
||||
### 1.4 Where beacons are sent: single-target and multi-target
|
||||
|
||||
The broadcaster can send to one set of radio settings or to several. **Single- and multi-target
|
||||
are equal options - neither is preferred and neither is legacy.** Pick whichever matches the
|
||||
deployment.
|
||||
|
||||
- **Single-target:** the scalar `broadcast_on_preset` / `broadcast_on_region` /
|
||||
`broadcast_on_channel` fields describe one destination. Used when `broadcast_targets` is empty.
|
||||
- **Multi-target:** `broadcast_targets` (repeated `BroadcastTarget`) describes several. When
|
||||
non-empty it takes over from the scalar `broadcast_on_*` fields, and the broadcaster sends **one
|
||||
beacon copy per entry**. Each `BroadcastTarget` is `{ optional preset, region, optional channel_index }`,
|
||||
where `channel_index` references a slot in the node's own channel table (the channel must already be
|
||||
configured locally - its key is needed to encrypt the beacon). Within one cycle, targets that
|
||||
resolve to the **same** effective preset/region/channel are de-duplicated - only the first is
|
||||
transmitted - so an accidentally repeated entry costs no extra airtime.
|
||||
|
||||
#### Same-settings vs. other-settings
|
||||
|
||||
Independent of single/multi, each destination can either reuse the node's **own current radio
|
||||
settings** or specify **different** ones:
|
||||
|
||||
- **Same-settings ("message of the day"):** leave the preset / region / channel unset. They fall
|
||||
back to the running config, so the beacon goes out on the node's current mesh with **no radio
|
||||
switch** - a plain periodic broadcast to whoever is already on this preset/region.
|
||||
- **Other-settings (cross-mesh invite):** set a preset / region / channel that differs from the
|
||||
running config. The radio is temporarily switched for that copy's TX, then restored (see
|
||||
[§1.3](#radio-switching-for-tx)).
|
||||
|
||||
Both modes support both styles: a single-target beacon with no `broadcast_on_*` overrides is a
|
||||
message-of-the-day on the current mesh; a multi-target list can mix one entry on the current
|
||||
settings with others on different presets/regions.
|
||||
|
||||
### 1.5 Legacy split (`FLAG_LEGACY_SPLIT`)
|
||||
|
||||
This one flag controls **two** independent legacy-compatibility behaviours. Both are about making
|
||||
beacons usable by firmware that predates this module.
|
||||
|
||||
**(a) Text/offer packet split.** A combined `MESH_BEACON_APP` packet carries both the text and the
|
||||
offer, but old firmware only decodes `TEXT_MESSAGE_APP` and would never show the text. When
|
||||
`FLAG_LEGACY_SPLIT` is set **and both text and offer content are present**, the broadcaster
|
||||
emits **two** packets on the same beacon radio settings instead of one:
|
||||
|
||||
- **Packet A** - `MESH_BEACON_APP` carrying the **offer only** (no text).
|
||||
- **Packet B** - `TEXT_MESSAGE_APP` carrying the **text only**.
|
||||
|
||||
This is an independent two-packet decision, not an either/or: offer-only and text-only payloads
|
||||
still go out as a single packet in their respective cases; only the both-present case splits.
|
||||
|
||||
**(b) `hop_start = 1` override.** When `FLAG_LEGACY_SPLIT` is set, **every** beacon packet it sends
|
||||
(combined, split-A, or split-B; even same-settings ones) is stamped with `hop_start = 1` while
|
||||
`hop_limit` stays `0`. Pre-2.7.20 firmware drops `hop_start == 0` packets in a pre-decryption check
|
||||
before it can read the bitfield, so `hop_start = 1` lets those nodes accept the beacon - and it
|
||||
remains genuinely zero-hop (`hop_limit = 0` still prevents any rebroadcast).
|
||||
|
||||
> **Side effect for clients:** with `hop_start = 1, hop_limit = 0`, receivers compute
|
||||
> `hops_away = hop_start − hop_limit = 1`, so a legacy-split beacon reads as **1 hop away** even
|
||||
> though it arrived over direct RF. Without legacy-split it reads as direct (0). Don't treat a
|
||||
> beacon's `hops_away` as a reliable distance signal.
|
||||
|
||||
### 1.6 `broadcast_send_as_node` (currently disabled)
|
||||
|
||||
The schema reserves `broadcast_send_as_node` (field 3) to send beacons _as_ another node ID. **The
|
||||
firmware application of this field is currently commented out pending review**, so beacons always
|
||||
go out as the local node today. The access-control rule is, however, already enforced in
|
||||
AdminModule and should be treated as canonical:
|
||||
|
||||
> A remote admin may only set `broadcast_send_as_node` to **their own** node ID
|
||||
> (`mp.from`). Any other value is rejected and reset to the stored value.
|
||||
|
||||
Design note for when it is re-enabled: it is a _node-ID_ spoof only - it rewrites `from` but forges
|
||||
no signature. Once `from` is not us, the packet is no longer `isFromUs()`, so the router skips
|
||||
XEdDSA signing and receivers get an unsigned packet attributed to another node.
|
||||
|
||||
### 1.7 Reception behaviour (listener)
|
||||
|
||||
When `FLAG_LISTEN_ENABLED` is **off**, the router drops incoming `MESH_BEACON_APP` packets up front
|
||||
(`Router::handleReceived`, same pattern as a disabled NeighborInfo module) - so they reach neither
|
||||
the modules nor the phone. When it is **on**, the packet flows normally and the listener's
|
||||
`wantPacket` accepts it (`has_mesh_beacon` + `FLAG_LISTEN_ENABLED` + `portnum == MESH_BEACON_APP`).
|
||||
On a valid beacon (`handleReceivedProtobuf`):
|
||||
|
||||
1. **Offer → cache.** Any offer (`offer_channel` / `offer_region` / `offer_preset`) is stored in
|
||||
the static `lastReceivedOffer` (sender, channel, region, preset, `received_at`). `received_at`
|
||||
is `0` if the node has no RTC fix yet - **consumers must not treat `0` as a valid timestamp.**
|
||||
2. **Never auto-applied.** The firmware does not switch channel/preset/region from a received
|
||||
offer. Acting on it is the client app's job.
|
||||
3. The handler returns `CONTINUE` (not `STOP`), so the original `MESH_BEACON_APP` packet **flows to
|
||||
the client unchanged** through the normal FromRadio path (see Part 2). The client reads the
|
||||
`message` field directly from that packet - there is no separate copy.
|
||||
|
||||
The firmware deliberately does **not** unwrap a combined beacon's text into a synthesized
|
||||
`TEXT_MESSAGE_APP`, and does **not** fire `EVENT_RECEIVED_MSG`: a beacon is an advisory broadcast,
|
||||
not a personal message, so it must not duplicate the text or wake the device from sleep. If a
|
||||
broadcaster needs non-beacon-aware clients to see the text, it uses `FLAG_LEGACY_SPLIT`, which sends
|
||||
a real `TEXT_MESSAGE_APP` over RF (see [§1.5](#15-legacy-split-flag_legacy_split)).
|
||||
|
||||
### 1.8 Settings reference (`ModuleConfig.MeshBeaconConfig`, tag 17)
|
||||
|
||||
| # | Field | Type | Meaning / constraints |
|
||||
| --- | ------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------ |
|
||||
| 1 | `flags` | uint32 (bitfield) | Bitwise-OR of `Flags` values (listen / broadcast / legacy-split toggles). See enum below. |
|
||||
| 3 | `broadcast_send_as_node` | uint32 | Send-as node ID. **Application disabled in firmware.** Remote admin may only set to own node ID. |
|
||||
| 4 | `broadcast_message` | string | Text in each broadcast. **Hard-capped at 100 bytes.** |
|
||||
| 5 | `broadcast_offer_channel` | ChannelSettings | Channel advertised in `offer_channel`. |
|
||||
| 6 | `broadcast_offer_region` | RegionCode | Region advertised in `offer_region`. Must be a known region or it is cleared. |
|
||||
| 7 | `broadcast_offer_preset` | optional ModemPreset | Preset advertised in `offer_preset`. Validated against offer region (else cleared). |
|
||||
| 8 | `broadcast_on_channel` | ChannelSettings | Channel to transmit on (single-target). Empty name → preset display name. |
|
||||
| 9 | `broadcast_on_region` | RegionCode | Region to transmit on (single-target). |
|
||||
| 10 | `broadcast_on_preset` | optional ModemPreset | Preset to transmit on (single-target). Validated against on-region (else this + `on_channel` cleared). |
|
||||
| 11 | `broadcast_interval_secs` | uint32 | Cadence. **Min 3600**, default 3600; `0` = default. |
|
||||
| 13 | `broadcast_targets` | repeated BroadcastTarget | Multi-target list; when non-empty overrides the single-target `broadcast_on_*` fields. |
|
||||
|
||||
> The three boolean toggles were folded into the `flags` bitfield; field tags 2 and 12 are now
|
||||
> unused (the branch is unreleased, so the old tags are left as gaps rather than reserved).
|
||||
|
||||
**`Flags` enum** (nested in `MeshBeaconConfig`; OR the values into `flags`):
|
||||
|
||||
| Bit value | Name | Meaning |
|
||||
| --------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| 0 | `FLAG_NONE` | No options enabled. |
|
||||
| 1 | `FLAG_LISTEN_ENABLED` | Receive beacons; cache the offer. The packet flows to the client, which reads `message` directly. |
|
||||
| 2 | `FLAG_BROADCAST_ENABLED` | Periodically broadcast beacons from this node. |
|
||||
| 4 | `FLAG_LEGACY_SPLIT` | Legacy compatibility: (a) split text+offer into separate `TEXT_MESSAGE_APP` + `MESH_BEACON_APP` packets, and (b) stamp `hop_start = 1` on every beacon so pre-2.7.20 firmware accepts it (see [§1.5](#15-legacy-split-flag_legacy_split)). |
|
||||
|
||||
`BroadcastTarget`: `1 preset` (optional, falls back to running config), `2 region` (`UNSET` = running config), `4 channel_index` (optional `uint32`, index into the node's channel table; if unset, the default channel for the preset is used). Tag `3` is an unused gap - it previously held an embedded `ChannelSettings`, dropped to keep `ModuleConfig` within the BLE `FromRadio` size budget.
|
||||
|
||||
---
|
||||
|
||||
## Part 2 - Client interface specification
|
||||
|
||||
This section is what a client app needs to integrate with the beacon module. Everything goes
|
||||
through the **standard admin / ToRadio / FromRadio protocol** - there is no bespoke transport.
|
||||
|
||||
### 2.1 Capability detection
|
||||
|
||||
The module is build-flag optional. Treat it as present when the node's `LocalModuleConfig`
|
||||
contains a `mesh_beacon` sub-message (`LocalModuleConfig.mesh_beacon`, tag 18). If absent, the
|
||||
firmware was built with `MESHTASTIC_EXCLUDE_BEACON` - hide the beacon UI.
|
||||
|
||||
### 2.2 Reading and writing configuration
|
||||
|
||||
Standard module-config flow - no new admin messages:
|
||||
|
||||
- **Read:** `AdminMessage.get_module_config_request = ModuleConfig.MeshBeaconConfig` (variant 17).
|
||||
Reply is `get_module_config_response` with the `mesh_beacon` payload.
|
||||
- **Write:** `AdminMessage.set_module_config { mesh_beacon = … }`.
|
||||
|
||||
The on/off toggles (listen, broadcast, legacy-split) are bits in the `flags` field, not separate
|
||||
booleans - read/write them with the `MeshBeaconConfig.Flags` values
|
||||
(`FLAG_LISTEN_ENABLED = 1`, `FLAG_BROADCAST_ENABLED = 2`, `FLAG_LEGACY_SPLIT = 4`). To toggle one
|
||||
bit, read the current `flags`, set/clear the bit, and write the whole config back.
|
||||
|
||||
The firmware **sanitises on write** - your value may be silently adjusted. Mirror these rules
|
||||
client-side so the UI doesn't disagree with the device:
|
||||
|
||||
| Rule | Firmware behaviour |
|
||||
| --------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
|
||||
| `broadcast_message` length | Truncated to 100 bytes. |
|
||||
| `broadcast_interval_secs` | If non-zero and `< 3600`, raised to 3600. |
|
||||
| `broadcast_on_preset` invalid for `broadcast_on_region` (or current region) | Cleared, **and `broadcast_on_channel` cleared too.** |
|
||||
| `broadcast_offer_preset` invalid for offer/current region | Cleared. |
|
||||
| `broadcast_offer_region` not a known region | Cleared to `UNSET`. |
|
||||
| `broadcast_targets[i].region` not a known region | That entry's region cleared to `UNSET` (TX falls back to running config). |
|
||||
| `broadcast_targets[i].preset` invalid for that entry's region | That entry's `preset` and `channel_index` cleared. |
|
||||
| `broadcast_targets[i].channel_index` ≥ `MAX_NUM_CHANNELS` (8) | That entry's `channel_index` cleared (existence is **not** checked - see §2.5). |
|
||||
| `broadcast_send_as_node` ≠ sender's node ID (remote admin) | Rejected, reset to stored value. |
|
||||
|
||||
Setting beacon config does **not** trigger a reboot (`shouldReboot = false`); changes take effect
|
||||
on the next broadcast cycle. After a successful write, **re-read** the config to display the
|
||||
effective (sanitised) values.
|
||||
|
||||
### 2.3 Receiving beacons
|
||||
|
||||
A received beacon reaches the client as a normal `FromRadio.packet` (`MeshPacket`) - the listener
|
||||
returns `CONTINUE`, so the packet is **not** consumed on-device. The client must:
|
||||
|
||||
1. Subscribe to the FromRadio packet stream as usual.
|
||||
2. For packets with `decoded.portnum == MESH_BEACON_APP (37)`, decode `decoded.payload` as a
|
||||
`meshtastic.MeshBeacon`.
|
||||
3. Read `message`, `offer_channel`, `offer_region`, `offer_preset` (presence-checked).
|
||||
4. `packet.from` is the **originating beaconer** (the firmware preserves it).
|
||||
|
||||
> **Requires `FLAG_LISTEN_ENABLED` set in `flags`.** With listening disabled the firmware drops
|
||||
> received `MESH_BEACON_APP` packets in the router - before they reach the phone or any on-device
|
||||
> handler - the same way it drops a disabled module's packets (e.g. NeighborInfo). The node still
|
||||
> physically receives the RF, but the client will not see beacons over the FromRadio stream until
|
||||
> listening is enabled.
|
||||
|
||||
#### Reading the text - no duplication
|
||||
|
||||
For a beacon-aware client the text is **simply the `message` field of the `MESH_BEACON_APP`
|
||||
packet** you already decode for the offer (step 3 above). One packet, one field - the firmware does
|
||||
**not** inject a separate `TEXT_MESSAGE_APP` copy, so there is nothing to deduplicate.
|
||||
|
||||
The only time a beacon's text arrives as a separate `TEXT_MESSAGE_APP` is when the broadcaster set
|
||||
`FLAG_LEGACY_SPLIT`: in that mode the `MESH_BEACON_APP` carries the **offer only** (empty `message`)
|
||||
and the text is sent as a normal `TEXT_MESSAGE_APP` over RF, so legacy/non-beacon-aware clients can
|
||||
display it. These two cases are mutually exclusive - a given beacon's text appears exactly once,
|
||||
either in `MESH_BEACON_APP.message` (combined) or as a `TEXT_MESSAGE_APP` (legacy-split) - so a
|
||||
client never needs to dedup. Render whichever it receives.
|
||||
|
||||
### 2.4 Acting on an offer (the core client responsibility)
|
||||
|
||||
When a `MESH_BEACON_APP` carries offer content, present it to the user as an **invitation** -
|
||||
e.g. _"Node ⟨from⟩ invites you to join '⟨offer_channel.name⟩' on ⟨preset⟩/⟨region⟩."_ Then, only on
|
||||
explicit user confirmation, apply it by writing normal config:
|
||||
|
||||
- `offer_channel` → add/replace a `Channel` (`set_channel`), typically as a secondary channel.
|
||||
- `offer_region` / `offer_preset` → `set_config { lora = … }` (`use_preset = true`, set
|
||||
`modem_preset` and `region`). **Note this changes the node's own radio and will drop it off its
|
||||
current mesh** - make that consequence explicit in the UI.
|
||||
|
||||
**The firmware will never do any of this for the user. No silent auto-apply.** The on-device
|
||||
`lastReceivedOffer` cache is a firmware-internal convenience and is **not** currently exposed via
|
||||
an admin message - clients should source offers from the live `MESH_BEACON_APP` packet stream
|
||||
(§2.3), not expect a "get last offer" RPC.
|
||||
|
||||
#### Offer trust model - read before applying
|
||||
|
||||
- **The advertised PSK is not a secret.** `offer_channel.psk` is a public join token sent in the
|
||||
clear inside a broadcast; it is a convenience, not a security boundary. An operator who wants a
|
||||
genuinely private channel must distribute the PSK out-of-band and leave `offer_channel` unset.
|
||||
Surface offered channels as **public/open** to the user.
|
||||
- **Validate before applying.** Reject or warn if `offer_preset` is not valid for `offer_region`,
|
||||
and **never** apply a licensed-only (ham) region for a user who is not a licensed operator -
|
||||
mirror the firmware's own guard.
|
||||
- Beacons are **unsigned** when sent as another node (the disabled send-as path), and even normal
|
||||
beacons assert nothing about the sender's authority. Treat `from` as informational.
|
||||
|
||||
### 2.5 Configuring this node as a broadcaster
|
||||
|
||||
To make a node advertise a mesh, write `MeshBeaconConfig` with `FLAG_BROADCAST_ENABLED` set in
|
||||
`flags` and at least one of: a non-empty `broadcast_message`, or offer content
|
||||
(`broadcast_offer_*`). With neither, the broadcaster has nothing to send and stays silent.
|
||||
|
||||
Typical multi-region invite beacon:
|
||||
|
||||
```text
|
||||
flags = FLAG_BROADCAST_ENABLED | FLAG_LEGACY_SPLIT // broadcast on; split so legacy nodes still see the text
|
||||
broadcast_message = "Join us on NarrowSlow!"
|
||||
broadcast_offer_preset = NARROW_SLOW
|
||||
broadcast_offer_region = EU_N_868
|
||||
broadcast_offer_channel = { name: "MyChannel", psk: <32-byte key> }
|
||||
broadcast_interval_secs = 3600
|
||||
// channel_index points at slots in THIS node's channel table - configure those channels first.
|
||||
broadcast_targets = [
|
||||
{ preset: LONG_FAST, region: EU_868, channel_index: 0 },
|
||||
{ preset: NARROW_SLOW, region: EU_N_868, channel_index: 1 },
|
||||
]
|
||||
```
|
||||
|
||||
The same fields can be baked in at build time via `userPrefs.jsonc`
|
||||
(`USERPREFS_MESH_BEACON_*`) - see that file for the full list, including
|
||||
`USERPREFS_MESH_BEACON_TARGET_<n>_*` for multi-target entries.
|
||||
|
||||
#### Single-target vs. multi-target - equal options, different channel representation
|
||||
|
||||
Single-target and multi-target are **equal, first-class options**. Neither is preferred,
|
||||
deprecated, or a "legacy" fallback - pick whichever matches the deployment (a single-target
|
||||
beacon with no overrides is a plain message-of-the-day; a multi-target list reaches several
|
||||
preset/region/channel combinations). The broadcaster uses `broadcast_targets` when it is
|
||||
non-empty and the scalar `broadcast_on_*` fields when it is empty.
|
||||
|
||||
The one **subtle implementation difference** is how each names its TX channel:
|
||||
|
||||
| Path | TX channel is specified by | Channel name/PSK live… |
|
||||
| ------------- | ------------------------------------------------------- | ----------------------------------------- |
|
||||
| Single-target | `broadcast_on_channel` - an embedded `ChannelSettings` | …inline in the beacon config |
|
||||
| Multi-target | `broadcast_targets[i].channel_index` - a `uint32` index | …in the node's channel table (referenced) |
|
||||
|
||||
This asymmetry is deliberate: embedding a full `ChannelSettings` in every one of the (up to
|
||||
four) targets would push `ModuleConfig` past the BLE `FromRadio` size limit, so a target
|
||||
references an already-configured channel-table slot instead. `broadcast_offer_channel` (the
|
||||
advertised join token) is **always** inline regardless of path - it is the advertisement payload
|
||||
and must carry the actual name/PSK.
|
||||
|
||||
#### Configuring a multi-target broadcaster (two-step)
|
||||
|
||||
Because a target's channel is a reference, configuring a multi-target broadcaster takes **two
|
||||
admin writes**, in order:
|
||||
|
||||
1. **Create/define each channel in the node's channel table** with the normal channel admin flow
|
||||
(the same `set_channel` your app already uses for adding channels):
|
||||
|
||||
```text
|
||||
AdminMessage.set_channel { index: 1, role: SECONDARY,
|
||||
settings: { name: "NarrowSlow", psk: <key>, channel_num: 0 } }
|
||||
```
|
||||
|
||||
2. **Write the beacon config**, pointing each target at the slot index from step 1:
|
||||
|
||||
```text
|
||||
AdminMessage.set_module_config { mesh_beacon: {
|
||||
flags = FLAG_BROADCAST_ENABLED
|
||||
broadcast_targets = [ { preset: NARROW_SLOW, region: EU_N_868, channel_index: 1 } ]
|
||||
} }
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- A target may **only** reference a channel that already exists locally - the node needs that
|
||||
channel's key to encrypt the beacon. A `channel_index` that is out of range, or points at a
|
||||
blank/unconfigured slot, is not an error: the beacon falls back to the node's **current/primary
|
||||
channel** (its name, PSK, and slot) on the target preset/region. The channel name only defaults
|
||||
to the preset's display name (e.g. `LongFast`) when the primary channel itself is unnamed - so
|
||||
the fallback is "broadcast on my home channel," **not** a freshly-synthesised default-PSK channel
|
||||
for that preset.
|
||||
- `channel_index` must be `< MAX_NUM_CHANNELS` (8); the firmware clears it on write otherwise (see
|
||||
§2.2 sanitise rules). This is the **only** check on write - the firmware does **not** verify that
|
||||
the referenced slot is actually populated, because you may legitimately write the beacon config
|
||||
before creating the channel. **Validating that a referenced channel exists is the client app's
|
||||
responsibility.** A dangling reference doesn't error; it silently falls back to the preset's
|
||||
default channel - so without a client-side check, the user can believe they're advertising
|
||||
channel _X_ while the node is really transmitting on the preset default. Before writing, confirm
|
||||
each `channel_index` maps to a configured `Channel`, and warn the user otherwise.
|
||||
- **No automatic deduplication of channels.** Neither the beacon config nor the channel table
|
||||
dedups by content: two `broadcast_targets` may carry the same `channel_index`, or different
|
||||
indices whose slots hold identical settings, and `set_channel` will happily store two slots with
|
||||
the same name/PSK. The broadcaster _does_ skip transmitting a target whose effective
|
||||
preset/region/channel duplicates an earlier one in the same cycle (so a duplicated entry wastes
|
||||
no airtime), but it does not rewrite or reject your config - keeping the target list free of
|
||||
redundant entries is up to the client.
|
||||
- The single-target path needs no separate `set_channel` step - its `broadcast_on_channel` is
|
||||
written inline in the same beacon-config message.
|
||||
|
||||
### 2.6 Quick reference
|
||||
|
||||
| Concern | Value |
|
||||
| ---------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| Port number | `MESH_BEACON_APP = 37` |
|
||||
| Wire message | `meshtastic.MeshBeacon` |
|
||||
| Config message | `ModuleConfig.MeshBeaconConfig` (variant tag 17) |
|
||||
| On/off toggles | `flags` bitfield (`MeshBeaconConfig.Flags`) |
|
||||
| Local config presence | `LocalModuleConfig.mesh_beacon` (tag 18) |
|
||||
| Min broadcast interval | 3600 s (1 h) |
|
||||
| Message max length | 100 bytes |
|
||||
| Hop behaviour | Zero-hop (`hop_limit = 0`), never rebroadcast; `hop_start = 1` under `FLAG_LEGACY_SPLIT` |
|
||||
| Auto-apply offers? | **Never** - client + user decide |
|
||||
| Offer PSK | Public join token, not a secret |
|
||||
| Disabled today | `broadcast_send_as_node` application |
|
||||
@@ -0,0 +1,456 @@
|
||||
# NextHop direct-message reliability on dense meshes - findings & plan
|
||||
|
||||
**Status:** Implemented - mitigations and tests in `PR3-tmm-nexthop`
|
||||
**Date:** 2026-06-13
|
||||
**Area:** `src/mesh` router stack (`NextHopRouter`, `ReliableRouter`, `FloodingRouter`, `Router`, `NodeDB`, `PacketHistory`)
|
||||
**Constraint:** No over-the-air / wire-format changes - `next_hop` and `relay_node` stay 1 byte, no `PacketHeader` changes, no breaking protobuf changes. All new state is RAM-only.
|
||||
|
||||
This document captures the analysis and the proposed mitigations so the work can be
|
||||
continued on this branch by anyone. It is intentionally code-grounded (file:line
|
||||
references throughout) and standalone - you should not need the original investigation
|
||||
context to pick it up.
|
||||
|
||||
---
|
||||
|
||||
## TL;DR
|
||||
|
||||
NextHop routing for direct messages (DMs) is unreliable on dense meshes. The headline
|
||||
cause is the **birthday problem**: `next_hop` and `relay_node` are each a single byte
|
||||
(the last byte of a 32-bit node number), so on a mesh of N nodes the probability that
|
||||
two share the same byte hits ~50% at **~19 nodes** and is near-certain by 50-100. But
|
||||
there are **other, equally important issues**: that single byte is trusted blindly at
|
||||
five different code sites, learned routes **never decay**, routes are learned from the
|
||||
**reverse (ACK) path** (asymmetric-link hazard), and collision-driven spurious
|
||||
rebroadcasts **amplify congestion** exactly when the mesh is busy.
|
||||
|
||||
Because we can't widen the on-wire field, the fix is **interpretation-side** ("don't
|
||||
trust a byte that doesn't map to a unique reachable neighbor - flood instead") plus
|
||||
**recovery-side** ("decay stale/failing routes so they get re-discovered"). Four
|
||||
mitigations, M1-M4, all RAM-only. The net behavioral change: on dense/mobile meshes a
|
||||
DM that today silently misroutes or black-holes instead falls back to managed flooding
|
||||
(which still delivers) and re-learns a fresh route quickly. Sparse-mesh happy paths are
|
||||
unchanged.
|
||||
|
||||
---
|
||||
|
||||
## How NextHop routing works today (mechanics)
|
||||
|
||||
Inheritance chain: `Router` → `FloodingRouter` → `NextHopRouter` → `ReliableRouter`.
|
||||
|
||||
**The single-byte identifiers.** Both routing bytes come from one helper:
|
||||
|
||||
```cpp
|
||||
// src/mesh/NodeDB.h:255
|
||||
uint8_t getLastByteOfNodeNum(NodeNum num) { return (uint8_t)((num & 0xFF) ? (num & 0xFF) : 0xFF); }
|
||||
```
|
||||
|
||||
It projects a 32-bit node number onto 255 values (`0x00` is remapped to `0xFF` so it
|
||||
never collides with the `0`-valued sentinels `NO_NEXT_HOP_PREFERENCE` / `NO_RELAY_NODE`,
|
||||
`src/mesh/MeshTypes.h:44-46`). `next_hop` and `relay_node` in the packet header are
|
||||
`uint8_t` (`src/mesh/mesh.pb.h`, comments "Last byte of the node number…"). The learned
|
||||
route stored per destination, `meshtastic_NodeInfoLite::next_hop`, is also a single byte
|
||||
(`src/mesh/generated/meshtastic/deviceonly.pb.h:83`).
|
||||
|
||||
**Sending a DM** - `NextHopRouter::send` (`src/mesh/NextHopRouter.cpp:23`):
|
||||
|
||||
1. `p->relay_node = getLastByteOfNodeNum(getNodeNum())` (mark ourselves as relayer).
|
||||
2. `p->next_hop = getNextHop(p->to, p->relay_node)` (`src/mesh/NextHopRouter.cpp:192`):
|
||||
look up `nodeDB->getMeshNode(to)->next_hop`; return it unless it equals the relayer
|
||||
byte; otherwise `NO_NEXT_HOP_PREFERENCE` (→ flood).
|
||||
|
||||
**Relaying** - `NextHopRouter::perhapsRebroadcast` (`src/mesh/NextHopRouter.cpp:133`):
|
||||
rebroadcast iff `next_hop == NO_NEXT_HOP_PREFERENCE` (flood) **or**
|
||||
`next_hop == getLastByteOfNodeNum(getNodeNum())` (we are the addressed next hop)
|
||||
(`:147`). Each node only ever compares against **its own** byte.
|
||||
|
||||
**Learning** - `NextHopRouter::sniffReceived` (`src/mesh/NextHopRouter.cpp:89`): on an
|
||||
ACK/reply (`request_id`/`reply_id` set), if the relayer of the ACK was also a relayer of
|
||||
the original packet (validated via `PacketHistory::checkRelayers`), set
|
||||
`origTx->next_hop = p->relay_node` (`:114`). I.e. the **forward** next-hop is learned
|
||||
from the **reverse** path's relayer.
|
||||
|
||||
**Retransmission / fallback** - `NextHopRouter::doRetransmissions`
|
||||
(`src/mesh/NextHopRouter.cpp:284`). Budgets: `NUM_RELIABLE_RETX=3` (originator: initial
|
||||
|
||||
- 2 retries), `NUM_INTERMEDIATE_RETX=2` (relayer: 1 retry). On the **last** retry
|
||||
(`numRetransmissions==1`) it resets `next_hop` to `NO_NEXT_HOP_PREFERENCE` on the packet
|
||||
**and** clears `sentTo->next_hop` in NodeDB, then floods (`:313-321`). Retransmit timing
|
||||
comes from `iface->getRetransmissionMsec`, whose contention window **grows with channel
|
||||
utilization** (`src/mesh/RadioInterface.cpp` `getTxDelayMsec`/`getTxDelayMsecWeighted`).
|
||||
|
||||
**Dedup / relayer history** - `PacketHistory` (`src/mesh/PacketHistory.cpp`): a bounded
|
||||
ring (`PACKETHISTORY_MAX = max(MAX_NUM_NODES*2, 100)`, 20 B/record) keyed by
|
||||
`(sender,id)`, tracking up to `NUM_RELAYERS=6` relayer **bytes** per packet in
|
||||
`relayed_by[]`. `wasRelayer` (`:490`) and `checkRelayers` (`:517`) match bytes against
|
||||
that array.
|
||||
|
||||
---
|
||||
|
||||
## Root-cause analysis
|
||||
|
||||
### 1. The single byte is trusted blindly at five sites (the birthday problem)
|
||||
|
||||
| # | Site | File:line | Failure on collision |
|
||||
| --- | -------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 1 | Rebroadcast self-check | `NextHopRouter.cpp:147` | A remote "impostor" node sharing the intended next-hop's byte also rebroadcasts → wasted airtime / congestion. |
|
||||
| 2 | Route learning | `NextHopRouter.cpp:111-114` | Stores an ambiguous byte as the route; later resolves to the wrong physical node. |
|
||||
| 3 | Relayer validation | `PacketHistory.cpp:490-538` | `wasRelayer(byte)` returns true for the wrong node → mis-validated ACK / mis-learn. |
|
||||
| 4 | Favorite-router hop preservation | `Router.cpp:120-145` | **First** NodeDB node whose last byte matches wins - non-deterministic; can preserve hops for the wrong relay (hop leak). |
|
||||
| 5 | Send-path lookup | `NextHopRouter.cpp:192-207` | Emits a byte that may address the wrong node; no check it still maps to a reachable neighbor. |
|
||||
|
||||
Collision math (uniform last byte over 255 buckets): P(collision) ≈ 50% at ~19 nodes,
|
||||
|
||||
> 99% by ~75 nodes. Dense meshes are squarely in the "always colliding" regime.
|
||||
|
||||
### 2. Stale routes never decay
|
||||
|
||||
The learned `next_hop` byte is cleared only on the **current DM's** last retry
|
||||
(`NextHopRouter.cpp:313-321`). A route learned hours ago that has since gone dead is
|
||||
still trusted on the **next** DM's first attempt - which on a congested mesh is also the
|
||||
slowest attempt. Result: silent black-hole at a dead hop until the retransmission budget
|
||||
drains, then a late flood. Intermediate nodes hold stale routes indefinitely.
|
||||
|
||||
### 3. Reverse-path (asymmetric-link) learning
|
||||
|
||||
`origTx->next_hop` is learned from the ACK's relayer (`NextHopRouter.cpp:110-114`) - the
|
||||
**reverse** direction. RF links are frequently asymmetric, so the best reverse relay can
|
||||
be a poor forward relay. Worse, the next reverse ACK immediately re-learns the same bad
|
||||
hop, so the route **flaps** back to the bad value even after a failure reset.
|
||||
|
||||
### 4. Congestion amplification
|
||||
|
||||
Collision-driven impostor rebroadcasts (issue 1) add airtime; the contention window
|
||||
grows with channel utilization, so retransmit intervals **lengthen** exactly when the
|
||||
mesh is busy. The 3-try reliable budget can then expire before delivery. On dense
|
||||
meshes, efficiency _is_ reliability.
|
||||
|
||||
### Note: pubkey-derived node numbers (develop / 2.8) - does not change the plan
|
||||
|
||||
develop derives the node number from the public key:
|
||||
`my_node_num = crc32Buffer(public_key)` (`src/mesh/NodeDB.cpp:481`), re-derived on key
|
||||
change in `createNewIdentity()` (`src/mesh/NodeDB.cpp:3113`). This **reinforces** the
|
||||
plan rather than changing it:
|
||||
|
||||
- **Birthday problem unchanged and now textbook-exact.** CRC32 mixes well → the last
|
||||
byte is uniformly distributed over 256 values. Derivation adds no wire bits.
|
||||
- **Node numbers are now immutable / identity-bound.** Pre-2.8 `pickNewNodeNum()` could
|
||||
renumber a node to dodge a conflict; now the number is fixed by the key, so a last-byte
|
||||
collision **cannot be resolved operationally by renumbering** → M1/M2/M3 become _more_
|
||||
necessary.
|
||||
- **Resolver gets cleaner inputs.** Stable node numbers keep a learned byte bound to one
|
||||
identity (good for M3 freshness). `createNewIdentity()` retires the old entry by marking
|
||||
it **ignored** and clearing its pubkey (`src/mesh/NodeDB.cpp:3123-3125`), which M1's
|
||||
candidate gate already skips - so key rotation can't pollute resolution.
|
||||
- **No wire-free disambiguation unlocked.** A receiver still gets only 1 byte and cannot
|
||||
recover which full node number a colliding value meant - so "detect ambiguity → flood"
|
||||
remains the correct strategy.
|
||||
|
||||
---
|
||||
|
||||
## Proposed mitigations
|
||||
|
||||
Key insight for all of M1/M2: **a 1-byte ID only needs to be unique among a node's
|
||||
direct neighbors / plausible relays, not the whole mesh.** That candidate set is small
|
||||
(typically 5-15), so a byte usually resolves unambiguously there; when it doesn't, fall
|
||||
back to the _safe_ behavior (flood / decrement / don't-learn).
|
||||
|
||||
### M1 - Ambiguity-aware last-byte resolution (new NodeDB primitive)
|
||||
|
||||
New types + methods in `src/mesh/NodeDB.h` (near line 255) / `src/mesh/NodeDB.cpp`
|
||||
(near `getMeshNode`, ~2936):
|
||||
|
||||
```cpp
|
||||
enum class LastByteResolution : uint8_t { None, Unique, Ambiguous };
|
||||
struct ResolvedNode { LastByteResolution status = LastByteResolution::None; NodeNum num = 0; };
|
||||
|
||||
// Resolve a single on-wire last-byte to a unique full NodeNum among relevant candidates.
|
||||
ResolvedNode resolveLastByte(uint8_t lastByte, bool requireDirectNeighbor);
|
||||
// Convenience: true iff exactly one relevant candidate (Ambiguous and None both -> false = SAFE).
|
||||
bool resolveUniqueLastByte(uint8_t lastByte, bool requireDirectNeighbor, NodeNum *outNum = nullptr);
|
||||
```
|
||||
|
||||
- **One linear pass** over `meshNodes`, reusing `getNumMeshNodes()`/`getMeshNodeByIndex()`,
|
||||
the bitfield helpers (`nodeInfoLiteIsFavorite/HasUser/IsIgnored`), `sinceLastSeen()`,
|
||||
and `getLastByteOfNodeNum()`. **Early-exit** on the 2nd match (return `Ambiguous`).
|
||||
- **Guard:** `if (lastByte == 0) return {None, 0};` (covers `NO_RELAY_NODE` / MQTT-invalid).
|
||||
- **Candidate gate** (skip): `num == getNodeNum()` (never resolve to ourselves), `num == 0`,
|
||||
`num == NODENUM_BROADCAST`, `nodeInfoLiteIsIgnored`. Then match
|
||||
`getLastByteOfNodeNum(node->num) == lastByte` (cheapest test last, mirroring `Router.cpp:119`).
|
||||
- **Relevance gate:**
|
||||
- `requireDirectNeighbor == true` (strict, for SEND): `has_hops_away && hops_away == 0`
|
||||
**and** `sinceLastSeen(node) < NEXTHOP_NEIGHBOR_FRESH_SECS`.
|
||||
- `requireDirectNeighbor == false` (lenient, for learn / hop-preserve): accept if direct
|
||||
neighbor **or** `nodeInfoLiteIsFavorite` **or** role ∈ {ROUTER, ROUTER_LATE, CLIENT_BASE}.
|
||||
- **No tie-break.** A collision must return `Ambiguous` - picking "best SNR" would
|
||||
resurrect the silent-misroute bug. (Deliberate non-goal; document in code.)
|
||||
|
||||
New constant in `src/mesh/MeshTypes.h` (near line 44):
|
||||
`#define NEXTHOP_NEIGHBOR_FRESH_SECS (60 * 60 * 2)` (mirrors `NUM_ONLINE_SECS`).
|
||||
|
||||
### M2 - Only route on bytes that resolve to a unique, reachable neighbor
|
||||
|
||||
In `getNextHop` (`src/mesh/NextHopRouter.cpp:192-207`), after the existing split-horizon
|
||||
check (`node->next_hop != relay_node`), require the stored byte to resolve to a **unique,
|
||||
currently-fresh direct neighbor**; else flood:
|
||||
|
||||
```cpp
|
||||
if (node->next_hop != relay_node) {
|
||||
ResolvedNode r = nodeDB->resolveLastByte(node->next_hop, /*requireDirectNeighbor=*/true);
|
||||
if (r.status == LastByteResolution::Unique) return node->next_hop;
|
||||
LOG_WARN("Next hop 0x%x for 0x%x %s -> flood", node->next_hop, to,
|
||||
r.status == LastByteResolution::Ambiguous ? "ambiguous among neighbors" : "no longer a neighbor");
|
||||
return std::nullopt;
|
||||
}
|
||||
```
|
||||
|
||||
This self-heals when a neighbor goes away (unicast-into-a-void becomes a flood). It
|
||||
applies to originating, relaying, and retrying, since all route through `getNextHop`.
|
||||
|
||||
Apply M1's safe fallback at the other sites:
|
||||
|
||||
- **Learning** (`NextHopRouter.cpp:111-114`): gate `origTx->next_hop = p->relay_node` on
|
||||
`resolveUniqueLastByte(p->relay_node, /*direct=*/false)`. Ambiguous/unknown → don't
|
||||
learn (leave route unset → flood).
|
||||
- **Favorite-router preservation** (`Router.cpp:120-145`): replace the "first match wins"
|
||||
loop with `resolveUniqueLastByte(p->relay_node, /*direct=*/false)` + a re-check that the
|
||||
resolved node is favorite/has_user/router. Ambiguous/none/not-favorite → **decrement**
|
||||
(safe). Net: removes one full DB scan, adds one resolver scan (wash).
|
||||
|
||||
**Left unchanged, by design (document why in code):**
|
||||
|
||||
- **Site 1** rebroadcast self-check (`NextHopRouter.cpp:147`) and self-identity checks
|
||||
(`ReliableRouter.cpp:127`): a node matches its **own** byte - no DB resolution helps. A
|
||||
remote impostor sharing the intended next-hop's byte will still rebroadcast. M1/M2
|
||||
shrink the blast radius by reducing how often an ambiguous byte is ever stored or
|
||||
originated; a true fix needs a wider field (out of scope). **This is the one residual
|
||||
the plan cannot fully close.**
|
||||
- **Site 3** `wasRelayer`/`checkRelayers` (`PacketHistory.cpp:490-538`): intentionally
|
||||
byte-domain (both sides are on-wire bytes); the consumer (learning) is now hardened.
|
||||
Add a one-line comment; do not change.
|
||||
|
||||
### M3 - Route freshness / failure memory (RAM table on NextHopRouter)
|
||||
|
||||
A bounded, LRU-evicted table keyed by destination, mirroring `PacketHistory`'s
|
||||
reuse-oldest discipline (not an unbounded map) to cap RAM.
|
||||
|
||||
`src/mesh/NextHopRouter.h` (near `pending`, line 99):
|
||||
|
||||
```cpp
|
||||
struct RouteHealth {
|
||||
NodeNum dest = 0; // 0 == empty slot
|
||||
uint32_t learnedAtMsec = 0; // millis() at last (re)learn; rollover-aware
|
||||
uint8_t consecutiveFailures = 0;
|
||||
uint8_t lastNextHop = NO_NEXT_HOP_PREFERENCE; // byte this health refers to
|
||||
};
|
||||
static constexpr uint8_t ROUTE_HEALTH_MAX = 32; // ~384B; drop to 16 if RAM-tight
|
||||
RouteHealth routeHealth[ROUTE_HEALTH_MAX] = {};
|
||||
// Helpers take `now` (pure/testable): findRouteHealth, getOrAllocRouteHealth,
|
||||
// noteRouteLearned, noteRouteSuccess, noteRouteFailure, isRouteStale, clearRouteHealth
|
||||
```
|
||||
|
||||
Policy:
|
||||
|
||||
| Constant | Value | Rationale |
|
||||
| ------------------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `ROUTE_TTL_MSEC` | 30 min | Survives a normal conversation; re-discovers a moved node within a telemetry interval. |
|
||||
| `ROUTE_FAILURE_THRESHOLD` | 3 | 1-2 consecutive failures are transient LoRa collisions; 3 to the same hop = dead. Accumulates **across** DMs (independent of the per-DM 3-try budget). |
|
||||
|
||||
`isRouteStale(h, now)` = `(now - h.learnedAtMsec) >= ROUTE_TTL_MSEC || h.consecutiveFailures >= ROUTE_FAILURE_THRESHOLD`.
|
||||
All age math uses **unsigned subtraction** (rollover-safe, matching
|
||||
`PacketHistory.cpp:364`); treat `learnedAtMsec == 0` as "set now".
|
||||
|
||||
Wiring (as built - `src/mesh/NextHopRouter.cpp`, `src/mesh/ReliableRouter.cpp`):
|
||||
|
||||
- `getNextHop`: if a health record matches the stored byte and `isRouteStale`, clear
|
||||
`node->next_hop` (NodeDB) **and** `clearRouteHealth`, return `nullopt` (flood). No
|
||||
record yet (cold path, first DM after boot) → trust NodeDB, but the M2 strict-neighbor
|
||||
gate still applies.
|
||||
- `sniffReceived` learn: gate the write through `resolveUniqueLastByte` (M2), then
|
||||
`noteRouteLearned(p->from, p->relay_node, millis())` - resets `consecutiveFailures`
|
||||
**only if the hop changed** (anti-flap for asymmetric re-learn); otherwise just refreshes
|
||||
`learnedAtMsec`. (No success signal is taken on the intermediate reverse-pass: an ACK
|
||||
merely passing through us is not proof that _we_ delivered, and resetting failures there
|
||||
would reintroduce the asymmetric flap.)
|
||||
- `doRetransmissions`: on the last-retransmission branch (`numRetransmissions == 1`, the
|
||||
point a directed delivery has gone un-ACKed for both originator and intermediate) →
|
||||
`noteRouteFailure(to)`, then the existing NodeDB `next_hop` reset + flood. We deliberately
|
||||
do **not** `clearRouteHealth` here: keeping the record is what lets the failure count
|
||||
accumulate across DMs so a flapping reverse-path-relearned dead hop eventually ages out.
|
||||
- `ReliableRouter::sniffReceived` ACK path → `noteRouteSuccess(getFrom(p), millis())`
|
||||
(an end-to-end ACK addressed to us is genuine forward-delivery proof; clears failures and
|
||||
refreshes freshness). `noteRouteSuccess`/`noteRouteFailure` are no-ops when no record
|
||||
exists, so flood-only destinations never pollute the table.
|
||||
|
||||
**Reconciliation (no double-handling):** `doRetransmissions` owns _in-flight_ failure of
|
||||
the current DM (reset NodeDB `next_hop` + flood, and bump the cross-DM failure counter);
|
||||
`getNextHop` owns _between-DM_ staleness (TTL or failure-threshold → flood + clear). The
|
||||
only place that erases a health record is the `getNextHop` decay path; the retransmission
|
||||
path leaves it intact so the counter survives a reverse-path re-learn.
|
||||
|
||||
### M4 - Earlier flood for unverified routes (gated, off by default)
|
||||
|
||||
Compile-gated so healthy sparse meshes are untouched. **Default is off** - the define
|
||||
lives in `NextHopRouter.h` and must be flipped to measure:
|
||||
`#define NEXTHOP_EARLY_FLOOD_ON_UNVERIFIED 1`.
|
||||
|
||||
In `doRetransmissions`, the directed-retry `else` branch: if the route is **not verified**
|
||||
(`!findRouteHealth(to) || consecutiveFailures > 0 || isRouteStale`), reset `next_hop` and
|
||||
flood on this attempt instead of spending another directed try. A **verified** route
|
||||
(record present, `consecutiveFailures == 0`, within TTL - i.e. recently ACKed) takes the
|
||||
unchanged directed-retry path, so the sparse-mesh happy path is untouched. Trade-off:
|
||||
airtime ↔ latency; the gate ensures we never pay the flood cost on a proven route, only on
|
||||
one we already distrust. Off by default precisely so it can be A/B-measured on the
|
||||
simulator before broad enable.
|
||||
|
||||
---
|
||||
|
||||
## Files to modify
|
||||
|
||||
| File | Change |
|
||||
| ------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `src/mesh/MeshTypes.h` | `NEXTHOP_NEIGHBOR_FRESH_SECS`, `ROUTE_TTL_MSEC`, `ROUTE_FAILURE_THRESHOLD`, `NEXTHOP_EARLY_FLOOD_ON_UNVERIFIED` |
|
||||
| `src/mesh/NodeDB.h` / `src/mesh/NodeDB.cpp` | `LastByteResolution`, `ResolvedNode`, `resolveLastByte`, `resolveUniqueLastByte` |
|
||||
| `src/mesh/NextHopRouter.h` | `RouteHealth` + array + helpers; `#ifdef PIO_UNIT_TESTING public:` for helpers and `getNextHop` |
|
||||
| `src/mesh/NextHopRouter.cpp` | `getNextHop` (M2 gate + M3 decay); `sniffReceived` (learn gate + health seed + success); `doRetransmissions` (failure counting + M4); comment site 1 |
|
||||
| `src/mesh/Router.cpp` | `shouldDecrementHopLimit` → resolver + favorite/router re-check |
|
||||
| `src/mesh/ReliableRouter.cpp` | ACK path → `noteRouteSuccess` |
|
||||
| `test/test_nexthop_routing/test_main.cpp` | **new** unit suite (auto-built under `[env:native]`) |
|
||||
|
||||
**Reuse, don't reinvent:** `getLastByteOfNodeNum`, `sinceLastSeen`, the bitfield helpers,
|
||||
`getMeshNodeByIndex`/`getNumMeshNodes`, PacketHistory's reuse-oldest eviction shape, and
|
||||
`MockNodeDB::addTestNode` (from `test/test_hop_scaling/test_main.cpp`).
|
||||
|
||||
---
|
||||
|
||||
## Edge cases
|
||||
|
||||
- **`0x00`↔`0xFF` projection:** the resolver compares via `getLastByteOfNodeNum` on both
|
||||
sides, so a `…00` node and a `…FF` node correctly collide on `0xFF` → `Ambiguous`. Test
|
||||
explicitly.
|
||||
- **MQTT packets:** `relay_node`/`next_hop` are forced invalid when `hop_start == 0`
|
||||
(`src/mesh/RadioLibInterface.cpp:603-605`) → byte 0 → resolver `None` → don't learn
|
||||
(correct).
|
||||
- **`has_hops_away == false`** nodes are excluded from the strict gate (never fabricate a
|
||||
Unique neighbor for M2); admitted to the lenient gate only via favorite/router role.
|
||||
Safe; self-corrects once `hops_away` is learned.
|
||||
- **Self / broadcast:** the resolver skips `getNodeNum()` and `NODENUM_BROADCAST`;
|
||||
`getNextHop` already early-returns for broadcast.
|
||||
- **Perf:** M2 adds one O(N) resolver scan per directed send/relay (early-exit on the 2nd
|
||||
match), cheaper than the crypto already on that path; site-4 is a wash. If ever hot, a
|
||||
future 256-entry last-byte index is the optimization (not now - RAM).
|
||||
|
||||
---
|
||||
|
||||
## Verification (all tiers)
|
||||
|
||||
### 1. Native unit tests - new `test/test_nexthop_routing/test_main.cpp`
|
||||
|
||||
`pio test -e native -f test_nexthop_routing`; on macOS `./bin/test-native-docker.sh -f test_nexthop_routing`.
|
||||
Design the RouteHealth helpers to take `now` as a parameter so the 30-min TTL logic is
|
||||
testable without a clock mock.
|
||||
|
||||
- **Resolver:** None / Unique / **Ambiguous (birthday collision)** / strict-excludes-stale /
|
||||
strict-excludes-far / lenient-includes-favorite-router / lenient-collision / skips-self /
|
||||
skips-ignored / **`0x00`↔`0xFF` collision** / early-exit.
|
||||
- **`getNextHop`:** unique→byte, **ambiguous→nullopt**, stale-neighbor→nullopt,
|
||||
split-horizon (relay==next_hop)→nullopt, broadcast→nullopt.
|
||||
- **RouteHealth:** TTL boundary, **rollover** (learn near `0xFFFFFFFF`, check after wrap),
|
||||
failure threshold, success-resets, **re-learn-same-hop keeps fails (anti-flap)**,
|
||||
re-learn-new-hop resets, LRU eviction bound, clear.
|
||||
- **Site-4:** preserve on unique favorite router; **decrement on two colliding favorites**;
|
||||
decrement when the resolved node is not a favorite.
|
||||
- **Sparse-mesh regression:** all-distinct last bytes → every resolve Unique, `getNextHop`
|
||||
returns the stored byte unchanged (proves no happy-path change).
|
||||
- Re-run `test_packet_history` and `test_hop_scaling` for no regression.
|
||||
|
||||
### 2. portduino SimRadio simulator
|
||||
|
||||
`pio run -e native && ./bin/test-simulator.sh`. Best vehicle for the **intermediate-node**
|
||||
path the 2-device bench can't reach. Line topology A - B - C: establish A→C (B learns a
|
||||
directed route), stop B relaying that dest, confirm A re-discovers via flood within
|
||||
`ROUTE_FAILURE_THRESHOLD` and that B's `noteRouteFailure`/`clearRouteHealth` fires (visible
|
||||
via the `LOG_INFO "Route to … stale"` / "Resetting next hop" lines). Use this to A/B M4
|
||||
(attempts-to-delivery, total airtime).
|
||||
|
||||
### 3. Hardware via meshtastic MCP (auto-detect; 3+ devices for a real hop)
|
||||
|
||||
- `meshtastic-mcp/tests/mesh/test_nexthop_multihop_recovery.py` - **the multi-hop validator
|
||||
for this work** (added on this branch). Self-discovers an A - relay - C line, asserts a
|
||||
directed DM is delivered across the relay (next_hop + M1/M2/M3 engaged), and asserts
|
||||
delivery recovers after the relay is power-cycled (M3). Skips unless the bench is a true
|
||||
multi-hop line (≥3 roles via `--hub-profile`, endpoints out of direct RF range).
|
||||
- `meshtastic-mcp/tests/mesh/test_direct_with_ack.py` - happy-path regression: a fresh/unique
|
||||
route still delivers a want_ack DM on the first/second try (M4's gate must keep this
|
||||
green).
|
||||
- `meshtastic-mcp/tests/mesh/test_peer_offline_recovery.py` - 2-device recovery validator: peer
|
||||
off mid-conversation then back. Must stay green and ideally recover in fewer attempts.
|
||||
|
||||
### 4. Build / format sanity
|
||||
|
||||
native-macos **and** Docker both ways; trunk clang-format@16.0.3; a release `pio run` to
|
||||
confirm the `#ifdef PIO_UNIT_TESTING` visibility widening does **not** leak into
|
||||
production; sanity-check RAM headroom on the smallest nRF52 build for the ~384 B table.
|
||||
|
||||
---
|
||||
|
||||
## Verification status (as built on `nexthop-redux`)
|
||||
|
||||
| Tier | What ran | Result |
|
||||
| -------------------------------- | ----------------------------------------------------------------------------------- | ------------------- |
|
||||
| Unit (native-macos) | `test_nexthop_routing` (31 cases) | ✅ 31/31 |
|
||||
| Unit (Docker / Linux, CI parity) | `test_nexthop_routing` | ✅ 31/31 |
|
||||
| Regression | `test_packet_history`, `test_hop_scaling`, `test_mqtt`, `test_traffic_management` | ✅ 105/105 |
|
||||
| Build | `pio run -e native-macos` (M4 off) and with `-DNEXTHOP_EARLY_FLOOD_ON_UNVERIFIED=1` | ✅ both link |
|
||||
| Format | trunk `clang-format@16.0.3` | ✅ no issues |
|
||||
| Simulator (CI `simulator-tests`) | `meshtasticd -s` + `meshtastic.test.testSimulator()` on native-macos | ✅ exit 0, no crash |
|
||||
|
||||
**Pending (environment-blocked, not yet run):**
|
||||
|
||||
- **Multi-hop A-B-C recovery sim** - the `simulator/` broker hub is **not git-tracked**
|
||||
(only stale local `.pyc`), and two `meshtasticd -s` instances can't hear each other
|
||||
without it. The intermediate-node failure-count path and the M4 A/B therefore have unit
|
||||
coverage of their logic but no end-to-end multi-node run yet.
|
||||
- **Hardware / multi-hop tier** - a committable bench test now exists:
|
||||
`meshtastic-mcp/tests/mesh/test_nexthop_multihop_recovery.py`. It self-discovers a real
|
||||
multi-hop pair (A - relay - C), asserts a directed DM is delivered across the relay, and
|
||||
asserts delivery recovers after the relay is power-cycled (the M3 path). It
|
||||
`pytest.skip`s cleanly unless the bench is a true line with endpoints out of direct RF
|
||||
range (≥3 roles via `--hub-profile`), so it's safe to commit and only asserts when the
|
||||
NextHop path is genuinely exercised. Collected + verified to skip without hardware;
|
||||
not yet run on a bench. `test_direct_with_ack.py` / `test_peer_offline_recovery.py`
|
||||
remain the 2-device happy-path/recovery regressions.
|
||||
|
||||
---
|
||||
|
||||
## Risks & limitations
|
||||
|
||||
- **Site-1 impostor rebroadcast** is unfixable without a wider field - documented; M1/M2
|
||||
only shrink its frequency.
|
||||
- **Dense meshes flood DMs more often** - intended (a flooded DM arrives; a mis-unicast one
|
||||
black-holes). Call out in the PR so reviewers expect a slightly higher DM flood rate on
|
||||
very dense meshes.
|
||||
- **M4 airtime** if the gate is too loose → default conservative + compile-gated +
|
||||
simulator A/B before broad enable.
|
||||
- **RAM** ~384 B (32 slots); 16 slots (~192 B) with graceful LRU degradation if tight.
|
||||
- **Asymmetric flap** not fully closed (a _new_ bad hop resets the counter); the TTL
|
||||
backstop bounds it. Per-hop failure history is future work (more RAM).
|
||||
|
||||
---
|
||||
|
||||
## How to continue this work (commit sequencing)
|
||||
|
||||
Each step is independently testable; land them as separate commits.
|
||||
|
||||
1. **M1 resolver + unit tests** - `NodeDB` only; no behavior change until wired. Lands the
|
||||
`resolveLastByte`/`resolveUniqueLastByte` primitive and its full unit-test matrix.
|
||||
2. **M2 + wiring + tests** - `getNextHop` strict gate, learning gate, favorite-router
|
||||
preservation rewrite. Adds the `getNextHop` and site-4 tests.
|
||||
3. **M3 health table + decay + tests** - RAM `RouteHealth` table, decay-on-read, failure/
|
||||
success accounting, reconciliation with the existing last-retry reset. Adds the
|
||||
route-health unit tests and the simulator recovery check.
|
||||
4. **M4 gated tuning** - early-flood-on-unverified behind the compile flag; simulator A/B
|
||||
and hardware regression.
|
||||
|
||||
Reference plan (with the same content) was developed at
|
||||
`~/.claude/plans/nexthop-routing-for-direct-lexical-shell.md` on the author's machine; this
|
||||
in-repo doc is the canonical handoff copy.
|
||||
@@ -33,7 +33,7 @@ Import("env")
|
||||
|
||||
env.Append(LINKFLAGS=["-flto", "-flto-partition=1to1"])
|
||||
|
||||
# The -fno-lto re-compiles below run with the global env, which lacks the framework's
|
||||
# The ISR -fno-lto re-compiles below run with the global env, which lacks the framework's
|
||||
# bundled-library include dirs -- and those libs cross-include each other (Wire pulls in
|
||||
# Adafruit_TinyUSB.h, which pulls in SPI.h, ...). Add every bundled-lib dir (+ its src/) so
|
||||
# the re-compiles resolve without chasing headers one at a time.
|
||||
@@ -51,6 +51,52 @@ USB_ISR = "Adafruit_TinyUSB_nrf" # USBD_IRQHandler
|
||||
LIB_ISR = ("/bluefruit.cpp", "/Wire_nRF52.cpp", "/PDM.cpp", "/RotaryEncoder.cpp")
|
||||
|
||||
|
||||
# The active board variant file owns strong overrides of the core's weak initVariant()/
|
||||
# earlyInitVariant()/lateInitVariant() stubs (cores/nRF5/main.cpp). Same failure mode as the ISRs
|
||||
# above: whole-image LTO mis-resolves the call (made from the -fno-lto framework core) to the empty
|
||||
# WEAK stub, so the board's early hardware setup -- e.g. driving PIN_3V3_EN to power the LoRa/sensor
|
||||
# rail -- silently never runs and the radio probe finds no chip. Compiling the variant without LTO
|
||||
# lets ordinary linking pick the strong override. HW-proven on nrf52_promicro_diy_tcxo (boot-trace
|
||||
# 2026-06-30). NOTE: only the real board variant (built from /variants/...) -- NOT the guarded
|
||||
# src/platform/extra_variants/*/variant.cpp no-op stubs, which don't tolerate the -fno-lto recompile.
|
||||
#
|
||||
# Anchor to THIS repo's variants/ tree: a board variant's abspath is
|
||||
# <PROJECT_DIR>/variants/<arch>/<board>/variant.cpp. Anchoring deliberately excludes
|
||||
# - PlatformIO's framework-owned .../framework-arduinoadafruitnrf52/variants/*/variant.cpp
|
||||
# - the src/platform/extra_variants/*/variant.cpp stubs (they live under src/, not variants/)
|
||||
# both of which also end in "/variant.cpp" but must NOT be recompiled here.
|
||||
_PROJECT_VARIANTS = (
|
||||
env.subst("$PROJECT_DIR").replace("\\", "/").rstrip("/") + "/variants/"
|
||||
)
|
||||
|
||||
|
||||
def _is_board_variant(path):
|
||||
return (
|
||||
path.endswith("/variant.cpp")
|
||||
and path.startswith(_PROJECT_VARIANTS)
|
||||
and "extra_variants" not in path
|
||||
)
|
||||
|
||||
|
||||
# projenv is the construction env PlatformIO uses to compile project sources (src/ + the board
|
||||
# variant). Unlike the bare framework env captured above, it carries the -DAPP_VERSION... flags
|
||||
# and the generated-protobuf include path (pb.h) that bin/platformio-custom.py appends *after*
|
||||
# this pre-script's eval. It isn't exported yet at eval time, but it is by the time the build
|
||||
# middleware fires -- so fetch it lazily from SCons' export registry, falling back to env.
|
||||
_projenv_cache = []
|
||||
|
||||
|
||||
def _get_projenv():
|
||||
if not _projenv_cache:
|
||||
try:
|
||||
from SCons.Script.SConscript import global_exports
|
||||
|
||||
_projenv_cache.append(global_exports.get("projenv", env))
|
||||
except Exception:
|
||||
_projenv_cache.append(env)
|
||||
return _projenv_cache[0]
|
||||
|
||||
|
||||
def _no_lto(node):
|
||||
try:
|
||||
path = node.get_abspath()
|
||||
@@ -69,6 +115,17 @@ def _no_lto(node):
|
||||
CCFLAGS=env["CCFLAGS"] + ["-fno-lto"],
|
||||
CPPPATH=env["CPPPATH"] + _extra_inc,
|
||||
)
|
||||
if _is_board_variant(path):
|
||||
# The board variant is a project source, not a framework object: it can #include
|
||||
# configuration.h/sleep.h, which need the -DAPP_VERSION... define and the generated-
|
||||
# protobuf include path (pb.h). Recompile it with projenv (which has both) -- using the
|
||||
# bare framework env here fails with "APP_VERSION must be set" / "pb.h: No such file".
|
||||
build_env = _get_projenv()
|
||||
return build_env.Object(
|
||||
node,
|
||||
CCFLAGS=build_env["CCFLAGS"] + ["-fno-lto"],
|
||||
CPPPATH=build_env["CPPPATH"] + _extra_inc,
|
||||
)
|
||||
return node
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
#!/usr/bin/env python3
|
||||
# trunk-ignore-all(ruff/F821)
|
||||
# trunk-ignore-all(flake8/F821): For SConstruct imports
|
||||
#
|
||||
# Post-link guard for the warm-node-store raw-flash region on nRF52840.
|
||||
#
|
||||
# The 3 app-region pages below LittleFS (0xEA000-0xED000, reclaimed by whole-image
|
||||
# LTO) are reserved for the WarmNodeStore record-ring (see WarmNodeStore.h). Our
|
||||
# linker scripts (nrf52840_s140_v6.ld and nrf52840_s140_v7.ld) cap the image at
|
||||
# 0xEA000, but boards on the framework-default script (FLASH ending at 0xED000) could
|
||||
# silently place code in those pages - the first warm-store save would then brick the
|
||||
# device. This turns that into a build failure.
|
||||
#
|
||||
# Image flash end = __etext + sizeof(.data) (loaded at LMA __etext); symbols from
|
||||
# the framework's nrf52_common.ld.
|
||||
import os
|
||||
|
||||
Import("env")
|
||||
|
||||
WARM_REGION_BASE = 0xEA000 # keep in sync with WARM_FLASH_REGION_BASE in WarmNodeStore.h (3 x 4 KB record-ring)
|
||||
|
||||
_tc = env.PioPlatform().get_package_dir("toolchain-gccarmnoneeabi") or ""
|
||||
_NM = os.path.join(_tc, "bin", "arm-none-eabi-nm")
|
||||
if not os.path.isfile(_NM):
|
||||
_NM = "arm-none-eabi-nm" # fall back to PATH
|
||||
|
||||
|
||||
def _assert_warm_region_clear(source, target, env):
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
try:
|
||||
elf = env.subst("$BUILD_DIR/${PROGNAME}.elf")
|
||||
out = subprocess.check_output([_NM, elf], universal_newlines=True)
|
||||
except Exception as exc: # tooling hiccup: warn loudly, don't wedge the build
|
||||
print("nrf52_warm_region: WARNING - guard skipped (nm failed: %s)" % exc)
|
||||
return
|
||||
|
||||
syms = {}
|
||||
for line in out.split("\n"):
|
||||
f = line.split()
|
||||
if len(f) >= 3 and f[-1] in ("__etext", "__data_start__", "__data_end__"):
|
||||
syms[f[-1]] = int(f[0], 16)
|
||||
if len(syms) != 3:
|
||||
print("nrf52_warm_region: WARNING - guard skipped (linker symbols not found)")
|
||||
return
|
||||
|
||||
flash_end = syms["__etext"] + (syms["__data_end__"] - syms["__data_start__"])
|
||||
if flash_end > WARM_REGION_BASE:
|
||||
sys.stderr.write(
|
||||
"\n*** nrf52 warm-region guard: image ends at 0x%X, past the reserved "
|
||||
"warm-store region at 0x%X ***\n"
|
||||
"The 12 KB region at 0xEA000 holds the WarmNodeStore record-ring; a warm-store\n"
|
||||
"save would overwrite this firmware's tail. Shrink the image, or shrink/move\n"
|
||||
"the region (WARM_FLASH_REGION_BASE in src/mesh/WarmNodeStore.h, the FLASH\n"
|
||||
"LENGTH in src/platform/nrf52/nrf52840_s140_v6.ld and _v7.ld, and this guard).\n\n"
|
||||
% (flash_end, WARM_REGION_BASE)
|
||||
)
|
||||
from SCons.Script import Exit
|
||||
|
||||
Exit(1)
|
||||
print(
|
||||
"nrf52_warm_region: guard OK -- image ends at 0x%X, %d KB clear of the warm region"
|
||||
% (flash_end, (WARM_REGION_BASE - flash_end) // 1024)
|
||||
)
|
||||
|
||||
|
||||
# Attach to the phony "buildprog" alias (not the .elf node) so the guard runs
|
||||
# on incremental relinks too -- same reasoning as nrf52_lto.py's guard.
|
||||
env.AddPostAction("buildprog", _assert_warm_region_clear)
|
||||
@@ -31,7 +31,7 @@ else:
|
||||
The rule format depends on the host:
|
||||
Windows (CMake's RunCMake wraps every command):
|
||||
COMMAND = cmd.exe /C "cd /D DIR && arm-none-eabi-gcc.exe ... -o linker.cmd && cmake.exe -E cmake_transform_depfile ..."
|
||||
POSIX (Linux/macOS — no wrapper):
|
||||
POSIX (Linux/macOS - no wrapper):
|
||||
COMMAND = cd DIR && arm-none-eabi-gcc ... -o linker.cmd && cmake -E cmake_transform_depfile ...
|
||||
|
||||
Returns (gcc_cmd_string, cwd_path) or raises RuntimeError.
|
||||
@@ -51,7 +51,7 @@ else:
|
||||
|
||||
command_val = stripped[len("COMMAND = ") :]
|
||||
|
||||
# On Windows the value is wrapped in `cmd.exe /C "..."` — strip
|
||||
# On Windows the value is wrapped in `cmd.exe /C "..."` - strip
|
||||
# the wrapper. On POSIX hosts the inner sequence is the value
|
||||
# itself (no quoting layer).
|
||||
m = re.search(r'/C\s+"(.*)"\s*$', command_val)
|
||||
@@ -88,7 +88,7 @@ else:
|
||||
linker_cmd = os.path.join(zephyr_dir, "linker.cmd")
|
||||
|
||||
if os.path.exists(linker_cmd):
|
||||
return # Already present — nothing to do
|
||||
return # Already present - nothing to do
|
||||
|
||||
ninja_build = os.path.join(build_dir, "build.ninja")
|
||||
if not os.path.exists(ninja_build):
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
#!/usr/bin/env python3
|
||||
# trunk-ignore-all(ruff/F821)
|
||||
# trunk-ignore-all(flake8/F821): For SConstruct imports
|
||||
#
|
||||
# Firmware-specific Emscripten *link* settings for [env:native-wasm].
|
||||
#
|
||||
# PlatformIO routes `build_flags` to the compile step, not the link step, so the
|
||||
# WASM node's link-time emscripten settings have to be appended to LINKFLAGS
|
||||
# here. The generic, app-agnostic flags (Asyncify, MODULARIZE, ALLOW_MEMORY_
|
||||
# GROWTH, the ES6 module shape) live in the platform-wasm builder; what belongs
|
||||
# to *this firmware* is:
|
||||
#
|
||||
# * EXPORT_NAME - the ES-module factory name consumers import.
|
||||
# * EXPORTED_RUNTIME_METHODS - ccall/cwrap/callMain + FS/IDBFS/NODEFS/PATH and
|
||||
# the string helpers the JS host + bridge drive.
|
||||
# * EXPORTED_FUNCTIONS - the C entry points (the wasm_* API is also kept via
|
||||
# EMSCRIPTEN_KEEPALIVE in source; _malloc/_free are
|
||||
# needed so the host can marshal protobuf buffers).
|
||||
# * ASYNCIFY_IMPORTS - the WebUSB seam: these imported C functions suspend
|
||||
# the stack (Asyncify) while a WebUSB transfer awaits.
|
||||
#
|
||||
# Only attached to the wasm env (see extra_scripts in [env:native-wasm]); a guard keeps
|
||||
# it inert if it is ever pulled into another env.
|
||||
#
|
||||
Import("env")
|
||||
|
||||
if env["PIOENV"] == "native-wasm":
|
||||
env.Append(
|
||||
LINKFLAGS=[
|
||||
"-sEXPORT_NAME=createMeshNode",
|
||||
"-sEXPORTED_RUNTIME_METHODS=ccall,cwrap,callMain,FS,IDBFS,NODEFS,PATH,HEAPU8,UTF8ToString,stringToUTF8",
|
||||
"-sEXPORTED_FUNCTIONS=_main,_wasm_setup,_wasm_loop_once,_wasm_fs_sync,"
|
||||
"_wasm_set_region,_wasm_api_to_radio,_wasm_api_from_radio,"
|
||||
"_wasm_api_available,_wasm_api_is_connected,_wasm_set_lora_module,"
|
||||
"_wasm_set_lora_usb_ids,_wasm_set_lora_usb_serial,"
|
||||
"_wasm_set_lora_dio_config,_wasm_set_lora_spi_speed,_wasm_set_lora_pin,"
|
||||
"_malloc,_free",
|
||||
"-sASYNCIFY_IMPORTS=webusb_open,webusb_transceive,webusb_digital_write,"
|
||||
"webusb_digital_read,webusb_close",
|
||||
]
|
||||
)
|
||||
@@ -1,35 +0,0 @@
|
||||
.venv/
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.egg-info/
|
||||
.pytest_cache/
|
||||
.mypy_cache/
|
||||
dist/
|
||||
build/
|
||||
|
||||
# Persistent device-log capture (recorder + Datadog cursor).
|
||||
# Cross-session JSONL streams written by the autouse Recorder singleton
|
||||
# (see src/meshtastic_mcp/recorder/). Lives outside tests/ so the pytest
|
||||
# fixture truncate doesn't touch it.
|
||||
.mtlog/
|
||||
|
||||
# Test harness artifacts
|
||||
tests/report.html
|
||||
tests/junit.xml
|
||||
tests/reportlog.jsonl
|
||||
tests/fwlog.jsonl
|
||||
# Subprocess-output tee from pio/esptool/nrfutil/picotool (live flash
|
||||
# progress for the TUI; also a post-run diagnostic for plain CLI runs).
|
||||
tests/flash.log
|
||||
tests/tool_coverage.json
|
||||
tests/.coverage
|
||||
htmlcov/
|
||||
# Persistent run counter for meshtastic-mcp-test-tui header.
|
||||
tests/.tui-runs
|
||||
# Cross-run history (TUI duration sparkline).
|
||||
tests/.history/
|
||||
# Reproducer bundles (TUI `x` export on failed tests).
|
||||
tests/reproducers/
|
||||
# UI-tier camera captures + per-test transcripts. Regenerated every run;
|
||||
# left on disk for human review between runs.
|
||||
tests/ui_captures/
|
||||
@@ -1,412 +0,0 @@
|
||||
# Meshtastic MCP Server
|
||||
|
||||
An [MCP](https://modelcontextprotocol.io) server for working with the Meshtastic firmware repo and connected devices. Lets Claude Code / Claude Desktop:
|
||||
|
||||
- Discover USB-connected Meshtastic devices
|
||||
- Enumerate PlatformIO board variants (166+) with Meshtastic metadata
|
||||
- Build, clean, flash, erase-and-flash (factory), and OTA-update firmware
|
||||
- Read serial logs via `pio device monitor` (with board-specific exception decoders)
|
||||
- Trigger 1200bps touch-reset for bootloader entry (nRF52, ESP32-S3, RP2040)
|
||||
- Query and administer a running node via the [`meshtastic` Python API](https://github.com/meshtastic/python): owner name, config (LocalConfig + ModuleConfig), channels, messaging, reboot/shutdown/factory-reset
|
||||
- Call `esptool`, `nrfutil`, `picotool` directly when PlatformIO doesn't cover the operation
|
||||
|
||||
## Design principle
|
||||
|
||||
**PlatformIO first.** Its `pio run -t upload` knows the correct protocol, offsets, and post-build chain for every variant in `variants/`. Direct vendor-tool wrappers (`esptool_*`, `nrfutil_*`, `picotool_*`) exist as escape hatches for operations pio doesn't cover (blank-chip erase, DFU `.zip` packages, BOOTSEL-mode inspection).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python ≥ 3.11
|
||||
- [PlatformIO Core](https://platformio.org/install/cli) — `pio` on `$PATH` or at `~/.platformio/penv/bin/pio`
|
||||
- The Meshtastic firmware repo checked out somewhere (set via `MESHTASTIC_FIRMWARE_ROOT`)
|
||||
- Optional: `esptool`, `nrfutil`, `picotool` on `$PATH` (or under the firmware venv at `.venv/bin/`) if you want to use the direct-tool wrappers
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
cd <firmware-repo>/mcp-server
|
||||
python3 -m venv .venv
|
||||
.venv/bin/pip install -e .
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
MESHTASTIC_FIRMWARE_ROOT=<firmware-repo> .venv/bin/python -m meshtastic_mcp
|
||||
```
|
||||
|
||||
The server blocks on stdin (that's correct — it speaks MCP over stdio). Ctrl-C to exit.
|
||||
|
||||
## Register with Claude Code
|
||||
|
||||
Edit `~/.claude/settings.json` (global) or `<firmware-repo>/.claude/settings.local.json` (project-only):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"meshtastic": {
|
||||
"command": "<firmware-repo>/mcp-server/.venv/bin/python",
|
||||
"args": ["-m", "meshtastic_mcp"],
|
||||
"env": {
|
||||
"MESHTASTIC_FIRMWARE_ROOT": "<firmware-repo>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Replace `<firmware-repo>` with the absolute path, e.g. `/Users/you/GitHub/firmware`. Restart Claude Code after editing.
|
||||
|
||||
## Register with Claude Desktop
|
||||
|
||||
Same `mcpServers` block, but in `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows).
|
||||
|
||||
## Tools (43)
|
||||
|
||||
### Discovery & metadata
|
||||
|
||||
| Tool | What it does |
|
||||
| -------------- | ------------------------------------------------------------------------------------------ |
|
||||
| `list_devices` | USB/serial port listing, flags likely-Meshtastic candidates |
|
||||
| `list_boards` | PlatformIO envs with `custom_meshtastic_*` metadata; filters by arch/supported/query/level |
|
||||
| `get_board` | Full env dict incl. raw pio config |
|
||||
|
||||
### Build & flash
|
||||
|
||||
| Tool | What it does |
|
||||
| ----------------- | -------------------------------------------------------------------- |
|
||||
| `build` | `pio run -e <env>` (+ mtjson target) |
|
||||
| `clean` | `pio run -e <env> -t clean` |
|
||||
| `pio_flash` | `pio run -e <env> -t upload --upload-port <port>` — any architecture |
|
||||
| `erase_and_flash` | ESP32 full factory flash via `bin/device-install.sh` |
|
||||
| `update_flash` | ESP32 OTA app-partition update via `bin/device-update.sh` |
|
||||
| `touch_1200bps` | 1200-baud open/close to trigger USB CDC bootloader entry |
|
||||
|
||||
### Serial log sessions
|
||||
|
||||
Backed by long-running `pio device monitor` subprocesses with a 10k-line ring buffer per session and board-specific filters (`esp32_exception_decoder` auto-selected when you pass `env=`).
|
||||
|
||||
| Tool | What it does |
|
||||
| -------------- | ------------------------------------------------------------------ |
|
||||
| `serial_open` | Start a monitor session; returns `session_id` |
|
||||
| `serial_read` | Cursor-based pull; reports `dropped` if lines aged out of the ring |
|
||||
| `serial_list` | All active sessions |
|
||||
| `serial_close` | Terminate a session |
|
||||
|
||||
### Device reads
|
||||
|
||||
| Tool | What it does |
|
||||
| ------------- | --------------------------------------------------------------------------- |
|
||||
| `device_info` | my_node_num, long/short name, firmware version, region, channel, node count |
|
||||
| `list_nodes` | Full node database with position, SNR, RSSI, last_heard, battery |
|
||||
|
||||
_The tool tables below document 38 currently registered MCP server tools._
|
||||
|
||||
### Device writes
|
||||
|
||||
| Tool | What it does |
|
||||
| ------------------- | -------------------------------------------------------------------------- |
|
||||
| `set_owner` | Long name + optional short name (≤4 chars) |
|
||||
| `get_config` | One section or all (LocalConfig + ModuleConfig) |
|
||||
| `set_config` | Dot-path field write: `lora.region`=`"US"`, `device.role`=`"ROUTER"`, etc. |
|
||||
| `get_channel_url` | Primary-only or include_all=admin URL |
|
||||
| `set_channel_url` | Import channels from a Meshtastic URL |
|
||||
| `set_debug_log_api` | Enable or disable debug logging for the Meshtastic Python API client |
|
||||
| `send_text` | Broadcast or direct text message |
|
||||
| `reboot` | `localNode.reboot(secs)` — requires `confirm=True` |
|
||||
| `shutdown` | `localNode.shutdown(secs)` — requires `confirm=True` |
|
||||
| `factory_reset` | `localNode.factoryReset(full?)` — requires `confirm=True` |
|
||||
|
||||
### Direct hardware tools (escape hatches)
|
||||
|
||||
| Tool | What it does |
|
||||
| --------------------- | --------------------------------------------------------- |
|
||||
| `esptool_chip_info` | Read chip, MAC, crystal, flash size |
|
||||
| `esptool_erase_flash` | Full-chip erase (destructive) |
|
||||
| `esptool_raw` | Pass-through; confirm=True required for write/erase/merge |
|
||||
| `nrfutil_dfu` | DFU-flash a `.zip` package |
|
||||
| `nrfutil_raw` | Pass-through |
|
||||
| `picotool_info` | Read Pico BOOTSEL-mode info |
|
||||
| `picotool_load` | Load a UF2 |
|
||||
| `picotool_raw` | Pass-through |
|
||||
|
||||
### USB power control (uhubctl)
|
||||
|
||||
| Tool | What it does |
|
||||
| --------------- | ----------------------------------------------------------- |
|
||||
| `uhubctl_list` | Enumerate USB hubs + attached-device VID/PID (read-only) |
|
||||
| `uhubctl_power` | Drive a hub port `on` or `off`; `off` requires confirm=True |
|
||||
| `uhubctl_cycle` | Off → wait `delay_s` → on; confirm=True required |
|
||||
|
||||
Target a port by explicit `(location, port)` (raw uhubctl syntax like
|
||||
`location="1-1.3", port=2`) or by `role` (`"nrf52"`, `"esp32s3"`). Role
|
||||
lookup checks `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` +
|
||||
`MESHTASTIC_UHUBCTL_PORT_<ROLE>` env vars first, then auto-detects via VID
|
||||
against `uhubctl`'s output.
|
||||
|
||||
Requires [`uhubctl`](https://github.com/mvp/uhubctl) on PATH:
|
||||
|
||||
```bash
|
||||
brew install uhubctl # macOS
|
||||
apt install uhubctl # Debian/Ubuntu
|
||||
```
|
||||
|
||||
Modern macOS + PPPS-capable hubs generally work without root. On Linux
|
||||
without udev rules, or on old macOS with driver quirks, you may need
|
||||
`sudo`. If uhubctl returns a permission error the MCP tool raises a
|
||||
clear `UhubctlError` pointing at the
|
||||
[udev-rules / sudo fallback](https://github.com/mvp/uhubctl#linux-usb-permissions)
|
||||
rather than auto-`sudo`'ing mid-run.
|
||||
|
||||
## Safety
|
||||
|
||||
- **All destructive flash/admin tools require `confirm=True`** as a tool-level gate, on top of any permission prompt from Claude.
|
||||
- **Serial port is exclusive.** If a `serial_*` session is active on a port, `device_info`/admin tools on the same port will fail fast with a pointer at the active `session_id`. Close the session first.
|
||||
- **Flash confirmation by architecture**: `erase_and_flash` / `update_flash` error if the env's architecture isn't ESP32 — use `pio_flash` for nRF52/RP2040/STM32.
|
||||
|
||||
## Environment variables
|
||||
|
||||
| Var | Default | Purpose |
|
||||
| -------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| `MESHTASTIC_FIRMWARE_ROOT` | walks up from cwd for `platformio.ini` | Pin the firmware repo |
|
||||
| `MESHTASTIC_PIO_BIN` | `~/.platformio/penv/bin/pio` → `$PATH` `pio` → `platformio` | Override `pio` location |
|
||||
| `MESHTASTIC_ESPTOOL_BIN` | `<firmware>/.venv/bin/esptool` → `$PATH` | Override esptool |
|
||||
| `MESHTASTIC_NRFUTIL_BIN` | `$PATH` | Override nrfutil |
|
||||
| `MESHTASTIC_PICOTOOL_BIN` | `$PATH` | Override picotool |
|
||||
| `MESHTASTIC_MCP_SEED` | `mcp-<user>-<host>` | PSK seed for test-harness session (CI override) |
|
||||
| `MESHTASTIC_MCP_FLASH_LOG` | `<mcp-server>/tests/flash.log` | Tee target for pio/esptool/nrfutil subprocess output (TUI tails it) |
|
||||
| `MESHTASTIC_MCP_TCP_HOST` | unset | `host` or `host:port` of a `meshtasticd` daemon to surface as a TCP device (see "TCP / native-host nodes" below) |
|
||||
|
||||
## TCP / native-host nodes
|
||||
|
||||
The `native-macos` and `native` PlatformIO envs build a headless `meshtasticd`
|
||||
binary that runs on the host (Apple Silicon / Intel macOS, or Linux Portduino).
|
||||
The daemon exposes the meshtastic TCP API on port `4403` rather than a USB
|
||||
serial endpoint — point the MCP server at it via `MESHTASTIC_MCP_TCP_HOST`:
|
||||
|
||||
```bash
|
||||
# 1. Build + run a daemon on this host (see variants/native/portduino/platformio.ini
|
||||
# for full Homebrew prereqs and CH341 LoRa-adapter setup).
|
||||
pio run -e native-macos
|
||||
~/.meshtasticd/meshtasticd
|
||||
|
||||
# 2. Point the MCP server at it.
|
||||
export MESHTASTIC_MCP_TCP_HOST=localhost # or host:port, default port 4403
|
||||
```
|
||||
|
||||
**First-run gotcha — MAC address.** `meshtasticd` derives its MAC from the
|
||||
USB adapter's serial-number / product strings. Many cheap CH341 dongles
|
||||
(MeshStick included — VID 0x1A86 / PID 0x5512) ship with `iSerialNumber=0`
|
||||
and `iProduct=0`, so the daemon aborts on boot with `*** Blank MAC Address
|
||||
not allowed!`. Set the MAC explicitly in `config.yaml`:
|
||||
|
||||
```yaml
|
||||
# Under General:
|
||||
MACAddress: 02:CA:FE:BA:BE:01
|
||||
```
|
||||
|
||||
Use a locally-administered address (first byte's second-LSB set, e.g.
|
||||
`02:*` / `06:*` / `0A:*` / `0E:*`) to avoid colliding with a real OUI.
|
||||
|
||||
There is also a `--hwid AA:BB:CC:DD:EE:FF` CLI flag visible in
|
||||
`meshtasticd --help`, but it is **currently broken** in
|
||||
`MAC_from_string()` (`src/platform/portduino/PortduinoGlue.cpp`): the
|
||||
function strips colons from its parameter but then reads bytes from the
|
||||
global `portduino_config.mac_address`, so `--hwid` is silently overridden
|
||||
when `MACAddress:` is also set, and crashes the daemon (uncaught
|
||||
`std::invalid_argument: stoi: no conversion`) when it isn't. Use the YAML
|
||||
form until that's fixed upstream.
|
||||
|
||||
`list_devices` will surface the daemon as `tcp://localhost:4403` with
|
||||
`likely_meshtastic=True`, so `device_info`, `list_nodes`, `get_config`,
|
||||
`set_config`, `set_owner`, `send_text`, `userprefs_*`, and the admin RPCs
|
||||
auto-select it when no `port` is passed. Pass `port="tcp://other-host:9999"`
|
||||
explicitly to target a different daemon.
|
||||
|
||||
**Tools that don't apply to a TCP/native node** (no USB hardware to operate
|
||||
on) raise a clear `ConnectionError` rather than failing mysteriously:
|
||||
`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`,
|
||||
`serial_open` (use info/admin tools directly), and the vendor escape hatches
|
||||
`esptool_*`, `nrfutil_*`, `picotool_*`. `pio_flash` against a `native*` env
|
||||
similarly raises — there's no upload step; use `build` and run the binary
|
||||
directly.
|
||||
|
||||
The pytest harness in `tests/` still assumes USB-attached devices per role —
|
||||
TCP-aware fixtures are not part of this surface yet.
|
||||
|
||||
## Hardware Test Suite
|
||||
|
||||
`mcp-server/tests/` holds a pytest-based integration suite that exercises
|
||||
real USB-connected Meshtastic devices against the MCP server surface. Separate
|
||||
from the native C++ unit tests in the firmware repo's top-level `test/`
|
||||
directory — this one validates the device-facing behavior end-to-end.
|
||||
|
||||
### Invocation
|
||||
|
||||
```bash
|
||||
./mcp-server/run-tests.sh # full suite (auto-detect + auto-bake-if-needed)
|
||||
./mcp-server/run-tests.sh --force-bake # reflash devices before testing
|
||||
./mcp-server/run-tests.sh --assume-baked # skip the bake step (caller vouches for state)
|
||||
./mcp-server/run-tests.sh tests/mesh # one tier
|
||||
./mcp-server/run-tests.sh tests/mesh/test_traceroute.py # one file
|
||||
./mcp-server/run-tests.sh -k telemetry # pytest name filter
|
||||
```
|
||||
|
||||
The wrapper auto-detects connected devices (VID `0x239A` → `nrf52` → env
|
||||
`rak4631`; `0x303A` or `0x10C4` → `esp32s3` → env `heltec-v3`), exports
|
||||
`MESHTASTIC_MCP_ENV_<ROLE>` env vars, and invokes pytest. Overrides via
|
||||
per-role env vars: `MESHTASTIC_MCP_ENV_NRF52=heltec-mesh-node-t114 ./run-tests.sh`.
|
||||
|
||||
No hardware connected? The wrapper narrows to `tests/unit/` only and says so
|
||||
in the pre-flight header.
|
||||
|
||||
### Tiers (run in this order)
|
||||
|
||||
- **`bake`** (`tests/test_00_bake.py`) — flashes both hub roles with the
|
||||
session's test profile. Has a skip-if-already-baked check (region + channel
|
||||
match); `--force-bake` overrides.
|
||||
- **`unit`** — pure Python, no hardware. boards / PIO wrapper /
|
||||
userPrefs-parse / testing-profile fixtures.
|
||||
- **`mesh`** — 2-device mesh: formation, broadcast delivery, direct+ACK,
|
||||
traceroute, bidirectional. Parametrized over both directions. Includes
|
||||
`test_peer_offline_recovery` which uses uhubctl to power-cycle one peer
|
||||
mid-conversation and verifies the mesh recovers (skips without uhubctl).
|
||||
- **`telemetry`** — periodic telemetry broadcast + on-demand request/reply
|
||||
(`TELEMETRY_APP` with `wantResponse=True`).
|
||||
- **`monitor`** — boot log has no panic markers within 60 s of reboot.
|
||||
- **`recovery`** — `uhubctl` power-cycle round-trip: verifies the hub port
|
||||
can be toggled off/on, the device re-enumerates with the same
|
||||
`my_node_num`, and NVS-resident config (region, channel, modem preset)
|
||||
survives a hard reset. Requires `uhubctl` on PATH; skips cleanly otherwise.
|
||||
- **`ui`** — input-broker-driven screen navigation (`AdminMessage.send_input_event`
|
||||
injection → `Screen::handleInputEvent` → frame transition). Parametrized
|
||||
on the screen-bearing role (heltec-v3 OLED). Captures images via USB
|
||||
webcam + OCRs them for HTML-report evidence. Requires `pip install -e '.[ui]'`
|
||||
and `MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=<index>`; tier is auto-deselected
|
||||
if `cv2` isn't importable.
|
||||
- **`fleet`** — PSK-seed isolation: two labs with different seeds never
|
||||
overlap.
|
||||
- **`admin`** — owner persistence across reboot, channel URL round-trip,
|
||||
`lora.hop_limit` persistence.
|
||||
- **`provisioning`** — region/channel baking, userPrefs survive
|
||||
`factory_reset(full=False)`.
|
||||
|
||||
#### UI tier setup
|
||||
|
||||
The `tests/ui/` tier drives the on-device OLED via the firmware's existing
|
||||
`AdminMessage.send_input_event` RPC (no firmware changes required) and
|
||||
verifies transitions via a macro-gated log line + camera + OCR. Summary:
|
||||
|
||||
1. Install extras: `pip install -e 'mcp-server/.[ui]'` — pulls in
|
||||
`opencv-python-headless`, `numpy`, `easyocr`, `Pillow`. First easyocr
|
||||
run downloads ~100 MB of models to `~/.EasyOCR/`; an autouse session
|
||||
fixture pre-warms the reader so per-test OCR is <100 ms after that.
|
||||
2. Point a USB webcam at the heltec-v3 OLED. Discover its index:
|
||||
```bash
|
||||
.venv/bin/python -c "import cv2; [print(i, cv2.VideoCapture(i).read()[0]) for i in range(5)]"
|
||||
```
|
||||
3. Export the per-role device env var:
|
||||
```bash
|
||||
export MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0
|
||||
```
|
||||
4. Run:
|
||||
```bash
|
||||
./run-tests.sh tests/ui -v
|
||||
```
|
||||
Captures land under `tests/ui_captures/<session_seed>/<test_id>/`, one
|
||||
PNG + `.ocr.txt` per `frame_capture()` call, with a per-test
|
||||
`transcript.md` stepping through event → frame → OCR. The HTML report
|
||||
embeds the full image strip inline (pass or fail).
|
||||
|
||||
On macOS, `cv2.VideoCapture(0)` triggers the TCC Camera permission prompt
|
||||
on first use. Pre-grant Terminal (or your IDE's terminal) before running.
|
||||
The `OpenCVBackend` fails fast on 10 consecutive black frames so a silent
|
||||
permission denial surfaces as a clear error, not an empty PNG strip.
|
||||
|
||||
No camera? Set `MESHTASTIC_UI_CAMERA_BACKEND=null` (or leave the device var
|
||||
unset). Tests still exercise the event-injection path and log assertions;
|
||||
captures just become 1×1 black PNGs.
|
||||
|
||||
### Artifacts (regenerated every run, under `tests/`)
|
||||
|
||||
- `report.html` — self-contained pytest-html report. Each test gets a
|
||||
**Meshtastic debug** section attached on failure with a 200-line firmware
|
||||
log tail + device-state dump. Open this first on failures.
|
||||
- `junit.xml` — CI-parseable.
|
||||
- `reportlog.jsonl` — `pytest-reportlog` event stream; consumed by the TUI.
|
||||
- `fwlog.jsonl` — firmware log mirror (`meshtastic.log.line` pubsub → JSONL).
|
||||
- `flash.log` — tee of all pio / esptool / nrfutil / picotool subprocess
|
||||
output during the run (driven by `MESHTASTIC_MCP_FLASH_LOG`).
|
||||
|
||||
### Live TUI
|
||||
|
||||
```bash
|
||||
.venv/bin/meshtastic-mcp-test-tui
|
||||
.venv/bin/meshtastic-mcp-test-tui tests/mesh # pytest args pass through
|
||||
```
|
||||
|
||||
Textual-based wrapper over `run-tests.sh` with a live test tree, tier
|
||||
counters, pytest output pane, firmware-log pane, and a device-status strip.
|
||||
Key bindings: `r` re-run focused, `f` filter, `d` failure detail, `g` open
|
||||
`report.html`, `x` export reproducer bundle, `l` cycle fw-log filter, `q`
|
||||
quit (SIGINT → SIGTERM → SIGKILL escalation).
|
||||
|
||||
Set `MESHTASTIC_UI_TUI_CAMERA=1` to mount a bottom-of-screen **UI camera**
|
||||
panel. Left side: the latest capture PNG rendered as Unicode half-blocks
|
||||
(via `rich-pixels`, works in any terminal — no kitty/sixel required).
|
||||
Right side: live transcript tail ("step 3 — frame 4/8 name=nodelist_nodes
|
||||
— OCR: Nodes 2/2") so you can see every event-injection and its result
|
||||
as each UI test runs. Requires the `[ui]` extras for image rendering; the
|
||||
transcript alone works without them.
|
||||
|
||||
### Slash commands
|
||||
|
||||
Three AI-assisted workflows are wired up for Claude Code operators
|
||||
(`.claude/commands/`) and Copilot operators (`.github/prompts/`):
|
||||
`/test` (run + interpret), `/diagnose` (read-only health report), `/repro`
|
||||
(flake triage, N-times re-run with log diff).
|
||||
|
||||
### House rules (for human + agent contributors)
|
||||
|
||||
- Session-scoped fixtures in `tests/conftest.py` snapshot + restore
|
||||
`userPrefs.jsonc`; **never edit `userPrefs.jsonc` from inside a test**.
|
||||
Use the `test_profile` / `no_region_profile` fixtures for ephemeral
|
||||
overrides.
|
||||
- `SerialInterface` holds an **exclusive port lock**; sequence calls
|
||||
open → mutate → close, then next device. No parallel calls to the
|
||||
same port.
|
||||
- Directed PKI-encrypted sends need **bilateral NodeInfo warmup** —
|
||||
both sides must hold the other's current pubkey. See
|
||||
`tests/mesh/_receive.py::nudge_nodeinfo_port` and the three directed-
|
||||
send tests (`test_direct_with_ack`, `test_traceroute`,
|
||||
`test_telemetry_request_reply`) for the canonical pattern.
|
||||
|
||||
## Layout
|
||||
|
||||
```text
|
||||
mcp-server/
|
||||
├── pyproject.toml
|
||||
├── README.md
|
||||
└── src/meshtastic_mcp/
|
||||
├── __main__.py # entry: python -m meshtastic_mcp
|
||||
├── server.py # FastMCP app + @app.tool() registrations (thin)
|
||||
├── config.py # firmware_root, pio_bin, esptool_bin, etc.
|
||||
├── pio.py # subprocess wrapper (timeouts, JSON, tail_lines)
|
||||
├── devices.py # list_devices (findPorts + comports)
|
||||
├── boards.py # list_boards / get_board (pio project config parse + cache)
|
||||
├── flash.py # build, clean, flash, erase_and_flash, update_flash, touch_1200bps
|
||||
├── serial_session.py # SerialSession + reader thread + ring buffer
|
||||
├── registry.py # session registry + per-port locks
|
||||
├── connection.py # connect(port) ctx mgr — SerialInterface + port lock
|
||||
├── info.py # device_info, list_nodes
|
||||
├── admin.py # set_owner, get/set_config, channels, send_text, reboot/shutdown/factory_reset
|
||||
└── hw_tools.py # esptool / nrfutil / picotool wrappers
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **"Could not locate Meshtastic firmware root"** — set `MESHTASTIC_FIRMWARE_ROOT`.
|
||||
- **"Could not find `pio`"** — install PlatformIO or set `MESHTASTIC_PIO_BIN`.
|
||||
- **"Port is held by serial session ..."** — call `serial_close(session_id)` or `serial_list` to find it.
|
||||
- **`factory.bin` not found after build** — the env may not be ESP32; only ESP32 envs produce a `.factory.bin`.
|
||||
- **`touch_1200bps` reported `new_port: null`** — the device may not have 1200bps-reset stdio, or the bootloader re-uses the same port name. Check `list_devices` manually.
|
||||
@@ -1,54 +0,0 @@
|
||||
[project]
|
||||
name = "meshtastic-mcp"
|
||||
version = "0.1.0"
|
||||
description = "MCP server for Meshtastic firmware development: device discovery, PlatformIO tooling, flashing, serial monitoring, and device administration via the meshtastic Python API."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
license = { text = "GPL-3.0-only" }
|
||||
authors = [{ name = "thebentern" }]
|
||||
dependencies = ["mcp>=1.2", "pyserial>=3.5", "meshtastic>=2.7.8"]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = ["pytest>=7"]
|
||||
test = [
|
||||
"pytest>=8",
|
||||
"pytest-html>=4",
|
||||
"pytest-reportlog>=0.4",
|
||||
"pytest-timeout>=2.3",
|
||||
"coverage[toml]>=7",
|
||||
"pyyaml>=6",
|
||||
# textual is required by the `meshtastic-mcp-test-tui` script (see
|
||||
# `src/meshtastic_mcp/cli/test_tui.py`). Bundled into `test` rather than a
|
||||
# separate `[tui]` extra because v1 expects test operators are the only
|
||||
# consumers; revisit if install cost pushes back.
|
||||
"textual>=0.50",
|
||||
]
|
||||
# UI test tier + `capture_screen` MCP tool. Optional because the ML OCR
|
||||
# model alone is ~100 MB and camera hardware is user-supplied.
|
||||
# pip install -e '.[ui]' — full (OpenCV + easyocr)
|
||||
# pip install -e '.[ui-min]' — image capture only, no OCR
|
||||
ui = [
|
||||
"opencv-python-headless>=4.9",
|
||||
"numpy>=1.26",
|
||||
"easyocr>=1.7",
|
||||
"Pillow>=10.0",
|
||||
# Renders the latest camera capture as Unicode half-blocks in the TUI
|
||||
# (MESHTASTIC_UI_TUI_CAMERA=1). Terminal-agnostic — no kitty / sixel
|
||||
# dependency. Pure Python, tiny.
|
||||
"rich-pixels>=3.0",
|
||||
]
|
||||
ui-min = ["opencv-python-headless>=4.9", "numpy>=1.26"]
|
||||
|
||||
[project.scripts]
|
||||
meshtastic-mcp = "meshtastic_mcp.__main__:main"
|
||||
# Live TUI wrapping run-tests.sh — shells out to the same script the plain
|
||||
# CLI uses, tails pytest-reportlog for per-test state, and polls the device
|
||||
# list at startup + post-run (port lock forces it to stay idle during the run).
|
||||
meshtastic-mcp-test-tui = "meshtastic_mcp.cli.test_tui:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/meshtastic_mcp"]
|
||||
@@ -1,274 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# mcp-server hardware test runner.
|
||||
#
|
||||
# Auto-detects connected Meshtastic devices, maps each to its PlatformIO env
|
||||
# via the same role table the pytest fixtures use, exports the right
|
||||
# MESHTASTIC_MCP_ENV_* env vars, and invokes pytest.
|
||||
#
|
||||
# Usage:
|
||||
# ./run-tests.sh # full suite, default pytest args
|
||||
# ./run-tests.sh tests/mesh # subset (any pytest args pass through)
|
||||
# ./run-tests.sh --force-bake # override one default with another
|
||||
# MESHTASTIC_MCP_ENV_NRF52=foo ./run-tests.sh # override env per role
|
||||
# MESHTASTIC_MCP_SEED=ci-run-42 ./run-tests.sh # override PSK seed
|
||||
#
|
||||
# If zero supported devices are detected, only the unit tier runs.
|
||||
#
|
||||
# Also restores `userPrefs.jsonc` from the session-backup sidecar if a prior
|
||||
# run exited abnormally (belt to conftest.py's atexit suspenders).
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# cd to the script's directory so relative paths resolve consistently no
|
||||
# matter where the user invoked from.
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
cd "$SCRIPT_DIR"
|
||||
|
||||
VENV_PY="$SCRIPT_DIR/.venv/bin/python"
|
||||
if [[ ! -x $VENV_PY ]]; then
|
||||
echo "error: $VENV_PY not found or not executable." >&2
|
||||
echo " Bootstrap the venv first:" >&2
|
||||
echo " cd $SCRIPT_DIR && python3 -m venv .venv && .venv/bin/pip install -e '.[test]'" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Resolve firmware root the same way conftest.py does (this script sits in
|
||||
# mcp-server/, firmware repo root is one level up).
|
||||
FIRMWARE_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
USERPREFS_PATH="$FIRMWARE_ROOT/userPrefs.jsonc"
|
||||
USERPREFS_SIDECAR="$USERPREFS_PATH.mcp-session-bak"
|
||||
|
||||
# ---------- Pre-flight: recover stale userPrefs.jsonc from prior crash ----
|
||||
# If conftest.py's atexit hook didn't fire (SIGKILL, kernel panic, OS
|
||||
# restart), the sidecar is the ground truth. Self-heal before running so we
|
||||
# don't bake the previous run's dirty state into this run's firmware.
|
||||
if [[ -f $USERPREFS_SIDECAR ]]; then
|
||||
echo "[pre-flight] found $USERPREFS_SIDECAR from a prior abnormal exit;" >&2
|
||||
echo " restoring userPrefs.jsonc before starting." >&2
|
||||
cp "$USERPREFS_SIDECAR" "$USERPREFS_PATH"
|
||||
rm -f "$USERPREFS_SIDECAR"
|
||||
fi
|
||||
|
||||
# If userPrefs.jsonc has uncommitted changes BEFORE the run starts, that's
|
||||
# worth warning about — tests will snapshot this dirty state and restore to
|
||||
# it at the end, which may not be what the operator wants.
|
||||
if command -v git >/dev/null 2>&1; then
|
||||
cd "$FIRMWARE_ROOT"
|
||||
# Capture the git status into a local first — SC2312 flags command
|
||||
# substitution inside `[[ -n ... ]]` because the exit code of `git
|
||||
# status` is masked. A two-step assignment makes the failure path
|
||||
# explicit (non-git, missing file) and keeps the bracket test clean.
|
||||
_git_status_porcelain="$(git status --porcelain userPrefs.jsonc 2>/dev/null || true)"
|
||||
if [[ -n $_git_status_porcelain ]]; then
|
||||
echo "[pre-flight] warning: userPrefs.jsonc has uncommitted changes." >&2
|
||||
echo " Tests will snapshot THIS state and restore to it" >&2
|
||||
echo " at teardown. If that's not intended, run:" >&2
|
||||
echo " git checkout userPrefs.jsonc" >&2
|
||||
echo " and re-invoke." >&2
|
||||
fi
|
||||
cd "$SCRIPT_DIR"
|
||||
fi
|
||||
|
||||
# ---------- Seed default --------------------------------------------------
|
||||
# Per-machine default so repeated runs from the same operator land on the
|
||||
# same PSK (makes --assume-baked valid across invocations). Operator can
|
||||
# override with an explicit env var if they want isolation (e.g. CI).
|
||||
if [[ -z ${MESHTASTIC_MCP_SEED-} ]]; then
|
||||
WHO="$(whoami 2>/dev/null || echo anon)"
|
||||
HOST="$(hostname -s 2>/dev/null || echo host)"
|
||||
export MESHTASTIC_MCP_SEED="mcp-${WHO}-${HOST}"
|
||||
fi
|
||||
|
||||
# ---------- Flash progress log --------------------------------------------
|
||||
# pio.py / hw_tools.py tee subprocess output (pio run -t upload, esptool,
|
||||
# nrfutil, picotool) to this file line-by-line as it arrives when this env
|
||||
# var is set. The TUI tails it so the operator sees live flash progress
|
||||
# instead of 3 minutes of silence during `test_00_bake.py`. Plain CLI users
|
||||
# also benefit — the log is a post-run diagnostic even without the TUI.
|
||||
# Truncate at session start so each run gets a clean log.
|
||||
export MESHTASTIC_MCP_FLASH_LOG="$SCRIPT_DIR/tests/flash.log"
|
||||
: >"$MESHTASTIC_MCP_FLASH_LOG"
|
||||
|
||||
# ---------- Detect connected hardware -------------------------------------
|
||||
# In-process call to the same Python API the test fixtures use, so the
|
||||
# script never drifts from what pytest sees. Returns a JSON object
|
||||
# {role: port, ...}.
|
||||
ROLES_JSON="$(
|
||||
"$VENV_PY" - <<'PY'
|
||||
import json
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, "src")
|
||||
from meshtastic_mcp import devices
|
||||
|
||||
# Role → canonical VID map. Kept in sync with
|
||||
# `tests/conftest.py::hub_profile` defaults; if that changes, this must too.
|
||||
ROLE_BY_VID = {
|
||||
0x239A: "nrf52", # Adafruit / RAK nRF52 native USB (app + DFU)
|
||||
0x303A: "esp32s3", # Espressif native USB (ESP32-S3)
|
||||
0x10C4: "esp32s3", # CP2102 USB-UART (common on Heltec/LilyGO ESP32 boards)
|
||||
}
|
||||
|
||||
out: dict[str, str] = {}
|
||||
for dev in devices.list_devices(include_unknown=True):
|
||||
vid_raw = dev.get("vid") or ""
|
||||
try:
|
||||
if isinstance(vid_raw, str) and vid_raw.startswith("0x"):
|
||||
vid = int(vid_raw, 16)
|
||||
else:
|
||||
vid = int(vid_raw)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
role = ROLE_BY_VID.get(vid)
|
||||
# First port wins per role — matches hub_devices fixture semantics.
|
||||
if role and role not in out:
|
||||
out[role] = dev["port"]
|
||||
|
||||
json.dump(out, sys.stdout)
|
||||
PY
|
||||
)"
|
||||
|
||||
# ---------- Map role → pio env --------------------------------------------
|
||||
# Honor MESHTASTIC_MCP_ENV_<ROLE> operator overrides; fall back to the
|
||||
# same defaults hardcoded in tests/conftest.py::_DEFAULT_ROLE_ENVS.
|
||||
resolve_env() {
|
||||
local role="$1"
|
||||
local default="$2"
|
||||
local upper
|
||||
upper="$(echo "$role" | tr '[:lower:]' '[:upper:]')"
|
||||
local var="MESHTASTIC_MCP_ENV_${upper}"
|
||||
eval "local override=\${$var:-}"
|
||||
if [[ -n $override ]]; then
|
||||
echo "$override"
|
||||
else
|
||||
echo "$default"
|
||||
fi
|
||||
}
|
||||
|
||||
NRF52_PORT="$(echo "$ROLES_JSON" | "$VENV_PY" -c 'import json,sys; print(json.loads(sys.stdin.read()).get("nrf52", ""))')"
|
||||
ESP32S3_PORT="$(echo "$ROLES_JSON" | "$VENV_PY" -c 'import json,sys; print(json.loads(sys.stdin.read()).get("esp32s3", ""))')"
|
||||
|
||||
DETECTED=""
|
||||
if [[ -n $NRF52_PORT ]]; then
|
||||
NRF52_ENV="$(resolve_env nrf52 rak4631)"
|
||||
export MESHTASTIC_MCP_ENV_NRF52="$NRF52_ENV"
|
||||
DETECTED="${DETECTED} nrf52 @ ${NRF52_PORT} -> env=${NRF52_ENV}\n"
|
||||
fi
|
||||
if [[ -n $ESP32S3_PORT ]]; then
|
||||
ESP32S3_ENV="$(resolve_env esp32s3 heltec-v3)"
|
||||
export MESHTASTIC_MCP_ENV_ESP32S3="$ESP32S3_ENV"
|
||||
DETECTED="${DETECTED} esp32s3 @ ${ESP32S3_PORT} -> env=${ESP32S3_ENV}\n"
|
||||
fi
|
||||
|
||||
# ---------- Pre-flight summary --------------------------------------------
|
||||
# Surface what pytest is about to do with respect to the bake phase: the
|
||||
# operator should see "will verify + bake if needed" by default, so a
|
||||
# 3-minute flash appearing mid-run isn't a surprise. Detection of the
|
||||
# explicit overrides is best-effort — we just scan $@ for the known flags.
|
||||
_bake_mode="auto (verify + bake if needed)"
|
||||
for _arg in "$@"; do
|
||||
case "$_arg" in
|
||||
--assume-baked) _bake_mode="skip (--assume-baked)" ;;
|
||||
--force-bake) _bake_mode="force (--force-bake)" ;;
|
||||
*) ;; # any other arg: pass-through; bake mode unchanged
|
||||
esac
|
||||
done
|
||||
|
||||
echo "mcp-server test runner"
|
||||
echo " firmware root : $FIRMWARE_ROOT"
|
||||
echo " seed : $MESHTASTIC_MCP_SEED"
|
||||
echo " bake : $_bake_mode"
|
||||
if [[ -n $DETECTED ]]; then
|
||||
echo " detected hub :"
|
||||
printf "%b" "$DETECTED"
|
||||
else
|
||||
echo " detected hub : (none)"
|
||||
fi
|
||||
echo
|
||||
|
||||
# ---------- Invoke pytest -------------------------------------------------
|
||||
# If no devices detected, only the unit tier would produce meaningful
|
||||
# PASS/FAIL — every hardware test would SKIP with "role not present". We
|
||||
# narrow to tests/unit explicitly so the summary reads as "no hardware,
|
||||
# unit suite only" instead of "big skip count looks suspicious".
|
||||
# Keep terminal output condensed (`-q -r fE`) so skip-heavy runs do not print
|
||||
# each skipped test in full; skip counts still appear in pytest's summary.
|
||||
if [[ -z $DETECTED && $# -eq 0 ]]; then
|
||||
echo "[pre-flight] no supported devices detected; running unit tier only."
|
||||
echo
|
||||
exec "$VENV_PY" -m pytest tests/unit -q -r fE --report-log=tests/reportlog.jsonl
|
||||
fi
|
||||
|
||||
# Default pytest args when the user passed none. Power users can invoke
|
||||
# `./run-tests.sh tests/mesh -v --tb=long` and skip all of these defaults.
|
||||
#
|
||||
# NOTE: `--assume-baked` is DELIBERATELY omitted here. `tests/test_00_bake.py`
|
||||
# has an internal skip-if-already-baked check (`_bake_role`: query device_info,
|
||||
# compare region + primary_channel to the session profile, skip on match).
|
||||
# So the fast path is ~8-10 s of verification overhead when the devices are
|
||||
# already baked — negligible next to the 2-6 min suite runtime. Letting
|
||||
# test_00_bake.py run means a fresh device, a re-seeded session, or a post-
|
||||
# factory-reset device gets flashed automatically instead of silently
|
||||
# skipping half the hardware tests with "not baked with session profile"
|
||||
# errors. Power users who know their hardware is current and want to shave
|
||||
# those seconds can pass `--assume-baked` explicitly.
|
||||
# Defaults also use condensed reporting (`-q -r fE`) to avoid listing every
|
||||
# skipped test verbatim while still surfacing failures/errors and summary data.
|
||||
if [[ $# -eq 0 ]]; then
|
||||
set -- tests/ \
|
||||
--html=tests/report.html --self-contained-html \
|
||||
--junitxml=tests/junit.xml \
|
||||
-q -r fE --tb=short
|
||||
fi
|
||||
|
||||
# UI tier requires opencv-python-headless (and ideally easyocr). If it's
|
||||
# not installed, auto-deselect tests/ui so operators without the [ui]
|
||||
# extra still get a green run. Printed in yellow; silent when cv2 is
|
||||
# present.
|
||||
_cv2_ok=0
|
||||
if "$VENV_PY" -c "import cv2" >/dev/null 2>&1; then
|
||||
_cv2_ok=1
|
||||
fi
|
||||
_running_ui=0
|
||||
for _arg in "$@"; do
|
||||
case "$_arg" in
|
||||
*tests/ui* | tests/) _running_ui=1 ;;
|
||||
*) ;;
|
||||
esac
|
||||
done
|
||||
if [[ $_running_ui -eq 1 && $_cv2_ok -eq 0 ]]; then
|
||||
printf '\033[33m[pre-flight] tests/ui tier detected, but opencv-python-headless is not installed — deselecting.\033[0m\n'
|
||||
printf ' install with: .venv/bin/pip install -e "mcp-server/.[ui]"\n'
|
||||
echo
|
||||
set -- "$@" --ignore=tests/ui
|
||||
fi
|
||||
|
||||
# Recovery tier needs `uhubctl` on PATH — it power-cycles devices via USB
|
||||
# hub PPPS. The tier's conftest already skips cleanly, so this is just a
|
||||
# friendly heads-up before the skip happens. `baked_single`'s auto-
|
||||
# recovery hook also benefits from having uhubctl available across the
|
||||
# whole suite.
|
||||
if ! command -v uhubctl >/dev/null 2>&1; then
|
||||
printf "\033[33m[pre-flight] uhubctl not found on PATH — recovery tier will skip, and\n"
|
||||
printf " wedged-device auto-recovery is disabled.\033[0m\n"
|
||||
printf " install with: brew install uhubctl (macOS) or apt install uhubctl (Debian/Ubuntu).\n"
|
||||
echo
|
||||
fi
|
||||
|
||||
# Always emit `tests/reportlog.jsonl` (unless the operator explicitly passed
|
||||
# their own `--report-log=...`). Consumers — notably the
|
||||
# `meshtastic-mcp-test-tui` TUI — tail the reportlog for live per-test state.
|
||||
# Appending here means power-user invocations like `./run-tests.sh tests/mesh`
|
||||
# also produce it, not just the all-defaults invocation.
|
||||
_has_report_log=0
|
||||
for _arg in "$@"; do
|
||||
case "$_arg" in
|
||||
--report-log | --report-log=*) _has_report_log=1 ;;
|
||||
*) ;; # any other arg: no-op; loop continues
|
||||
esac
|
||||
done
|
||||
if [[ $_has_report_log -eq 0 ]]; then
|
||||
set -- "$@" --report-log=tests/reportlog.jsonl
|
||||
fi
|
||||
|
||||
exec "$VENV_PY" -m pytest "$@"
|
||||
@@ -1,217 +0,0 @@
|
||||
{
|
||||
"title": "Meshtastic Firmware — Recorder Stream",
|
||||
"description": "Live view of `.mtlog/` streams shipped by `mtlog_to_datadog.py`. Heap, packet volume, log levels, errors. One row per port.",
|
||||
"widgets": [
|
||||
{
|
||||
"definition": {
|
||||
"title": "Free heap (bytes)",
|
||||
"type": "timeseries",
|
||||
"show_legend": true,
|
||||
"requests": [
|
||||
{
|
||||
"queries": [
|
||||
{
|
||||
"name": "free_heap",
|
||||
"data_source": "metrics",
|
||||
"query": "avg:mesh.local.heap_free_bytes{service:meshtastic-firmware} by {port}"
|
||||
}
|
||||
],
|
||||
"response_format": "timeseries",
|
||||
"display_type": "line"
|
||||
}
|
||||
],
|
||||
"yaxis": { "label": "bytes" }
|
||||
}
|
||||
},
|
||||
{
|
||||
"definition": {
|
||||
"title": "Heap slope (bytes/min) — last 1h",
|
||||
"type": "query_value",
|
||||
"precision": 0,
|
||||
"requests": [
|
||||
{
|
||||
"queries": [
|
||||
{
|
||||
"name": "slope",
|
||||
"data_source": "metrics",
|
||||
"query": "derivative(avg:mesh.local.heap_free_bytes{service:meshtastic-firmware})",
|
||||
"aggregator": "avg"
|
||||
}
|
||||
],
|
||||
"response_format": "scalar"
|
||||
}
|
||||
],
|
||||
"conditional_formats": [
|
||||
{ "comparator": "<", "value": -100, "palette": "white_on_red" },
|
||||
{ "comparator": "<", "value": 0, "palette": "white_on_yellow" },
|
||||
{ "comparator": ">=", "value": 0, "palette": "white_on_green" }
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"definition": {
|
||||
"title": "Total heap (bytes)",
|
||||
"type": "timeseries",
|
||||
"requests": [
|
||||
{
|
||||
"queries": [
|
||||
{
|
||||
"name": "total_heap",
|
||||
"data_source": "metrics",
|
||||
"query": "avg:mesh.local.heap_total_bytes{service:meshtastic-firmware} by {port}"
|
||||
}
|
||||
],
|
||||
"response_format": "timeseries",
|
||||
"display_type": "line"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"definition": {
|
||||
"title": "Battery level (%)",
|
||||
"type": "timeseries",
|
||||
"requests": [
|
||||
{
|
||||
"queries": [
|
||||
{
|
||||
"name": "battery",
|
||||
"data_source": "metrics",
|
||||
"query": "avg:mesh.device.battery_level{service:meshtastic-firmware} by {port}"
|
||||
}
|
||||
],
|
||||
"response_format": "timeseries",
|
||||
"display_type": "line"
|
||||
}
|
||||
],
|
||||
"yaxis": { "min": "0", "max": "105" }
|
||||
}
|
||||
},
|
||||
{
|
||||
"definition": {
|
||||
"title": "Air utilization (TX %)",
|
||||
"type": "timeseries",
|
||||
"requests": [
|
||||
{
|
||||
"queries": [
|
||||
{
|
||||
"name": "airutil",
|
||||
"data_source": "metrics",
|
||||
"query": "avg:mesh.device.air_util_tx{service:meshtastic-firmware} by {port}"
|
||||
}
|
||||
],
|
||||
"response_format": "timeseries",
|
||||
"display_type": "line"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"definition": {
|
||||
"title": "Channel utilization (%)",
|
||||
"type": "timeseries",
|
||||
"requests": [
|
||||
{
|
||||
"queries": [
|
||||
{
|
||||
"name": "chutil",
|
||||
"data_source": "metrics",
|
||||
"query": "avg:mesh.device.channel_utilization{service:meshtastic-firmware} by {port}"
|
||||
}
|
||||
],
|
||||
"response_format": "timeseries",
|
||||
"display_type": "line"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"definition": {
|
||||
"title": "Log volume by level",
|
||||
"type": "timeseries",
|
||||
"show_legend": true,
|
||||
"requests": [
|
||||
{
|
||||
"response_format": "timeseries",
|
||||
"display_type": "bars",
|
||||
"queries": [
|
||||
{
|
||||
"name": "log_count",
|
||||
"data_source": "logs",
|
||||
"indexes": ["*"],
|
||||
"compute": { "aggregation": "count" },
|
||||
"search": { "query": "service:meshtastic-firmware" },
|
||||
"group_by": [
|
||||
{
|
||||
"facet": "@level",
|
||||
"limit": 10,
|
||||
"sort": { "order": "desc", "aggregation": "count" }
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"definition": {
|
||||
"title": "Recent ERROR / CRIT firmware logs",
|
||||
"type": "list_stream",
|
||||
"requests": [
|
||||
{
|
||||
"response_format": "event_list",
|
||||
"query": {
|
||||
"data_source": "logs_stream",
|
||||
"query_string": "service:meshtastic-firmware (status:error OR @level:ERROR OR @level:CRIT)",
|
||||
"indexes": [],
|
||||
"sort": { "column": "timestamp", "order": "desc" }
|
||||
},
|
||||
"columns": [
|
||||
{ "field": "timestamp", "width": "auto" },
|
||||
{ "field": "host", "width": "auto" },
|
||||
{ "field": "@port", "width": "auto" },
|
||||
{ "field": "@level", "width": "auto" },
|
||||
{ "field": "@thread", "width": "auto" },
|
||||
{ "field": "message", "width": "stretch" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"definition": {
|
||||
"title": "Recorder marker events",
|
||||
"type": "list_stream",
|
||||
"requests": [
|
||||
{
|
||||
"response_format": "event_list",
|
||||
"query": {
|
||||
"data_source": "logs_stream",
|
||||
"query_string": "service:meshtastic-firmware @level:MARK",
|
||||
"indexes": [],
|
||||
"sort": { "column": "timestamp", "order": "desc" }
|
||||
},
|
||||
"columns": [
|
||||
{ "field": "timestamp", "width": "auto" },
|
||||
{ "field": "host", "width": "auto" },
|
||||
{ "field": "message", "width": "stretch" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"template_variables": [
|
||||
{
|
||||
"name": "port",
|
||||
"prefix": "port",
|
||||
"available_values": [],
|
||||
"default": "*"
|
||||
},
|
||||
{ "name": "host", "prefix": "host", "available_values": [], "default": "*" }
|
||||
],
|
||||
"layout_type": "ordered",
|
||||
"notify_list": [],
|
||||
"reflow_type": "auto"
|
||||
}
|
||||
@@ -1,389 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Forward selected recorder JSONL streams to Datadog.
|
||||
|
||||
Reads `.mtlog/logs.jsonl` and `.mtlog/telemetry.jsonl`, ships logs to the
|
||||
Logs Intake API and telemetry numerics to the Metrics v2 series API.
|
||||
Resumes from `.mtlog/.dd-cursor.json` so a daemon restart doesn't
|
||||
duplicate rows already shipped from the current live files.
|
||||
|
||||
This forwarder does not currently backfill rotated `.jsonl.gz` archives.
|
||||
If the recorder rotates before this process drains the live file, or the
|
||||
forwarder is down across a rotation, those older rows are skipped.
|
||||
|
||||
Usage:
|
||||
DD_API_KEY=... ./scripts/mtlog_to_datadog.py --tail
|
||||
./scripts/mtlog_to_datadog.py --once # catch up + exit
|
||||
./scripts/mtlog_to_datadog.py --since 3600 # backfill last hour from start
|
||||
|
||||
Default `DD_SITE` is `us5.datadoghq.com` — the team's Datadog instance.
|
||||
Override via `DD_SITE=...` env var or `--site` flag for one-offs.
|
||||
|
||||
The forwarder is a separate process by design — a Datadog outage or
|
||||
auth failure must not backpressure the recorder. We exit non-zero on
|
||||
fatal config errors (missing API key) and keep retrying on transient
|
||||
network/HTTP errors.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any, Iterator
|
||||
|
||||
try:
|
||||
import requests
|
||||
except ImportError:
|
||||
print(
|
||||
"requests is required. Install it in the mcp-server venv: "
|
||||
"uv pip install requests",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(2)
|
||||
|
||||
|
||||
_DEFAULT_LOG_DIR = Path(__file__).resolve().parents[1] / ".mtlog"
|
||||
_LOG_INTAKE_TPL = "https://http-intake.logs.{site}/api/v2/logs"
|
||||
_METRICS_TPL = "https://api.{site}/api/v2/series"
|
||||
_LOG_BATCH = 50
|
||||
_METRICS_BATCH = 100
|
||||
_MAX_RETRIES = 5
|
||||
_RETRY_BASE_S = 1.5
|
||||
|
||||
|
||||
# --- streaming JSONL with byte-position cursor -------------------------
|
||||
|
||||
|
||||
class _StreamReader:
|
||||
"""Reads a single rotating JSONL with cursor-based resume.
|
||||
|
||||
This tails only the live `.jsonl` file. The recorder rotates files
|
||||
(live `.jsonl` → `.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`), which means
|
||||
the live file shrinks abruptly. We detect that via inode change OR live
|
||||
size < cursor position, and reset the live-file cursor to 0.
|
||||
"""
|
||||
|
||||
def __init__(self, path: Path, cursor: dict[str, Any]):
|
||||
self.path = path
|
||||
self.cursor = cursor
|
||||
|
||||
def _state(self) -> tuple[int, int]:
|
||||
"""Return (inode, size) for the live file. (0, 0) if missing."""
|
||||
try:
|
||||
st = self.path.stat()
|
||||
return (st.st_ino, st.st_size)
|
||||
except FileNotFoundError:
|
||||
return (0, 0)
|
||||
|
||||
def iter_new_records(self) -> Iterator[dict[str, Any]]:
|
||||
ino, size = self._state()
|
||||
last_ino = self.cursor.get("ino")
|
||||
last_pos = int(self.cursor.get("pos") or 0)
|
||||
if ino == 0:
|
||||
return
|
||||
if last_ino is not None and last_ino != ino:
|
||||
# Rotation happened. Start over.
|
||||
last_pos = 0
|
||||
if last_pos > size:
|
||||
# Live file truncated/shrunk under us — recorder rotated.
|
||||
last_pos = 0
|
||||
try:
|
||||
with self.path.open("r", encoding="utf-8") as fh:
|
||||
fh.seek(last_pos)
|
||||
for line in fh:
|
||||
line = line.rstrip("\n")
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
yield json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
last_pos = fh.tell()
|
||||
except FileNotFoundError:
|
||||
return
|
||||
self.cursor["ino"] = ino
|
||||
self.cursor["pos"] = last_pos
|
||||
|
||||
|
||||
def _load_cursor(path: Path) -> dict[str, Any]:
|
||||
if not path.exists():
|
||||
return {}
|
||||
try:
|
||||
return json.loads(path.read_text())
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return {}
|
||||
|
||||
|
||||
def _save_cursor(path: Path, data: dict[str, Any]) -> None:
|
||||
tmp = path.with_suffix(".json.tmp")
|
||||
tmp.write_text(json.dumps(data, separators=(",", ":")))
|
||||
tmp.replace(path)
|
||||
|
||||
|
||||
# --- Datadog clients ---------------------------------------------------
|
||||
|
||||
|
||||
class _DDSession:
|
||||
"""Pool one HTTPS session, share retry logic."""
|
||||
|
||||
def __init__(self, api_key: str, site: str, hostname: str) -> None:
|
||||
self.api_key = api_key
|
||||
self.site = site
|
||||
self.hostname = hostname
|
||||
self.session = requests.Session()
|
||||
self.session.headers.update(
|
||||
{
|
||||
"DD-API-KEY": api_key,
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
)
|
||||
|
||||
def _post(self, url: str, payload: Any) -> bool:
|
||||
for attempt in range(_MAX_RETRIES):
|
||||
try:
|
||||
resp = self.session.post(url, json=payload, timeout=30)
|
||||
except requests.RequestException as e:
|
||||
_wait_retry(attempt, f"network error: {e}")
|
||||
continue
|
||||
if 200 <= resp.status_code < 300:
|
||||
return True
|
||||
if resp.status_code in (408, 429, 500, 502, 503, 504):
|
||||
_wait_retry(
|
||||
attempt,
|
||||
f"HTTP {resp.status_code} retrying",
|
||||
)
|
||||
continue
|
||||
print(
|
||||
f"datadog refused: {resp.status_code} {resp.text[:200]}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return False
|
||||
return False
|
||||
|
||||
def send_logs(self, records: list[dict[str, Any]]) -> int:
|
||||
if not records:
|
||||
return 0
|
||||
url = _LOG_INTAKE_TPL.format(site=self.site)
|
||||
sent = 0
|
||||
for i in range(0, len(records), _LOG_BATCH):
|
||||
batch = records[i : i + _LOG_BATCH]
|
||||
if self._post(url, batch):
|
||||
sent += len(batch)
|
||||
return sent
|
||||
|
||||
def send_metrics(self, series: list[dict[str, Any]]) -> int:
|
||||
if not series:
|
||||
return 0
|
||||
url = _METRICS_TPL.format(site=self.site)
|
||||
sent = 0
|
||||
for i in range(0, len(series), _METRICS_BATCH):
|
||||
batch = series[i : i + _METRICS_BATCH]
|
||||
if self._post(url, {"series": batch}):
|
||||
sent += len(batch)
|
||||
return sent
|
||||
|
||||
|
||||
def _wait_retry(attempt: int, reason: str) -> None:
|
||||
wait = _RETRY_BASE_S * (2**attempt)
|
||||
print(
|
||||
f" retry {attempt + 1}/{_MAX_RETRIES} in {wait:.1f}s ({reason})",
|
||||
file=sys.stderr,
|
||||
)
|
||||
time.sleep(wait)
|
||||
|
||||
|
||||
# --- record → datadog payload ------------------------------------------
|
||||
|
||||
|
||||
def _log_record_to_dd(rec: dict[str, Any], host: str) -> dict[str, Any]:
|
||||
line = rec.get("line") or ""
|
||||
tags = [
|
||||
f"role:{rec.get('role')}",
|
||||
f"port:{rec.get('port')}",
|
||||
]
|
||||
level = rec.get("level")
|
||||
if level:
|
||||
tags.append(f"level:{level}")
|
||||
tag = rec.get("tag")
|
||||
if tag:
|
||||
tags.append(f"thread:{tag}")
|
||||
return {
|
||||
"ddsource": "meshtastic-firmware",
|
||||
"service": "meshtastic-firmware",
|
||||
"hostname": host,
|
||||
"message": line,
|
||||
"ddtags": ",".join(t for t in tags if t and "None" not in t),
|
||||
"timestamp": int((rec.get("ts") or time.time()) * 1000),
|
||||
"level": level,
|
||||
}
|
||||
|
||||
|
||||
def _telemetry_record_to_metrics(
|
||||
rec: dict[str, Any], host: str
|
||||
) -> list[dict[str, Any]]:
|
||||
fields = rec.get("fields") or {}
|
||||
if not isinstance(fields, dict):
|
||||
return []
|
||||
variant = rec.get("variant") or "unknown"
|
||||
ts = int(rec.get("ts") or time.time())
|
||||
out: list[dict[str, Any]] = []
|
||||
tags = []
|
||||
if rec.get("port"):
|
||||
tags.append(f"port:{rec['port']}")
|
||||
if rec.get("role"):
|
||||
tags.append(f"role:{rec['role']}")
|
||||
if rec.get("from_node"):
|
||||
tags.append(f"from_node:{rec['from_node']}")
|
||||
tags.append(f"variant:{variant}")
|
||||
for field, value in fields.items():
|
||||
if not isinstance(value, (int, float)) or isinstance(value, bool):
|
||||
continue
|
||||
metric = f"mesh.{variant}.{_metric_safe(field)}"
|
||||
out.append(
|
||||
{
|
||||
"metric": metric,
|
||||
"type": 3, # GAUGE
|
||||
"points": [{"timestamp": ts, "value": float(value)}],
|
||||
"tags": tags,
|
||||
"resources": [{"type": "host", "name": host}],
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def _metric_safe(name: str) -> str:
|
||||
# Lowercase, replace non-alnum with underscore for safe metric names.
|
||||
return "".join(c.lower() if c.isalnum() else "_" for c in name)
|
||||
|
||||
|
||||
# --- main loop ---------------------------------------------------------
|
||||
|
||||
|
||||
def run(
|
||||
log_dir: Path,
|
||||
*,
|
||||
once: bool,
|
||||
since_seconds: float | None,
|
||||
poll_interval: float,
|
||||
dd: _DDSession,
|
||||
) -> int:
|
||||
cursor_path = log_dir / ".dd-cursor.json"
|
||||
cursors = _load_cursor(cursor_path)
|
||||
|
||||
# `--since` overrides cursor: rewind to (now-since) timestamp.
|
||||
# We can't seek by timestamp directly (cursor is byte position), so
|
||||
# we just reset cursors to 0 and let the time filter in iter_new
|
||||
# drop older records.
|
||||
cutoff_ts: float | None = None
|
||||
if since_seconds is not None:
|
||||
cursors = {}
|
||||
cutoff_ts = time.time() - since_seconds
|
||||
|
||||
sent_total = {"logs": 0, "telemetry": 0}
|
||||
|
||||
while True:
|
||||
# logs.jsonl → DD logs
|
||||
log_cursor = cursors.setdefault("logs", {})
|
||||
log_batch: list[dict[str, Any]] = []
|
||||
for rec in _StreamReader(log_dir / "logs.jsonl", log_cursor).iter_new_records():
|
||||
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
|
||||
continue
|
||||
log_batch.append(_log_record_to_dd(rec, dd.hostname))
|
||||
if log_batch:
|
||||
n = dd.send_logs(log_batch)
|
||||
sent_total["logs"] += n
|
||||
print(f"logs: sent {n}/{len(log_batch)}")
|
||||
|
||||
# telemetry.jsonl → DD metrics
|
||||
telem_cursor = cursors.setdefault("telemetry", {})
|
||||
metric_series: list[dict[str, Any]] = []
|
||||
for rec in _StreamReader(
|
||||
log_dir / "telemetry.jsonl", telem_cursor
|
||||
).iter_new_records():
|
||||
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
|
||||
continue
|
||||
metric_series.extend(_telemetry_record_to_metrics(rec, dd.hostname))
|
||||
if metric_series:
|
||||
n = dd.send_metrics(metric_series)
|
||||
sent_total["telemetry"] += n
|
||||
print(f"telemetry: sent {n}/{len(metric_series)} metric points")
|
||||
|
||||
_save_cursor(cursor_path, cursors)
|
||||
|
||||
if once:
|
||||
print(f"done. logs={sent_total['logs']} metrics={sent_total['telemetry']}")
|
||||
return 0
|
||||
time.sleep(poll_interval)
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"--log-dir",
|
||||
default=str(_DEFAULT_LOG_DIR),
|
||||
help="Path to .mtlog/ (default: mcp-server/.mtlog)",
|
||||
)
|
||||
mode = parser.add_mutually_exclusive_group()
|
||||
mode.add_argument("--once", action="store_true", help="Catch up then exit")
|
||||
mode.add_argument(
|
||||
"--tail",
|
||||
action="store_true",
|
||||
help="Daemon: poll forever (default)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--since",
|
||||
type=float,
|
||||
default=None,
|
||||
help="Backfill last N seconds. Resets cursor.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--poll-interval",
|
||||
type=float,
|
||||
default=5.0,
|
||||
help="Seconds between tail polls (default 5)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--site",
|
||||
default=os.environ.get("DD_SITE", "us5.datadoghq.com"),
|
||||
help=(
|
||||
"Datadog site. Default is the team's instance (us5.datadoghq.com). "
|
||||
"Override via DD_SITE env var or this flag."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--host",
|
||||
default=socket.gethostname(),
|
||||
help="Hostname tag (default: socket.gethostname())",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
api_key = os.environ.get("DD_API_KEY")
|
||||
if not api_key:
|
||||
print("DD_API_KEY env var required.", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
log_dir = Path(args.log_dir)
|
||||
if not log_dir.exists():
|
||||
print(
|
||||
f"log dir {log_dir} does not exist — start the mcp-server first.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
|
||||
dd = _DDSession(api_key=api_key, site=args.site, hostname=args.host)
|
||||
once = args.once and not args.tail
|
||||
return run(
|
||||
log_dir,
|
||||
once=once,
|
||||
since_seconds=args.since,
|
||||
poll_interval=args.poll_interval,
|
||||
dd=dd,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,3 +0,0 @@
|
||||
"""Meshtastic MCP server — device discovery, PlatformIO tooling, and device admin."""
|
||||
|
||||
__version__ = "0.1.0"
|
||||
@@ -1,11 +0,0 @@
|
||||
"""Entry point for `python -m meshtastic_mcp`."""
|
||||
|
||||
from meshtastic_mcp.server import app
|
||||
|
||||
|
||||
def main() -> None:
|
||||
app.run()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,417 +0,0 @@
|
||||
"""Device administration: owner, config, channels, messaging, admin actions.
|
||||
|
||||
All operations use the same `connect()` context manager so port selection,
|
||||
port-busy detection, and cleanup are handled uniformly.
|
||||
|
||||
Config writes use a dot-path: the first segment names a section (e.g.
|
||||
`"lora"` in LocalConfig or `"mqtt"` in LocalModuleConfig), remaining segments
|
||||
walk protobuf fields. Enum fields accept their string names (`"US"` for
|
||||
`lora.region`) so callers don't need to know the numeric values.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from google.protobuf import descriptor as pb_descriptor
|
||||
from google.protobuf import json_format
|
||||
from meshtastic.protobuf import localonly_pb2
|
||||
|
||||
from .connection import connect
|
||||
|
||||
|
||||
class AdminError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
LOCAL_CONFIG_SECTIONS = {f.name for f in localonly_pb2.LocalConfig.DESCRIPTOR.fields}
|
||||
MODULE_CONFIG_SECTIONS = {
|
||||
f.name for f in localonly_pb2.LocalModuleConfig.DESCRIPTOR.fields
|
||||
}
|
||||
|
||||
|
||||
def _require_confirm(confirm: bool, operation: str) -> None:
|
||||
if not confirm:
|
||||
raise AdminError(f"{operation} is destructive and requires confirm=True.")
|
||||
|
||||
|
||||
def _message_to_dict(msg: Any) -> dict[str, Any]:
|
||||
# `including_default_value_fields` was renamed to
|
||||
# `always_print_fields_with_no_presence` in protobuf 5.26+. Pick whichever
|
||||
# kwarg the installed version accepts so we work against both.
|
||||
kwargs: dict[str, Any] = {"preserving_proto_field_name": True}
|
||||
import inspect
|
||||
|
||||
sig = inspect.signature(json_format.MessageToDict)
|
||||
if "always_print_fields_with_no_presence" in sig.parameters:
|
||||
kwargs["always_print_fields_with_no_presence"] = False
|
||||
elif "including_default_value_fields" in sig.parameters:
|
||||
kwargs["including_default_value_fields"] = False
|
||||
return json_format.MessageToDict(msg, **kwargs)
|
||||
|
||||
|
||||
# ---------- owner ----------------------------------------------------------
|
||||
|
||||
|
||||
def set_owner(
|
||||
long_name: str,
|
||||
short_name: str | None = None,
|
||||
port: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
if short_name is not None and len(short_name) > 4:
|
||||
raise AdminError("short_name must be 4 characters or fewer")
|
||||
with connect(port=port) as iface:
|
||||
iface.localNode.setOwner(long_name=long_name, short_name=short_name)
|
||||
return {
|
||||
"ok": True,
|
||||
"long_name": long_name,
|
||||
"short_name": short_name,
|
||||
}
|
||||
|
||||
|
||||
# ---------- config reads ---------------------------------------------------
|
||||
|
||||
|
||||
def _section_container(node, section: str) -> tuple[Any, str]:
|
||||
"""Return (container_message, parent_name) for a section name.
|
||||
|
||||
Parent is 'localConfig' or 'moduleConfig' so callers know where to call
|
||||
writeConfig() after mutating.
|
||||
"""
|
||||
if section in LOCAL_CONFIG_SECTIONS:
|
||||
return getattr(node.localConfig, section), "localConfig"
|
||||
if section in MODULE_CONFIG_SECTIONS:
|
||||
return getattr(node.moduleConfig, section), "moduleConfig"
|
||||
raise AdminError(
|
||||
f"Unknown config section: {section!r}. "
|
||||
f"Valid sections: {sorted(LOCAL_CONFIG_SECTIONS | MODULE_CONFIG_SECTIONS)}"
|
||||
)
|
||||
|
||||
|
||||
def get_config(section: str | None = None, port: str | None = None) -> dict[str, Any]:
|
||||
"""Read one or all config sections.
|
||||
|
||||
`section` may be any name in LocalConfig (device, lora, position, power,
|
||||
network, display, bluetooth, security) or LocalModuleConfig (mqtt, serial,
|
||||
telemetry, ...). Omit `section` or pass `"all"` for everything.
|
||||
"""
|
||||
with connect(port=port) as iface:
|
||||
node = iface.localNode
|
||||
if section in (None, "all"):
|
||||
lc = _message_to_dict(node.localConfig)
|
||||
mc = _message_to_dict(node.moduleConfig)
|
||||
return {
|
||||
"config": {
|
||||
"localConfig": lc,
|
||||
"moduleConfig": mc,
|
||||
}
|
||||
}
|
||||
container, _parent = _section_container(node, section)
|
||||
return {"config": {section: _message_to_dict(container)}}
|
||||
|
||||
|
||||
# ---------- config writes --------------------------------------------------
|
||||
|
||||
|
||||
def _coerce_enum(field: pb_descriptor.FieldDescriptor, value: Any) -> int:
|
||||
"""Accept an enum value as either its int or its string name."""
|
||||
enum_type = field.enum_type
|
||||
if isinstance(value, bool):
|
||||
raise AdminError(f"{field.name}: expected enum {enum_type.name}, got bool")
|
||||
if isinstance(value, int):
|
||||
if enum_type.values_by_number.get(value) is None:
|
||||
raise AdminError(
|
||||
f"{field.name}: {value} is not a valid {enum_type.name} value"
|
||||
)
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
upper = value.upper()
|
||||
ev = enum_type.values_by_name.get(upper)
|
||||
if ev is None:
|
||||
valid = sorted(enum_type.values_by_name.keys())
|
||||
raise AdminError(
|
||||
f"{field.name}: {value!r} is not a valid {enum_type.name}. "
|
||||
f"Valid: {valid}"
|
||||
)
|
||||
return ev.number
|
||||
raise AdminError(
|
||||
f"{field.name}: expected enum {enum_type.name}, got {type(value).__name__}"
|
||||
)
|
||||
|
||||
|
||||
def _coerce_scalar(field: pb_descriptor.FieldDescriptor, value: Any) -> Any:
|
||||
t = field.type
|
||||
FT = pb_descriptor.FieldDescriptor
|
||||
if t == FT.TYPE_ENUM:
|
||||
return _coerce_enum(field, value)
|
||||
if t == FT.TYPE_BOOL:
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
return value.strip().lower() in ("true", "yes", "1", "on")
|
||||
if isinstance(value, int):
|
||||
return bool(value)
|
||||
if t in (
|
||||
FT.TYPE_INT32,
|
||||
FT.TYPE_INT64,
|
||||
FT.TYPE_UINT32,
|
||||
FT.TYPE_UINT64,
|
||||
FT.TYPE_SINT32,
|
||||
FT.TYPE_SINT64,
|
||||
FT.TYPE_FIXED32,
|
||||
FT.TYPE_FIXED64,
|
||||
):
|
||||
return int(value)
|
||||
if t in (FT.TYPE_FLOAT, FT.TYPE_DOUBLE):
|
||||
return float(value)
|
||||
if t == FT.TYPE_STRING:
|
||||
return str(value)
|
||||
if t == FT.TYPE_BYTES:
|
||||
if isinstance(value, (bytes, bytearray)):
|
||||
return bytes(value)
|
||||
return str(value).encode("utf-8")
|
||||
raise AdminError(
|
||||
f"{field.name}: unsupported field type {t}. Use raw protobuf for this field."
|
||||
)
|
||||
|
||||
|
||||
def _walk_to_field(
|
||||
root_msg: Any, path_segments: list[str]
|
||||
) -> tuple[Any, pb_descriptor.FieldDescriptor]:
|
||||
"""Walk `root_msg` by field names until the leaf; return (parent_msg, leaf_field_descriptor)."""
|
||||
msg = root_msg
|
||||
for i, name in enumerate(path_segments):
|
||||
desc = msg.DESCRIPTOR
|
||||
field = desc.fields_by_name.get(name)
|
||||
if field is None:
|
||||
trail = ".".join(path_segments[:i] or ["<root>"])
|
||||
valid = [f.name for f in desc.fields]
|
||||
raise AdminError(f"No field {name!r} in {trail}. Valid: {valid}")
|
||||
is_last = i == len(path_segments) - 1
|
||||
if is_last:
|
||||
return msg, field
|
||||
if field.type != pb_descriptor.FieldDescriptor.TYPE_MESSAGE:
|
||||
raise AdminError(
|
||||
f"{'.'.join(path_segments[:i+1])} is a scalar; cannot descend into it"
|
||||
)
|
||||
msg = getattr(msg, name)
|
||||
# path_segments was empty
|
||||
raise AdminError("Empty config path")
|
||||
|
||||
|
||||
def set_config(path: str, value: Any, port: str | None = None) -> dict[str, Any]:
|
||||
"""Set a single config field by dot-path and write it to the device.
|
||||
|
||||
Examples:
|
||||
set_config("lora.region", "US")
|
||||
set_config("lora.modem_preset", "LONG_FAST")
|
||||
set_config("device.role", "ROUTER")
|
||||
set_config("mqtt.enabled", True)
|
||||
set_config("mqtt.address", "mqtt.example.com")
|
||||
|
||||
"""
|
||||
segments = [s for s in path.split(".") if s]
|
||||
if not segments:
|
||||
raise AdminError("path cannot be empty")
|
||||
section = segments[0]
|
||||
|
||||
with connect(port=port) as iface:
|
||||
node = iface.localNode
|
||||
container, parent_name = _section_container(node, section)
|
||||
|
||||
# Treat the section as the root; the rest of the path walks into it.
|
||||
leaf_parent, field = _walk_to_field(container, segments[1:] or [])
|
||||
# Use `is_repeated` (modern upb protobuf API) rather than the
|
||||
# deprecated `label == LABEL_REPEATED` check — the C-extension
|
||||
# FieldDescriptor in protobuf >= 5.x doesn't expose `.label` at
|
||||
# all, and `is_repeated` is the supported replacement that works
|
||||
# across both the pure-python and upb backends.
|
||||
if field.is_repeated:
|
||||
raise AdminError(
|
||||
f"{path!r} is a repeated field; v1 only supports scalar sets. "
|
||||
"Use the raw meshtastic CLI for now."
|
||||
)
|
||||
old_raw = getattr(leaf_parent, field.name)
|
||||
coerced = _coerce_scalar(field, value)
|
||||
try:
|
||||
setattr(leaf_parent, field.name, coerced)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise AdminError(f"{path}: {exc}") from exc
|
||||
|
||||
node.writeConfig(section)
|
||||
|
||||
# Stringify enums for the response (so the caller can see the change in
|
||||
# the same vocabulary they used to set it).
|
||||
if field.type == pb_descriptor.FieldDescriptor.TYPE_ENUM:
|
||||
try:
|
||||
old_display = field.enum_type.values_by_number[old_raw].name
|
||||
new_display = field.enum_type.values_by_number[coerced].name
|
||||
except Exception:
|
||||
old_display, new_display = old_raw, coerced
|
||||
else:
|
||||
old_display, new_display = old_raw, coerced
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
"path": path,
|
||||
"section": section,
|
||||
"parent": parent_name,
|
||||
"old_value": old_display,
|
||||
"new_value": new_display,
|
||||
}
|
||||
|
||||
|
||||
# ---------- channels -------------------------------------------------------
|
||||
|
||||
|
||||
def get_channel_url(
|
||||
include_all: bool = False, port: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
with connect(port=port) as iface:
|
||||
url = iface.localNode.getURL(includeAll=include_all)
|
||||
return {"url": url}
|
||||
|
||||
|
||||
def set_channel_url(url: str, port: str | None = None) -> dict[str, Any]:
|
||||
with connect(port=port) as iface:
|
||||
# setURL replaces the channel set from the URL's contents. It does not
|
||||
# return a count; we infer by counting non-DISABLED channels after.
|
||||
iface.localNode.setURL(url)
|
||||
channels = iface.localNode.channels or []
|
||||
active = sum(1 for c in channels if getattr(c, "role", 0) != 0)
|
||||
return {"ok": True, "channels_imported": active}
|
||||
|
||||
|
||||
# ---------- messaging ------------------------------------------------------
|
||||
|
||||
|
||||
def send_text(
|
||||
text: str,
|
||||
to: str | int | None = None,
|
||||
channel_index: int = 0,
|
||||
want_ack: bool = False,
|
||||
port: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
destination = to if to is not None else "^all"
|
||||
with connect(port=port) as iface:
|
||||
packet = iface.sendText(
|
||||
text,
|
||||
destinationId=destination,
|
||||
wantAck=want_ack,
|
||||
channelIndex=channel_index,
|
||||
)
|
||||
packet_id = getattr(packet, "id", None)
|
||||
return {"ok": True, "packet_id": packet_id, "destination": destination}
|
||||
|
||||
|
||||
# ---------- diagnostics ----------------------------------------------------
|
||||
|
||||
|
||||
def set_debug_log_api(enabled: bool, port: str | None = None) -> dict[str, Any]:
|
||||
"""Toggle `config.security.debug_log_api_enabled` on the local node.
|
||||
|
||||
When enabled, firmware emits log lines as protobuf `LogRecord` messages
|
||||
over the StreamAPI instead of raw text. meshtastic-python surfaces them
|
||||
on pubsub topic `meshtastic.log.line`, which flows through the SAME
|
||||
SerialInterface our tests already hold open — no `pio device monitor`
|
||||
needed, no port-contention with admin/info calls.
|
||||
|
||||
Firmware gate: `src/SerialConsole.cpp` (`usingProtobufs &&
|
||||
config.security.debug_log_api_enabled`). Setting persists in NVS; it
|
||||
survives reboot. `factory_reset(full=False)` clears it unless it's
|
||||
re-applied after reset.
|
||||
|
||||
Previously-documented concurrency hazard (emitLogRecord sharing the
|
||||
main packet-emission buffers) has been fixed — see `StreamAPI.h`
|
||||
where the log path now owns dedicated `fromRadioScratchLog` /
|
||||
`txBufLog` buffers, and `StreamAPI::emitTxBuffer` +
|
||||
`StreamAPI::emitLogRecord` both serialize their `stream->write`
|
||||
calls via `streamLock`. Leaving the flag on under traffic is safe.
|
||||
"""
|
||||
with connect(port=port) as iface:
|
||||
sec = iface.localNode.localConfig.security
|
||||
sec.debug_log_api_enabled = bool(enabled)
|
||||
iface.localNode.writeConfig("security")
|
||||
return {"ok": True, "debug_log_api_enabled": bool(enabled)}
|
||||
|
||||
|
||||
# ---------- admin actions --------------------------------------------------
|
||||
|
||||
|
||||
def reboot(
|
||||
port: str | None = None, confirm: bool = False, seconds: int = 10
|
||||
) -> dict[str, Any]:
|
||||
_require_confirm(confirm, "reboot")
|
||||
with connect(port=port) as iface:
|
||||
iface.localNode.reboot(secs=seconds)
|
||||
return {"ok": True, "rebooting_in_s": seconds}
|
||||
|
||||
|
||||
def shutdown(
|
||||
port: str | None = None, confirm: bool = False, seconds: int = 10
|
||||
) -> dict[str, Any]:
|
||||
_require_confirm(confirm, "shutdown")
|
||||
with connect(port=port) as iface:
|
||||
iface.localNode.shutdown(secs=seconds)
|
||||
return {"ok": True, "shutting_down_in_s": seconds}
|
||||
|
||||
|
||||
def send_input_event(
|
||||
event_code: int | str,
|
||||
kb_char: int = 0,
|
||||
touch_x: int = 0,
|
||||
touch_y: int = 0,
|
||||
port: str | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""Inject an InputBroker event (button press / key / gesture) into the UI.
|
||||
|
||||
Wraps `AdminMessage.send_input_event` (handled in firmware at
|
||||
src/modules/AdminModule.cpp::handleSendInputEvent). Local-only — no PKI
|
||||
warmup needed since the admin message is addressed to `my_node_num`.
|
||||
|
||||
`event_code` accepts an int, a case-insensitive name
|
||||
(`"RIGHT"` / `"input_broker_right"`), or an `InputEventCode`. The
|
||||
firmware-side enum lives in src/input/InputBroker.h and is mirrored in
|
||||
`meshtastic_mcp.input_events`.
|
||||
"""
|
||||
from meshtastic.protobuf import admin_pb2 # type: ignore[import-untyped]
|
||||
|
||||
from .input_events import coerce_event_code
|
||||
|
||||
code = coerce_event_code(event_code)
|
||||
if not 0 <= kb_char <= 255:
|
||||
raise ValueError(f"kb_char out of u8 range: {kb_char}")
|
||||
if not 0 <= touch_x <= 65535:
|
||||
raise ValueError(f"touch_x out of u16 range: {touch_x}")
|
||||
if not 0 <= touch_y <= 65535:
|
||||
raise ValueError(f"touch_y out of u16 range: {touch_y}")
|
||||
|
||||
with connect(port=port) as iface:
|
||||
msg = admin_pb2.AdminMessage()
|
||||
msg.send_input_event.event_code = code
|
||||
msg.send_input_event.kb_char = kb_char
|
||||
msg.send_input_event.touch_x = touch_x
|
||||
msg.send_input_event.touch_y = touch_y
|
||||
iface.localNode._sendAdmin(msg)
|
||||
return {"ok": True, "event_code": code, "kb_char": kb_char}
|
||||
|
||||
|
||||
def factory_reset(
|
||||
port: str | None = None, confirm: bool = False, full: bool = False
|
||||
) -> dict[str, Any]:
|
||||
"""Tell the node to factory-reset its config.
|
||||
|
||||
Works around a meshtastic-python 2.7.8 bug: `Node.factoryReset(full=True)`
|
||||
internally does `p.factory_reset_config = True` where the field is
|
||||
int32. protobuf 5.x rejects bool→int assignment as a TypeError. We build
|
||||
the AdminMessage directly with int values (1=non-full, 2=full) and call
|
||||
`_sendAdmin` to sidestep the SDK bug entirely.
|
||||
"""
|
||||
_require_confirm(confirm, "factory_reset")
|
||||
from meshtastic.protobuf import admin_pb2 # type: ignore[import-untyped]
|
||||
|
||||
with connect(port=port) as iface:
|
||||
msg = admin_pb2.AdminMessage()
|
||||
msg.factory_reset_config = 2 if full else 1
|
||||
iface.localNode._sendAdmin(msg)
|
||||
return {"ok": True, "full": full}
|
||||
@@ -1,159 +0,0 @@
|
||||
"""Board / PlatformIO env enumeration.
|
||||
|
||||
Parses `pio project config --json-output` — a nested list of
|
||||
`[section_name, [[key, value], ...]]` pairs — into a dict keyed by env name,
|
||||
extracting the `custom_meshtastic_*` metadata the firmware variants expose.
|
||||
|
||||
The parsed config is cached and invalidated when `platformio.ini`'s mtime
|
||||
changes, so subsequent calls don't pay the 1–2s pio startup cost.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
from typing import Any
|
||||
|
||||
from . import config, pio
|
||||
|
||||
_CACHE_LOCK = threading.Lock()
|
||||
_CACHE: dict[str, Any] = {"mtime": None, "envs": None}
|
||||
|
||||
|
||||
def _parse_bool(value: Any) -> bool:
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
return value.strip().lower() in ("true", "yes", "1", "on")
|
||||
return bool(value)
|
||||
|
||||
|
||||
def _parse_int(value: Any) -> int | None:
|
||||
try:
|
||||
return int(value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _parse_tags(value: Any) -> list[str]:
|
||||
if value is None:
|
||||
return []
|
||||
if isinstance(value, list):
|
||||
return [str(v).strip() for v in value if str(v).strip()]
|
||||
return [t.strip() for t in str(value).replace(",", " ").split() if t.strip()]
|
||||
|
||||
|
||||
def _env_record(env_name: str, items: list[list[Any]]) -> dict[str, Any]:
|
||||
"""Build a normalized dict for one env section."""
|
||||
d = dict(items)
|
||||
return {
|
||||
"env": env_name,
|
||||
"architecture": d.get("custom_meshtastic_architecture"),
|
||||
"hw_model": _parse_int(d.get("custom_meshtastic_hw_model")),
|
||||
"hw_model_slug": d.get("custom_meshtastic_hw_model_slug"),
|
||||
"display_name": d.get("custom_meshtastic_display_name"),
|
||||
"actively_supported": _parse_bool(
|
||||
d.get("custom_meshtastic_actively_supported")
|
||||
),
|
||||
"support_level": _parse_int(d.get("custom_meshtastic_support_level")),
|
||||
"board_level": d.get("board_level"), # "pr", "extra", or None
|
||||
"tags": _parse_tags(d.get("custom_meshtastic_tags")),
|
||||
"images": _parse_tags(d.get("custom_meshtastic_images")),
|
||||
"board": d.get("board"),
|
||||
"upload_speed": _parse_int(d.get("upload_speed")),
|
||||
"upload_protocol": d.get("upload_protocol"),
|
||||
"monitor_speed": _parse_int(d.get("monitor_speed")),
|
||||
"monitor_filters": d.get("monitor_filters") or [],
|
||||
"_raw": d, # Full dict for get_board
|
||||
}
|
||||
|
||||
|
||||
def _load_all() -> dict[str, dict[str, Any]]:
|
||||
"""Parse `pio project config` into `{env_name: record}`."""
|
||||
raw = pio.run_json(["project", "config"], timeout=pio.TIMEOUT_PROJECT_CONFIG)
|
||||
result: dict[str, dict[str, Any]] = {}
|
||||
for section_name, items in raw:
|
||||
if not isinstance(section_name, str) or not section_name.startswith("env:"):
|
||||
continue
|
||||
env_name = section_name.split(":", 1)[1]
|
||||
result[env_name] = _env_record(env_name, items)
|
||||
return result
|
||||
|
||||
|
||||
def _get_cached() -> dict[str, dict[str, Any]]:
|
||||
root = config.firmware_root()
|
||||
platformio_ini = root / "platformio.ini"
|
||||
try:
|
||||
mtime = platformio_ini.stat().st_mtime
|
||||
except FileNotFoundError:
|
||||
mtime = None
|
||||
|
||||
with _CACHE_LOCK:
|
||||
if _CACHE["envs"] is not None and _CACHE["mtime"] == mtime:
|
||||
return _CACHE["envs"]
|
||||
envs = _load_all()
|
||||
_CACHE["envs"] = envs
|
||||
_CACHE["mtime"] = mtime
|
||||
return envs
|
||||
|
||||
|
||||
def invalidate_cache() -> None:
|
||||
with _CACHE_LOCK:
|
||||
_CACHE["envs"] = None
|
||||
_CACHE["mtime"] = None
|
||||
|
||||
|
||||
def _public_record(rec: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Strip the `_raw` field for list outputs."""
|
||||
return {k: v for k, v in rec.items() if not k.startswith("_")}
|
||||
|
||||
|
||||
def list_boards(
|
||||
architecture: str | None = None,
|
||||
actively_supported_only: bool = False,
|
||||
query: str | None = None,
|
||||
board_level: str | None = None, # "release" | "pr" | "extra"
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Enumerate PlatformIO envs with Meshtastic metadata.
|
||||
|
||||
Filters are cumulative (AND). `board_level="release"` means envs with no
|
||||
explicit `board_level` set (the default release targets).
|
||||
"""
|
||||
envs = _get_cached()
|
||||
q = query.lower().strip() if query else None
|
||||
|
||||
out = []
|
||||
for rec in envs.values():
|
||||
if architecture and rec.get("architecture") != architecture:
|
||||
continue
|
||||
if actively_supported_only and not rec.get("actively_supported"):
|
||||
continue
|
||||
if board_level is not None:
|
||||
rec_level = rec.get("board_level")
|
||||
if board_level == "release":
|
||||
if rec_level not in (None, ""):
|
||||
continue
|
||||
elif rec_level != board_level:
|
||||
continue
|
||||
if q:
|
||||
display = (rec.get("display_name") or "").lower()
|
||||
env_name = rec.get("env", "").lower()
|
||||
slug = (rec.get("hw_model_slug") or "").lower()
|
||||
if q not in display and q not in env_name and q not in slug:
|
||||
continue
|
||||
out.append(_public_record(rec))
|
||||
|
||||
out.sort(key=lambda r: (r.get("architecture") or "", r.get("env")))
|
||||
return out
|
||||
|
||||
|
||||
def get_board(env: str) -> dict[str, Any]:
|
||||
"""Full metadata for one env, including the raw pio config dict."""
|
||||
envs = _get_cached()
|
||||
rec = envs.get(env)
|
||||
if rec is None:
|
||||
raise KeyError(
|
||||
f"Unknown env: {env!r}. Use list_boards() to see available envs."
|
||||
)
|
||||
public = _public_record(rec)
|
||||
public["raw_config"] = rec["_raw"]
|
||||
return public
|
||||
@@ -1,286 +0,0 @@
|
||||
"""Cross-platform USB-webcam capture for UI tests + the `capture_screen` tool.
|
||||
|
||||
Backends:
|
||||
- `opencv` — cv2.VideoCapture (AVFoundation on macOS, V4L2 on Linux).
|
||||
- `ffmpeg` — subprocess shelling out to the system `ffmpeg` binary. Slower
|
||||
per frame, but zero Python deps beyond stdlib.
|
||||
- `null` — no-op stub returning a 1×1 black PNG. Used when no camera is
|
||||
configured; keeps code paths alive without forcing every operator to
|
||||
hook up hardware.
|
||||
|
||||
Environment variables (read at `get_camera()` call time):
|
||||
- `MESHTASTIC_UI_CAMERA_BACKEND` — one of `opencv` / `ffmpeg` / `null` /
|
||||
`auto` (default). `auto` picks opencv if `cv2` imports, else ffmpeg if
|
||||
`ffmpeg --version` resolves, else null.
|
||||
- `MESHTASTIC_UI_CAMERA_DEVICE` — generic default (index or path).
|
||||
- `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` — per-role override, e.g.
|
||||
`MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0` for the OLED-bearing heltec-v3.
|
||||
Role suffix is uppercased before lookup.
|
||||
|
||||
Dependencies land in the optional `[ui]` extra; imports are lazy so clients
|
||||
without `opencv-python-headless` installed can still import this module.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import warnings
|
||||
from pathlib import Path
|
||||
from typing import Protocol
|
||||
|
||||
|
||||
class CameraError(RuntimeError):
|
||||
"""Raised when a camera backend fails to initialize or capture."""
|
||||
|
||||
|
||||
class CameraBackend(Protocol):
|
||||
name: str
|
||||
|
||||
def capture(self) -> bytes:
|
||||
"""Return one PNG-encoded frame."""
|
||||
...
|
||||
|
||||
def close(self) -> None: ...
|
||||
|
||||
|
||||
# ---------- OpenCV backend -------------------------------------------------
|
||||
|
||||
|
||||
class OpenCVBackend:
|
||||
name = "opencv"
|
||||
|
||||
def __init__(self, device: int | str, warmup_frames: int = 5) -> None:
|
||||
try:
|
||||
import cv2 # type: ignore[import-untyped] # noqa: PLC0415
|
||||
except ImportError as exc:
|
||||
raise CameraError(
|
||||
"opencv backend requested but `cv2` is not installed. "
|
||||
"Install the mcp-server [ui] extra: pip install -e '.[ui]'"
|
||||
) from exc
|
||||
|
||||
self._cv2 = cv2
|
||||
device_arg: int | str
|
||||
if isinstance(device, str) and device.isdigit():
|
||||
device_arg = int(device)
|
||||
else:
|
||||
device_arg = device
|
||||
self._cap = cv2.VideoCapture(device_arg)
|
||||
if not self._cap.isOpened():
|
||||
raise CameraError(
|
||||
f"cv2.VideoCapture({device_arg!r}) failed to open. "
|
||||
"On macOS check TCC Camera permission; on Linux check /dev/video* and v4l2 access."
|
||||
)
|
||||
|
||||
# Drop the first few frames — auto-exposure + white-balance settle.
|
||||
for _ in range(warmup_frames):
|
||||
self._cap.read()
|
||||
# Detect a stuck black-frame camera early rather than silently
|
||||
# producing all-black captures.
|
||||
ok, frame = self._cap.read()
|
||||
if not ok or frame is None:
|
||||
self._cap.release()
|
||||
raise CameraError(f"camera {device_arg!r} opened but returned no frames")
|
||||
|
||||
def capture(self) -> bytes:
|
||||
cv2 = self._cv2
|
||||
ok, frame = self._cap.read()
|
||||
if not ok or frame is None:
|
||||
raise CameraError("cv2.VideoCapture.read() returned no frame")
|
||||
success, buf = cv2.imencode(".png", frame)
|
||||
if not success:
|
||||
raise CameraError("cv2.imencode('.png', ...) failed")
|
||||
return bytes(buf)
|
||||
|
||||
def close(self) -> None:
|
||||
try:
|
||||
self._cap.release()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
|
||||
# ---------- ffmpeg subprocess backend --------------------------------------
|
||||
|
||||
|
||||
class FfmpegBackend:
|
||||
name = "ffmpeg"
|
||||
|
||||
def __init__(self, device: int | str) -> None:
|
||||
if shutil.which("ffmpeg") is None:
|
||||
raise CameraError("ffmpeg backend requested but `ffmpeg` is not on PATH")
|
||||
|
||||
self._device = str(device)
|
||||
# Platform-specific -f flag:
|
||||
# macOS → avfoundation (index like "0")
|
||||
# Linux → v4l2 (device like "/dev/video0" or "0")
|
||||
if sys.platform == "darwin":
|
||||
self._input_format = "avfoundation"
|
||||
self._input_spec = self._device # bare index for avfoundation
|
||||
else:
|
||||
self._input_format = "v4l2"
|
||||
self._input_spec = (
|
||||
self._device
|
||||
if self._device.startswith("/dev/")
|
||||
else f"/dev/video{self._device}"
|
||||
)
|
||||
|
||||
def capture(self) -> bytes:
|
||||
cmd = [
|
||||
"ffmpeg",
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-f",
|
||||
self._input_format,
|
||||
"-i",
|
||||
self._input_spec,
|
||||
"-frames:v",
|
||||
"1",
|
||||
"-f",
|
||||
"image2pipe",
|
||||
"-vcodec",
|
||||
"png",
|
||||
"-",
|
||||
]
|
||||
try:
|
||||
out = subprocess.run(
|
||||
cmd, capture_output=True, check=True, timeout=15 # noqa: S603
|
||||
)
|
||||
except subprocess.CalledProcessError as exc:
|
||||
raise CameraError(
|
||||
f"ffmpeg capture failed (rc={exc.returncode}): {exc.stderr.decode(errors='replace')[:200]}"
|
||||
) from exc
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
raise CameraError("ffmpeg capture timed out after 15s") from exc
|
||||
return out.stdout
|
||||
|
||||
def close(self) -> None:
|
||||
pass # stateless — each capture spawns a new process
|
||||
|
||||
|
||||
# ---------- Null backend ---------------------------------------------------
|
||||
|
||||
|
||||
# A tiny valid 1×1 transparent PNG so callers always get a decodable image.
|
||||
_BLACK_1X1_PNG = bytes.fromhex(
|
||||
"89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c489"
|
||||
"0000000d49444154789c6300010000000500010d0a2db40000000049454e44ae426082"
|
||||
)
|
||||
|
||||
|
||||
class NullBackend:
|
||||
name = "null"
|
||||
|
||||
def capture(self) -> bytes:
|
||||
return _BLACK_1X1_PNG
|
||||
|
||||
def close(self) -> None:
|
||||
pass
|
||||
|
||||
|
||||
# ---------- Factory --------------------------------------------------------
|
||||
|
||||
|
||||
def _resolve_device(role: str | None) -> str | None:
|
||||
if role:
|
||||
specific = os.environ.get(f"MESHTASTIC_UI_CAMERA_DEVICE_{role.upper()}")
|
||||
if specific:
|
||||
return specific
|
||||
return os.environ.get("MESHTASTIC_UI_CAMERA_DEVICE")
|
||||
|
||||
|
||||
def get_camera(role: str | None = None) -> CameraBackend:
|
||||
"""Return a CameraBackend for the given device role (e.g. `"esp32s3"`).
|
||||
|
||||
Falls back to `NullBackend` if no camera is configured or the selected
|
||||
backend fails to init — tests should treat captures as best-effort
|
||||
evidence, not a blocker.
|
||||
"""
|
||||
backend = os.environ.get("MESHTASTIC_UI_CAMERA_BACKEND", "auto").lower()
|
||||
device = _resolve_device(role)
|
||||
|
||||
if backend in ("null", "none") or device is None:
|
||||
return NullBackend()
|
||||
|
||||
if backend == "auto":
|
||||
# Prefer opencv if importable; fall back to ffmpeg; else null.
|
||||
try:
|
||||
import cv2 # type: ignore[import-untyped] # noqa: F401,PLC0415
|
||||
|
||||
backend = "opencv"
|
||||
except ImportError:
|
||||
backend = "ffmpeg" if shutil.which("ffmpeg") else "null"
|
||||
|
||||
if backend == "opencv":
|
||||
try:
|
||||
return OpenCVBackend(device)
|
||||
except CameraError as exc:
|
||||
warnings.warn(
|
||||
f"camera backend {backend!r} failed to initialize for device "
|
||||
f"{device!r}: {exc}; falling back to null backend",
|
||||
RuntimeWarning,
|
||||
stacklevel=2,
|
||||
)
|
||||
return NullBackend()
|
||||
if backend == "ffmpeg":
|
||||
try:
|
||||
return FfmpegBackend(device)
|
||||
except CameraError as exc:
|
||||
warnings.warn(
|
||||
f"camera backend {backend!r} failed to initialize for device "
|
||||
f"{device!r}: {exc}; falling back to null backend",
|
||||
RuntimeWarning,
|
||||
stacklevel=2,
|
||||
)
|
||||
return NullBackend()
|
||||
if backend == "null":
|
||||
return NullBackend()
|
||||
|
||||
raise CameraError(f"unknown MESHTASTIC_UI_CAMERA_BACKEND: {backend!r}")
|
||||
|
||||
|
||||
def save_capture(png_bytes: bytes, path: Path) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_bytes(png_bytes)
|
||||
|
||||
|
||||
def capture_to_file(role: str | None, path: Path) -> dict[str, object]:
|
||||
"""One-shot: open camera, capture, write PNG, close. Returns metadata."""
|
||||
started = time.monotonic()
|
||||
cam = get_camera(role)
|
||||
try:
|
||||
data = cam.capture()
|
||||
finally:
|
||||
cam.close()
|
||||
save_capture(data, path)
|
||||
return {
|
||||
"backend": cam.name,
|
||||
"path": str(path),
|
||||
"bytes": len(data),
|
||||
"elapsed_s": round(time.monotonic() - started, 3),
|
||||
}
|
||||
|
||||
|
||||
def _is_png(data: bytes) -> bool:
|
||||
return data.startswith(b"\x89PNG\r\n\x1a\n")
|
||||
|
||||
|
||||
# Exposed so callers can sanity-check a capture without a full PIL import.
|
||||
__all__ = [
|
||||
"CameraBackend",
|
||||
"CameraError",
|
||||
"FfmpegBackend",
|
||||
"NullBackend",
|
||||
"OpenCVBackend",
|
||||
"capture_to_file",
|
||||
"get_camera",
|
||||
"save_capture",
|
||||
]
|
||||
|
||||
# Keep `io` import used (pyflakes is picky) via a small guard used at import
|
||||
# time to normalize stdin/stdout if a subclass ever needs it.
|
||||
_ = io.BytesIO # noqa: SLF001
|
||||
@@ -1,6 +0,0 @@
|
||||
"""Command-line entry points that sit alongside the MCP server.
|
||||
|
||||
Modules here are loaded on-demand by `[project.scripts]` entries in
|
||||
`pyproject.toml`. They are NOT imported by `meshtastic_mcp.server` or the
|
||||
admin/info tool surface — the MCP server stays pure stdio JSON-RPC.
|
||||
"""
|
||||
@@ -1,73 +0,0 @@
|
||||
"""Flash progress log tailer for ``meshtastic-mcp-test-tui``.
|
||||
|
||||
``pio.py`` / ``hw_tools.py`` tee subprocess output (``pio run -t upload``,
|
||||
``esptool erase_flash``, ``nrfutil dfu``, etc.) to ``tests/flash.log``
|
||||
line-by-line as it arrives — controlled by the ``MESHTASTIC_MCP_FLASH_LOG``
|
||||
env var that ``run-tests.sh`` sets. The TUI tails that file so the operator
|
||||
sees live flash progress in the pytest pane instead of 3 minutes of silence
|
||||
during ``test_00_bake``.
|
||||
|
||||
Separate from ``_fwlog.py`` because that one parses JSONL, this one
|
||||
streams plain text lines. Same daemon-thread + EOF-backoff structure.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pathlib
|
||||
import threading
|
||||
import time
|
||||
from typing import Callable
|
||||
|
||||
|
||||
class FlashLogTailer(threading.Thread):
|
||||
"""Tail a plain-text log file, publish each stripped line via ``post``.
|
||||
|
||||
``post`` is invoked with a single ``str`` for every new line. Lines are
|
||||
stripped of trailing newlines; empty lines after stripping are dropped.
|
||||
|
||||
The file may not exist yet when this thread starts — it's truncated by
|
||||
``run-tests.sh`` at session start, but if the tailer races the shell,
|
||||
we tolerate FileNotFoundError for up to ``wait_s`` seconds.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
path: pathlib.Path,
|
||||
post: Callable[[str], None],
|
||||
stop: threading.Event,
|
||||
*,
|
||||
wait_s: float = 30.0,
|
||||
) -> None:
|
||||
super().__init__(daemon=True, name="flashlog-tail")
|
||||
self._path = path
|
||||
self._post = post
|
||||
self._stop = stop
|
||||
self._wait_s = wait_s
|
||||
|
||||
def run(self) -> None:
|
||||
deadline = time.monotonic() + self._wait_s
|
||||
while not self._path.is_file():
|
||||
if self._stop.is_set() or time.monotonic() > deadline:
|
||||
return
|
||||
time.sleep(0.1)
|
||||
try:
|
||||
fh = self._path.open("r", encoding="utf-8", errors="replace")
|
||||
except OSError:
|
||||
return
|
||||
try:
|
||||
while not self._stop.is_set():
|
||||
line = fh.readline()
|
||||
if not line:
|
||||
time.sleep(0.05)
|
||||
continue
|
||||
line = line.rstrip("\r\n")
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
self._post(line)
|
||||
except Exception:
|
||||
# A post failure (e.g. closed app) is terminal for this
|
||||
# thread but we still want to close the file handle.
|
||||
return
|
||||
finally:
|
||||
fh.close()
|
||||
@@ -1,96 +0,0 @@
|
||||
"""Firmware log tail worker for ``meshtastic-mcp-test-tui``.
|
||||
|
||||
Complements v1's reportlog-tail worker. ``tests/conftest.py`` owns a
|
||||
session-scoped autouse fixture (``_firmware_log_stream``) that mirrors
|
||||
every ``meshtastic.log.line`` pubsub event to ``tests/fwlog.jsonl`` —
|
||||
one JSON object per line:
|
||||
|
||||
{"ts": 1729100000.123, "port": "/dev/cu.usbmodem1101", "line": "..."}
|
||||
|
||||
The TUI tails that file from a worker thread; each new line becomes a
|
||||
:class:`FirmwareLogLine` message posted to the App. Same pattern as the
|
||||
reportlog tail worker — truncate on launch, tolerate missing file for
|
||||
30 s, back off at EOF.
|
||||
|
||||
Kept in its own module so the (large) ``test_tui.py`` stays focused on
|
||||
the Textual App shell.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import pathlib
|
||||
import threading
|
||||
import time
|
||||
from typing import Any, Callable
|
||||
|
||||
|
||||
class FirmwareLogTailer(threading.Thread):
|
||||
"""Tail ``tests/fwlog.jsonl``, publish parsed records via ``post``.
|
||||
|
||||
``post`` is the App's ``post_message`` (or any callable that accepts a
|
||||
single payload arg). We pass parsed dicts rather than constructing
|
||||
Textual Message objects here — keeps this module free of the
|
||||
textual dependency so it's unit-testable in a bare venv.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
path:
|
||||
Path to ``tests/fwlog.jsonl``. The file may not exist yet at
|
||||
startup — pytest only creates it once the session fixture runs.
|
||||
post:
|
||||
Callable invoked with a dict ``{"ts", "port", "line"}`` for every
|
||||
new line parsed from the file.
|
||||
stop:
|
||||
An event the App sets to signal shutdown.
|
||||
wait_s:
|
||||
How long to poll for the file's creation before giving up. Default
|
||||
30 s; pytest collection on a cold cache can be slow.
|
||||
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
path: pathlib.Path,
|
||||
post: Callable[[dict[str, Any]], None],
|
||||
stop: threading.Event,
|
||||
*,
|
||||
wait_s: float = 30.0,
|
||||
) -> None:
|
||||
super().__init__(daemon=True, name="fwlog-tail")
|
||||
self._path = path
|
||||
self._post = post
|
||||
self._stop = stop
|
||||
self._wait_s = wait_s
|
||||
|
||||
def run(self) -> None:
|
||||
deadline = time.monotonic() + self._wait_s
|
||||
while not self._path.is_file():
|
||||
if self._stop.is_set() or time.monotonic() > deadline:
|
||||
return
|
||||
time.sleep(0.1)
|
||||
try:
|
||||
fh = self._path.open("r", encoding="utf-8")
|
||||
except OSError:
|
||||
return
|
||||
try:
|
||||
while not self._stop.is_set():
|
||||
line = fh.readline()
|
||||
if not line:
|
||||
time.sleep(0.05)
|
||||
continue
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
record = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
# Defensive: require the three fields we rely on.
|
||||
if not isinstance(record, dict):
|
||||
continue
|
||||
if "line" not in record:
|
||||
continue
|
||||
self._post(record)
|
||||
finally:
|
||||
fh.close()
|
||||
@@ -1,127 +0,0 @@
|
||||
"""Cross-run history for ``meshtastic-mcp-test-tui``.
|
||||
|
||||
Persists one JSON object per pytest run to
|
||||
``mcp-server/tests/.history/runs.jsonl``. The TUI reads the last N
|
||||
entries on launch to render a duration sparkline in the header — a
|
||||
quick read on whether the suite is slowing down over time.
|
||||
|
||||
Schema (keep small; the file can grow for months):
|
||||
|
||||
{"run": 42, "ts": 1729100000.0, "duration_s": 387.2,
|
||||
"passed": 52, "failed": 0, "skipped": 23, "exit_code": 0,
|
||||
"seed": "mcp-user-host"}
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import pathlib
|
||||
import time
|
||||
from dataclasses import asdict, dataclass
|
||||
from typing import Iterable
|
||||
|
||||
# Sparkline glyphs, low → high. 8 levels is the Unicode convention.
|
||||
_SPARK_BLOCKS = "▁▂▃▄▅▆▇█"
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunRecord:
|
||||
run: int
|
||||
ts: float
|
||||
duration_s: float
|
||||
passed: int
|
||||
failed: int
|
||||
skipped: int
|
||||
exit_code: int
|
||||
seed: str
|
||||
|
||||
|
||||
class HistoryStore:
|
||||
"""Append-only JSONL store with bounded read.
|
||||
|
||||
Writes are fsynced after each append (the file is tiny; fsync cost
|
||||
is negligible and protects against truncation on a crash).
|
||||
"""
|
||||
|
||||
def __init__(self, path: pathlib.Path, *, keep_last: int = 50) -> None:
|
||||
self._path = path
|
||||
self._keep_last = keep_last
|
||||
|
||||
def append(self, record: RunRecord) -> None:
|
||||
try:
|
||||
self._path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with self._path.open("a", encoding="utf-8") as fh:
|
||||
fh.write(json.dumps(asdict(record)) + "\n")
|
||||
fh.flush()
|
||||
except Exception:
|
||||
# Non-fatal: history is cosmetic.
|
||||
pass
|
||||
|
||||
def read_recent(self) -> list[RunRecord]:
|
||||
"""Return the last ``keep_last`` records in chronological order."""
|
||||
if not self._path.is_file():
|
||||
return []
|
||||
try:
|
||||
lines = self._path.read_text(encoding="utf-8").splitlines()
|
||||
except OSError:
|
||||
return []
|
||||
out: list[RunRecord] = []
|
||||
# Parse tail-first so we don't waste work on a huge history.
|
||||
for line in lines[-self._keep_last :]:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
raw = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
try:
|
||||
out.append(RunRecord(**raw))
|
||||
except TypeError:
|
||||
# Schema drift; skip the record rather than crash.
|
||||
continue
|
||||
return out
|
||||
|
||||
def record_run(
|
||||
self,
|
||||
*,
|
||||
run: int,
|
||||
duration_s: float,
|
||||
passed: int,
|
||||
failed: int,
|
||||
skipped: int,
|
||||
exit_code: int,
|
||||
seed: str,
|
||||
) -> RunRecord:
|
||||
rec = RunRecord(
|
||||
run=run,
|
||||
ts=time.time(),
|
||||
duration_s=float(duration_s),
|
||||
passed=int(passed),
|
||||
failed=int(failed),
|
||||
skipped=int(skipped),
|
||||
exit_code=int(exit_code),
|
||||
seed=seed,
|
||||
)
|
||||
self.append(rec)
|
||||
return rec
|
||||
|
||||
|
||||
def sparkline(values: Iterable[float], *, width: int = 20) -> str:
|
||||
"""Render a Unicode block-character sparkline from the last ``width`` values.
|
||||
|
||||
Returns an empty string for empty input so the header handles
|
||||
"no history yet" gracefully.
|
||||
"""
|
||||
buf = [v for v in values if v >= 0][-width:]
|
||||
if not buf:
|
||||
return ""
|
||||
lo, hi = min(buf), max(buf)
|
||||
if hi - lo < 1e-9:
|
||||
return _SPARK_BLOCKS[len(_SPARK_BLOCKS) // 2] * len(buf)
|
||||
n = len(_SPARK_BLOCKS) - 1
|
||||
out = []
|
||||
for v in buf:
|
||||
idx = int(round((v - lo) / (hi - lo) * n))
|
||||
out.append(_SPARK_BLOCKS[max(0, min(n, idx))])
|
||||
return "".join(out)
|
||||
@@ -1,214 +0,0 @@
|
||||
"""Reproducer bundle builder for ``meshtastic-mcp-test-tui``.
|
||||
|
||||
When the operator presses ``x`` on a failed test leaf, we package the
|
||||
minimum viable failure context into a tarball under
|
||||
``mcp-server/tests/reproducers/``:
|
||||
|
||||
::
|
||||
|
||||
repro-<ts>-<short_nodeid>.tar.gz
|
||||
├── README.md human-readable overview
|
||||
├── test_report.json the failing TestReport event from reportlog
|
||||
├── fwlog.jsonl firmware log filtered to the failure window
|
||||
├── devices.json per-device device_info + lora config snapshot
|
||||
└── env.json seed, run #, pytest version, platform, hostname
|
||||
|
||||
Separate module so the logic can be unit-tested without Textual. The
|
||||
TUI glue is thin — one key binding calls :func:`build_reproducer_bundle`
|
||||
with the focused test's state and shows the path in a modal.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import json
|
||||
import pathlib
|
||||
import platform
|
||||
import re
|
||||
import socket
|
||||
import tarfile
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Iterable
|
||||
|
||||
|
||||
@dataclass
|
||||
class ReproContext:
|
||||
"""Everything :func:`build_reproducer_bundle` needs. Shaped to map
|
||||
cleanly onto the state the TUI already tracks — no extra data
|
||||
collection required at export time."""
|
||||
|
||||
nodeid: str
|
||||
longrepr: str
|
||||
sections: list[tuple[str, str]]
|
||||
start_ts: float | None
|
||||
stop_ts: float | None
|
||||
seed: str
|
||||
run_number: int
|
||||
exit_code: int | None
|
||||
fwlog_path: pathlib.Path
|
||||
output_dir: pathlib.Path
|
||||
extra_device_rows: list[dict[str, Any]] # [{role, port, info, ...}, ...]
|
||||
|
||||
|
||||
def _short_nodeid(nodeid: str) -> str:
|
||||
"""Collapse a pytest nodeid into a filename-safe slug (<= 60 chars)."""
|
||||
# Drop the file path prefix; keep test name + parametrization.
|
||||
tail = nodeid.split("::", 1)[-1] if "::" in nodeid else nodeid
|
||||
slug = re.sub(r"[^A-Za-z0-9_.\-]", "_", tail)
|
||||
return slug[:60].strip("_.-") or "test"
|
||||
|
||||
|
||||
def _filtered_fwlog(
|
||||
fwlog_path: pathlib.Path,
|
||||
start_ts: float | None,
|
||||
stop_ts: float | None,
|
||||
*,
|
||||
pad_s: float = 5.0,
|
||||
) -> bytes:
|
||||
"""Return fwlog.jsonl lines whose ``ts`` lies in [start-pad, stop+pad]."""
|
||||
if not fwlog_path.is_file():
|
||||
return b""
|
||||
if start_ts is None or stop_ts is None:
|
||||
# Without a time window, include the whole file — rare; happens
|
||||
# when a test fails in setup before pytest emitted a start ts.
|
||||
try:
|
||||
return fwlog_path.read_bytes()
|
||||
except OSError:
|
||||
return b""
|
||||
lo, hi = start_ts - pad_s, stop_ts + pad_s
|
||||
out = io.BytesIO()
|
||||
try:
|
||||
with fwlog_path.open("r", encoding="utf-8") as fh:
|
||||
for line in fh:
|
||||
stripped = line.strip()
|
||||
if not stripped:
|
||||
continue
|
||||
try:
|
||||
record = json.loads(stripped)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
ts = record.get("ts")
|
||||
if not isinstance(ts, (int, float)):
|
||||
continue
|
||||
if lo <= ts <= hi:
|
||||
out.write(line.encode("utf-8"))
|
||||
except OSError:
|
||||
return b""
|
||||
return out.getvalue()
|
||||
|
||||
|
||||
def _readme(ctx: ReproContext) -> str:
|
||||
t = time.strftime("%Y-%m-%d %H:%M:%S %Z", time.localtime())
|
||||
return f"""# Reproducer bundle
|
||||
|
||||
Exported by `meshtastic-mcp-test-tui` on {t}.
|
||||
|
||||
## Failing test
|
||||
|
||||
- **nodeid:** `{ctx.nodeid}`
|
||||
- **seed:** `{ctx.seed}`
|
||||
- **run #:** {ctx.run_number}
|
||||
- **suite exit code (at export time):** {ctx.exit_code if ctx.exit_code is not None else "in progress"}
|
||||
|
||||
## Files in this archive
|
||||
|
||||
| File | Contents |
|
||||
|---|---|
|
||||
| `test_report.json` | The pytest-reportlog `TestReport` event for the failing test — includes `longrepr`, captured `sections` (stdout/stderr/log), `duration`, `location`, `keywords`. |
|
||||
| `fwlog.jsonl` | Firmware log lines (from `meshtastic.log.line` pubsub) filtered to [start−5s, stop+5s] around the test's run window. Each line is `{{ts, port, line}}`. |
|
||||
| `devices.json` | Per-device snapshot at export time: `device_info` + `lora` config per detected role. |
|
||||
| `env.json` | Python version, platform, hostname, seed, run number. |
|
||||
|
||||
## How to triage
|
||||
|
||||
1. Open `test_report.json` and read `longrepr` + `sections` — most failures explain themselves there.
|
||||
2. If the failure is a mesh/telemetry assertion, `fwlog.jsonl` is where the answer usually lives. Grep for `Error=`, `NAK`, `PKI_UNKNOWN_PUBKEY`, `Skip send`, `Guru Meditation`, or the uptime timestamps around the assertion event.
|
||||
3. Compare `devices.json` against the expected state (e.g. `num_nodes >= 2`, `primary_channel == "McpTest"`, `region == "US"`). If fields disagree with the seed-derived USERPREFS profile, the device probably wasn't baked with this session's profile.
|
||||
|
||||
## Reproducing locally
|
||||
|
||||
```bash
|
||||
cd mcp-server
|
||||
MESHTASTIC_MCP_SEED='{ctx.seed}' .venv/bin/pytest '{ctx.nodeid}' --tb=long -v
|
||||
```
|
||||
"""
|
||||
|
||||
|
||||
def build_reproducer_bundle(ctx: ReproContext) -> pathlib.Path:
|
||||
"""Build a tarball under ``ctx.output_dir`` and return its path.
|
||||
|
||||
Parent dirs are created as needed. Errors during optional sections
|
||||
(devices, env) are swallowed — the bundle is still useful without
|
||||
them; refusing to export because the device poller had a hiccup
|
||||
would be worse than the export missing a file.
|
||||
"""
|
||||
ctx.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
ts = int(time.time())
|
||||
slug = _short_nodeid(ctx.nodeid)
|
||||
archive_path = ctx.output_dir / f"repro-{ts}-{slug}.tar.gz"
|
||||
|
||||
with tarfile.open(archive_path, "w:gz") as tar:
|
||||
|
||||
def _add(name: str, data: bytes) -> None:
|
||||
info = tarfile.TarInfo(name=name)
|
||||
info.size = len(data)
|
||||
info.mtime = ts
|
||||
tar.addfile(info, io.BytesIO(data))
|
||||
|
||||
# README
|
||||
_add("README.md", _readme(ctx).encode("utf-8"))
|
||||
|
||||
# test_report.json — reconstruct from the fields the TUI stashes.
|
||||
test_report = {
|
||||
"nodeid": ctx.nodeid,
|
||||
"outcome": "failed",
|
||||
"longrepr": ctx.longrepr,
|
||||
"sections": [list(s) for s in ctx.sections],
|
||||
"start": ctx.start_ts,
|
||||
"stop": ctx.stop_ts,
|
||||
}
|
||||
_add(
|
||||
"test_report.json",
|
||||
json.dumps(test_report, indent=2, default=str).encode("utf-8"),
|
||||
)
|
||||
|
||||
# fwlog.jsonl (filtered)
|
||||
_add("fwlog.jsonl", _filtered_fwlog(ctx.fwlog_path, ctx.start_ts, ctx.stop_ts))
|
||||
|
||||
# devices.json
|
||||
try:
|
||||
devices_payload = json.dumps(
|
||||
ctx.extra_device_rows or [], indent=2, default=str
|
||||
)
|
||||
except Exception:
|
||||
devices_payload = "[]"
|
||||
_add("devices.json", devices_payload.encode("utf-8"))
|
||||
|
||||
# env.json
|
||||
try:
|
||||
from importlib.metadata import version as _pkg_version
|
||||
|
||||
pytest_version = _pkg_version("pytest")
|
||||
except Exception:
|
||||
pytest_version = "unknown"
|
||||
env_payload = {
|
||||
"seed": ctx.seed,
|
||||
"run": ctx.run_number,
|
||||
"exit_code": ctx.exit_code,
|
||||
"export_ts": ts,
|
||||
"python": platform.python_version(),
|
||||
"pytest": pytest_version,
|
||||
"platform": f"{platform.system()} {platform.release()} {platform.machine()}",
|
||||
"hostname": socket.gethostname(),
|
||||
}
|
||||
_add("env.json", json.dumps(env_payload, indent=2).encode("utf-8"))
|
||||
|
||||
return archive_path
|
||||
|
||||
|
||||
def iter_entries(archive_path: pathlib.Path) -> Iterable[str]:
|
||||
"""Yield member names — used by callers that want to confirm the bundle shape."""
|
||||
with tarfile.open(archive_path, "r:gz") as tar:
|
||||
for m in tar.getmembers():
|
||||
yield m.name
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user