GPS 115200 baud

Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>

.claude/commands/diagnose.md

@@ -49,11 +49,17 @@ Call the meshtastic MCP tool bundle and format a structured health report for on
    - Do the LoRa configs match? (region, channel_num, modem_preset should all agree; mismatch = no mesh)
    - Do the primary channel NAMES match? Mismatch = different PSK = no decode.
 
-7. **Suggest next actions only for specific, recognisable failure modes**:
+7. **Recorder slice (cheap, always available).** The mcp-server runs an autouse log recorder that's been collecting from every connected device. Pull two short slices to surface anything weird that's already happened:
+   - `mcp__meshtastic__logs_window(start="-2m", level="WARN|ERROR|CRIT", max_lines=20)` — recent firmware errors. If empty, say "no recent errors"; don't manufacture concern.
+   - `mcp__meshtastic__telemetry_timeline(window="1h", field="free_heap", max_points=60)` — heap trend. If `slope_per_min < -50`, flag it and recommend `/leakhunt window=6h` for a deeper read; otherwise just note the current free heap.
+   - If `recorder_status` shows `running:false` or `files.telemetry.last_ts` is null, note "recorder has no telemetry yet — enable `set_debug_log_api(True)` to populate" and skip this step gracefully.
+
+8. **Suggest next actions only for specific, recognisable failure modes**:
    - Stale PKI pubkey one-way → "run `/test tests/mesh/test_direct_with_ack.py` — the retry + nodeinfo-ping heals this in the test path."
    - Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
    - Device unreachable, reachable via DFU → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds AND the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
    - CP2102-wedged-driver on macOS → see the note in `run-tests.sh`.
+   - Heap slope strongly negative → "run `/leakhunt window=6h` for a full timeline + classification."
 
 ## What NOT to do
 

.claude/commands/leakhunt.md

@@ -0,0 +1,103 @@
+---
+description: Hunt for memory leaks (and other slow degradations) by reading the persistent recorder's heap timeline + log slice over a window
+argument-hint: [window=1h] [field=free_heap] [variant=local]
+---
+
+<!-- markdownlint-disable MD029 -->
+
+# `/leakhunt` — read the recorder, classify a memory leak
+
+Use the always-on recorder (`mcp-server/.mtlog/`) to read a heap timeline plus the matching log slice and produce a one-page verdict: **steady / slow leak / fragmentation / OOM-imminent**. No firmware changes, no special build flags — the LocalStats telemetry packet that the firmware already broadcasts every ~60 s carries `heap_free_bytes` and `heap_total_bytes`.
+
+## Two signal paths — pick the right one
+
+| Path                  | Build flag       | Cadence        | Per-thread attribution | Cost                      |
+| --------------------- | ---------------- | -------------- | ---------------------- | ------------------------- |
+| LocalStats packet     | (default)        | ~60 s          | No                     | Free — always on          |
+| `[heap N]` log prefix | `-DDEBUG_HEAP=1` | every log line | Yes (Thread X leaked)  | Bigger flash + log volume |
+
+Both feed the same `telemetry_timeline(field="free_heap")` query — when DEBUG_HEAP is on, the recorder synthesizes telemetry rows from log prefixes (tagged `source: debug_heap`), so a single timeline call gets whichever signal is available. **For a slow leak diagnosis, the default path is plenty** (60 s cadence over 6 h = 360 points; linear regression over that nails sub-100-byte/min slopes). **DEBUG_HEAP is for attribution** — when the slope is real and you need to know which thread is leaking.
+
+## What to do
+
+1. **Parse `$ARGUMENTS`**: optional `window` (default `1h`, accepts `30m`/`6h`/`-3d`/etc.), optional `field` (default `free_heap`; alternates: `total_heap`, `battery_level`, anything in the LocalStats variant), optional `variant` (default `local`; alternates: `device`, `environment`, `power`, `airQuality`, `health`).
+
+2. **Verify the recorder is alive** — call `mcp__meshtastic__recorder_status`. Check:
+   - `running == True`
+   - `files.telemetry.lines > 0` (at least one telemetry packet recorded — if zero, the device hasn't broadcast LocalStats yet OR `set_debug_log_api` has never been on; tell the operator to run `mcp__meshtastic__set_debug_log_api(enabled=True)` and wait one device-update interval)
+   - `files.telemetry.last_ts` within the last 5 minutes (if older, the device is silent — log that, not "leak detected")
+
+3. **Detect whether DEBUG_HEAP is active** — `mcp__meshtastic__logs_window(start="-2m", grep=r"\\[heap \\d+\\]", max_lines=3)`. If any line matches, the firmware has the prefix → DEBUG_HEAP is on, expect higher-cadence data and `heap_event` rows. If zero matches over the last 2 minutes, you're on the LocalStats-only path.
+
+4. **Pull the timeline** — `mcp__meshtastic__telemetry_timeline(window=$window, variant=$variant, field=$field, max_points=200)`. Read:
+   - `samples` — how many raw points contributed
+   - `min`, `max` — total swing
+   - `slope_per_min` — units per minute (linear regression over the whole window)
+
+5. **Pull the log context for the same window** — `mcp__meshtastic__logs_window(start="-${window}", grep="Heap status|leaked heap|freed heap|out of memory|Alloc an err|panic|abort", max_lines=200)`. These are the strings the firmware emits when something memory-related happens (`DEBUG_HEAP` builds emit `"Heap status:"` and `"leaked heap"` lines; production builds emit `"Alloc an err"` on failure and `"out of memory"` on OOM).
+
+6. **Pull marker events** so we know if the operator labeled phases — `mcp__meshtastic__events_window(start="-${window}", kind="mark|connection_lost|connection_established")`. If a `connection_lost` overlaps a sharp drop, that's not a leak; that's a reboot.
+
+6a. **(DEBUG_HEAP only) Per-thread attribution** — `mcp__meshtastic__logs_window(start="-${window}", grep="leaked heap", max_lines=200)`. Each row has a structured `heap_event` field with `{kind, thread, before, after, delta}`. Aggregate by thread: sum the `delta` over the window per thread name. The thread with the largest cumulative negative delta is your suspect. Note the count too — a thread with 50× small leaks is different from 1× big leak.
+
+7. **Classify** based on what the data says, NOT on what you wish it said. Use these rules in order:
+   - **Insufficient data** (< 5 samples): say so. Suggest a longer window or longer wait. Stop.
+   - **Reboot mid-window**: if any `connection_lost` event is present AND `free_heap` jumped UP at that timestamp, the device rebooted. Note it; pre-reboot trend may be a leak but you only have part of the curve.
+   - **OOM-imminent**: any `Alloc an err=` or `out of memory` line in the log slice. This trumps everything; flag urgently.
+   - **Slow leak**: `slope_per_min < -50` AND `max - min > 1000` AND no reboot. The heap is monotonically (or near-monotonically) declining. Estimate time-to-zero: `min / -slope_per_min` minutes. Surface it.
+   - **Fragmentation suspect**: `slope_per_min` close to zero (|x| < 50) BUT min trends down across the window AND the log slice shows `Alloc an err` warnings WITHOUT total OOM. Means free total is OK but largest contiguous block is shrinking. Recommend a `DEBUG_HEAP` build to confirm.
+   - **Steady**: |slope_per_min| < 50, no error lines. Heap is fine.
+   - **Recovery curve**: slope is POSITIVE — heap recovered. Either a workload completed or GC fired. Note it; not a leak.
+
+8. **Report**:
+
+   ```text
+   /leakhunt window=6h field=free_heap variant=local
+   ────────────────────────────────────────────────────
+   recorder        : running, telem last_ts 8s ago
+   build           : DEBUG_HEAP=ON (per-line prefix detected)
+   samples         : 14,200 over 6h (cadence ~1.5s, log-line synth)
+   free_heap       : min 92,344 / max 124,008 / range 31,664
+   slope           : -82 bytes/min (negative — heap declining)
+   reboots         : none in window
+   OOM events      : none
+   error lines     : 3× "Alloc an err=ESP_ERR_NO_MEM" at +4h12m, +5h08m, +5h44m
+   thread leaks    : (DEBUG_HEAP) MeshPacket -3,124 B over 18 events
+                                  Router       -1,408 B over 4 events
+                                  others       -240 B
+   verdict         : SLOW LEAK — primary suspect MeshPacket thread
+   est. time-to-OOM: ~1,127 min (~18.8 h) at current slope
+   evidence        : (3 log line citations with uptimes)
+   ```
+
+   Then: **what to do next.**
+   - SLOW LEAK, **DEBUG_HEAP off** → recommend rebuilding with the flag and re-running this skill. Concrete one-liner the operator can copy:
+     ```text
+     mcp__meshtastic__build(env="<env>", build_flags={"DEBUG_HEAP": 1})
+     mcp__meshtastic__pio_flash(env="<env>", port="<port>", confirm=True)
+     ```
+     After flash, set debug_log_api back on and wait one window; re-run `/leakhunt`.
+   - SLOW LEAK, **DEBUG_HEAP on** → cite the top-leaking thread name from step 6a. Point at the corresponding source file (`grep -rn "ThreadName(\"<name>\")" src/`); the operator decides what to fix.
+   - FRAGMENTATION SUSPECT → propose pre-allocating any per-packet buffers; or rebuilding with `CONFIG_HEAP_TASK_TRACKING=y` on ESP32 to see who's holding the largest blocks.
+   - OOM-IMMINENT → flag for immediate attention; don't wait for the next telemetry interval.
+   - STEADY → say so; stop. Don't invent problems.
+
+## What NOT to do
+
+- Don't assume a leak from a single dip. LocalStats fires every ~60 s and the firmware naturally allocates+frees on each broadcast cycle; one packet sees the trough. Look at the slope, not the deltas.
+- Don't recommend code changes. This skill diagnoses; the operator decides what to fix.
+- Don't enable `set_debug_log_api` automatically — if it's off, telemetry isn't reaching pubsub anyway, and the recorder will be empty. Tell the operator to flip it on and wait, then re-run.
+- Don't run heavy workloads to "trigger the leak." The recorder is passive; we read what's there.
+
+## Companion: `mark_event` for stress runs
+
+If the operator wants to test under stimulus (e.g. blast 50 broadcasts and see what the heap does), they can frame the experiment with markers:
+
+```text
+mark_event("burst-start")
+… run the workload …
+mark_event("burst-end")
+/leakhunt window=15m
+```
+
+The markers land in both `events.jsonl` and `logs.jsonl`, so the report can show "free_heap dipped 8 KB during the burst window, recovered to baseline within 2 LocalStats cycles" → not a leak.

.claude/commands/repro.md

@@ -3,6 +3,8 @@ description: Re-run a specific test N times in isolation to triage flakes, diff
 argument-hint: <test-node-id> [count=5]
 ---
 
+<!-- markdownlint-disable MD029 -->
+
 # `/repro` — flakiness triage for one test
 
 Re-run a single pytest node ID N times in isolation, track pass rate, and surface what's _different_ in the firmware logs between the passing attempts and the failing ones. Turns "it's flaky, I guess" into "it fails when X, passes when Y."
@@ -40,6 +42,8 @@ Re-run a single pytest node ID N times in isolation, track pass rate, and surfac
 
    Surface the top 3 differences as a "passes when / fails when" table. Don't dump full logs — pull specific lines with uptime timestamps.
 
+5a. **Archive recorder slices per attempt** (no extra device interaction; the recorder runs autouse). Right after each attempt finishes, capture its `(start_ts, end_ts)` and call `mcp__meshtastic__recorder_export(start=<start>, end=<end>, dest_dir="mcp-server/tests/repro_artifacts/<safe-test-id>/attempt_<n>/")`. This drops a `logs.jsonl`, `telemetry.jsonl`, `packets.jsonl`, and `events.jsonl` snapshot scoped to the attempt window. Use these for cross-attempt diffs in step 5: `jq '.line' logs.jsonl` is faster than re-running the test, and the telemetry slice lets you compare heap behavior across attempts.
+
 6. **Classify the flake** into one of:
    - **LoRa airtime collision** → pass rate improves with fewer concurrent transmitters; propose a `time.sleep` gap or retry bump in the test body.
    - **PKI key staleness** → fails on first attempt, passes after self-heal; existing retry loop in `test_direct_with_ack.py` handles this.

.devcontainer/devcontainer.json

@@ -8,17 +8,21 @@
   "features": {
     "ghcr.io/devcontainers/features/python:1": {
       "installTools": true,
-      "version": "3.14"
+      "version": "3.13"
     }
   },
   "customizations": {
     "vscode": {
       "extensions": [
         "ms-vscode.cpptools",
-        "platformio.platformio-ide",
+        "Jason2866.esp-decoder",
+        "pioarduino.pioarduino-ide",
         "Trunk.io"
       ],
-      "unwantedRecommendations": ["ms-azuretools.vscode-docker"],
+      "unwantedRecommendations": [
+        "ms-azuretools.vscode-docker",
+        "platformio.platformio-ide"
+      ],
       "settings": {
         "extensions.ignoreRecommendations": true
       }

.github/copilot-instructions.md

@@ -193,22 +193,26 @@ Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which d
 
 Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly.
 
-### Gradient sync (opt-in via special nonces)
+### Sync flow: thin NodeInfo + post-COMPLETE_ID replay (no opt-in)
 
-`client_capabilities` is **not** a thing in this branch. Phone clients opt into the new sync flow by sending one of two values in the `ToRadio.want_config_id`:
+There is no capability flag and no special "gradient" nonce. The **default** sync flow is:
 
-- `SPECIAL_NONCE_GRADIENT_SYNC` (69422) — full config + thin NodeInfo + replay phases.
-- `SPECIAL_NONCE_GRADIENT_ONLY_NODES` (69423) — skip config segments, NodeInfo + replay only.
+1. Config / module-config / channel / metadata segments (same as before).
+2. `STATE_SEND_OWN_NODEINFO` — **our own** NodeInfo, still bundled with our position and device_metrics (because the replay snapshot excludes our own NodeNum). Emitted via `ConvertToNodeInfo(lite)`.
+3. `STATE_SEND_OTHER_NODEINFOS` — every other peer's NodeInfo, **always thin** (no `position`, no `device_metrics`). Emitted via `ConvertToNodeInfoThin(lite)`.
+4. `STATE_SEND_FILEMANIFEST` → `STATE_SEND_COMPLETE_ID` — the phone sees `config_complete_id` and treats sync as done.
+5. `STATE_SEND_PACKETS` — live mesh packets, with a trailing replay drain interleaved. The replay drain walks four cached satellite stores in order (positions → telemetry → environment → status) and emits each cached entry as an ordinary `MeshPacket` on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` device + environment variants, `NODE_STATUS_APP`). These are indistinguishable on the wire from live mesh traffic, so clients need no special handling — any code that already updates UI on `POSITION_APP` etc. works.
 
-`PhoneAPI::clientWantsGradientSync()` is the single switch. When true, `STATE_SEND_OTHER_NODEINFOS` is followed by:
+`PhoneAPI::sendConfigComplete()` arms `replayPhase = REPLAY_PHASE_POSITIONS` for default/full sync and `SPECIAL_NONCE_ONLY_NODES`, while `SPECIAL_NONCE_ONLY_CONFIG` skips replay. The drain runs inside `STATE_SEND_PACKETS` via `popReplayPacket()`, lower priority than live traffic. When all four phases drain, `replayPhase` flips back to `REPLAY_PHASE_IDLE` and the snapshot vectors get `shrink_to_fit`ed.
 
-```text
-STATE_REPLAY_POSITIONS → STATE_REPLAY_TELEMETRY → STATE_REPLAY_ENVIRONMENT → STATE_REPLAY_STATUS
-```
+STM32WL and any other build with all four `MESHTASTIC_EXCLUDE_*DB` flags set produces zero replay packets — `popReplayPacket` advances through each phase in microseconds without emitting anything.
 
-Each replay phase walks the corresponding satellite map and emits synthetic `MeshPacket`s on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` for both device + environment variants, `STATUS_MESSAGE_APP`). Legacy clients (no special nonce) get the bundled-NodeInfo path with position/device_metrics joined back in by `ConvertToNodeInfo(lite, pos*, dm*)` — wire bytes are byte-identical to pre-v25 for them.
+Special nonces that still mean something:
 
-`ConvertToNodeInfoThin(lite)` is the gradient-sync emitter (no position/telemetry).
+- `SPECIAL_NONCE_ONLY_CONFIG` (69420) — skip node sync entirely, just config.
+- `SPECIAL_NONCE_ONLY_NODES` (69421) — skip config segments, jump straight to `STATE_SEND_OWN_NODEINFO`. Still gets the post-COMPLETE_ID replay drain.
+
+There are no other reserved nonces; everything else is a fresh random `want_config_id` from the client.
 
 ### v24 → v25 migration
 
@@ -285,6 +289,8 @@ firmware/
 - Prefer `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR` for logging
 - Use `assert()` for invariants that should never fail
 - C++17 features are available (`std::optional`, structured bindings, `if constexpr`, etc.)
+- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
+- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
 
 ### Naming Conventions
 

.gitignore

@@ -47,6 +47,10 @@ data/boot/logo.*
 managed_components/*
 arduino-lib-builder*
 dependencies.lock
+
+# JLink / RTT debug artifacts (nRF SoCs)
+flash.jlink
+rtt_*.txt
 idf_component.yml
 CMakeLists.txt
 /sdkconfig.*
@@ -56,3 +60,9 @@ CMakeLists.txt
 .python3
 .claude/scheduled_tasks.lock
 userPrefs.jsonc.mcp-session-bak
+
+# Fake-NodeDB fixture pipeline (bin/regen-fake-nodedbs.sh)
+# JSONL seeds are committed (test/fixtures/nodedb/seed_v25_*.jsonl);
+# compiled .proto outputs are ephemeral build artifacts.
+build/fixtures/
+bin/_generated/

.trunk/trunk.yaml

@@ -4,19 +4,19 @@ cli:
 plugins:
   sources:
     - id: trunk
-      ref: v1.8.0
+      ref: v1.10.0
       uri: https://github.com/trunk-io/plugins
 lint:
   enabled:
-    - checkov@3.2.526
+    - checkov@3.2.529
     - renovate@43.150.0
     - prettier@3.8.3
-    - trufflehog@3.95.2
+    - trufflehog@3.95.3
     - yamllint@1.38.0
     - bandit@1.9.4
     - trivy@0.70.0
     - taplo@0.10.0
-    - ruff@0.15.12
+    - ruff@0.15.13
     - isort@8.0.1
     - markdownlint@0.48.0
     - oxipng@10.1.1
@@ -26,7 +26,7 @@ lint:
     - hadolint@2.14.0
     - shfmt@3.6.0
     - shellcheck@0.11.0
-    - black@26.3.1
+    - black@26.5.1
     - git-diff-check
     - gitleaks@8.30.1
     - clang-format@16.0.3
@@ -34,6 +34,13 @@ lint:
     - linters: [ALL]
       paths:
         - bin/**
+    # Fake-NodeDB fixture JSONL files contain deterministic synthetic
+    # public_key_hex (64-char hex) values that gitleaks misidentifies as
+    # generic-api-key. These are not secrets — they're test fixtures
+    # produced by bin/gen-fake-nodedb-seed.py with a fixed RNG seed.
+    - linters: [gitleaks]
+      paths:
+        - test/fixtures/nodedb/seed_v25_*.jsonl
 runtimes:
   enabled:
     - python@3.14.4

.vscode/extensions.json

@@ -1,10 +1,10 @@
 {
-    // See http://go.microsoft.com/fwlink/?LinkId=827846
-    // for the documentation about the extensions.json format
     "recommendations": [
-        "platformio.platformio-ide"
+        "Jason2866.esp-decoder",
+        "pioarduino.pioarduino-ide"
     ],
     "unwantedRecommendations": [
-        "ms-vscode.cpptools-extension-pack"
+        "ms-vscode.cpptools-extension-pack",
+        "platformio.platformio-ide"
     ]
 }

AGENTS.md

@@ -66,6 +66,8 @@ Key rotation to never trigger casually: only the **full** factory reset (`factor
 - **Don't speculate about firmware root causes.** When evidence doesn't support a classification, say "unknown" and list what would disambiguate.
 - **Run `trunk fmt` before proposing a commit.** The `trunk_check` CI gate will reject unformatted code.
 - **`confirm=True` on destructive MCP tools is a real gate, not a formality.** Don't bypass it via auto-approve settings.
+- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
+- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
 
 ## Typical agent workflows
 

bin/_rewrite_proto_namespace.py

@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+"""Post-process protoc-generated Python files to live under a local namespace.
+
+Called by bin/regen-py-protos.sh. Walks the generated *_pb2.py files in the
+target directory and rewrites every `meshtastic` reference (imports, dotted
+attribute access) to use the new namespace (e.g., `meshtastic_v25`).
+
+Why: the .proto files declare `package meshtastic;`, so protoc emits
+`from meshtastic import mesh_pb2 as ...` lines. That would shadow the PyPI
+`meshtastic` package which other parts of the mcp-server depend on. Renaming
+to a local namespace keeps both available.
+
+Usage:
+    _rewrite_proto_namespace.py <generated_dir> <new_namespace>
+"""
+
+from __future__ import annotations
+
+import pathlib
+import re
+import sys
+
+
+def rewrite(dir_path: pathlib.Path, new_ns: str) -> int:
+    # Standard protoc import forms:
+    #   from meshtastic.X_pb2 import ...     (rare, for direct symbol pulls)
+    #   from meshtastic import X_pb2 as ...  (common, the cross-file ref)
+    #   import meshtastic.X_pb2              (also possible)
+    pattern_dotted_from = re.compile(r"^from meshtastic\.", re.MULTILINE)
+    pattern_bare_from = re.compile(r"^from meshtastic import ", re.MULTILINE)
+    pattern_dotted_import = re.compile(r"^import meshtastic\.", re.MULTILINE)
+
+    count = 0
+    for p in dir_path.glob("*.py"):
+        text = p.read_text(encoding="utf-8")
+        new = pattern_dotted_from.sub(f"from {new_ns}.", text)
+        new = pattern_bare_from.sub(f"from {new_ns} import ", new)
+        new = pattern_dotted_import.sub(f"import {new_ns}.", new)
+        # NOTE: we deliberately leave `meshtastic/X.proto` source-filename
+        # references inside descriptor strings alone. The descriptor pool is
+        # keyed by source filename (independent of Python package layout), so
+        # those don't collide with the PyPI package's descriptors.
+        if new != text:
+            p.write_text(new, encoding="utf-8")
+            count += 1
+    return count
+
+
+def main(argv: list[str]) -> int:
+    if len(argv) != 2:
+        print("usage: _rewrite_proto_namespace.py <generated_dir> <new_namespace>", file=sys.stderr)
+        return 2
+    dir_path = pathlib.Path(argv[0])
+    new_ns = argv[1]
+    if not dir_path.is_dir():
+        print(f"directory not found: {dir_path}", file=sys.stderr)
+        return 2
+    n = rewrite(dir_path, new_ns)
+    print(f"rewrote {n} file(s) in {dir_path} → namespace {new_ns}", file=sys.stderr)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))

bin/build-esp32.sh

@@ -38,4 +38,4 @@ cp bin/device-install.* $OUTDIR/
 cp bin/device-update.* $OUTDIR/
 
 echo "Copying manifest"
-cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json || true
+cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json

bin/config.d/lora-station-g3.yaml

@@ -0,0 +1,18 @@
+# Station G3 motherboard with a Raspberry Pi Zero 2W as the MCU daughterboard.
+# Verify spidev / I2C device paths for your OS — they may differ.
+Meta:
+  name: Station G3
+  support: community
+  compatible:
+    - raspberry-pi
+
+Lora:
+  Module: sx1262
+  IRQ: 22                       # BCM pin — wiki spec
+  Reset: 16                     # BCM pin — wiki spec
+  Busy: 24                      # BCM pin — wiki spec
+  # CS: 8                       # BCM 8 = SPI0 CE0 (default); uncomment only to override
+  DIO2_AS_RF_SWITCH: true
+  DIO3_TCXO_VOLTAGE: true
+  spidev: spidev0.0
+  # SX126X_MAX_POWER: 19        # matches Station G2 firmware cap; raise carefully per PA jumper mode

bin/gen-fake-nodedb-seed.py

@@ -0,0 +1,439 @@
+#!/usr/bin/env python3
+"""Deterministic seed-data generator for the fake NodeDB fixture pipeline.
+
+Writes a JSONL file describing N fake-but-realistic Meshtastic peers.
+The output is hand-editable and committed; a sibling compile step
+(bin/seed-json-to-proto.py) turns it into a binary `meshtastic_NodeDatabase`
+v25 protobuf with fresh "now-relative" timestamps.
+
+Determinism contract:
+  Same --seed -> byte-identical JSONL output, regardless of wall clock.
+  All timestamps are stored as `*_offset_sec` (seconds before "now"); the
+  compile step resolves them to absolute epochs at compile time.
+
+Structural fields covered:
+  * NodeInfoLite header: num, long_name, short_name, hw_model, role,
+    public_key, snr, channel, hops_away, next_hop, bitfield flags
+  * PositionLite: lat/long Gaussian around --centroid, altitude, source
+  * DeviceMetrics: battery/voltage/util/uptime
+  * EnvironmentMetrics: temp/humidity/pressure/iaq
+  * StatusMessage: error_code (usually zero)
+
+Active-board allow-list:
+  hw_model values are restricted to the intersection of
+    (a) variants with `custom_meshtastic_support_level = 1` in
+        variants/*/*/platformio.ini, AND
+    (b) values present in the `HardwareModel` enum in mesh.proto.
+  See HW_MODEL_WEIGHTS below. Deprecated boards (legacy TLORA / Heltec V1-2 /
+  classic TBEAM / TBEAM_V0P7 / Nano G1 / etc.) and fuzzer-only sentinels
+  (PORTDUINO, ANDROID_SIM, DIY_V1, ...) are excluded.
+
+Active-role allow-list:
+  Excludes ROUTER_CLIENT (deprecated v2.3.15) and REPEATER (deprecated v2.7.11).
+"""
+
+from __future__ import annotations
+
+import argparse
+import datetime as _dt
+import json
+import math
+import pathlib
+import random
+import sys
+
+# --------------------------------------------------------------------------
+# Active-board allow-list (intersection of tier-1 variants + HardwareModel enum).
+# Refresh by running:
+#   for f in $(find variants -name 'platformio.ini' | xargs grep -lE 'custom_meshtastic_support_level = 1'); do
+#       grep custom_meshtastic_hw_model_slug $f | awk -F= '{print $2}' | tr -d ' ';
+#   done | sort -u | comm -12 - <(python3 -c "from meshtastic.protobuf.mesh_pb2 import HardwareModel; print('\\n'.join(HardwareModel.keys()))" | sort)
+# --------------------------------------------------------------------------
+HW_MODEL_WEIGHTS: dict[str, float] = {
+    "HELTEC_V3": 14.0,
+    "T_DECK": 9.0,
+    "HELTEC_V4": 8.0,
+    "RAK4631": 8.0,
+    "HELTEC_MESH_POCKET": 6.0,
+    "TRACKER_T1000_E": 5.0,
+    "HELTEC_MESH_NODE_T114": 5.0,
+    "T_DECK_PRO": 5.0,
+    "LILYGO_TBEAM_S3_CORE": 4.0,
+    "HELTEC_WIRELESS_PAPER": 4.0,
+    "HELTEC_WSL_V3": 3.0,
+    "T_ECHO": 3.0,
+    "HELTEC_WIRELESS_TRACKER": 3.0,
+    "HELTEC_WIRELESS_TRACKER_V2": 2.0,
+    "HELTEC_VISION_MASTER_E290": 2.0,
+    "HELTEC_MESH_SOLAR": 2.0,
+    "SEEED_WIO_TRACKER_L1": 2.0,
+    "T_LORA_PAGER": 1.5,
+    "HELTEC_VISION_MASTER_E213": 1.5,
+    "T_ECHO_PLUS": 1.0,
+    "MUZI_BASE": 1.0,
+    "WISMESH_TAP_V2": 1.0,
+    "THINKNODE_M2": 1.0,
+    "THINKNODE_M5": 1.0,
+    "TLORA_T3_S3": 1.0,
+    # Long tail (uniform low weight across remaining tier-1 boards):
+    "HELTEC_V4_R8": 0.3,
+    "HELTEC_VISION_MASTER_T190": 0.3,
+    "HELTEC_HT62": 0.3,
+    "HELTEC_MESH_NODE_T096": 0.3,
+    "M5STACK_C6L": 0.3,
+    "MINI_EPAPER_S3": 0.3,
+    "MUZI_R1_NEO": 0.3,
+    "NOMADSTAR_METEOR_PRO": 0.3,
+    "RAK3312": 0.3,
+    "RAK3401": 0.3,
+    "SEEED_SOLAR_NODE": 0.3,
+    "SEEED_WIO_TRACKER_L1_EINK": 0.3,
+    "SENSECAP_INDICATOR": 0.3,
+    "TBEAM_1_WATT": 0.3,
+    "THINKNODE_M1": 0.3,
+    "THINKNODE_M3": 0.3,
+    "THINKNODE_M6": 0.3,
+    "T_ECHO_LITE": 0.3,
+    "WISMESH_TAG": 0.3,
+    "WISMESH_TAP": 0.3,
+    "XIAO_NRF52_KIT": 0.3,
+    "CROWPANEL": 0.3,
+}
+
+# Non-deprecated roles only.
+ROLE_WEIGHTS: dict[str, float] = {
+    "CLIENT": 75.0,
+    "CLIENT_MUTE": 5.0,
+    "ROUTER": 7.0,
+    "TRACKER": 3.0,
+    "SENSOR": 2.0,
+    "CLIENT_HIDDEN": 2.0,
+    "ROUTER_LATE": 2.0,
+    "CLIENT_BASE": 2.0,
+    "TAK": 1.0,
+    "TAK_TRACKER": 0.5,
+    "LOST_AND_FOUND": 0.5,
+}
+
+# Name pools — 60 firsts × 60 lasts = 3600 combinations.
+FIRSTS = [
+    "Quick", "Brave", "Silent", "Wild", "Lone", "Bright", "Red", "Blue",
+    "Green", "Black", "White", "Iron", "Steel", "Copper", "Silver", "Gold",
+    "Stone", "River", "Forest", "Mountain", "Canyon", "Desert", "Storm", "Sky",
+    "Solar", "Lunar", "Dawn", "Dusk", "Misty", "Frosty", "Sunny", "Shady",
+    "Happy", "Sleepy", "Drowsy", "Sneaky", "Sharp", "Smooth", "Rough", "Loud",
+    "Soft", "Slow", "Fast", "Tall", "Short", "Old", "New", "Tiny",
+    "Giant", "Hidden", "Lost", "Found", "Wandering", "Roving", "Drifting", "Floating",
+    "Burning", "Frozen", "Whispering", "Howling",
+]
+LASTS = [
+    "Phoenix", "Lion", "Bear", "Wolf", "Hawk", "Eagle", "Fox", "Lynx",
+    "Cougar", "Coyote", "Raven", "Owl", "Crow", "Falcon", "Heron", "Crane",
+    "Otter", "Badger", "Bison", "Elk", "Moose", "Stag", "Doe", "Hare",
+    "Marmot", "Mole", "Beaver", "Squirrel", "Mustang", "Bronco", "Pony", "Colt",
+    "Cobra", "Viper", "Mamba", "Adder", "Gecko", "Iguana", "Tortoise", "Turtle",
+    "Salmon", "Trout", "Bass", "Pike", "Shark", "Whale", "Dolphin", "Seal",
+    "Cactus", "Yucca", "Sage", "Juniper", "Pine", "Cedar", "Aspen", "Oak",
+    "Bluff", "Mesa", "Arroyo", "Ridge",
+]
+
+# Brief callsign pool for licensed-looking suffixes.
+CALLSIGN_PREFIXES = ["KX", "WD", "N5", "KE", "AB", "W5", "K1", "KQ", "AE", "NM"]
+
+# Only emojis that fit in 4 UTF-8 bytes (no variation selectors). short_name's
+# nanopb max_size:5 (incl. NUL) limits content to 4 bytes. ❄️ / ☀️ would be
+# 6 bytes due to U+FE0F variation selector — explicitly excluded.
+EMOJI_SHORTNAMES = ["🦊", "🐺", "🦅", "🐢", "🌵", "🔥", "🌙",
+                    "🌊", "🗻", "🌲", "🦌", "🐝", "🦂", "🦉",
+                    "🦇", "🦋"]
+
+# --------------------------------------------------------------------------
+# Helpers
+# --------------------------------------------------------------------------
+
+NUM_RESERVED = 4         # firmware reserves 0..3 (per NodeDB constants)
+NUM_MAX_EXCLUSIVE = 0x80000000  # restrict to positive int32 range for readability
+
+
+def _weighted_choice(rng: random.Random, weights: dict[str, float]) -> str:
+    """Deterministic weighted pick. Uses sorted keys so dict order is fixed."""
+    keys = sorted(weights.keys())
+    totals = [weights[k] for k in keys]
+    return rng.choices(keys, weights=totals, k=1)[0]
+
+
+def _gen_long_name(rng: random.Random, is_licensed: bool) -> str:
+    base = f"{rng.choice(FIRSTS)} {rng.choice(LASTS)}"
+    if is_licensed:
+        prefix = rng.choice(CALLSIGN_PREFIXES)
+        # Two trailing alpha chars after the digit; keep within 25 - len(base) - 1
+        suffix = f" {prefix}{rng.randint(0,9)}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}"
+        # nanopb max_size:25 means C string fits 24 bytes + NUL.
+        if len(base) + len(suffix) <= 24:
+            base = base + suffix
+    # Hard cap to 24 chars (nanopb max_size:25 minus NUL).
+    return base[:24]
+
+
+def _gen_short_name(rng: random.Random, long_name: str) -> str:
+    # 10% emoji-only short_name
+    if rng.random() < 0.10:
+        return rng.choice(EMOJI_SHORTNAMES)
+    first_char = long_name[0].upper() if long_name else "X"
+    alphanums = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
+    return first_char + "".join(rng.choices(alphanums, k=3))
+
+
+def _gen_hops_away(rng: random.Random) -> int:
+    # Geometric-ish: 0→55%, 1→25%, 2→12%, 3→5%, 4→2%, 5+→1%
+    r = rng.random()
+    if r < 0.55:
+        return 0
+    if r < 0.80:
+        return 1
+    if r < 0.92:
+        return 2
+    if r < 0.97:
+        return 3
+    if r < 0.99:
+        return 4
+    return rng.randint(5, 7)
+
+
+def _gen_position(
+    rng: random.Random,
+    centroid_lat: float,
+    centroid_lon: float,
+    spread_km: float,
+    last_heard_offset_sec: int,
+) -> dict:
+    # 1 deg ≈ 111 km at the equator; we use this as a flat approximation.
+    lat = centroid_lat + rng.gauss(0.0, spread_km / 111.0)
+    lon = centroid_lon + rng.gauss(0.0, spread_km / 111.0)
+    altitude = max(0, round(rng.gauss(1376.0, 250.0)))  # T or C valley floor + relief
+    # Position was reported up to 300s before last_heard.
+    time_offset_sec = last_heard_offset_sec + rng.randint(0, 300)
+    return {
+        "latitude": round(lat, 6),
+        "longitude": round(lon, 6),
+        "altitude": altitude,
+        "time_offset_sec": time_offset_sec,
+        "location_source": "LOC_INTERNAL",
+    }
+
+
+def _gen_telemetry(rng: random.Random) -> dict:
+    # 5% plugged-in (battery_level == 101); rest uniform [10..100].
+    if rng.random() < 0.05:
+        battery_level = 101
+        voltage = 4.20
+    else:
+        battery_level = rng.randint(10, 100)
+        voltage = round(3.3 + (battery_level / 100.0) * 0.9, 3)
+    # Beta distributions for low/right-skewed metrics; randomly draw via gammavariate.
+    def _beta(a: float, b: float) -> float:
+        x = rng.gammavariate(a, 1.0)
+        y = rng.gammavariate(b, 1.0)
+        return x / (x + y)
+    channel_utilization = round(_beta(2.0, 15.0) * 100.0, 2)
+    air_util_tx = round(_beta(1.5, 20.0) * 10.0, 3)
+    uptime_seconds = int(rng.expovariate(1.0 / 86400.0))
+    return {
+        "battery_level": battery_level,
+        "voltage": voltage,
+        "channel_utilization": channel_utilization,
+        "air_util_tx": air_util_tx,
+        "uptime_seconds": uptime_seconds,
+    }
+
+
+def _gen_environment(rng: random.Random) -> dict:
+    return {
+        "temperature": round(rng.gauss(22.0, 8.0), 2),
+        "relative_humidity": round(min(100.0, max(0.0, rng.gauss(55.0, 20.0))), 2),
+        "barometric_pressure": round(rng.gauss(1013.0, 8.0), 2),
+        "iaq": int(min(500, max(0, round(rng.gauss(50.0, 30.0))))),
+    }
+
+
+def _gen_status(rng: random.Random) -> dict:
+    # `StatusMessage` (mesh.proto:1445) has a single free-form `string status`.
+    # Most peers report a healthy short status; occasional alert string.
+    healthy = ["OK", "online", "active", "running", "ready", "nominal"]
+    alert = ["low-batt", "no-gps", "weak-signal", "rebooted", "offline-soon"]
+    if rng.random() < 0.92:
+        return {"status": rng.choice(healthy)}
+    return {"status": rng.choice(alert)}
+
+
+def _gen_node(
+    rng: random.Random,
+    num: int,
+    centroid_lat: float,
+    centroid_lon: float,
+    spread_km: float,
+    coverage: dict[str, float],
+    last_heard_mean_sec: int,
+    last_heard_max_sec: int,
+) -> dict:
+    is_licensed = rng.random() < 0.05
+    long_name = _gen_long_name(rng, is_licensed)
+    short_name = _gen_short_name(rng, long_name)
+    hw_model = _weighted_choice(rng, HW_MODEL_WEIGHTS)
+    role = _weighted_choice(rng, ROLE_WEIGHTS)
+    has_public_key = rng.random() < 0.92
+    public_key_hex = (
+        "".join(f"{rng.randint(0,255):02x}" for _ in range(32)) if has_public_key else ""
+    )
+    snr = round(max(-20.0, min(12.0, rng.gauss(6.0, 4.0))), 2)
+    channel = 0 if rng.random() < 0.90 else rng.randint(1, 7)
+    hops_away = _gen_hops_away(rng)
+    next_hop = rng.randint(0, 255) if hops_away > 0 else 0
+    last_heard_offset_sec = int(min(rng.expovariate(1.0 / last_heard_mean_sec), last_heard_max_sec))
+
+    bitfield = {
+        "has_user": True,
+        "is_favorite": rng.random() < 0.08,
+        "is_muted": rng.random() < 0.03,
+        "via_mqtt": rng.random() < 0.12,
+        "is_ignored": rng.random() < 0.01,
+        "is_licensed": is_licensed,
+        "has_is_unmessagable": True,
+        "is_unmessagable": rng.random() < 0.02,
+        "is_key_manually_verified": rng.random() < 0.04,
+    }
+
+    node: dict = {
+        "num": f"0x{num:08x}",
+        "long_name": long_name,
+        "short_name": short_name,
+        "hw_model": hw_model,
+        "role": role,
+        "public_key_hex": public_key_hex,
+        "snr": snr,
+        "channel": channel,
+        "hops_away": hops_away,
+        "next_hop": next_hop,
+        "last_heard_offset_sec": last_heard_offset_sec,
+        "bitfield": bitfield,
+        "position": (
+            _gen_position(rng, centroid_lat, centroid_lon, spread_km, last_heard_offset_sec)
+            if rng.random() < coverage["position"]
+            else None
+        ),
+        "telemetry": _gen_telemetry(rng) if rng.random() < coverage["telemetry"] else None,
+        "environment": _gen_environment(rng) if rng.random() < coverage["environment"] else None,
+        "status": _gen_status(rng) if rng.random() < coverage["status"] else None,
+    }
+    return node
+
+
+def _parse_my_node_num(s: str | None) -> int | None:
+    if s is None:
+        return None
+    s = s.strip()
+    if s.startswith("0x") or s.startswith("0X"):
+        return int(s, 16)
+    return int(s)
+
+
+def main(argv: list[str]) -> int:
+    p = argparse.ArgumentParser(
+        description="Deterministic JSONL seed for the fake NodeDB fixture.",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    p.add_argument("--count", type=int, required=True, help="Number of fake nodes to emit.")
+    p.add_argument("--seed", type=int, required=True, help="Deterministic seed.")
+    p.add_argument("--out", required=True, help="Output JSONL path.")
+    p.add_argument(
+        "--centroid",
+        default="33.1284,-107.2528",
+        help="LAT,LON centroid (default: Truth or Consequences, NM).",
+    )
+    p.add_argument("--spread-km", type=float, default=60.0, help="Gaussian std-dev in km.")
+    p.add_argument("--position-coverage", type=float, default=0.85)
+    p.add_argument("--telemetry-coverage", type=float, default=0.70)
+    p.add_argument("--environment-coverage", type=float, default=0.25)
+    p.add_argument("--status-coverage", type=float, default=0.40)
+    p.add_argument("--my-node-num", default=None, help="Exclude this NodeNum from generated set (hex or dec).")
+    p.add_argument("--last-heard-mean-sec", type=int, default=3600)
+    p.add_argument("--last-heard-max-sec", type=int, default=7 * 86400)
+    args = p.parse_args(argv)
+
+    if args.count <= 0:
+        print("--count must be positive", file=sys.stderr)
+        return 2
+
+    try:
+        centroid_lat, centroid_lon = (float(s) for s in args.centroid.split(","))
+    except ValueError:
+        print(f"--centroid must be LAT,LON; got {args.centroid!r}", file=sys.stderr)
+        return 2
+
+    my_node_num = _parse_my_node_num(args.my_node_num)
+
+    rng = random.Random(args.seed)
+
+    # 1) Generate a unique deterministic set of NodeNums.
+    nums: set[int] = set()
+    while len(nums) < args.count:
+        n = rng.randrange(NUM_RESERVED, NUM_MAX_EXCLUSIVE)
+        if my_node_num is not None and n == my_node_num:
+            continue
+        nums.add(n)
+    ordered_nums = sorted(nums)  # sort to fix output order independent of set hash
+
+    # 2) Per-node generation (in num order, single RNG continues).
+    coverage = {
+        "position": args.position_coverage,
+        "telemetry": args.telemetry_coverage,
+        "environment": args.environment_coverage,
+        "status": args.status_coverage,
+    }
+    nodes = [
+        _gen_node(
+            rng,
+            n,
+            centroid_lat,
+            centroid_lon,
+            args.spread_km,
+            coverage,
+            args.last_heard_mean_sec,
+            args.last_heard_max_sec,
+        )
+        for n in ordered_nums
+    ]
+
+    # 3) Write JSONL.
+    out_path = pathlib.Path(args.out)
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    # `generated_at_iso` is informational; it does NOT affect determinism because
+    # we derive it from the seed, not from wall clock. (Same seed -> same string.)
+    generated_at = _dt.datetime.fromtimestamp(args.seed, tz=_dt.timezone.utc).isoformat().replace("+00:00", "Z")
+    meta = {
+        "_meta": {
+            "version": 25,
+            "seed": args.seed,
+            "count": args.count,
+            "centroid": [centroid_lat, centroid_lon],
+            "spread_km": args.spread_km,
+            "generated_at_iso": generated_at,
+            "my_node_num_excluded": (None if my_node_num is None else f"0x{my_node_num:08x}"),
+            "coverage": coverage,
+            "last_heard_mean_sec": args.last_heard_mean_sec,
+            "last_heard_max_sec": args.last_heard_max_sec,
+        }
+    }
+    with out_path.open("w", encoding="utf-8") as f:
+        # `ensure_ascii=False` so emoji short_names survive. `sort_keys=True` for
+        # determinism (insertion order varies by Python version otherwise).
+        f.write(json.dumps(meta, ensure_ascii=False, sort_keys=True) + "\n")
+        for node in nodes:
+            f.write(json.dumps(node, ensure_ascii=False, sort_keys=True) + "\n")
+
+    print(f"wrote {args.count} nodes to {out_path} ({out_path.stat().st_size} bytes)", file=sys.stderr)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))

bin/platformio-custom.py

@@ -293,9 +293,12 @@ if ("HAS_TFT", 1) in env.get("CPPDEFINES", []):
 board_arch = infer_architecture(env.BoardConfig())
 should_skip_manifest = board_arch is None
 
-# For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
-mtjson_deps = [] if should_skip_manifest else ["buildprog"]
-if not should_skip_manifest and platform.name == "espressif32":
+# Most platforms can generate the manifest as part of the default 'buildprog' target.
+# Typically this passes success/failure properly.
+mtjson_deps = ["buildprog"]
+if platform.name == "espressif32":
+    # On ESP32, we need to explicitly depend upon the binary to prevent fake-success upon failure.
+    mtjson_deps = ["$BUILD_DIR/${PROGNAME}.bin"]
     # Build littlefs image as part of mtjson target
     # Equivalent to `pio run -t buildfs`
     target_lfs = env.DataToBin(
@@ -309,7 +312,8 @@ if should_skip_manifest:
 
     env.AddCustomTarget(
         name="mtjson",
-        dependencies=mtjson_deps,
+        # For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
+        dependencies=[],
         actions=[skip_manifest],
         title="Meshtastic Manifest (skipped)",
         description="mtjson generation is skipped for native environments",

bin/regen-fake-nodedbs.sh

@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+# Regenerate the fake-NodeDB fixtures: produces 250 / 500 / 1000 / 2000-node
+# JSONL seed files + their compiled v25 protobufs.
+#
+# Layout:
+#   test/fixtures/nodedb/seed_v25_<N>.jsonl   — COMMITTED, hand-editable.
+#   build/fixtures/nodedb/nodes_v25_<N>.proto — .gitignored, build artifact.
+#                                               Drop into /prefs/nodes.proto.
+#
+# Daily use:  ./bin/regen-fake-nodedbs.sh
+#   - Recompiles protos from committed seeds (fresh wall-clock timestamps).
+# Intentional seed bump:  REGEN_SEEDS=yes ./bin/regen-fake-nodedbs.sh
+#   - Overwrites the committed JSONL files with freshly-seeded data.
+
+set -euo pipefail
+cd "$(dirname "$0")/.."
+
+# 1) Make sure the Python protobuf bindings exist (in-tree generation; .gitignored).
+if [[ ! -d bin/_generated/meshtastic ]]; then
+    echo "regenerating Python protobuf bindings (one-time)..."
+    ./bin/regen-py-protos.sh
+fi
+
+# 2) Pick a Python interpreter that has the meshtastic deps installed.
+#    Prefer the mcp-server venv (most likely to be set up by the operator).
+PY="python3"
+for cand in mcp-server/.venv/bin/python3 .venv/bin/python3; do
+    if [[ -x "$cand" ]]; then
+        PY="$cand"
+        break
+    fi
+done
+
+# 3) Pinned seeds per size — bump only when you intentionally want different
+#    structural data committed. Parallel arrays so the script works on
+#    macOS bash 3.2 (no `declare -A`).
+SIZES=(250 500 1000 2000)
+SEEDS=(20260511 20260512 20260513 20260514)
+
+REGEN_SEEDS="${REGEN_SEEDS:-no}"
+
+mkdir -p build/fixtures/nodedb test/fixtures/nodedb
+
+for i in 0 1 2 3; do
+    n="${SIZES[$i]}"
+    seed="${SEEDS[$i]}"
+    jsonl=$(printf "test/fixtures/nodedb/seed_v25_%04d.jsonl" "$n")
+    proto=$(printf "build/fixtures/nodedb/nodes_v25_%04d.proto" "$n")
+
+    if [[ "$REGEN_SEEDS" == "yes" || ! -f "$jsonl" ]]; then
+        $PY bin/gen-fake-nodedb-seed.py \
+            --count "$n" \
+            --seed "$seed" \
+            --out "$jsonl" \
+            --centroid 33.1284,-107.2528 \
+            --spread-km 60 \
+            --position-coverage 0.85 \
+            --telemetry-coverage 0.70 \
+            --environment-coverage 0.25 \
+            --status-coverage 0.40
+        echo "  seed:   $jsonl ($(wc -c < "$jsonl") bytes)"
+    fi
+
+    $PY bin/seed-json-to-proto.py --in "$jsonl" --out "$proto"
+    echo "  proto:  $proto ($(wc -c < "$proto") bytes)"
+done
+
+echo ""
+echo "Done. To load on Portduino native:"
+echo "  cp build/fixtures/nodedb/nodes_v25_1000.proto ~/.portduino/default/prefs/nodes.proto"
+echo ""
+echo "To push to a hardware device:"
+echo "  Use the mcp-server tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"

bin/regen-py-protos.sh

@@ -0,0 +1,51 @@
+#!/usr/bin/env bash
+# Regenerate Python protobuf bindings from the in-tree `protobufs/` submodule
+# into `bin/_generated/`. Called by bin/regen-fake-nodedbs.sh; also useful as
+# a standalone refresh after any change to a .proto file.
+#
+# Output is .gitignored — bindings are a build artifact.
+#
+# Namespace rewrite:
+# The .proto files declare `package meshtastic;`, which makes protoc emit
+# imports like `from meshtastic import mesh_pb2`. That conflicts with the
+# PyPI `meshtastic` package (which the mcp-server relies on for its
+# SerialInterface/BLEInterface transport). We post-process the generated
+# files to live under `meshtastic_v25` instead — both the directory layout
+# and all internal imports — so they coexist cleanly with the PyPI package.
+
+set -euo pipefail
+cd "$(dirname "$0")/.."
+
+if ! command -v protoc >/dev/null 2>&1; then
+    echo "ERROR: protoc not found in PATH." >&2
+    echo "  macOS: brew install protobuf" >&2
+    echo "  Ubuntu/Debian: apt install protobuf-compiler" >&2
+    exit 1
+fi
+
+OUT=bin/_generated
+LOCAL_NS=meshtastic_v25
+
+rm -rf "$OUT"
+mkdir -p "$OUT"
+
+# 1) Generate from the in-tree protos. nanopb.proto first so its descriptor
+#    is available for the [(nanopb).*] options on other messages.
+protoc \
+    --proto_path=protobufs \
+    --python_out="$OUT" \
+    protobufs/nanopb.proto \
+    protobufs/meshtastic/*.proto
+
+# 2) Move the generated `meshtastic/` directory to `meshtastic_v25/`.
+mv "$OUT/meshtastic" "$OUT/$LOCAL_NS"
+
+# 3) Rewrite internal imports: any reference to `meshtastic.X_pb2` or
+#    `from meshtastic import X_pb2` becomes `meshtastic_v25.*`.
+python3 bin/_rewrite_proto_namespace.py "$OUT/$LOCAL_NS" "$LOCAL_NS"
+
+# 4) Make the package importable.
+touch "$OUT/__init__.py"
+touch "$OUT/$LOCAL_NS/__init__.py"
+
+echo "regenerated Python protobuf bindings -> $OUT/$LOCAL_NS/ (namespace: $LOCAL_NS)" >&2

bin/seed-json-to-proto.py

@@ -0,0 +1,342 @@
+#!/usr/bin/env python3
+"""Compile a committed seed JSONL into a binary meshtastic_NodeDatabase v25 proto.
+
+The input is produced by `bin/gen-fake-nodedb-seed.py`. Timestamps in the JSONL
+are stored as `*_offset_sec` (seconds before "now"); this script resolves them
+to absolute epochs using `--now-epoch` (default: current wall clock).
+
+Output is a raw `pb_encode`-compatible binary that can be dropped at
+`/prefs/nodes.proto` on the device (Portduino prefs dir or hardware via
+XModem) and loaded by `NodeDB::loadFromDisk` at boot.
+
+Wire format reference:
+  protobufs/meshtastic/deviceonly.proto  (NodeDatabase, NodeInfoLite, sat entries)
+  src/mesh/NodeDB.h:467-484              (bitfield bit positions)
+  src/mesh/NodeDB.cpp:1523-1524          (pb_decode entry point)
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import pathlib
+import sys
+import time
+from typing import Any
+
+# Prefer the in-tree generated Python protobuf bindings (bin/_generated/meshtastic_v25/)
+# because the firmware branch's protos (v25 NodeDatabase satellite arrays, slim
+# NodeInfoLite) are typically newer than what the PyPI `meshtastic` package
+# ships. Run `bin/regen-py-protos.sh` to (re)generate.
+#
+# Namespace note: the local bindings live under `meshtastic_v25` (NOT `meshtastic`)
+# to avoid shadowing the PyPI `meshtastic` package — bin/regen-py-protos.sh
+# post-processes the protoc output to rename the package.
+_HERE = pathlib.Path(__file__).resolve().parent
+_LOCAL_PROTO_DIR = _HERE / "_generated"
+if _LOCAL_PROTO_DIR.is_dir():
+    sys.path.insert(0, str(_LOCAL_PROTO_DIR))
+
+try:
+    from meshtastic_v25.deviceonly_pb2 import (  # type: ignore[import-not-found]
+        NodeDatabase,
+        NodeInfoLite,
+        NodePositionEntry,
+        NodeTelemetryEntry,
+        NodeEnvironmentEntry,
+        NodeStatusEntry,
+        PositionLite,
+    )
+    from meshtastic_v25.mesh_pb2 import HardwareModel, Position, StatusMessage  # type: ignore[import-not-found]
+    from meshtastic_v25.config_pb2 import Config  # type: ignore[import-not-found]
+    from meshtastic_v25.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics  # type: ignore[import-not-found]
+except ImportError as local_err:
+    # Fall back to the PyPI package if in-tree bindings haven't been generated.
+    # Will fail the v25 assertion below if the PyPI package predates the
+    # satellite-DB schema, but at least gives a clear "run regen-py-protos.sh"
+    # error message instead of an opaque ImportError.
+    try:
+        from meshtastic.protobuf.deviceonly_pb2 import (
+            NodeDatabase,
+            NodeInfoLite,
+            NodePositionEntry,
+            NodeTelemetryEntry,
+            NodeEnvironmentEntry,
+            NodeStatusEntry,
+            PositionLite,
+        )
+        from meshtastic.protobuf.mesh_pb2 import HardwareModel, Position, StatusMessage
+        from meshtastic.protobuf.config_pb2 import Config
+        from meshtastic.protobuf.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics
+    except ImportError as pypi_err:
+        print(
+            "ERROR: could not import meshtastic protobuf bindings.\n"
+            "  In-tree generation: run `bin/regen-py-protos.sh` (requires protoc).\n"
+            "  PyPI fallback: `pip install meshtastic` (may lag firmware branch).\n"
+            f"  local error (meshtastic_v25): {local_err}\n"
+            f"  pypi error  (meshtastic.protobuf): {pypi_err}",
+            file=sys.stderr,
+        )
+        sys.exit(1)
+
+# Fail loudly if bindings predate v25 (no satellite arrays).
+assert (
+    hasattr(NodeDatabase, "DESCRIPTOR")
+    and "positions" in NodeDatabase.DESCRIPTOR.fields_by_name
+), (
+    "Loaded meshtastic bindings are older than v25 (NodeDatabase.positions missing). "
+    "Run `bin/regen-py-protos.sh` against the in-tree protobufs/ submodule."
+)
+
+# ---------------------------------------------------------------------------
+# Bitfield bit positions (mirror src/mesh/NodeDB.h:467-484).
+# ---------------------------------------------------------------------------
+BIT_IS_KEY_MANUALLY_VERIFIED = 0
+BIT_IS_MUTED = 1
+BIT_VIA_MQTT = 2
+BIT_IS_FAVORITE = 3
+BIT_IS_IGNORED = 4
+BIT_HAS_USER = 5
+BIT_IS_LICENSED = 6
+BIT_IS_UNMESSAGABLE = 7
+BIT_HAS_IS_UNMESSAGABLE = 8
+
+BITFIELD_LAYOUT = (
+    # JSON key            bit position
+    ("is_key_manually_verified", BIT_IS_KEY_MANUALLY_VERIFIED),
+    ("is_muted", BIT_IS_MUTED),
+    ("via_mqtt", BIT_VIA_MQTT),
+    ("is_favorite", BIT_IS_FAVORITE),
+    ("is_ignored", BIT_IS_IGNORED),
+    ("has_user", BIT_HAS_USER),
+    ("is_licensed", BIT_IS_LICENSED),
+    ("is_unmessagable", BIT_IS_UNMESSAGABLE),
+    ("has_is_unmessagable", BIT_HAS_IS_UNMESSAGABLE),
+)
+
+
+def _pack_bitfield(bf: dict[str, bool]) -> int:
+    out = 0
+    for key, shift in BITFIELD_LAYOUT:
+        if bf.get(key, False):
+            out |= (1 << shift)
+    return out
+
+
+def _validate_node(node: dict[str, Any]) -> None:
+    """Friendly errors so hand-editors get clear feedback."""
+    if "num" not in node or not isinstance(node["num"], str):
+        raise ValueError(f"node missing/invalid 'num' (must be hex string): {node!r}")
+    if "long_name" not in node:
+        raise ValueError(f"node {node['num']}: missing 'long_name'")
+    if len(node["long_name"]) > 24:
+        raise ValueError(
+            f"node {node['num']}: long_name {node['long_name']!r} is "
+            f"{len(node['long_name'])} chars; max 24 (nanopb max_size:25 minus NUL)"
+        )
+    if "short_name" in node:
+        # short_name max_size:5 (incl. NUL) → 4 bytes of content.
+        # Char count is irrelevant — emojis with variation selectors (e.g. ❄️ = 6 B)
+        # would slip past a `len(str) > 4` check. Always measure bytes.
+        b = node["short_name"].encode("utf-8")
+        if len(b) > 4:
+            raise ValueError(
+                f"node {node['num']}: short_name {node['short_name']!r} is "
+                f"{len(b)} bytes UTF-8; max 4 (nanopb max_size:5 minus NUL)"
+            )
+    pk = node.get("public_key_hex", "")
+    if pk and len(pk) != 64:
+        raise ValueError(
+            f"node {node['num']}: public_key_hex must be 64 hex chars or empty; "
+            f"got {len(pk)} chars"
+        )
+    if pk:
+        try:
+            bytes.fromhex(pk)
+        except ValueError as e:
+            raise ValueError(f"node {node['num']}: public_key_hex is not valid hex: {e}")
+
+
+def _resolve_time(
+    node: dict[str, Any],
+    field_absolute: str,
+    field_offset: str,
+    now_epoch: int,
+) -> int:
+    """If `field_absolute` is set, use it; else compute `now_epoch - offset`."""
+    if field_absolute in node and node[field_absolute] is not None:
+        return int(node[field_absolute])
+    offset = node.get(field_offset, 0)
+    return max(0, int(now_epoch) - int(offset))
+
+
+def _build_node_info_lite(node: dict[str, Any], now_epoch: int) -> NodeInfoLite:
+    _validate_node(node)
+    info = NodeInfoLite()
+    info.num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
+    info.long_name = node.get("long_name", "")
+    info.short_name = node.get("short_name", "")
+    # Enum lookups will raise ValueError on unknown names — that's exactly what we want.
+    info.hw_model = HardwareModel.Value(node.get("hw_model", "UNSET"))
+    info.role = Config.DeviceConfig.Role.Value(node.get("role", "CLIENT"))
+    pk_hex = node.get("public_key_hex", "")
+    if pk_hex:
+        info.public_key = bytes.fromhex(pk_hex)
+    info.snr = float(node.get("snr", 0.0))
+    info.channel = int(node.get("channel", 0))
+    if "hops_away" in node:
+        # `optional uint32 hops_away = 9;` — in Python protobuf, assigning the
+        # field implicitly sets HasField("hops_away") to True. No has_hops_away
+        # setter exists (unlike the C++ nanopb-generated header).
+        info.hops_away = int(node["hops_away"])
+    info.next_hop = int(node.get("next_hop", 0))
+    info.last_heard = _resolve_time(node, "last_heard", "last_heard_offset_sec", now_epoch)
+    info.bitfield = _pack_bitfield(node.get("bitfield", {}))
+    return info
+
+
+def _build_position_entry(num: int, pos: dict[str, Any], now_epoch: int) -> NodePositionEntry:
+    entry = NodePositionEntry()
+    entry.num = num
+    pl = PositionLite()
+    # Firmware stores lat/long as int32 in 1e-7 degrees.
+    pl.latitude_i = int(round(float(pos["latitude"]) * 1e7))
+    pl.longitude_i = int(round(float(pos["longitude"]) * 1e7))
+    pl.altitude = int(pos.get("altitude", 0))
+    pl.time = _resolve_time(pos, "time", "time_offset_sec", now_epoch)
+    pl.location_source = Position.LocSource.Value(pos.get("location_source", "LOC_UNSET"))
+    entry.position.CopyFrom(pl)
+    return entry
+
+
+def _build_telemetry_entry(num: int, tel: dict[str, Any]) -> NodeTelemetryEntry:
+    entry = NodeTelemetryEntry()
+    entry.num = num
+    dm = DeviceMetrics()
+    if "battery_level" in tel:
+        dm.battery_level = int(tel["battery_level"])
+    if "voltage" in tel:
+        dm.voltage = float(tel["voltage"])
+    if "channel_utilization" in tel:
+        dm.channel_utilization = float(tel["channel_utilization"])
+    if "air_util_tx" in tel:
+        dm.air_util_tx = float(tel["air_util_tx"])
+    if "uptime_seconds" in tel:
+        dm.uptime_seconds = int(tel["uptime_seconds"])
+    entry.device_metrics.CopyFrom(dm)
+    return entry
+
+
+def _build_environment_entry(num: int, env: dict[str, Any]) -> NodeEnvironmentEntry:
+    entry = NodeEnvironmentEntry()
+    entry.num = num
+    em = EnvironmentMetrics()
+    if "temperature" in env:
+        em.temperature = float(env["temperature"])
+    if "relative_humidity" in env:
+        em.relative_humidity = float(env["relative_humidity"])
+    if "barometric_pressure" in env:
+        em.barometric_pressure = float(env["barometric_pressure"])
+    if "iaq" in env:
+        em.iaq = int(env["iaq"])
+    entry.environment_metrics.CopyFrom(em)
+    return entry
+
+
+def _build_status_entry(num: int, status: dict[str, Any]) -> NodeStatusEntry:
+    # `StatusMessage` (mesh.proto:1445) has a single `string status` field.
+    entry = NodeStatusEntry()
+    entry.num = num
+    sm = StatusMessage()
+    if "status" in status:
+        sm.status = str(status["status"])
+    entry.status.CopyFrom(sm)
+    return entry
+
+
+def compile_jsonl_to_proto(jsonl_path: pathlib.Path, now_epoch: int) -> bytes:
+    """Read a seed JSONL and return the encoded NodeDatabase bytes."""
+    lines = jsonl_path.read_text(encoding="utf-8").splitlines()
+    if not lines:
+        raise ValueError(f"{jsonl_path} is empty")
+    meta_line = lines[0]
+    meta_obj = json.loads(meta_line)
+    meta = meta_obj.get("_meta", {})
+    version = meta.get("version")
+    if version != 25:
+        raise ValueError(
+            f"{jsonl_path}: meta version is {version!r}; this compiler "
+            f"requires version=25. Regenerate the seed with the matching tooling."
+        )
+
+    db = NodeDatabase()
+    db.version = 25
+
+    for ln, raw in enumerate(lines[1:], start=2):
+        raw = raw.strip()
+        if not raw:
+            continue
+        try:
+            node = json.loads(raw)
+        except json.JSONDecodeError as e:
+            raise ValueError(f"{jsonl_path}:{ln} JSON parse error: {e}")
+
+        num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
+
+        # Header
+        info = _build_node_info_lite(node, now_epoch)
+        db.nodes.append(info)
+
+        # Satellites (nullable)
+        if node.get("position"):
+            db.positions.append(_build_position_entry(num, node["position"], now_epoch))
+        if node.get("telemetry"):
+            db.telemetry.append(_build_telemetry_entry(num, node["telemetry"]))
+        if node.get("environment"):
+            db.environment.append(_build_environment_entry(num, node["environment"]))
+        if node.get("status"):
+            db.status.append(_build_status_entry(num, node["status"]))
+
+    return db.SerializeToString()
+
+
+def main(argv: list[str]) -> int:
+    p = argparse.ArgumentParser(
+        description="Compile a seed JSONL into a binary v25 NodeDatabase proto.",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    p.add_argument("--in", dest="in_path", required=True, help="Input seed JSONL.")
+    p.add_argument("--out", required=True, help="Output binary .proto path.")
+    p.add_argument(
+        "--now-epoch",
+        type=int,
+        default=None,
+        help="Pin 'now' to this Unix epoch (for byte-identical CI). Default: time.time().",
+    )
+    args = p.parse_args(argv)
+
+    in_path = pathlib.Path(args.in_path)
+    if not in_path.is_file():
+        print(f"input not found: {in_path}", file=sys.stderr)
+        return 2
+
+    now_epoch = args.now_epoch if args.now_epoch is not None else int(time.time())
+
+    try:
+        encoded = compile_jsonl_to_proto(in_path, now_epoch)
+    except ValueError as e:
+        print(f"ERROR: {e}", file=sys.stderr)
+        return 3
+
+    out_path = pathlib.Path(args.out)
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    out_path.write_bytes(encoded)
+    print(
+        f"compiled {in_path} -> {out_path} ({len(encoded)} bytes, now_epoch={now_epoch})",
+        file=sys.stderr,
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))

boards/CDEBYTE_EoRa-S3.json

@@ -7,7 +7,7 @@
     "extra_flags": [
       "-D CDEBYTE_EORA_S3",
       "-D ARDUINO_USB_CDC_ON_BOOT=1",
-      "-D ARDUINO_USB_MODE=0",
+      "-D ARDUINO_USB_MODE=1",
       "-D ARDUINO_RUNNING_CORE=1",
       "-D ARDUINO_EVENT_RUNNING_CORE=1",
       "-D BOARD_HAS_PSRAM"

boards/ThinkNode-M7.json

@@ -0,0 +1,42 @@
+{
+  "build": {
+    "arduino": {
+      "ldscript": "esp32s3_out.ld",
+      "memory_type": "qio_opi"
+    },
+    "core": "esp32",
+    "extra_flags": [
+      "-D BOARD_HAS_PSRAM",
+      "-D ARDUINO_USB_CDC_ON_BOOT=0",
+      "-D ARDUINO_USB_MODE=0",
+      "-D ARDUINO_RUNNING_CORE=1",
+      "-D ARDUINO_EVENT_RUNNING_CORE=0"
+    ],
+    "f_cpu": "240000000L",
+    "f_flash": "80000000L",
+    "flash_mode": "qio",
+    "psram_type": "qio_opi",
+    "hwids": [["0x303A", "0x1001"]],
+    "mcu": "esp32s3",
+    "variant": "ELECROW-ThinkNode-M7"
+  },
+  "connectivity": ["wifi", "bluetooth", "lora"],
+  "debug": {
+    "default_tool": "esp-builtin",
+    "onboard_tools": ["esp-builtin"],
+    "openocd_target": "esp32s3.cfg"
+  },
+  "frameworks": ["arduino", "espidf"],
+  "name": "ELECROW ThinkNode M7",
+  "upload": {
+    "flash_size": "8MB",
+    "maximum_ram_size": 524288,
+    "maximum_size": 8388608,
+    "use_1200bps_touch": true,
+    "wait_for_upload_port": true,
+    "require_upload_port": true,
+    "speed": 921600
+  },
+  "url": "https://www.elecrow.com",
+  "vendor": "ELECROW"
+}

boards/bpi_picow_esp32_s3.json

@@ -6,7 +6,7 @@
     "core": "esp32",
     "extra_flags": [
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1",
       "-DBOARD_HAS_PSRAM"

boards/hackaday-communicator.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_vision_master_e213.json

@@ -9,7 +9,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_vision_master_e290.json

@@ -9,7 +9,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_vision_master_t190.json

@@ -9,7 +9,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_wireless_tracker.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DHELTEC_WIRELESS_TRACKER",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_wireless_tracker_v2.json

@@ -7,7 +7,7 @@
     "core": "esp32",
     "extra_flags": [
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/nrf54l15dk.json

@@ -0,0 +1,26 @@
+{
+  "build": {
+    "cpu": "cortex-m33",
+    "f_cpu": "128000000L",
+    "mcu": "nrf54l15",
+    "zephyr": {
+      "variant": "nrf54l15dk/nrf54l15/cpuapp"
+    }
+  },
+  "connectivity": ["bluetooth"],
+  "debug": {
+    "default_tools": ["jlink"],
+    "jlink_device": "nRF54L15_M33",
+    "svd_path": "nrf54l15.svd"
+  },
+  "frameworks": ["zephyr"],
+  "name": "Nordic nRF54L15-DK (PCA10156)",
+  "upload": {
+    "maximum_ram_size": 262144,
+    "maximum_size": 1572864,
+    "protocol": "jlink",
+    "protocols": ["jlink"]
+  },
+  "url": "https://www.nordicsemi.com/Products/nRF54L15",
+  "vendor": "Nordic Semiconductor"
+}

boards/seeed-xiao-s3.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=0"
     ],

boards/station-g3.json

@@ -0,0 +1,41 @@
+{
+  "build": {
+    "arduino": {
+      "ldscript": "esp32s3_out.ld",
+      "memory_type": "qio_opi"
+    },
+    "core": "esp32",
+    "extra_flags": [
+      "-DBOARD_HAS_PSRAM",
+      "-DARDUINO_USB_CDC_ON_BOOT=1",
+      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_RUNNING_CORE=1",
+      "-DARDUINO_EVENT_RUNNING_CORE=0"
+    ],
+    "f_cpu": "240000000L",
+    "f_flash": "80000000L",
+    "flash_mode": "qio",
+    "hwids": [["0x303A", "0x1001"]],
+    "mcu": "esp32s3",
+    "variant": "station-g3"
+  },
+  "connectivity": ["wifi", "bluetooth", "lora"],
+  "debug": {
+    "default_tool": "esp-builtin",
+    "onboard_tools": ["esp-builtin"],
+    "openocd_target": "esp32s3.cfg"
+  },
+  "frameworks": ["arduino", "espidf"],
+  "name": "BQ Station G3",
+  "upload": {
+    "flash_size": "16MB",
+    "maximum_ram_size": 327680,
+    "maximum_size": 16777216,
+    "use_1200bps_touch": true,
+    "wait_for_upload_port": true,
+    "require_upload_port": true,
+    "speed": 921600
+  },
+  "url": "",
+  "vendor": "BQ Consulting"
+}

boards/t-beam-1w.json

@@ -9,7 +9,7 @@
       "-DBOARD_HAS_PSRAM",
       "-DLILYGO_TBEAM_1W",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/t-deck.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/tbeam-s3-core.json

@@ -8,7 +8,7 @@
       "-DBOARD_HAS_PSRAM",
       "-DLILYGO_TBEAM_S3_CORE",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/tlora-t3s3-v1.json

@@ -7,7 +7,7 @@
     "extra_flags": [
       "-DLILYGO_T3S3_V1",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1",
       "-DBOARD_HAS_PSRAM"

boards/unphone.json

@@ -10,7 +10,7 @@
       "-DBOARD_HAS_PSRAM",
       "-DUNPHONE_SPIN=9",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=0",
+      "-DARDUINO_USB_MODE=1",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

default_16MB.csv

@@ -0,0 +1,7 @@
+# Name,   Type, SubType, Offset,  Size, Flags
+nvs,      data, nvs,     0x9000,  0x5000,
+otadata,  data, ota,     0xe000,  0x2000,
+app0,     app,  ota_0,   0x10000, 0x640000,
+app1,     app,  ota_1,   0x650000,0x640000,
+spiffs,   data, spiffs,  0xc90000,0x360000,
+coredump, data, coredump,0xFF0000,0x10000,

default_8MB.csv

@@ -0,0 +1,7 @@
+# Name,   Type, SubType, Offset,  Size, Flags
+nvs,      data, nvs,     0x9000,  0x5000,
+otadata,  data, ota,     0xe000,  0x2000,
+app0,     app,  ota_0,   0x10000, 0x330000,
+app1,     app,  ota_1,   0x340000,0x330000,
+spiffs,   data, spiffs,  0x670000,0x180000,
+coredump, data, coredump,0x7F0000,0x10000,

extra_scripts/esp32_extra.py

@@ -70,17 +70,6 @@ def esp32_create_combined_bin(source, target, env):
 
 env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_combined_bin)
 
-esp32_kind = env.GetProjectOption("custom_esp32_kind")
-if esp32_kind == "esp32":
-    # Free up some IRAM by removing auxiliary SPI flash chip drivers.
-    # Wrapped stub symbols are defined in src/platform/esp32/iram-quirk.c.
-    env.Append(
-        LINKFLAGS=[
-            "-Wl,--wrap=esp_flash_chip_gd",
-            "-Wl,--wrap=esp_flash_chip_issi",
-            "-Wl,--wrap=esp_flash_chip_winbond",
-        ]
-    )
-else:
-    # For newer ESP32 targets, using newlib nano works better.
-    env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])
+# Enable Newlib Nano formatting to save space
+# ...but allow printf float support (compromise)
+env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])

extra_scripts/ld_response_file.py

@@ -0,0 +1,23 @@
+#!/usr/bin/env python3
+# trunk-ignore-all(ruff/F821)
+# trunk-ignore-all(flake8/F821): For SConstruct imports
+
+# force linker response file instead of command line arguments
+
+Import("env")
+
+
+def wrap_with_tempfile(command_key):
+    command = env.get(command_key)
+    if not command or not isinstance(command, str):
+        return
+    if "TEMPFILE(" in command:
+        return
+    env.Replace(**{command_key: "${TEMPFILE('%s')}" % command})
+
+
+# Force SCons to spill long commands into response files on this target.
+env.Replace(MAXLINELENGTH=8192)
+
+for key in ("LINKCOM", "CXXLINKCOM", "SHLINKCOM", "SHCXXLINKCOM"):
+    wrap_with_tempfile(key)

extra_scripts/nrf54l15_linker.py

@@ -0,0 +1,140 @@
+#!/usr/bin/env python3
+# trunk-ignore-all(ruff/F821)
+# trunk-ignore-all(flake8/F821): For SConstruct imports
+#
+# post:extra_scripts/nrf54l15_linker.py
+#
+# Fix for Zephyr two-pass link on nRF54L15:
+# platformio-build.py registers env.Depends("$PROG_PATH", final_ld_script) but
+# the SCons dependency chain is broken (final_ld_script Command never runs).
+# This script adds a PreAction on the final firmware binary that runs the gcc
+# preprocessing command directly (extracted from build.ninja) to generate
+# zephyr/linker.cmd before the link step.
+#
+# PlatformIO bundles an old Ninja that can't handle multi-output depslog rules,
+# so we parse the COMMAND line from build.ninja and run just the gcc -E part,
+# skipping the cmake_transform_depfile step (only needed for Ninja deps tracking).
+
+import os
+import re
+import subprocess
+
+Import("env")
+
+if env.get("PIOENV") != "nrf54l15dk":
+    pass  # Only for the nrf54l15dk environment
+else:
+
+    def _extract_gcc_command(ninja_build):
+        """Parse build.ninja to find the gcc -E command that generates linker.cmd.
+
+        The rule format depends on the host:
+          Windows (CMake's RunCMake wraps every command):
+            COMMAND = cmd.exe /C "cd /D DIR && arm-none-eabi-gcc.exe ... -o linker.cmd && cmake.exe -E cmake_transform_depfile ..."
+          POSIX (Linux/macOS — no wrapper):
+            COMMAND = cd DIR && arm-none-eabi-gcc ... -o linker.cmd && cmake -E cmake_transform_depfile ...
+
+        Returns (gcc_cmd_string, cwd_path) or raises RuntimeError.
+        """
+        in_rule = False
+        with open(ninja_build, "r", encoding="utf-8", errors="replace") as f:
+            for line in f:
+                # Detect start of the linker.cmd custom command rule
+                if not in_rule:
+                    if "build zephyr/linker.cmd" in line and "CUSTOM_COMMAND" in line:
+                        in_rule = True
+                    continue
+
+                stripped = line.strip()
+                if not stripped.startswith("COMMAND = "):
+                    continue
+
+                command_val = stripped[len("COMMAND = ") :]
+
+                # On Windows the value is wrapped in `cmd.exe /C "..."` — strip
+                # the wrapper. On POSIX hosts the inner sequence is the value
+                # itself (no quoting layer).
+                m = re.search(r'/C\s+"(.*)"\s*$', command_val)
+                inner = m.group(1) if m else command_val
+                parts = inner.split(" && ")
+
+                cwd = None
+                gcc_cmd = None
+                for part in parts:
+                    part = part.strip()
+                    if part.startswith("cd /D "):  # Windows form
+                        cwd = part[len("cd /D ") :]
+                    elif part.startswith("cd "):  # POSIX form
+                        cwd = part[len("cd ") :]
+                    elif "arm-none-eabi-gcc" in part:
+                        gcc_cmd = part
+
+                if not gcc_cmd:
+                    raise RuntimeError(
+                        "nRF54L15 linker fix: arm-none-eabi-gcc command not found in:\n%s"
+                        % inner[:400]
+                    )
+
+                return gcc_cmd, cwd
+
+        raise RuntimeError(
+            "nRF54L15 linker fix: 'build zephyr/linker.cmd' rule not found in build.ninja"
+        )
+
+    def _generate_linker_cmd(target, source, env):
+        """Generate zephyr/linker.cmd via direct gcc invocation before the final link."""
+        build_dir = env.subst("$BUILD_DIR")
+        zephyr_dir = os.path.join(build_dir, "zephyr")
+        linker_cmd = os.path.join(zephyr_dir, "linker.cmd")
+
+        if os.path.exists(linker_cmd):
+            return  # Already present — nothing to do
+
+        ninja_build = os.path.join(build_dir, "build.ninja")
+        if not os.path.exists(ninja_build):
+            raise RuntimeError(
+                "nRF54L15 linker fix: build.ninja not found at %s\n"
+                "Run a full build first so CMake generates the Ninja files."
+                % ninja_build
+            )
+
+        gcc_cmd, cwd = _extract_gcc_command(ninja_build)
+        run_cwd = cwd if cwd else zephyr_dir
+
+        print(
+            "==> nRF54L15: Generating zephyr/linker.cmd (LINKER_ZEPHYR_FINAL) via GCC"
+        )
+        # gcc_cmd comes verbatim from our own build.ninja (never user input) and
+        # contains Windows-style paths with spaces that cannot be safely argv-split
+        # with shlex, so we run it via the platform shell. nosec/nosemgrep below
+        # acknowledge this deliberate, scoped use of shell=True.
+        result = subprocess.run(  # nosec B602
+            gcc_cmd,
+            shell=True,  # nosemgrep: python.lang.security.audit.subprocess-shell-true.subprocess-shell-true
+            cwd=run_cwd,
+            capture_output=True,
+            text=True,
+        )
+        if result.returncode != 0:
+            print("GCC stdout:", result.stdout[:2000])
+            print("GCC stderr:", result.stderr[:2000])
+            raise RuntimeError(
+                "nRF54L15 linker fix: GCC failed to generate linker.cmd (rc=%d)"
+                % result.returncode
+            )
+        if not os.path.exists(linker_cmd):
+            raise RuntimeError(
+                "nRF54L15 linker fix: GCC returned 0 but linker.cmd was not created at %s"
+                % linker_cmd
+            )
+        print("==> linker.cmd generated successfully")
+
+    # Use PIOMAINPROG (set by ZephyrBuildProgram) to get the exact SCons node
+    prog = env.get("PIOMAINPROG")
+    if prog:
+        env.AddPreAction(prog, _generate_linker_cmd)
+    else:
+        print(
+            "[nrf54l15_linker] WARNING: PIOMAINPROG not set, falling back to $PROG_PATH"
+        )
+        env.AddPreAction(env.subst("$PROG_PATH"), _generate_linker_cmd)

mcp-server/.gitignore

@@ -7,6 +7,12 @@ __pycache__/
 dist/
 build/
 
+# Persistent device-log capture (recorder + Datadog cursor).
+# Cross-session JSONL streams written by the autouse Recorder singleton
+# (see src/meshtastic_mcp/recorder/). Lives outside tests/ so the pytest
+# fixture truncate doesn't touch it.
+.mtlog/
+
 # Test harness artifacts
 tests/report.html
 tests/junit.xml

mcp-server/scripts/datadog-dashboard.json

@@ -0,0 +1,217 @@
+{
+  "title": "Meshtastic Firmware — Recorder Stream",
+  "description": "Live view of `.mtlog/` streams shipped by `mtlog_to_datadog.py`. Heap, packet volume, log levels, errors. One row per port.",
+  "widgets": [
+    {
+      "definition": {
+        "title": "Free heap (bytes)",
+        "type": "timeseries",
+        "show_legend": true,
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "free_heap",
+                "data_source": "metrics",
+                "query": "avg:mesh.local.heap_free_bytes{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ],
+        "yaxis": { "label": "bytes" }
+      }
+    },
+    {
+      "definition": {
+        "title": "Heap slope (bytes/min) — last 1h",
+        "type": "query_value",
+        "precision": 0,
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "slope",
+                "data_source": "metrics",
+                "query": "derivative(avg:mesh.local.heap_free_bytes{service:meshtastic-firmware})",
+                "aggregator": "avg"
+              }
+            ],
+            "response_format": "scalar"
+          }
+        ],
+        "conditional_formats": [
+          { "comparator": "<", "value": -100, "palette": "white_on_red" },
+          { "comparator": "<", "value": 0, "palette": "white_on_yellow" },
+          { "comparator": ">=", "value": 0, "palette": "white_on_green" }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Total heap (bytes)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "total_heap",
+                "data_source": "metrics",
+                "query": "avg:mesh.local.heap_total_bytes{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Battery level (%)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "battery",
+                "data_source": "metrics",
+                "query": "avg:mesh.device.battery_level{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ],
+        "yaxis": { "min": "0", "max": "105" }
+      }
+    },
+    {
+      "definition": {
+        "title": "Air utilization (TX %)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "airutil",
+                "data_source": "metrics",
+                "query": "avg:mesh.device.air_util_tx{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Channel utilization (%)",
+        "type": "timeseries",
+        "requests": [
+          {
+            "queries": [
+              {
+                "name": "chutil",
+                "data_source": "metrics",
+                "query": "avg:mesh.device.channel_utilization{service:meshtastic-firmware} by {port}"
+              }
+            ],
+            "response_format": "timeseries",
+            "display_type": "line"
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Log volume by level",
+        "type": "timeseries",
+        "show_legend": true,
+        "requests": [
+          {
+            "response_format": "timeseries",
+            "display_type": "bars",
+            "queries": [
+              {
+                "name": "log_count",
+                "data_source": "logs",
+                "indexes": ["*"],
+                "compute": { "aggregation": "count" },
+                "search": { "query": "service:meshtastic-firmware" },
+                "group_by": [
+                  {
+                    "facet": "@level",
+                    "limit": 10,
+                    "sort": { "order": "desc", "aggregation": "count" }
+                  }
+                ]
+              }
+            ]
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Recent ERROR / CRIT firmware logs",
+        "type": "list_stream",
+        "requests": [
+          {
+            "response_format": "event_list",
+            "query": {
+              "data_source": "logs_stream",
+              "query_string": "service:meshtastic-firmware (status:error OR @level:ERROR OR @level:CRIT)",
+              "indexes": [],
+              "sort": { "column": "timestamp", "order": "desc" }
+            },
+            "columns": [
+              { "field": "timestamp", "width": "auto" },
+              { "field": "host", "width": "auto" },
+              { "field": "@port", "width": "auto" },
+              { "field": "@level", "width": "auto" },
+              { "field": "@thread", "width": "auto" },
+              { "field": "message", "width": "stretch" }
+            ]
+          }
+        ]
+      }
+    },
+    {
+      "definition": {
+        "title": "Recorder marker events",
+        "type": "list_stream",
+        "requests": [
+          {
+            "response_format": "event_list",
+            "query": {
+              "data_source": "logs_stream",
+              "query_string": "service:meshtastic-firmware @level:MARK",
+              "indexes": [],
+              "sort": { "column": "timestamp", "order": "desc" }
+            },
+            "columns": [
+              { "field": "timestamp", "width": "auto" },
+              { "field": "host", "width": "auto" },
+              { "field": "message", "width": "stretch" }
+            ]
+          }
+        ]
+      }
+    }
+  ],
+  "template_variables": [
+    {
+      "name": "port",
+      "prefix": "port",
+      "available_values": [],
+      "default": "*"
+    },
+    { "name": "host", "prefix": "host", "available_values": [], "default": "*" }
+  ],
+  "layout_type": "ordered",
+  "notify_list": [],
+  "reflow_type": "auto"
+}

mcp-server/scripts/mtlog_to_datadog.py

@@ -0,0 +1,389 @@
+#!/usr/bin/env python3
+"""Forward selected recorder JSONL streams to Datadog.
+
+Reads `.mtlog/logs.jsonl` and `.mtlog/telemetry.jsonl`, ships logs to the
+Logs Intake API and telemetry numerics to the Metrics v2 series API.
+Resumes from `.mtlog/.dd-cursor.json` so a daemon restart doesn't
+duplicate rows already shipped from the current live files.
+
+This forwarder does not currently backfill rotated `.jsonl.gz` archives.
+If the recorder rotates before this process drains the live file, or the
+forwarder is down across a rotation, those older rows are skipped.
+
+Usage:
+    DD_API_KEY=... ./scripts/mtlog_to_datadog.py --tail
+    ./scripts/mtlog_to_datadog.py --once          # catch up + exit
+    ./scripts/mtlog_to_datadog.py --since 3600   # backfill last hour from start
+
+Default `DD_SITE` is `us5.datadoghq.com` — the team's Datadog instance.
+Override via `DD_SITE=...` env var or `--site` flag for one-offs.
+
+The forwarder is a separate process by design — a Datadog outage or
+auth failure must not backpressure the recorder. We exit non-zero on
+fatal config errors (missing API key) and keep retrying on transient
+network/HTTP errors.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import socket
+import sys
+import time
+from pathlib import Path
+from typing import Any, Iterator
+
+try:
+    import requests
+except ImportError:
+    print(
+        "requests is required. Install it in the mcp-server venv: "
+        "uv pip install requests",
+        file=sys.stderr,
+    )
+    sys.exit(2)
+
+
+_DEFAULT_LOG_DIR = Path(__file__).resolve().parents[1] / ".mtlog"
+_LOG_INTAKE_TPL = "https://http-intake.logs.{site}/api/v2/logs"
+_METRICS_TPL = "https://api.{site}/api/v2/series"
+_LOG_BATCH = 50
+_METRICS_BATCH = 100
+_MAX_RETRIES = 5
+_RETRY_BASE_S = 1.5
+
+
+# --- streaming JSONL with byte-position cursor -------------------------
+
+
+class _StreamReader:
+    """Reads a single rotating JSONL with cursor-based resume.
+
+    This tails only the live `.jsonl` file. The recorder rotates files
+    (live `.jsonl` → `.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`), which means
+    the live file shrinks abruptly. We detect that via inode change OR live
+    size < cursor position, and reset the live-file cursor to 0.
+    """
+
+    def __init__(self, path: Path, cursor: dict[str, Any]):
+        self.path = path
+        self.cursor = cursor
+
+    def _state(self) -> tuple[int, int]:
+        """Return (inode, size) for the live file. (0, 0) if missing."""
+        try:
+            st = self.path.stat()
+            return (st.st_ino, st.st_size)
+        except FileNotFoundError:
+            return (0, 0)
+
+    def iter_new_records(self) -> Iterator[dict[str, Any]]:
+        ino, size = self._state()
+        last_ino = self.cursor.get("ino")
+        last_pos = int(self.cursor.get("pos") or 0)
+        if ino == 0:
+            return
+        if last_ino is not None and last_ino != ino:
+            # Rotation happened. Start over.
+            last_pos = 0
+        if last_pos > size:
+            # Live file truncated/shrunk under us — recorder rotated.
+            last_pos = 0
+        try:
+            with self.path.open("r", encoding="utf-8") as fh:
+                fh.seek(last_pos)
+                for line in fh:
+                    line = line.rstrip("\n")
+                    if not line:
+                        continue
+                    try:
+                        yield json.loads(line)
+                    except json.JSONDecodeError:
+                        continue
+                    last_pos = fh.tell()
+        except FileNotFoundError:
+            return
+        self.cursor["ino"] = ino
+        self.cursor["pos"] = last_pos
+
+
+def _load_cursor(path: Path) -> dict[str, Any]:
+    if not path.exists():
+        return {}
+    try:
+        return json.loads(path.read_text())
+    except (OSError, json.JSONDecodeError):
+        return {}
+
+
+def _save_cursor(path: Path, data: dict[str, Any]) -> None:
+    tmp = path.with_suffix(".json.tmp")
+    tmp.write_text(json.dumps(data, separators=(",", ":")))
+    tmp.replace(path)
+
+
+# --- Datadog clients ---------------------------------------------------
+
+
+class _DDSession:
+    """Pool one HTTPS session, share retry logic."""
+
+    def __init__(self, api_key: str, site: str, hostname: str) -> None:
+        self.api_key = api_key
+        self.site = site
+        self.hostname = hostname
+        self.session = requests.Session()
+        self.session.headers.update(
+            {
+                "DD-API-KEY": api_key,
+                "Content-Type": "application/json",
+            }
+        )
+
+    def _post(self, url: str, payload: Any) -> bool:
+        for attempt in range(_MAX_RETRIES):
+            try:
+                resp = self.session.post(url, json=payload, timeout=30)
+            except requests.RequestException as e:
+                _wait_retry(attempt, f"network error: {e}")
+                continue
+            if 200 <= resp.status_code < 300:
+                return True
+            if resp.status_code in (408, 429, 500, 502, 503, 504):
+                _wait_retry(
+                    attempt,
+                    f"HTTP {resp.status_code} retrying",
+                )
+                continue
+            print(
+                f"datadog refused: {resp.status_code} {resp.text[:200]}",
+                file=sys.stderr,
+            )
+            return False
+        return False
+
+    def send_logs(self, records: list[dict[str, Any]]) -> int:
+        if not records:
+            return 0
+        url = _LOG_INTAKE_TPL.format(site=self.site)
+        sent = 0
+        for i in range(0, len(records), _LOG_BATCH):
+            batch = records[i : i + _LOG_BATCH]
+            if self._post(url, batch):
+                sent += len(batch)
+        return sent
+
+    def send_metrics(self, series: list[dict[str, Any]]) -> int:
+        if not series:
+            return 0
+        url = _METRICS_TPL.format(site=self.site)
+        sent = 0
+        for i in range(0, len(series), _METRICS_BATCH):
+            batch = series[i : i + _METRICS_BATCH]
+            if self._post(url, {"series": batch}):
+                sent += len(batch)
+        return sent
+
+
+def _wait_retry(attempt: int, reason: str) -> None:
+    wait = _RETRY_BASE_S * (2**attempt)
+    print(
+        f"  retry {attempt + 1}/{_MAX_RETRIES} in {wait:.1f}s ({reason})",
+        file=sys.stderr,
+    )
+    time.sleep(wait)
+
+
+# --- record → datadog payload ------------------------------------------
+
+
+def _log_record_to_dd(rec: dict[str, Any], host: str) -> dict[str, Any]:
+    line = rec.get("line") or ""
+    tags = [
+        f"role:{rec.get('role')}",
+        f"port:{rec.get('port')}",
+    ]
+    level = rec.get("level")
+    if level:
+        tags.append(f"level:{level}")
+    tag = rec.get("tag")
+    if tag:
+        tags.append(f"thread:{tag}")
+    return {
+        "ddsource": "meshtastic-firmware",
+        "service": "meshtastic-firmware",
+        "hostname": host,
+        "message": line,
+        "ddtags": ",".join(t for t in tags if t and "None" not in t),
+        "timestamp": int((rec.get("ts") or time.time()) * 1000),
+        "level": level,
+    }
+
+
+def _telemetry_record_to_metrics(
+    rec: dict[str, Any], host: str
+) -> list[dict[str, Any]]:
+    fields = rec.get("fields") or {}
+    if not isinstance(fields, dict):
+        return []
+    variant = rec.get("variant") or "unknown"
+    ts = int(rec.get("ts") or time.time())
+    out: list[dict[str, Any]] = []
+    tags = []
+    if rec.get("port"):
+        tags.append(f"port:{rec['port']}")
+    if rec.get("role"):
+        tags.append(f"role:{rec['role']}")
+    if rec.get("from_node"):
+        tags.append(f"from_node:{rec['from_node']}")
+    tags.append(f"variant:{variant}")
+    for field, value in fields.items():
+        if not isinstance(value, (int, float)) or isinstance(value, bool):
+            continue
+        metric = f"mesh.{variant}.{_metric_safe(field)}"
+        out.append(
+            {
+                "metric": metric,
+                "type": 3,  # GAUGE
+                "points": [{"timestamp": ts, "value": float(value)}],
+                "tags": tags,
+                "resources": [{"type": "host", "name": host}],
+            }
+        )
+    return out
+
+
+def _metric_safe(name: str) -> str:
+    # Lowercase, replace non-alnum with underscore for safe metric names.
+    return "".join(c.lower() if c.isalnum() else "_" for c in name)
+
+
+# --- main loop ---------------------------------------------------------
+
+
+def run(
+    log_dir: Path,
+    *,
+    once: bool,
+    since_seconds: float | None,
+    poll_interval: float,
+    dd: _DDSession,
+) -> int:
+    cursor_path = log_dir / ".dd-cursor.json"
+    cursors = _load_cursor(cursor_path)
+
+    # `--since` overrides cursor: rewind to (now-since) timestamp.
+    # We can't seek by timestamp directly (cursor is byte position), so
+    # we just reset cursors to 0 and let the time filter in iter_new
+    # drop older records.
+    cutoff_ts: float | None = None
+    if since_seconds is not None:
+        cursors = {}
+        cutoff_ts = time.time() - since_seconds
+
+    sent_total = {"logs": 0, "telemetry": 0}
+
+    while True:
+        # logs.jsonl → DD logs
+        log_cursor = cursors.setdefault("logs", {})
+        log_batch: list[dict[str, Any]] = []
+        for rec in _StreamReader(log_dir / "logs.jsonl", log_cursor).iter_new_records():
+            if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
+                continue
+            log_batch.append(_log_record_to_dd(rec, dd.hostname))
+        if log_batch:
+            n = dd.send_logs(log_batch)
+            sent_total["logs"] += n
+            print(f"logs: sent {n}/{len(log_batch)}")
+
+        # telemetry.jsonl → DD metrics
+        telem_cursor = cursors.setdefault("telemetry", {})
+        metric_series: list[dict[str, Any]] = []
+        for rec in _StreamReader(
+            log_dir / "telemetry.jsonl", telem_cursor
+        ).iter_new_records():
+            if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
+                continue
+            metric_series.extend(_telemetry_record_to_metrics(rec, dd.hostname))
+        if metric_series:
+            n = dd.send_metrics(metric_series)
+            sent_total["telemetry"] += n
+            print(f"telemetry: sent {n}/{len(metric_series)} metric points")
+
+        _save_cursor(cursor_path, cursors)
+
+        if once:
+            print(f"done. logs={sent_total['logs']} metrics={sent_total['telemetry']}")
+            return 0
+        time.sleep(poll_interval)
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--log-dir",
+        default=str(_DEFAULT_LOG_DIR),
+        help="Path to .mtlog/ (default: mcp-server/.mtlog)",
+    )
+    mode = parser.add_mutually_exclusive_group()
+    mode.add_argument("--once", action="store_true", help="Catch up then exit")
+    mode.add_argument(
+        "--tail",
+        action="store_true",
+        help="Daemon: poll forever (default)",
+    )
+    parser.add_argument(
+        "--since",
+        type=float,
+        default=None,
+        help="Backfill last N seconds. Resets cursor.",
+    )
+    parser.add_argument(
+        "--poll-interval",
+        type=float,
+        default=5.0,
+        help="Seconds between tail polls (default 5)",
+    )
+    parser.add_argument(
+        "--site",
+        default=os.environ.get("DD_SITE", "us5.datadoghq.com"),
+        help=(
+            "Datadog site. Default is the team's instance (us5.datadoghq.com). "
+            "Override via DD_SITE env var or this flag."
+        ),
+    )
+    parser.add_argument(
+        "--host",
+        default=socket.gethostname(),
+        help="Hostname tag (default: socket.gethostname())",
+    )
+    args = parser.parse_args(argv)
+
+    api_key = os.environ.get("DD_API_KEY")
+    if not api_key:
+        print("DD_API_KEY env var required.", file=sys.stderr)
+        return 2
+
+    log_dir = Path(args.log_dir)
+    if not log_dir.exists():
+        print(
+            f"log dir {log_dir} does not exist — start the mcp-server first.",
+            file=sys.stderr,
+        )
+        return 2
+
+    dd = _DDSession(api_key=api_key, site=args.site, hostname=args.host)
+    once = args.once and not args.tail
+    return run(
+        log_dir,
+        once=once,
+        since_seconds=args.since,
+        poll_interval=args.poll_interval,
+        dd=dd,
+    )
+
+
+if __name__ == "__main__":
+    sys.exit(main())

mcp-server/src/meshtastic_mcp/fixtures.py

@@ -0,0 +1,382 @@
+"""Fake NodeDB fixture push — Portduino file copy + hardware XModem upload.
+
+The fixture pipeline is two-stage:
+  1. `bin/gen-fake-nodedb-seed.py` produces a deterministic JSONL describing N
+     fake-but-realistic peers. Committed under `test/fixtures/nodedb/`.
+  2. `bin/seed-json-to-proto.py` compiles JSONL → binary v25 NodeDatabase
+     protobuf with fresh wall-clock timestamps.
+
+This module exposes `push_fake_nodedb(...)`, the MCP tool that:
+  - target="portduino": compiles the JSONL into the device's prefs dir on
+    the local filesystem (`~/.portduino/<config>/prefs/nodes.proto`).
+  - target="hardware":  compiles to a temp file, then streams it over the
+    XModem protocol (via the meshtastic SerialInterface/BLEInterface +
+    `meshtastic.xmodempacket` pubsub topic) to `/prefs/nodes.proto` on the
+    device. Triggers a reboot so the firmware loads the new state on next
+    boot.
+
+XModem wire details (mirrors firmware impl at src/xmodem.cpp:115-260):
+  * 128-byte chunks; final chunk padded to 128 B with 0x1A (SUB) bytes.
+  * CRC16-CCITT (poly 0x1021, init 0x0000).
+  * SOH/seq=0 carries the destination filename in `buffer.bytes`. ACK if
+    `FSCom.open(filename, FILE_O_WRITE)` succeeds; NAK otherwise.
+  * SOH/seq≥1 carries a 128-byte chunk. ACK = advance; NAK = retransmit.
+  * EOT after the last chunk flushes + closes the file on-device.
+
+Hardware push requires `confirm=True` (mirrors factory_reset / erase_and_flash
+in the .github/copilot-instructions.md "never do these without asking" list).
+"""
+
+from __future__ import annotations
+
+import dataclasses
+import hashlib
+import pathlib
+import queue
+import shutil
+import subprocess
+import sys
+import tempfile
+import time
+from typing import Any, Literal
+
+from .connection import connect, is_tcp_port
+
+# Resolve repo root so the tool works regardless of mcp-server cwd.
+_REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
+_SEED_DIR = _REPO_ROOT / "test" / "fixtures" / "nodedb"
+_COMPILE_SCRIPT = _REPO_ROOT / "bin" / "seed-json-to-proto.py"
+
+_DEFAULT_NODES_FILENAME = "/prefs/nodes.proto"
+_XMODEM_CHUNK = 128
+_XMODEM_SUB = 0x1A
+_ACK_TIMEOUT_INIT_S = 5.0
+_ACK_TIMEOUT_CHUNK_S = 2.0
+_MAX_CHUNK_RETRIES = 5
+
+_VALID_SIZES = (250, 500, 1000, 2000)
+
+
+class FixtureError(RuntimeError):
+    """Raised for any fixture-push failure (compile, transport, ack timeout, …)."""
+
+
+# ---------------------------------------------------------------------------
+# CRC16-CCITT (poly 0x1021, init 0x0000). Matches the firmware's `crc16_ccitt`.
+# Hand-rolled to avoid the optional `crcmod` dep.
+# ---------------------------------------------------------------------------
+def _crc16_ccitt(data: bytes, *, init: int = 0x0000) -> int:
+    crc = init
+    for b in data:
+        crc ^= b << 8
+        for _ in range(8):
+            if crc & 0x8000:
+                crc = ((crc << 1) ^ 0x1021) & 0xFFFF
+            else:
+                crc = (crc << 1) & 0xFFFF
+    return crc
+
+
+# ---------------------------------------------------------------------------
+# Compile step — shells out to bin/seed-json-to-proto.py so the MCP module
+# doesn't have to duplicate the proto-encoding logic.
+# ---------------------------------------------------------------------------
+def _compile_proto(jsonl_path: pathlib.Path, out_path: pathlib.Path) -> None:
+    if not _COMPILE_SCRIPT.is_file():
+        raise FixtureError(f"compile script missing at {_COMPILE_SCRIPT}")
+    cmd = [
+        sys.executable,
+        str(_COMPILE_SCRIPT),
+        "--in",
+        str(jsonl_path),
+        "--out",
+        str(out_path),
+    ]
+    try:
+        subprocess.run(cmd, check=True, capture_output=True, text=True)
+    except subprocess.CalledProcessError as exc:
+        raise FixtureError(
+            f"seed-json-to-proto.py failed (exit {exc.returncode}):\n"
+            f"  stdout: {exc.stdout}\n  stderr: {exc.stderr}"
+        ) from exc
+
+
+def _resolve_seed_jsonl(size: int, custom: str | None) -> pathlib.Path:
+    if custom is not None:
+        p = pathlib.Path(custom).expanduser().resolve()
+        if not p.is_file():
+            raise FixtureError(f"custom_seed_jsonl not found: {p}")
+        return p
+    p = _SEED_DIR / f"seed_v25_{size:04d}.jsonl"
+    if not p.is_file():
+        raise FixtureError(
+            f"missing committed seed at {p}. "
+            f"Run `./bin/regen-fake-nodedbs.sh` to generate it."
+        )
+    return p
+
+
+# ---------------------------------------------------------------------------
+# Portduino push — file copy into ~/.portduino/<config>/prefs/
+# ---------------------------------------------------------------------------
+def _portduino_prefs_dir(config_name: str) -> pathlib.Path:
+    home = pathlib.Path.home()
+    return home / ".portduino" / config_name / "prefs"
+
+
+def _push_portduino(
+    size: int,
+    jsonl: pathlib.Path,
+    portduino_config: str,
+    backup_existing: bool,
+) -> dict[str, Any]:
+    prefs = _portduino_prefs_dir(portduino_config)
+    prefs.mkdir(parents=True, exist_ok=True)
+    target = prefs / "nodes.proto"
+    backed_up_to: str | None = None
+    if backup_existing and target.is_file():
+        ts = int(time.time())
+        backup = prefs / f"nodes.proto.bak.{ts}"
+        shutil.move(str(target), str(backup))
+        backed_up_to = str(backup)
+    _compile_proto(jsonl, target)
+    raw = target.read_bytes()
+    return {
+        "transport": "portduino",
+        "path": str(target),
+        "bytes": len(raw),
+        "sha256": hashlib.sha256(raw).hexdigest(),
+        "jsonl_source": str(jsonl),
+        "backed_up_to": backed_up_to,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Hardware push — XModem over BLE/serial via the meshtastic Python interface.
+# ---------------------------------------------------------------------------
+@dataclasses.dataclass
+class _AckEvent:
+    control: int
+    seq: int
+
+
+def _wait_for_response(q: "queue.Queue[_AckEvent]", timeout_s: float) -> _AckEvent:
+    try:
+        return q.get(timeout=timeout_s)
+    except queue.Empty as exc:
+        raise FixtureError(
+            f"XModem response timeout after {timeout_s:.1f}s — device not responding"
+        ) from exc
+
+
+def _push_hardware(
+    size: int,
+    jsonl: pathlib.Path,
+    port: str | None,
+    reboot_after: bool,
+) -> dict[str, Any]:
+    # Lazy imports so the module loads even when the meshtastic deps aren't
+    # available (e.g. CI in a Python env without the package installed).
+    try:
+        from meshtastic.protobuf import mesh_pb2, xmodem_pb2
+        from pubsub import pub
+    except ImportError as exc:  # pragma: no cover — dep missing
+        raise FixtureError(
+            f"hardware push requires the meshtastic + pypubsub packages: {exc}"
+        ) from exc
+
+    if is_tcp_port(port):
+        raise FixtureError(
+            "hardware push over TCP/portduino is not supported — use "
+            "target='portduino' to drop the fixture directly into the prefs dir."
+        )
+
+    # Compile the fixture to a temp file with fresh timestamps.
+    with tempfile.NamedTemporaryFile(suffix=".proto", delete=False) as tf:
+        proto_path = pathlib.Path(tf.name)
+    try:
+        _compile_proto(jsonl, proto_path)
+        payload = proto_path.read_bytes()
+    finally:
+        proto_path.unlink(missing_ok=True)
+
+    sha256 = hashlib.sha256(payload).hexdigest()
+    total_bytes = len(payload)
+
+    # Subscribe to XModem responses BEFORE we open the interface, so we don't
+    # race the first ACK that arrives during the SOH/seq=0 handshake.
+    #
+    # NB: the signature MUST declare every kwarg pypubsub will see for this
+    # topic, or pubsub locks the topic spec to a smaller set (whichever
+    # subscribe arrives first) and then *rejects* the meshtastic library's
+    # publish call with `SenderUnknownMsgDataError: unknown ... interface`.
+    # The meshtastic lib publishes both `packet=` and `interface=`
+    # (mesh_interface.py:1389-1395), so both must appear here.
+    response_q: "queue.Queue[_AckEvent]" = queue.Queue()
+
+    def _on_xmodem(packet: Any = None, interface: Any = None, **_kw: Any) -> None:
+        if packet is None:
+            return
+        response_q.put(_AckEvent(control=int(packet.control), seq=int(packet.seq)))
+
+    pub.subscribe(_on_xmodem, "meshtastic.xmodempacket")
+
+    chunks_sent = 0
+    retried = 0
+    rebooted = False
+
+    XMC = xmodem_pb2.XModem.Control
+    try:
+        with connect(port=port) as iface:
+            # 1) Send the filename (SOH, seq=0).
+            init_pkt = xmodem_pb2.XModem(
+                control=XMC.Value("SOH"),
+                seq=0,
+                buffer=_DEFAULT_NODES_FILENAME.encode("utf-8"),
+            )
+            iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=init_pkt))
+            ack = _wait_for_response(response_q, _ACK_TIMEOUT_INIT_S)
+            if ack.control != XMC.Value("ACK"):
+                raise FixtureError(
+                    f"device refused filename {_DEFAULT_NODES_FILENAME!r} "
+                    f"(got control={ack.control}, expected ACK). "
+                    f"Filesystem full or permissions issue?"
+                )
+
+            # 2) Stream the payload in 128 B chunks.
+            for offset in range(0, total_bytes, _XMODEM_CHUNK):
+                chunk = payload[offset : offset + _XMODEM_CHUNK]
+                if len(chunk) < _XMODEM_CHUNK:
+                    # Pad final chunk to 128 B with SUB. The trailing 0x1A bytes
+                    # become part of the file on-device, but nanopb ignores
+                    # bytes past the end of the top-level message.
+                    chunk = chunk + bytes([_XMODEM_SUB] * (_XMODEM_CHUNK - len(chunk)))
+                seq = ((offset // _XMODEM_CHUNK) + 1) % 256
+                # Retry loop on NAK / timeout.
+                attempts = 0
+                while True:
+                    pkt = xmodem_pb2.XModem(
+                        control=XMC.Value("SOH"),
+                        seq=seq,
+                        buffer=chunk,
+                        crc16=_crc16_ccitt(chunk),
+                    )
+                    iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=pkt))
+                    ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
+                    if ack.control == XMC.Value("ACK"):
+                        chunks_sent += 1
+                        break
+                    if ack.control == XMC.Value("NAK"):
+                        attempts += 1
+                        retried += 1
+                        if attempts >= _MAX_CHUNK_RETRIES:
+                            # Abort: send CAN so the firmware removes the half-
+                            # written file via FSCom.remove(filename).
+                            iface._sendToRadio(
+                                mesh_pb2.ToRadio(
+                                    xmodemPacket=xmodem_pb2.XModem(
+                                        control=XMC.Value("CAN")
+                                    )
+                                )
+                            )
+                            raise FixtureError(
+                                f"chunk seq={seq} NAK'd {attempts} times; "
+                                f"aborted transfer (file removed on-device)."
+                            )
+                        continue  # retry the same chunk
+                    raise FixtureError(
+                        f"unexpected XModem control={ack.control} on seq={seq}"
+                    )
+
+            # 3) Tell the device we're done.
+            iface._sendToRadio(
+                mesh_pb2.ToRadio(
+                    xmodemPacket=xmodem_pb2.XModem(control=XMC.Value("EOT"))
+                )
+            )
+            ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
+            if ack.control != XMC.Value("ACK"):
+                raise FixtureError(f"EOT not ACKed (got control={ack.control})")
+
+            # 4) Reboot so loadFromDisk picks up the new file.
+            if reboot_after:
+                iface.localNode.reboot(secs=1)
+                rebooted = True
+    finally:
+        try:
+            pub.unsubscribe(_on_xmodem, "meshtastic.xmodempacket")
+        except Exception:
+            pass
+
+    return {
+        "transport": "hardware",
+        "port": port,
+        "filename_on_device": _DEFAULT_NODES_FILENAME,
+        "bytes": total_bytes,
+        "chunks_sent": chunks_sent,
+        "retried": retried,
+        "sha256": sha256,
+        "jsonl_source": str(jsonl),
+        "rebooted": rebooted,
+    }
+
+
+# ---------------------------------------------------------------------------
+# Public entry point — registered as an MCP tool in server.py.
+# ---------------------------------------------------------------------------
+def push_fake_nodedb(
+    size: int,
+    target: Literal["portduino", "hardware"] = "portduino",
+    *,
+    port: str | None = None,
+    portduino_config: str = "default",
+    backup_existing: bool = True,
+    confirm: bool = False,
+    reboot_after: bool = True,
+    custom_seed_jsonl: str | None = None,
+) -> dict[str, Any]:
+    """Compile a fresh-timestamp NodeDatabase fixture and push it to a device.
+
+    Args:
+      size: 250, 500, 1000, or 2000 — selects which committed seed JSONL to use.
+      target: "portduino" (file copy to ~/.portduino/<config>/prefs/) or
+              "hardware" (XModem upload to /prefs/nodes.proto + reboot).
+      port: required for target="hardware". Serial path (e.g. /dev/cu.usbmodemXXXX)
+            or BLE identifier. TCP endpoints are rejected — use target="portduino"
+            instead.
+      portduino_config: which Portduino instance dir under ~/.portduino/. Default "default".
+      backup_existing: portduino only. Move nodes.proto -> nodes.proto.bak.<ts>
+                        if present, so you can roll back.
+      confirm: required True for target="hardware" (writes flash + reboots).
+      reboot_after: hardware only. If True, send a 1-second reboot after the
+                    final ACK so loadFromDisk picks up the new file at next boot.
+      custom_seed_jsonl: override the committed JSONL. Use to push a hand-edited
+                         test scenario.
+
+    Returns:
+        dict with transport, bytes, sha256, etc. — depends on target.
+
+    """
+    if size not in _VALID_SIZES:
+        raise FixtureError(
+            f"size must be one of {_VALID_SIZES}; got {size!r}. "
+            f"Add a new committed seed if you need a different cardinality."
+        )
+
+    jsonl = _resolve_seed_jsonl(size, custom_seed_jsonl)
+
+    if target == "portduino":
+        return _push_portduino(size, jsonl, portduino_config, backup_existing)
+
+    if target == "hardware":
+        if not confirm:
+            raise FixtureError(
+                "hardware push writes flash and triggers a reboot — pass confirm=True."
+            )
+        if not port:
+            raise FixtureError(
+                "target='hardware' requires a port (e.g. /dev/cu.usbmodemXXXX)."
+            )
+        return _push_hardware(size, jsonl, port, reboot_after)
+
+    raise FixtureError(f"unknown target {target!r}; expected 'portduino' or 'hardware'")

mcp-server/src/meshtastic_mcp/flash.py

@@ -108,18 +108,33 @@ def build(
     env: str,
     with_manifest: bool = True,
     userprefs_overrides: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Run `pio run -e <env>` and return artifact paths.
 
     `userprefs_overrides` (optional): dict of `USERPREFS_<KEY>: value` to inject
     into userPrefs.jsonc for this build only. File is restored byte-for-byte
     on exit. Use `userprefs_set()` for persistent changes.
+
+    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros to set for
+    this build only via `PLATFORMIO_BUILD_FLAGS`. Common useful flag:
+    `{"DEBUG_HEAP": 1}` enables per-thread leak detection + `[heap N]`
+    prefix on every log line. Combines with the recorder so heap shows
+    up at log cadence (much higher resolution than the ~60 s LocalStats
+    packet) — see `recorder/parsers.py:_HEAP_PREFIX_RE`. Bool values
+    expand to bare `-D<NAME>` (presence-only flags).
     """
     args = ["run", "-e", env]
     if with_manifest:
         args.extend(["-t", "mtjson"])
+    extra_env = _build_flags_env(build_flags) if build_flags else None
     with userprefs.temporary_overrides(userprefs_overrides) as effective:
-        result = pio.run(args, timeout=pio.TIMEOUT_BUILD, check=False)
+        result = pio.run(
+            args,
+            timeout=pio.TIMEOUT_BUILD,
+            check=False,
+            extra_env=extra_env,
+        )
     return {
         "exit_code": result.returncode,
         "artifacts": [str(p) for p in _artifacts_for(env)],
@@ -127,9 +142,27 @@ def build(
         "stderr_tail": pio.tail_lines(result.stderr, 200),
         "duration_s": round(result.duration_s, 2),
         "userprefs": _userprefs_summary(effective),
+        "build_flags": dict(build_flags) if build_flags else None,
     }
 
 
+def _build_flags_env(build_flags: dict[str, Any]) -> dict[str, str]:
+    """Translate `{"DEBUG_HEAP": 1, "FOO": "bar"}` → `{"PLATFORMIO_BUILD_FLAGS":
+    "-DDEBUG_HEAP=1 -DFOO=bar"}`. Bool True → bare `-D<NAME>`; False/None drop
+    the flag entirely. Other types stringify."""
+    parts: list[str] = []
+    for key, value in build_flags.items():
+        if value is False or value is None:
+            continue
+        if value is True:
+            parts.append(f"-D{key}")
+        else:
+            parts.append(f"-D{key}={value}")
+    if not parts:
+        return {}
+    return {"PLATFORMIO_BUILD_FLAGS": " ".join(parts)}
+
+
 def clean(env: str) -> dict[str, Any]:
     """Run `pio run -e <env> -t clean`."""
     result = pio.run(["run", "-e", env, "-t", "clean"], timeout=120, check=False)
@@ -146,20 +179,29 @@ def flash(
     port: str,
     confirm: bool = False,
     userprefs_overrides: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """`pio run -e <env> -t upload --upload-port <port>`. All architectures.
 
     `userprefs_overrides` (optional): see `build()` — the rebuild-before-upload
     that pio performs will pick up the injected values.
+
+    `build_flags` (optional): same shape as `build()` — `PLATFORMIO_BUILD_FLAGS`
+    is exported for the rebuild-before-upload, so the uploaded firmware
+    actually carries the flags. Without this propagation, `pio run -t upload`
+    would relink without the env var and silently drop them. Common use:
+    `build_flags={"DEBUG_HEAP": 1}` for the leak-hunt path.
     """
     _require_confirm(confirm, "flash")
     _reject_native_env(env, "flash")
     connection.reject_if_tcp(port, "flash")
+    extra_env = _build_flags_env(build_flags) if build_flags else None
     with userprefs.temporary_overrides(userprefs_overrides) as effective:
         result = pio.run(
             ["run", "-e", env, "-t", "upload", "--upload-port", port],
             timeout=pio.TIMEOUT_UPLOAD,
             check=False,
+            extra_env=extra_env,
         )
     return {
         "exit_code": result.returncode,
@@ -167,6 +209,7 @@ def flash(
         "stderr_tail": pio.tail_lines(result.stderr, 200),
         "duration_s": round(result.duration_s, 2),
         "userprefs": _userprefs_summary(effective),
+        "build_flags": dict(build_flags) if build_flags else None,
     }
 
 

mcp-server/src/meshtastic_mcp/log_query.py

@@ -0,0 +1,410 @@
+"""Read-side queries over the recorder's JSONL streams.
+
+Pure functions over `mcp-server/.mtlog/`. Streaming JSONL reader: never
+loads a whole file. Time-bound queries short-circuit as soon as `ts`
+exceeds the requested end. The recorder writes monotonically, so a
+forward scan is cheap; we don't need an index.
+
+All time arguments accept:
+  - epoch seconds (int/float)
+  - relative strings: "-15m", "-2h", "-3d", "now"
+  - ISO-ish absolute strings: "2026-05-07T14:30:00" (naive timestamps are
+    treated as UTC)
+
+Tools that return data ALWAYS cap their output (max_lines / max_points
+/ max), and report whether more matched than was returned.
+"""
+
+from __future__ import annotations
+
+import gzip
+import json
+import re
+import statistics
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Iterator
+
+from .recorder.recorder import get_recorder
+
+_REL_RE = re.compile(r"^\s*-\s*(\d+(?:\.\d+)?)\s*([smhd])\s*$")
+_REGEX_PREVIEW_MAX = 100
+_REGEX_PREVIEW_TRUNCATE = 97
+
+
+def _parse_time(value: Any, *, now: float | None = None) -> float:
+    """Coerce to epoch seconds. Defaults `now` to `time.time()`."""
+    if value is None:
+        return time.time()
+    if isinstance(value, (int, float)):
+        return float(value)
+    if not isinstance(value, str):
+        raise ValueError(f"invalid time: {value!r}")
+    s = value.strip().lower()
+    if s in ("", "now"):
+        return time.time() if now is None else now
+    m = _REL_RE.match(s)
+    if m:
+        n = float(m.group(1))
+        unit = m.group(2)
+        secs = n * {"s": 1, "m": 60, "h": 3600, "d": 86400}[unit]
+        base = time.time() if now is None else now
+        return base - secs
+    # Try ISO 8601. Accept naive (assume UTC) and Z-suffixed.
+    try:
+        if s.endswith("z"):
+            s = s[:-1] + "+00:00"
+        dt = datetime.fromisoformat(s)
+        if dt.tzinfo is None:
+            dt = dt.replace(tzinfo=timezone.utc)
+        return dt.timestamp()
+    except ValueError as e:
+        raise ValueError(f"unparseable time: {value!r}") from e
+
+
+def _iter_jsonl(path: Path, *, since: float, until: float) -> Iterator[dict[str, Any]]:
+    """Stream records in chronological order: rotated archives first
+    (oldest → newest by lex sort, which is chronological for our
+    `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` archive naming), then the live file
+    last. The "keep last N" pop-front logic in the window queries
+    relies on records arriving in time order across files.
+    """
+    files: list[Path] = []
+    # Gzipped archives are named "<stem>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz".
+    for archive in sorted(path.parent.glob(f"{path.stem}.*.jsonl.gz")):
+        files.append(archive)
+    if path.exists():
+        files.append(path)
+    for f in files:
+        opener = gzip.open if f.suffix == ".gz" else open
+        try:
+            with opener(f, "rt", encoding="utf-8") as fh:  # type: ignore[arg-type]
+                for line in fh:
+                    line = line.strip()
+                    if not line:
+                        continue
+                    try:
+                        rec = json.loads(line)
+                    except json.JSONDecodeError:
+                        continue
+                    ts = rec.get("ts")
+                    if not isinstance(ts, (int, float)):
+                        continue
+                    if ts < since:
+                        continue
+                    if ts > until:
+                        # Records are append-monotonic within a file, so
+                        # the rest of this file is also past `until`.
+                        # Archives can still overlap each other, so only
+                        # short-circuit this file, not the whole scan.
+                        break
+                    yield rec
+        except (FileNotFoundError, OSError):
+            continue
+
+
+# -- queries ------------------------------------------------------------
+
+
+def logs_window(
+    start: Any = "-15m",
+    end: Any = "now",
+    *,
+    grep: str | None = None,
+    level: str | None = None,
+    tag: str | None = None,
+    port: str | None = None,
+    max_lines: int = 200,
+) -> dict[str, Any]:
+    """Recent firmware log lines, filtered.
+
+    `level` accepts a single level name or pipe-separated set
+    ("WARN|ERROR|CRIT"). `grep` is a regex (Python re) over the raw
+    `line` field. Returns the last `max_lines` matches.
+    """
+    s = _parse_time(start)
+    e = _parse_time(end)
+    levels = _split_set(level)
+    if grep:
+        try:
+            grep_re = re.compile(grep)
+        except re.error as exc:
+            preview = (
+                grep
+                if len(grep) <= _REGEX_PREVIEW_MAX
+                else f"{grep[:_REGEX_PREVIEW_TRUNCATE]}..."
+            )
+            raise ValueError(f"invalid grep regex {preview!r}: {exc}") from exc
+    else:
+        grep_re = None
+
+    base = get_recorder().base_dir
+    matched = 0
+    out: list[dict[str, Any]] = []
+    for rec in _iter_jsonl(base / "logs.jsonl", since=s, until=e):
+        if levels and rec.get("level") not in levels:
+            continue
+        if tag and rec.get("tag") != tag:
+            continue
+        if port and rec.get("port") != port:
+            continue
+        if grep_re and not grep_re.search(rec.get("line") or ""):
+            continue
+        matched += 1
+        out.append(rec)
+        if len(out) > max_lines:
+            out.pop(0)  # keep the most recent N
+    return {
+        "lines": out,
+        "total_matched": matched,
+        "dropped": max(0, matched - max_lines),
+        "window": {"start": s, "end": e},
+    }
+
+
+def telemetry_timeline(
+    window: Any = "1h",
+    *,
+    variant: str = "local",
+    field: str = "free_heap",
+    port: str | None = None,
+    max_points: int = 200,
+) -> dict[str, Any]:
+    """Timeseries of one telemetry field, downsampled.
+
+    `field` matches both the protobuf snake_case name (`free_heap`,
+    `heap_free_bytes`, `battery_level`) and camelCase (`freeHeap`).
+    Server-side bucket-mean downsamples to ≤ `max_points`. Returns
+    `slope_per_min` (linear regression slope, units/min) so a leak
+    detector can read one number.
+    """
+    end = time.time()
+    if isinstance(window, (int, float)):
+        # Numeric `window` is a duration in seconds — "last N seconds".
+        # Without this branch, `_parse_time(-N)` would treat -N as an
+        # absolute epoch timestamp (i.e., Jan 1 1970 minus N seconds),
+        # producing a wildly negative `start` and matching nothing.
+        start = end - float(window)
+    elif isinstance(window, str) and not window.startswith("-"):
+        # Bare string like "1h" is sugar for "-1h".
+        start = _parse_time(f"-{window}", now=end)
+    else:
+        start = _parse_time(window, now=end)
+
+    base = get_recorder().base_dir
+    raw: list[tuple[float, float]] = []
+    field_aliases = _field_aliases(field)
+    for rec in _iter_jsonl(base / "telemetry.jsonl", since=start, until=end):
+        if rec.get("variant") != variant:
+            continue
+        if port and rec.get("port") != port:
+            continue
+        fields = rec.get("fields") or {}
+        value: Any = None
+        for alias in field_aliases:
+            if alias in fields:
+                value = fields[alias]
+                break
+        if not isinstance(value, (int, float)):
+            continue
+        raw.append((float(rec["ts"]), float(value)))
+
+    if not raw:
+        return {
+            "points": [],
+            "samples": 0,
+            "min": None,
+            "max": None,
+            "slope_per_min": None,
+            "window": {"start": start, "end": end, "variant": variant, "field": field},
+        }
+
+    points = _downsample(raw, max_points=max_points)
+    values = [v for _, v in raw]
+    return {
+        "points": [{"ts": ts, "value": v} for ts, v in points],
+        "samples": len(raw),
+        "min": min(values),
+        "max": max(values),
+        "slope_per_min": _slope_per_min(raw),
+        "window": {"start": start, "end": end, "variant": variant, "field": field},
+    }
+
+
+def packets_window(
+    start: Any = "-5m",
+    end: Any = "now",
+    *,
+    portnum: str | None = None,
+    from_node: str | None = None,
+    to_node: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    s = _parse_time(start)
+    e = _parse_time(end)
+    portnums = _split_set(portnum)
+    base = get_recorder().base_dir
+    matched = 0
+    out: list[dict[str, Any]] = []
+    for rec in _iter_jsonl(base / "packets.jsonl", since=s, until=e):
+        if portnums and rec.get("portnum") not in portnums:
+            continue
+        if from_node and str(rec.get("from_node")) != str(from_node):
+            continue
+        if to_node and str(rec.get("to_node")) != str(to_node):
+            continue
+        matched += 1
+        out.append(rec)
+        if len(out) > max:
+            out.pop(0)
+    return {
+        "packets": out,
+        "total_matched": matched,
+        "dropped": matched - max if matched > max else 0,
+        "window": {"start": s, "end": e},
+    }
+
+
+def events_window(
+    start: Any = "-1h",
+    end: Any = "now",
+    *,
+    kind: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    s = _parse_time(start)
+    e = _parse_time(end)
+    kinds = _split_set(kind)
+    base = get_recorder().base_dir
+    matched = 0
+    out: list[dict[str, Any]] = []
+    for rec in _iter_jsonl(base / "events.jsonl", since=s, until=e):
+        if kinds and rec.get("kind") not in kinds:
+            continue
+        matched += 1
+        out.append(rec)
+        if len(out) > max:
+            out.pop(0)
+    return {
+        "events": out,
+        "total_matched": matched,
+        "dropped": matched - max if matched > max else 0,
+        "window": {"start": s, "end": e},
+    }
+
+
+def export(
+    start: Any,
+    end: Any,
+    dest_dir: str,
+    *,
+    streams: list[str] | None = None,
+) -> dict[str, Any]:
+    """Bundle a slice of each requested stream into `dest_dir`.
+
+    For a notebook, a bug report, or a Datadog backfill. Output files
+    are uncompressed JSONL (callers gzip themselves if they want to).
+    """
+    s = _parse_time(start)
+    e = _parse_time(end)
+    selected = streams or ["logs", "telemetry", "packets", "events"]
+    dest = Path(dest_dir)
+    dest.mkdir(parents=True, exist_ok=True)
+
+    base = get_recorder().base_dir
+    paths: dict[str, str] = {}
+    for stream in selected:
+        src = base / f"{stream}.jsonl"
+        if not src.exists() and not list(base.glob(f"{stream}.*.jsonl.gz")):
+            continue
+        out_path = dest / f"{stream}.jsonl"
+        n = 0
+        with out_path.open("w", encoding="utf-8") as fh:
+            for rec in _iter_jsonl(src, since=s, until=e):
+                fh.write(json.dumps(rec, separators=(",", ":")) + "\n")
+                n += 1
+        paths[stream] = str(out_path)
+        paths[f"{stream}_count"] = str(n)
+    return {"dest_dir": str(dest), "paths": paths, "window": {"start": s, "end": e}}
+
+
+# -- helpers ------------------------------------------------------------
+
+
+def _split_set(value: str | None) -> set[str] | None:
+    if not value:
+        return None
+    return {v.strip() for v in value.split("|") if v.strip()}
+
+
+def _field_aliases(field: str) -> list[str]:
+    """Accept snake_case OR camelCase, plus a few legacy aliases."""
+    snake = field
+    camel = _snake_to_camel(field)
+    aliases = {snake, camel}
+    # Old protobuf fields (pre-LocalStats) used different names
+    legacy = {
+        "free_heap": ["free_heap", "freeHeap", "heap_free_bytes", "heapFreeBytes"],
+        "heap_free_bytes": [
+            "heap_free_bytes",
+            "heapFreeBytes",
+            "free_heap",
+            "freeHeap",
+        ],
+        "total_heap": ["total_heap", "totalHeap", "heap_total_bytes", "heapTotalBytes"],
+        "heap_total_bytes": [
+            "heap_total_bytes",
+            "heapTotalBytes",
+            "total_heap",
+            "totalHeap",
+        ],
+    }
+    if field in legacy:
+        aliases.update(legacy[field])
+    return list(aliases)
+
+
+def _snake_to_camel(name: str) -> str:
+    parts = name.split("_")
+    return parts[0] + "".join(p.title() for p in parts[1:])
+
+
+def _downsample(
+    points: list[tuple[float, float]], *, max_points: int
+) -> list[tuple[float, float]]:
+    if len(points) <= max_points:
+        return points
+    # Even-bucket mean. Preserves shape better than nth-sample picking.
+    n = len(points)
+    bucket = n / max_points
+    out: list[tuple[float, float]] = []
+    i = 0
+    for k in range(max_points):
+        end = int((k + 1) * bucket)
+        end = min(end, n)
+        if end <= i:
+            continue
+        chunk = points[i:end]
+        ts = chunk[len(chunk) // 2][0]
+        val = statistics.fmean(v for _, v in chunk)
+        out.append((ts, val))
+        i = end
+    return out
+
+
+def _slope_per_min(points: list[tuple[float, float]]) -> float | None:
+    """Least-squares slope (units per minute). None if too few points."""
+    if len(points) < 2:
+        return None
+    xs = [t for t, _ in points]
+    ys = [v for _, v in points]
+    n = len(xs)
+    mean_x = sum(xs) / n
+    mean_y = sum(ys) / n
+    num = sum((xs[i] - mean_x) * (ys[i] - mean_y) for i in range(n))
+    den = sum((x - mean_x) ** 2 for x in xs)
+    if den == 0:
+        return None
+    slope_per_sec = num / den
+    return slope_per_sec * 60.0

mcp-server/src/meshtastic_mcp/pio.py

@@ -92,6 +92,7 @@ def _run_capturing(
     cwd: Path | None = None,
     timeout: float | None = None,
     tee_header: str | None = None,
+    extra_env: dict[str, str] | None = None,
 ) -> tuple[int, str, str, float]:
     """Run a subprocess, capture stdout+stderr, optionally tee to the flash log.
 
@@ -99,6 +100,9 @@ def _run_capturing(
     `subprocess.TimeoutExpired` on timeout (callers map this to their own
     domain-specific error).
 
+    `extra_env` merges into the subprocess environment (parent env stays
+    intact). Used for `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar.
+
     Fast path: `subprocess.run(capture_output=True)` when no flash log is
     configured (unchanged behavior).
 
@@ -110,6 +114,9 @@ def _run_capturing(
     """
     log_path = _flash_log_path()
     t0 = time.monotonic()
+    env = None
+    if extra_env:
+        env = {**os.environ, **extra_env}
 
     if log_path is None:
         # Fast path — unchanged.
@@ -119,6 +126,7 @@ def _run_capturing(
             capture_output=True,
             text=True,
             timeout=timeout,
+            env=env,
         )
         return (
             proc.returncode,
@@ -145,6 +153,7 @@ def _run_capturing(
         stderr=subprocess.PIPE,
         text=True,
         bufsize=1,  # line-buffered
+        env=env,
     )
     stdout_chunks: list[str] = []
     stderr_chunks: list[str] = []
@@ -232,12 +241,17 @@ def run(
     cwd: Path | None = None,
     timeout: float | None = TIMEOUT_DEFAULT,
     check: bool = True,
+    extra_env: dict[str, str] | None = None,
 ) -> PioResult:
     """Invoke `pio <args>` and return captured output.
 
     `cwd` defaults to the firmware root. `check=True` raises `PioError` on
     non-zero exit; set `check=False` to inspect `returncode` manually.
 
+    `extra_env` merges into the subprocess environment — used for
+    `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar build-time
+    toggles that can't be expressed as command-line args.
+
     If `MESHTASTIC_MCP_FLASH_LOG` is set, output is also tee'd to that file
     line-by-line as it arrives (for live flash progress in the TUI).
     """
@@ -250,6 +264,7 @@ def run(
             cwd=work_dir,
             timeout=timeout,
             tee_header=f"pio {' '.join(args)}",
+            extra_env=extra_env,
         )
     except subprocess.TimeoutExpired as exc:
         raise PioTimeout(f"pio {' '.join(args)} timed out after {timeout}s") from exc

mcp-server/src/meshtastic_mcp/recorder/__init__.py

@@ -0,0 +1,19 @@
+"""Persistent device-log capture.
+
+Singleton `Recorder` subscribes once to the meshtastic pubsub fan-out
+(`meshtastic.log.line`, `meshtastic.receive.*`, `meshtastic.connection.*`)
+and appends to four JSONL files under `mcp-server/.mtlog/`. Pubsub is
+process-global so a single subscription captures every active interface
+(serial / TCP / BLE) without any per-connection bookkeeping.
+
+The recorder is opt-in-by-import: importing this package is a no-op; call
+`get_recorder().start()` (which `server.py` does at FastMCP app init) to
+begin writing. `pause()` / `resume()` exist for the rare case the user
+wants a clean stretch of file (e.g. capturing a known-good baseline).
+"""
+
+from __future__ import annotations
+
+from .recorder import Recorder, get_recorder
+
+__all__ = ["Recorder", "get_recorder"]

mcp-server/src/meshtastic_mcp/recorder/parsers.py

@@ -0,0 +1,309 @@
+"""Best-effort parsers for log lines and telemetry packets.
+
+Two flavors of log line cross our pubsub subscription:
+  1. Text-mode path (debug_log_api disabled): the meshtastic Python lib
+     accumulates bytes between protobuf frames and emits the full
+     firmware-formatted line, e.g.
+         "INFO  | 12:34:56 12345 [Main] Booting"
+     — level, HH:MM:SS, uptime seconds, thread bracket, then message.
+  2. LogRecord protobuf path (debug_log_api enabled): the lib calls
+     `_handleLogLine(record.message)` with ONLY the message body. The
+     level/source/time fields on the LogRecord are dropped before
+     pubsub fan-out. We get e.g. just "Booting".
+
+Both arrive on `meshtastic.log.line`. The parser tries to recover a
+level + thread when the prefix is present and falls back to level=None
+otherwise. Consumers who want level filtering on protobuf-mode hosts
+should grep the raw `line` field instead.
+
+Telemetry: `meshtastic.receive.telemetry` packets carry one of several
+metric variants in `packet["decoded"]["telemetry"]`. We flatten the
+chosen variant into a {field: value} dict so callers don't have to
+know the protobuf shape.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import Any
+
+# Match: LEVEL | HH:MM:SS UPTIME [Thread] message
+# HH:MM:SS may be ??:??:?? when RTC isn't valid. The level alternation
+# below is the canonical list — DebugConfiguration.h's MESHTASTIC_LOG_LEVEL_*
+# macros must stay in sync with these strings.
+_LINE_RE = re.compile(
+    r"""
+    ^
+    (?P<level>DEBUG|INFO\ |WARN\ |ERROR|CRIT\ |TRACE|HEAP\ )
+    \s*\|\s*
+    (?P<clock>(?:\d{2}:\d{2}:\d{2})|(?:\?{2}:\?{2}:\?{2}))
+    \s+
+    (?P<uptime>\d+)
+    \s+
+    (?:\[(?P<thread>[^\]]+)\]\s+)?
+    (?P<msg>.*)
+    $
+    """,
+    re.VERBOSE,
+)
+
+# DEBUG_HEAP build prepends `[heap N] ` to every message body, AFTER the
+# thread bracket. See src/RedirectablePrint.cpp:175.
+_HEAP_PREFIX_RE = re.compile(r"^\[heap\s+(?P<heap>\d+)\]\s+(?P<rest>.*)$")
+
+# OSThread leak/free detection. See src/concurrency/OSThread.cpp:89-91.
+# Format: "------ Thread NAME leaked heap A -> B (delta) ------"
+#         "++++++ Thread NAME freed heap A -> B (delta) ++++++"
+_THREAD_HEAP_RE = re.compile(
+    r"""
+    ^[\-+]+\s*
+    Thread\s+(?P<thread>\S+)\s+
+    (?P<kind>leaked|freed)\s+heap\s+
+    (?P<before>-?\d+)\s*->\s*(?P<after>-?\d+)\s+
+    \((?P<delta>-?\d+)\)
+    """,
+    re.VERBOSE,
+)
+
+# Power.cpp:908 periodic heap status (DEBUG_HEAP only).
+# Format: "Heap status: FREE/TOTAL bytes free (DELTA), running R/N threads"
+_HEAP_STATUS_RE = re.compile(
+    r"""
+    Heap\s+status:\s+
+    (?P<free>\d+)\s*/\s*(?P<total>\d+)\s+bytes\s+free
+    (?:\s+\((?P<delta>-?\d+)\))?
+    """,
+    re.VERBOSE,
+)
+
+
+_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
+_HEAP_BRACKET_RE = re.compile(r"^heap\s+(?P<heap>\d+)$")
+
+
+def parse_log_line(line: str) -> dict[str, Any]:
+    """Best-effort decompose a raw firmware log line.
+
+    Returns a dict with at least `line` (the original, unmodified — ANSI
+    codes preserved for fidelity). Adds `level`, `tag`, `clock`,
+    `uptime_s`, and `msg` when the full prefix is present.
+
+    Handles two firmware quirks:
+      - LogRecord.message can carry ANSI color escapes from RedirectablePrint
+        (the BLE/StreamAPI path inherited the colored body in some builds).
+        We strip ANSI before regex matching so the prefix survives.
+      - DEBUG_HEAP injects `[heap N]` after the thread bracket. When NO
+        thread name is set, the heap takes the thread bracket position —
+        looks like `[heap 12345] msg`. We detect that shape and move it
+        out of `tag` and into `heap_free`.
+
+    DEBUG_HEAP-build extras (when `[heap N]` is injected): `heap_free`
+    (bytes), and when a `Thread X leaked|freed heap` line is recognized,
+    `heap_event` = {kind, thread, before, after, delta}.
+
+    Never raises.
+    """
+    out: dict[str, Any] = {"line": line}
+    if not line:
+        return out
+
+    # Strip ANSI escapes BEFORE any regex matching. The original `line`
+    # stays in `out["line"]` for fidelity / future grep.
+    clean = _ANSI_RE.sub("", line)
+
+    m = _LINE_RE.match(clean)
+    msg: str | None = None
+    if m:
+        level = m.group("level").rstrip()
+        out["level"] = level
+        out["clock"] = m.group("clock")
+        try:
+            out["uptime_s"] = int(m.group("uptime"))
+        except (TypeError, ValueError):
+            out["uptime_s"] = None
+        thread = m.group("thread")
+        if thread:
+            # If "thread" is actually the heap prefix taking the bracket
+            # position (DEBUG_HEAP build, no thread set), capture heap
+            # and leave tag unset.
+            hb = _HEAP_BRACKET_RE.match(thread.strip())
+            if hb:
+                try:
+                    out["heap_free"] = int(hb.group("heap"))
+                except (TypeError, ValueError):
+                    pass
+            else:
+                out["tag"] = thread
+        msg = m.group("msg")
+        out["msg"] = msg
+    else:
+        # No prefix — bare LogRecord.message body. Inspect the whole
+        # line for DEBUG_HEAP-style content; the heap-prefix and
+        # thread-leak patterns can survive on either path.
+        msg = clean
+
+    # DEBUG_HEAP per-line heap prefix: `[heap 92344] message`.
+    # Sits AFTER the thread bracket and BEFORE the message body, but
+    # for bare LogRecord lines it's at the start. Match it at the
+    # head of `msg`.
+    if msg:
+        hp = _HEAP_PREFIX_RE.match(msg)
+        if hp:
+            try:
+                out["heap_free"] = int(hp.group("heap"))
+            except (TypeError, ValueError):
+                pass
+            else:
+                # Strip the prefix from `msg` so a grep on the message
+                # body doesn't have to know about it.
+                out["msg"] = hp.group("rest")
+                msg = hp.group("rest")
+
+        # Thread-level leak/free detection.
+        thr = _THREAD_HEAP_RE.search(msg)
+        if thr:
+            try:
+                out["heap_event"] = {
+                    "kind": thr.group("kind"),
+                    "thread": thr.group("thread"),
+                    "before": int(thr.group("before")),
+                    "after": int(thr.group("after")),
+                    "delta": int(thr.group("delta")),
+                }
+            except (TypeError, ValueError):
+                pass
+
+        # Power.cpp periodic "Heap status: F/T bytes free (D), running ..."
+        hs = _HEAP_STATUS_RE.search(msg)
+        if hs:
+            try:
+                out["heap_free"] = int(hs.group("free"))
+                out["heap_total"] = int(hs.group("total"))
+                if hs.group("delta") is not None:
+                    out["heap_delta"] = int(hs.group("delta"))
+            except (TypeError, ValueError):
+                pass
+
+    return out
+
+
+# -- Telemetry ----------------------------------------------------------
+
+# Order matters: meshtastic-python decoded packets use the protobuf
+# `oneof variant` field name (snake_case) as the dict key.
+_TELEMETRY_VARIANTS = (
+    ("device_metrics", "device"),
+    ("local_stats", "local"),
+    ("environment_metrics", "environment"),
+    ("power_metrics", "power"),
+    ("air_quality_metrics", "airQuality"),
+    ("health_metrics", "health"),
+    ("host_metrics", "host"),
+)
+
+
+def extract_telemetry(packet: dict[str, Any]) -> dict[str, Any] | None:
+    """Pull the telemetry variant + flat fields out of a `meshtastic.receive.telemetry`
+    packet. Returns None when the shape isn't what we expect — so the
+    caller can fall back to a generic packets.jsonl row.
+    """
+    if not isinstance(packet, dict):
+        return None
+    decoded = packet.get("decoded")
+    if not isinstance(decoded, dict):
+        return None
+    telem = decoded.get("telemetry")
+    if not isinstance(telem, dict):
+        return None
+    # The Python lib produces dict-of-camelCase keys via MessageToDict.
+    # Try both camelCase and snake_case to be robust to lib version drift.
+    for snake, label in _TELEMETRY_VARIANTS:
+        camel = _snake_to_camel(snake)
+        for key in (snake, camel):
+            value = telem.get(key)
+            if isinstance(value, dict):
+                return {
+                    "variant": label,
+                    "fields": {k: _scalarize(v) for k, v in value.items()},
+                    "time": telem.get("time"),
+                }
+    return None
+
+
+def _snake_to_camel(name: str) -> str:
+    parts = name.split("_")
+    return parts[0] + "".join(p.title() for p in parts[1:])
+
+
+def _scalarize(value: Any) -> Any:
+    """Keep telemetry fields JSON-friendly. Lists/dicts pass through
+    untouched; bytes -> hex string; protobuf enums occasionally arrive
+    as ints (fine) or strings (also fine)."""
+    if isinstance(value, (bytes, bytearray, memoryview)):
+        return bytes(value).hex()
+    return value
+
+
+# -- Generic packet summary ---------------------------------------------
+
+
+def summarize_packet(
+    packet: dict[str, Any], *, payload_hex_len: int = 64
+) -> dict[str, Any]:
+    """Reduce a packet dict to a stable, queryable summary. Drops the
+    full payload bytes — the recorder records summaries, not pcaps.
+    """
+    if not isinstance(packet, dict):
+        return {"raw_type": type(packet).__name__}
+    decoded = packet.get("decoded") if isinstance(packet.get("decoded"), dict) else {}
+    portnum = decoded.get("portnum") if isinstance(decoded, dict) else None
+    payload = decoded.get("payload") if isinstance(decoded, dict) else None
+    payload_hex = None
+    payload_size = None
+    if isinstance(payload, (bytes, bytearray, memoryview)):
+        b = bytes(payload)
+        payload_size = len(b)
+        payload_hex = b[:payload_hex_len].hex() if b else ""
+    elif isinstance(payload, str):
+        # Some decoded payloads (text messages) come as decoded strings.
+        payload_size = len(payload)
+        payload_hex = None  # not bytes
+    return {
+        "from_node": packet.get("fromId") or packet.get("from"),
+        "to_node": packet.get("toId") or packet.get("to"),
+        "portnum": portnum,
+        "hop_limit": packet.get("hopLimit"),
+        "want_ack": packet.get("wantAck"),
+        "rx_rssi": packet.get("rxRssi"),
+        "rx_snr": packet.get("rxSnr"),
+        "channel": packet.get("channel"),
+        "id": packet.get("id"),
+        "payload_size": payload_size,
+        "payload_hex_prefix": payload_hex,
+    }
+
+
+# -- Interface identification ------------------------------------------
+
+
+def interface_label(interface: Any) -> dict[str, Any]:
+    """Stable identifier for the meshtastic interface that emitted an event.
+
+    Used as the `port`/`role` tag on every recorded row. SerialInterface
+    has `devPath`; TCPInterface has `hostname`+`portNumber`; BLEInterface
+    has `address`. Falls back to the class name when none of those exist.
+    """
+    if interface is None:
+        return {"port": None, "role": None}
+    dev_path = getattr(interface, "devPath", None)
+    if dev_path:
+        return {"port": str(dev_path), "role": "serial"}
+    hostname = getattr(interface, "hostname", None)
+    if hostname:
+        port_num = getattr(interface, "portNumber", None)
+        endpoint = f"tcp://{hostname}:{port_num}" if port_num else f"tcp://{hostname}"
+        return {"port": endpoint, "role": "tcp"}
+    address = getattr(interface, "address", None)
+    if address:
+        return {"port": str(address), "role": "ble"}
+    return {"port": type(interface).__name__, "role": None}

mcp-server/src/meshtastic_mcp/recorder/recorder.py

@@ -0,0 +1,467 @@
+"""Process-global recorder singleton.
+
+Subscribes once to the meshtastic pubsub fan-out and writes four append-only
+JSONL streams under `mcp-server/.mtlog/`. The pubsub fan-out is
+process-global — a single subscription captures every active interface
+without per-connection bookkeeping.
+
+Files:
+  logs.jsonl      — every `meshtastic.log.line` event (best-effort prefix
+                    parsed for level/tag/uptime; raw `line` always preserved)
+  telemetry.jsonl — `meshtastic.receive.telemetry` packets, flattened by
+                    variant (device / local / environment / power / etc.)
+  packets.jsonl   — every other `meshtastic.receive.*` packet, summarized
+                    (portnum, hops, RSSI/SNR, payload size + 64-byte hex)
+  events.jsonl    — connection lifecycle, node-DB updates, and manual
+                    `mark_event` rows. Lower volume; useful for aligning
+                    timelines.
+
+Pause/resume: `pause()` flips a flag; subscriptions stay registered. The
+write methods short-circuit when paused, so we don't lose ordering when
+resumed (we just have a gap). No queueing.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import threading
+import time
+from pathlib import Path
+from typing import Any
+
+from . import parsers
+from .rotating import _RotatingJsonl
+
+_DEFAULT_DIR = Path(__file__).resolve().parents[3] / ".mtlog"
+log = logging.getLogger(__name__)
+
+
+class Recorder:
+    """Singleton write-side of the persistent log capture system."""
+
+    def __init__(self, base_dir: Path | None = None) -> None:
+        self.base_dir = Path(base_dir) if base_dir else _DEFAULT_DIR
+        self._lock = threading.RLock()
+        self._started = False
+        self._paused = False
+        self._pause_reason: str | None = None
+        self._started_at: float | None = None
+        self._handlers: list[tuple[str, Any]] = []
+        self._files: dict[str, _RotatingJsonl] = {}
+
+    # -- lifecycle ----------------------------------------------------
+
+    def start(self) -> None:
+        """Idempotent. Safe to call from FastMCP app startup."""
+        with self._lock:
+            if self._started:
+                return
+            self.base_dir.mkdir(parents=True, exist_ok=True)
+            self._files = {
+                "logs": _RotatingJsonl(self.base_dir / "logs.jsonl"),
+                "telemetry": _RotatingJsonl(self.base_dir / "telemetry.jsonl"),
+                "packets": _RotatingJsonl(self.base_dir / "packets.jsonl"),
+                "events": _RotatingJsonl(self.base_dir / "events.jsonl"),
+            }
+            self._wire_pubsub()
+            self._started = True
+            self._started_at = time.time()
+        # Write the recorder_start marker after the initialization block.
+        # `_write_event()` re-checks recorder state via `_files_snapshot()`,
+        # so keeping this out of the setup block avoids nested lifecycle work.
+        self._write_event(kind="recorder_start", label="recorder_started")
+
+    def stop(self) -> None:
+        with self._lock:
+            if not self._started:
+                return
+            self._unwire_pubsub()
+            for f in self._files.values():
+                f.close()
+            self._files = {}
+            self._started = False
+
+    def pause(self, reason: str | None = None) -> None:
+        # Write the pause marker BEFORE flipping the flag — `_write_event`
+        # short-circuits when paused, so the order matters for this event
+        # to actually land in events.jsonl.
+        self._write_event(
+            kind="recorder_pause",
+            label="paused",
+            note=reason,
+        )
+        with self._lock:
+            self._paused = True
+            self._pause_reason = reason
+
+    def resume(self) -> None:
+        # Mirror of `pause()`: clear the flag first, then write the marker
+        # so it isn't suppressed by the still-paused short-circuit.
+        with self._lock:
+            self._paused = False
+            self._pause_reason = None
+        self._write_event(kind="recorder_resume", label="resumed")
+
+    # -- pubsub wiring ------------------------------------------------
+
+    def _wire_pubsub(self) -> None:
+        from pubsub import pub  # type: ignore[import-untyped]
+
+        # Subscribers — one per topic. Each pubsub publisher sends
+        # keyword args matching its handler's signature; pubsub
+        # introspects the function signature to route args.
+        bindings = [
+            ("meshtastic.log.line", self._on_log_line),
+            ("meshtastic.serial.line", self._on_serial_line),
+            ("meshtastic.receive", self._on_receive),
+            ("meshtastic.receive.telemetry", self._on_telemetry),
+            ("meshtastic.connection.established", self._on_connection_established),
+            ("meshtastic.connection.lost", self._on_connection_lost),
+            ("meshtastic.node.updated", self._on_node_updated),
+        ]
+        for topic, handler in bindings:
+            try:
+                pub.subscribe(handler, topic)
+                self._handlers.append((topic, handler))
+            except Exception as exc:
+                # If pubsub refuses one binding (signature mismatch on
+                # an old lib version), log it and keep the rest.
+                log.warning("Recorder failed to subscribe to %s: %s", topic, exc)
+
+    def _unwire_pubsub(self) -> None:
+        from pubsub import pub  # type: ignore[import-untyped]
+
+        for topic, handler in self._handlers:
+            try:
+                pub.unsubscribe(handler, topic)
+            except Exception:
+                pass
+        self._handlers.clear()
+
+    # -- handlers -----------------------------------------------------
+    #
+    # Pubsub callbacks must never raise. Every handler is wrapped in a
+    # try/except that swallows so a bug here can't take down the
+    # SerialInterface receive thread.
+    #
+    # Threading: handlers fire on whatever thread the meshtastic library
+    # dispatches from (varies by interface), while `stop()` clears
+    # `self._files` under `self._lock`. We snapshot `_files` under the
+    # lock at the top of each handler so a concurrent stop can't
+    # KeyError us mid-write. The actual file write goes through
+    # `_RotatingJsonl` which has its own lock.
+
+    def _files_snapshot(self) -> dict[str, _RotatingJsonl] | None:
+        """Atomic-ish view of `self._files`. Returns None when the recorder
+        is paused or stopped, so handlers can early-exit cleanly without
+        racing `stop()`'s clear."""
+        with self._lock:
+            if not self._started or self._paused:
+                return None
+            return dict(self._files)
+
+    def _on_log_line(self, line: str, interface: Any = None) -> None:
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            tags = parsers.interface_label(interface)
+            parsed = parsers.parse_log_line(str(line))
+            ts = time.time()
+            record: dict[str, Any] = {
+                "ts": ts,
+                "port": tags["port"],
+                "role": tags["role"],
+                "level": parsed.get("level"),
+                "tag": parsed.get("tag"),
+                "uptime_s": parsed.get("uptime_s"),
+                "line": parsed["line"],
+            }
+            # DEBUG_HEAP enrichments (only present when the firmware
+            # was built with -DDEBUG_HEAP=1). Surface as first-class
+            # fields so logs_window can grep/filter on them and so
+            # heap_free synthesizes a telemetry point below.
+            if "heap_free" in parsed:
+                record["heap_free"] = parsed["heap_free"]
+            if "heap_total" in parsed:
+                record["heap_total"] = parsed["heap_total"]
+            if "heap_delta" in parsed:
+                record["heap_delta"] = parsed["heap_delta"]
+            heap_event = parsed.get("heap_event")
+            if heap_event:
+                record["heap_event"] = heap_event
+            files["logs"].write(record)
+
+            # If the line carried a heap snapshot, also write it as a
+            # synthesized LocalStats-shaped row so telemetry_timeline
+            # picks it up at log cadence (much higher resolution than
+            # the ~60 s LocalStats packet). Tagged source=debug_heap so
+            # consumers can filter if mixing scales is unwanted.
+            heap_free = parsed.get("heap_free")
+            if isinstance(heap_free, int):
+                fields: dict[str, Any] = {"heap_free_bytes": heap_free}
+                heap_total = parsed.get("heap_total")
+                if isinstance(heap_total, int):
+                    fields["heap_total_bytes"] = heap_total
+                files["telemetry"].write(
+                    {
+                        "ts": ts,
+                        "port": tags["port"],
+                        "role": tags["role"],
+                        "from_node": None,
+                        "variant": "local",
+                        "fields": fields,
+                        "source": "debug_heap",
+                    }
+                )
+        except Exception:
+            pass
+
+    def _on_serial_line(self, line: str, port: str | None = None) -> None:
+        """Text-mode passive tap. Fired from `serial_session._drain` when a
+        `pio device monitor` subprocess is running.
+
+        Same parse + heap-synthesis path as `_on_log_line`, but receives
+        the raw text-formatted line (full level/clock/uptime/thread/`[heap N]`/
+        body). On DEBUG_HEAP builds in text mode this gives us per-log-line
+        heap data — far higher cadence than LocalStats, and works without
+        protobuf API mode (no SerialInterface required).
+        """
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            parsed = parsers.parse_log_line(str(line))
+            ts = time.time()
+            record: dict[str, Any] = {
+                "ts": ts,
+                "port": port,
+                "role": "serial_session",
+                "level": parsed.get("level"),
+                "tag": parsed.get("tag"),
+                "uptime_s": parsed.get("uptime_s"),
+                "line": parsed["line"],
+            }
+            if "heap_free" in parsed:
+                record["heap_free"] = parsed["heap_free"]
+            if "heap_total" in parsed:
+                record["heap_total"] = parsed["heap_total"]
+            if "heap_delta" in parsed:
+                record["heap_delta"] = parsed["heap_delta"]
+            heap_event = parsed.get("heap_event")
+            if heap_event:
+                record["heap_event"] = heap_event
+            files["logs"].write(record)
+
+            # Synthesize a heap_free telemetry sample whenever the line
+            # carries one — same logic as _on_log_line, tagged source so
+            # consumers can distinguish text-mode tap from protobuf path.
+            heap_free = parsed.get("heap_free")
+            if isinstance(heap_free, int):
+                fields: dict[str, Any] = {"heap_free_bytes": heap_free}
+                heap_total = parsed.get("heap_total")
+                if isinstance(heap_total, int):
+                    fields["heap_total_bytes"] = heap_total
+                files["telemetry"].write(
+                    {
+                        "ts": ts,
+                        "port": port,
+                        "role": "serial_session",
+                        "from_node": None,
+                        "variant": "local",
+                        "fields": fields,
+                        "source": "debug_heap_serial",
+                    }
+                )
+        except Exception:
+            pass
+
+    def _on_telemetry(self, packet: dict[str, Any], interface: Any = None) -> None:
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            tags = parsers.interface_label(interface)
+            extracted = parsers.extract_telemetry(packet)
+            if extracted is None:
+                # Couldn't extract a known variant — fall through to the
+                # generic `_on_receive` path, which will still fire for
+                # this packet via the parent topic.
+                return
+            record = {
+                "ts": time.time(),
+                "port": tags["port"],
+                "role": tags["role"],
+                "from_node": packet.get("fromId") or packet.get("from"),
+                "variant": extracted["variant"],
+                "fields": extracted["fields"],
+                "device_time": extracted.get("time"),
+            }
+            files["telemetry"].write(record)
+        except Exception:
+            pass
+
+    def _on_receive(self, packet: dict[str, Any], interface: Any = None) -> None:
+        # Generic-receive fires for EVERY packet. Telemetry packets get
+        # recorded twice (here and in _on_telemetry) — that's intentional:
+        # packets.jsonl is the universal record, telemetry.jsonl is the
+        # structured timeseries view.
+        files = self._files_snapshot()
+        if files is None:
+            return
+        try:
+            tags = parsers.interface_label(interface)
+            summary = parsers.summarize_packet(packet)
+            record = {
+                "ts": time.time(),
+                "port": tags["port"],
+                "role": tags["role"],
+                **summary,
+            }
+            files["packets"].write(record)
+        except Exception:
+            pass
+
+    def _on_connection_established(self, interface: Any = None) -> None:
+        self._write_event(
+            kind="connection_established",
+            interface=interface,
+        )
+
+    def _on_connection_lost(self, interface: Any = None) -> None:
+        self._write_event(
+            kind="connection_lost",
+            interface=interface,
+        )
+
+    def _on_node_updated(
+        self, node: dict[str, Any] | None = None, interface: Any = None
+    ) -> None:
+        # Lower-volume than packets but informative — node ID, hops away,
+        # last heard. Skip the user dict if absent.
+        try:
+            user = (node or {}).get("user") if isinstance(node, dict) else None
+            self._write_event(
+                kind="node_updated",
+                interface=interface,
+                data={
+                    "num": (node or {}).get("num"),
+                    "id": (user or {}).get("id"),
+                    "short": (user or {}).get("shortName"),
+                    "long": (user or {}).get("longName"),
+                    "hops_away": (node or {}).get("hopsAway"),
+                    "snr": (node or {}).get("snr"),
+                    "last_heard": (node or {}).get("lastHeard"),
+                },
+            )
+        except Exception:
+            pass
+
+    # -- public write helpers -----------------------------------------
+
+    def mark_event(
+        self,
+        label: str,
+        note: str | None = None,
+        data: dict[str, Any] | None = None,
+    ) -> dict[str, Any]:
+        """User-facing marker. Writes to events.jsonl AND emits a
+        synthetic logs.jsonl row tagged level=MARK so timelines align.
+        """
+        ts = self._write_event(kind="mark", label=label, note=note, data=data)
+        # Mirror into logs so a single logs_window grep finds it.
+        files = self._files_snapshot()
+        if files is not None:
+            try:
+                files["logs"].write(
+                    {
+                        "ts": ts,
+                        "port": None,
+                        "role": "marker",
+                        "level": "MARK",
+                        "tag": "mark_event",
+                        "line": f"[mark] {label}" + (f" — {note}" if note else ""),
+                    }
+                )
+            except Exception:
+                pass
+        return {"ts": ts, "label": label}
+
+    def _write_event(
+        self,
+        *,
+        kind: str,
+        label: str | None = None,
+        note: str | None = None,
+        interface: Any = None,
+        data: dict[str, Any] | None = None,
+    ) -> float:
+        ts = time.time()
+        # Lifecycle markers (recorder_start, recorder_pause, recorder_resume)
+        # arrive at choreographed moments — `pause()` writes BEFORE flipping
+        # the flag and `resume()` writes AFTER clearing it, so those calls
+        # see _paused=False here. Other event kinds short-circuit when
+        # paused via the snapshot guard below.
+        files = self._files_snapshot()
+        if files is None:
+            return ts
+        try:
+            tags = parsers.interface_label(interface)
+            files["events"].write(
+                {
+                    "ts": ts,
+                    "kind": kind,
+                    "label": label,
+                    "note": note,
+                    "port": tags["port"],
+                    "role": tags["role"],
+                    "data": data,
+                }
+            )
+        except Exception:
+            pass
+        return ts
+
+    # -- introspection ------------------------------------------------
+
+    def status(self) -> dict[str, Any]:
+        with self._lock:
+            return {
+                "running": self._started,
+                "paused": self._paused,
+                "pause_reason": self._pause_reason,
+                "started_at": self._started_at,
+                "base_dir": str(self.base_dir),
+                "files": {name: f.status() for name, f in self._files.items()},
+            }
+
+    def force_rotate_all(self) -> dict[str, Any]:
+        """Test/admin hook: rotate every stream right now."""
+        with self._lock:
+            files = list(self._files.values())
+        for f in files:
+            f.force_rotate()
+        # `status()` re-acquires `self._lock`; release before calling it.
+        return self.status()
+
+
+# -- module-level singleton accessor ------------------------------------
+
+_INSTANCE_LOCK = threading.Lock()
+_INSTANCE: Recorder | None = None
+
+
+def get_recorder() -> Recorder:
+    """Return the process-global Recorder. Created on first call.
+
+    Honors `MESHTASTIC_MCP_LOG_DIR` env var for the base directory
+    (used by tests to redirect to a tmpdir).
+    """
+    global _INSTANCE
+    with _INSTANCE_LOCK:
+        if _INSTANCE is None:
+            override = os.environ.get("MESHTASTIC_MCP_LOG_DIR")
+            base = Path(override) if override else None
+            _INSTANCE = Recorder(base_dir=base)
+        return _INSTANCE

mcp-server/src/meshtastic_mcp/recorder/rotating.py

@@ -0,0 +1,163 @@
+"""Append-only JSONL writer with size-capped rotation.
+
+A `_RotatingJsonl` owns one live `.jsonl` file. Writes are line-delimited
+JSON objects (one row per call). When the live file exceeds `max_bytes`,
+it is closed, gzipped to `<name>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`,
+and the live file resets to empty. Old archives past `keep_archives` are
+unlinked oldest-first.
+
+Size check is amortized — `os.fstat` runs every `check_every` writes,
+not per-write, so the hot path stays at one `fh.write` + one `fh.flush`.
+
+Threading: every public method acquires `self._lock`. The recorder runs
+several pubsub handlers on whatever thread the meshtastic library
+dispatches from (varies by interface), and queries from MCP tool calls
+arrive on the FastMCP request thread, so this lock is not optional.
+"""
+
+from __future__ import annotations
+
+import gzip
+import json
+import os
+import shutil
+import threading
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any
+
+
+class _RotatingJsonl:
+    """Append-only JSONL with size rotation. Thread-safe."""
+
+    def __init__(
+        self,
+        path: Path,
+        *,
+        max_bytes: int = 100 * 1024 * 1024,
+        keep_archives: int = 5,
+        check_every: int = 1000,
+    ) -> None:
+        self.path = path
+        self.max_bytes = max_bytes
+        self.keep_archives = keep_archives
+        self.check_every = check_every
+        self._lock = threading.Lock()
+        self._fh: Any = None
+        self._writes_since_check = 0
+        self._rotations = 0
+        self._lines_written = 0
+        self._last_ts: float | None = None
+        self._open()
+
+    # -- lifecycle ----------------------------------------------------
+
+    def _open(self) -> None:
+        self.path.parent.mkdir(parents=True, exist_ok=True)
+        self._fh = self.path.open("a", encoding="utf-8")
+
+    def close(self) -> None:
+        with self._lock:
+            if self._fh is not None:
+                try:
+                    self._fh.close()
+                finally:
+                    self._fh = None
+
+    # -- write --------------------------------------------------------
+
+    def write(self, record: dict[str, Any]) -> None:
+        """Append one JSON object as a line. Triggers rotation if oversized."""
+        line = json.dumps(record, separators=(",", ":"), default=str) + "\n"
+        with self._lock:
+            if self._fh is None:
+                return
+            try:
+                self._fh.write(line)
+                self._fh.flush()
+            except Exception:
+                # Best-effort: a failed write must not crash the pubsub
+                # handler. Caller has no way to react anyway.
+                return
+            self._lines_written += 1
+            ts = record.get("ts")
+            if isinstance(ts, (int, float)):
+                self._last_ts = float(ts)
+            self._writes_since_check += 1
+            if self._writes_since_check >= self.check_every:
+                self._writes_since_check = 0
+                self._maybe_rotate()
+
+    # -- rotation -----------------------------------------------------
+
+    def _maybe_rotate(self) -> None:
+        # Caller holds self._lock.
+        try:
+            size = os.fstat(self._fh.fileno()).st_size
+        except OSError:
+            return
+        if size < self.max_bytes:
+            return
+        self._rotate_locked()
+
+    def _rotate_locked(self) -> None:
+        # Close, gzip-rename, reopen empty, prune oldest archives.
+        try:
+            self._fh.close()
+        except Exception:
+            pass
+        self._fh = None
+        # Microsecond-resolution timestamp + per-instance counter so back-
+        # to-back rotations (small max_bytes, repeated `force_rotate()`,
+        # or chatty test loops) get unique archive filenames. The lex
+        # sort order of `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` is chronological,
+        # which `_prune_archives()` and `log_query._iter_jsonl()` both
+        # rely on.
+        stamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S-%f")
+        archive = self.path.with_suffix(f".{stamp}-{self._rotations:05d}.jsonl.gz")
+        try:
+            with self.path.open("rb") as src, gzip.open(archive, "wb") as dst:
+                shutil.copyfileobj(src, dst, length=1024 * 1024)
+            self.path.unlink()
+        except Exception:
+            # Rotation is best-effort. If gzip fails, leave the file
+            # in place and re-open it; we'll try again next check.
+            pass
+        self._open()
+        self._rotations += 1
+        self._prune_archives()
+
+    def _prune_archives(self) -> None:
+        # Match siblings of self.path.name with `.jsonl.gz` suffix.
+        prefix = self.path.stem  # "logs" for "logs.jsonl"
+        # Archive filenames are already lexicographically chronological.
+        # Prune by name, not mtime, so copied/restored files don't reorder.
+        archives = sorted(self.path.parent.glob(f"{prefix}.*.jsonl.gz"))
+        excess = len(archives) - self.keep_archives
+        for old in archives[: max(0, excess)]:
+            try:
+                old.unlink()
+            except OSError:
+                pass
+
+    def force_rotate(self) -> None:
+        """Test/admin hook: rotate immediately regardless of size."""
+        with self._lock:
+            if self._fh is not None:
+                self._rotate_locked()
+
+    # -- introspection ------------------------------------------------
+
+    def status(self) -> dict[str, Any]:
+        with self._lock:
+            try:
+                size = os.fstat(self._fh.fileno()).st_size if self._fh else 0
+            except OSError:
+                size = 0
+            return {
+                "path": str(self.path),
+                "size": size,
+                "lines": self._lines_written,
+                "last_ts": self._last_ts,
+                "rotations": self._rotations,
+            }

mcp-server/src/meshtastic_mcp/serial_session.py

@@ -46,7 +46,23 @@ class SerialSession:
 
 
 def _drain(session: SerialSession) -> None:
-    """Reader thread: line-by-line pull stdout into buffer."""
+    """Reader thread: line-by-line pull stdout into buffer.
+
+    Each line is also published to the `meshtastic.serial.line` pubsub
+    topic so the persistent recorder can capture it without holding its
+    own port. This is the text-mode tap path: when no SerialInterface is
+    open, the firmware emits full formatted lines (level + clock + uptime
+    + thread + `[heap N]` prefix on DEBUG_HEAP builds + body), and we
+    fan them out to whoever is listening. Pubsub is best-effort —
+    publish failures must never block the reader.
+    """
+    # Lazy import: pubsub isn't required just to import this module
+    # (e.g., during static analysis), and we want a clean test surface.
+    try:
+        from pubsub import pub  # type: ignore[import-untyped]
+    except Exception:  # pragma: no cover - defensive
+        pub = None
+
     assert session.proc.stdout is not None
     try:
         for line in session.proc.stdout:
@@ -54,6 +70,16 @@ def _drain(session: SerialSession) -> None:
             with session.lock:
                 session.buffer.append(line_stripped)
                 session.total_lines += 1
+            if pub is not None:
+                try:
+                    pub.sendMessage(
+                        "meshtastic.serial.line",
+                        line=line_stripped,
+                        port=session.port,
+                    )
+                except Exception:
+                    # A subscriber raising must not break the reader.
+                    pass
     except Exception:  # pragma: no cover - defensive
         pass
     finally:

mcp-server/src/meshtastic_mcp/server.py

@@ -6,6 +6,7 @@ etc.). Business logic does not live here.
 
 from __future__ import annotations
 
+import logging
 from typing import Any
 
 from mcp.server.fastmcp import FastMCP
@@ -14,17 +15,38 @@ from . import (
     admin,
     boards,
     devices,
+    fixtures,
     flash,
     hw_tools,
     info,
+    log_query,
     registry,
     serial_session,
 )
 from . import userprefs as userprefs_mod
+from .recorder import get_recorder
+
+log = logging.getLogger(__name__)
 
 app = FastMCP("meshtastic-mcp")
 
 
+def _start_recorder() -> None:
+    # Persistent device-log capture. Starts on first import — pubsub fan-out
+    # is process-global, so subscribing here captures every active interface
+    # (whether opened by an MCP tool, a pytest fixture, or a serial_session).
+    # Files land in mcp-server/.mtlog/ (gitignored). See recorder/recorder.py
+    # for the full design. Recorder startup is best-effort: an unwritable
+    # log dir or pubsub mismatch should not take the MCP server down.
+    try:
+        get_recorder().start()
+    except Exception as exc:
+        log.warning("Failed to start persistent recorder: %s", exc)
+
+
+_start_recorder()
+
+
 # ---------- Discovery & metadata ------------------------------------------
 
 
@@ -75,6 +97,7 @@ def build(
     env: str,
     with_manifest: bool = True,
     userprefs: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Build firmware for one env via `pio run -e <env>`.
 
@@ -86,8 +109,21 @@ def build(
     build via userPrefs.jsonc injection. The file is restored after the build
     completes. Use `userprefs_manifest` to discover available keys. Use
     `userprefs_set` for persistent changes.
+
+    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for this build
+    only, injected via `PLATFORMIO_BUILD_FLAGS`. Common pattern:
+    `build_flags={"DEBUG_HEAP": 1}` enables per-thread leak detection + a
+    `[heap N]` prefix on every log line. The recorder picks the prefix up
+    automatically and synthesizes a high-resolution heap timeline that
+    `telemetry_timeline(field="free_heap")` can read alongside the normal
+    ~60 s LocalStats packets. Pair with `/leakhunt` for classification.
     """
-    return flash.build(env, with_manifest=with_manifest, userprefs_overrides=userprefs)
+    return flash.build(
+        env,
+        with_manifest=with_manifest,
+        userprefs_overrides=userprefs,
+        build_flags=build_flags,
+    )
 
 
 @app.tool()
@@ -105,6 +141,7 @@ def pio_flash(
     port: str,
     confirm: bool = False,
     userprefs: dict[str, Any] | None = None,
+    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Flash firmware via `pio run -e <env> -t upload --upload-port <port>`.
 
@@ -114,8 +151,19 @@ def pio_flash(
 
     `userprefs` (optional): dict of `USERPREFS_<KEY>: value` baked into this
     build via userPrefs.jsonc injection; restored after upload.
+
+    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for the
+    rebuild-before-upload, e.g. `{"DEBUG_HEAP": 1}`. Required for the flags
+    to actually land in the uploaded firmware — without it, the implicit
+    rebuild relinks without the env var and silently drops them.
     """
-    return flash.flash(env, port, confirm=confirm, userprefs_overrides=userprefs)
+    return flash.flash(
+        env,
+        port,
+        confirm=confirm,
+        userprefs_overrides=userprefs,
+        build_flags=build_flags,
+    )
 
 
 @app.tool()
@@ -734,3 +782,227 @@ def picotool_load(uf2_path: str, confirm: bool = False) -> dict[str, Any]:
 def picotool_raw(args: list[str], confirm: bool = False) -> dict[str, Any]:
     """Pass-through to `picotool`. load/reboot/save/erase require confirm=True."""
     return hw_tools.picotool_raw(args, confirm=confirm)
+
+
+# ---------- Persistent device-log capture (recorder) ----------------------
+#
+# The recorder is autouse — it starts at server import and continuously
+# writes every meshtastic pubsub event to JSONL files under .mtlog/. These
+# tools are query-only over those files, plus a few lifecycle controls.
+
+
+@app.tool()
+def logs_window(
+    start: str = "-15m",
+    end: str = "now",
+    grep: str | None = None,
+    level: str | None = None,
+    tag: str | None = None,
+    port: str | None = None,
+    max_lines: int = 200,
+) -> dict[str, Any]:
+    """Recent firmware log lines from the persistent recorder.
+
+    Filters by time window, regex over the line, level (single or
+    pipe-separated set like "WARN|ERROR|CRIT"), thread-name tag, and
+    interface port. Returns up to max_lines most-recent matches.
+
+    Time strings: "-15m", "-2h", "-3d", "now", or ISO 8601.
+
+    Note: lines arriving via the LogRecord protobuf path (when
+    set_debug_log_api(True) is on) come without level prefix — the
+    meshtastic Python lib drops record.level before fan-out. For those,
+    `level` filter won't match; use `grep` instead.
+    """
+    return log_query.logs_window(
+        start=start,
+        end=end,
+        grep=grep,
+        level=level,
+        tag=tag,
+        port=port,
+        max_lines=max_lines,
+    )
+
+
+@app.tool()
+def telemetry_timeline(
+    window: str = "1h",
+    variant: str = "local",
+    field: str = "free_heap",
+    port: str | None = None,
+    max_points: int = 200,
+) -> dict[str, Any]:
+    """Time series of one telemetry field, downsampled to <= max_points.
+
+    `variant` ∈ device, local, environment, power, airQuality, health, host.
+    `field` accepts snake_case or camelCase; common aliases (free_heap ↔
+    heap_free_bytes) are normalized.
+
+    Returns slope_per_min (linear-regression slope, units/minute) so a
+    leak detector can read one number — negative slope on free_heap over
+    a long window indicates a real leak.
+
+    LocalStats variant ("local") cadence is ~60 s (whatever the device's
+    `device_update_interval` is set to), so a 1 h window gives ~60 raw
+    points. Bucket-mean downsampling preserves shape.
+    """
+    return log_query.telemetry_timeline(
+        window=window,
+        variant=variant,
+        field=field,
+        port=port,
+        max_points=max_points,
+    )
+
+
+@app.tool()
+def packets_window(
+    start: str = "-5m",
+    end: str = "now",
+    portnum: str | None = None,
+    from_node: str | None = None,
+    to_node: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    """Recent mesh packets recorded by the recorder.
+
+    Each row is a summary (portnum, from/to, hop_limit, RSSI/SNR, payload
+    size + first 64 bytes hex) — full payload bytes are not stored.
+    `portnum` accepts a pipe-separated set like "TEXT_MESSAGE_APP|POSITION_APP".
+    """
+    return log_query.packets_window(
+        start=start,
+        end=end,
+        portnum=portnum,
+        from_node=from_node,
+        to_node=to_node,
+        max=max,
+    )
+
+
+@app.tool()
+def events_window(
+    start: str = "-1h",
+    end: str = "now",
+    kind: str | None = None,
+    max: int = 200,
+) -> dict[str, Any]:
+    """Return recorder events: connection lifecycle, node updates, and `mark_event` markers.
+
+    `kind` ∈ recorder_start, recorder_pause, recorder_resume,
+    connection_established, connection_lost, node_updated, mark.
+    Pipe-separated sets ("connection_lost|connection_established") work.
+    """
+    return log_query.events_window(start=start, end=end, kind=kind, max=max)
+
+
+@app.tool()
+def mark_event(
+    label: str,
+    note: str | None = None,
+    data: dict[str, Any] | None = None,
+) -> dict[str, Any]:
+    """Drop a named marker into events.jsonl AND logs.jsonl.
+
+    Useful for aligning a timeline around a known stimulus: call before
+    and after a stress workload, then query telemetry_timeline /
+    logs_window with the markers' timestamps as bounds.
+
+    The marker also lands in logs.jsonl with level=MARK so a single
+    grep over logs picks it up.
+    """
+    return get_recorder().mark_event(label=label, note=note, data=data)
+
+
+@app.tool()
+def recorder_status() -> dict[str, Any]:
+    """Return recorder runtime info: running, paused, file sizes, last_ts per stream.
+
+    Use this to sanity-check that capture is working before you trust a
+    `logs_window` / `telemetry_timeline` result.
+    """
+    return get_recorder().status()
+
+
+@app.tool()
+def recorder_pause(reason: str | None = None) -> dict[str, Any]:
+    """Pause writes to all four streams. Pubsub subscriptions stay active —
+    we just drop events on the floor while paused. Resume with `recorder_resume`.
+
+    Use when capturing a known-good baseline that you don't want to
+    pollute with pre-test noise. Default state is recording; this is
+    rarely needed.
+    """
+    get_recorder().pause(reason=reason)
+    return {"ok": True, "paused": True, "reason": reason}
+
+
+@app.tool()
+def recorder_resume() -> dict[str, Any]:
+    """Resume writes after `recorder_pause`. No-op if already running."""
+    get_recorder().resume()
+    return {"ok": True, "paused": False}
+
+
+@app.tool()
+def recorder_export(
+    start: str,
+    end: str,
+    dest_dir: str,
+    streams: list[str] | None = None,
+) -> dict[str, Any]:
+    """Bundle a slice of the recorder's streams into `dest_dir`.
+
+    Writes one uncompressed JSONL per requested stream (logs / telemetry /
+    packets / events). Useful for: attaching to a bug report, feeding a
+    notebook, or backfilling Datadog after the fact.
+    """
+    return log_query.export(
+        start=start,
+        end=end,
+        dest_dir=dest_dir,
+        streams=streams,
+    )
+
+
+# ---------- Fixture / test-data push --------------------------------------
+
+
+@app.tool()
+def push_fake_nodedb(
+    size: int,
+    target: str = "portduino",
+    port: str | None = None,
+    portduino_config: str = "default",
+    backup_existing: bool = True,
+    confirm: bool = False,
+    reboot_after: bool = True,
+    custom_seed_jsonl: str | None = None,
+) -> dict[str, Any]:
+    """Push a fake-NodeDB v25 fixture (250/500/1000/2000 nodes) onto a device.
+
+    Two transports:
+      target="portduino" — file copy to ~/.portduino/<portduino_config>/prefs/nodes.proto.
+                            Fast, no device connection needed.
+      target="hardware"  — XModem upload over serial/BLE to /prefs/nodes.proto.
+                            Requires `port` + `confirm=True`. Triggers a reboot
+                            so loadFromDisk picks up the new file at next boot.
+
+    Compiles a fresh-timestamp proto from the committed JSONL seed under
+    test/fixtures/nodedb/seed_v25_<N>.jsonl each invocation, so the loaded
+    NodeDB always looks "recent" to the connecting phone. Structural data
+    (names, IDs, positions, telemetries) is deterministic per the seed.
+
+    Override the JSONL via `custom_seed_jsonl` to push a hand-edited scenario.
+    """
+    return fixtures.push_fake_nodedb(
+        size=size,
+        target=target,  # type: ignore[arg-type]
+        port=port,
+        portduino_config=portduino_config,
+        backup_existing=backup_existing,
+        confirm=confirm,
+        reboot_after=reboot_after,
+        custom_seed_jsonl=custom_seed_jsonl,
+    )

mcp-server/tests/unit/test_build_flags.py

@@ -0,0 +1,88 @@
+"""Unit tests for the `build_flags` injection on `flash.build()`.
+
+We don't actually run pio here — too slow, requires hardware-aware envs.
+We test the translation layer (`_build_flags_env`) and that the env vars
+are threaded through pio.run correctly via mock.
+"""
+
+from __future__ import annotations
+
+from unittest.mock import patch
+
+from meshtastic_mcp import flash, pio
+
+
+class TestBuildFlagsEnv:
+    def test_simple_value(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": 1})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"}
+
+    def test_string_value(self) -> None:
+        out = flash._build_flags_env({"FOO": "bar"})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DFOO=bar"}
+
+    def test_bool_true_is_bare_flag(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": True})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP"}
+
+    def test_bool_false_dropped(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": False, "OTHER": 1})
+        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DOTHER=1"}
+
+    def test_none_dropped(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": None})
+        assert out == {}
+
+    def test_multiple_combined(self) -> None:
+        out = flash._build_flags_env({"DEBUG_HEAP": 1, "FOO": "x", "BAR": True})
+        # Order isn't guaranteed in dict iteration, so check membership.
+        flags = out["PLATFORMIO_BUILD_FLAGS"].split()
+        assert set(flags) == {"-DDEBUG_HEAP=1", "-DFOO=x", "-DBAR"}
+
+
+class TestBuildPropagatesFlags:
+    def test_extra_env_passed_to_pio_run(self) -> None:
+        # Mock pio.run so we don't actually invoke pio. Capture extra_env.
+        captured = {}
+
+        class _StubResult:
+            returncode = 0
+            stdout = ""
+            stderr = ""
+            duration_s = 0.1
+
+        def _stub(args, **kwargs):
+            captured["args"] = args
+            captured["kwargs"] = kwargs
+            return _StubResult()
+
+        with patch.object(pio, "run", side_effect=_stub):
+            with patch.object(flash, "_artifacts_for", return_value=[]):
+                out = flash.build(
+                    "fake-env",
+                    with_manifest=False,
+                    build_flags={"DEBUG_HEAP": 1},
+                )
+        assert captured["args"] == ["run", "-e", "fake-env"]
+        assert captured["kwargs"]["extra_env"] == {
+            "PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"
+        }
+        assert out["build_flags"] == {"DEBUG_HEAP": 1}
+
+    def test_no_flags_means_no_extra_env(self) -> None:
+        captured = {}
+
+        class _StubResult:
+            returncode = 0
+            stdout = ""
+            stderr = ""
+            duration_s = 0.1
+
+        def _stub(args, **kwargs):
+            captured["kwargs"] = kwargs
+            return _StubResult()
+
+        with patch.object(pio, "run", side_effect=_stub):
+            with patch.object(flash, "_artifacts_for", return_value=[]):
+                flash.build("fake-env", with_manifest=False)
+        assert captured["kwargs"]["extra_env"] is None

mcp-server/tests/unit/test_fake_nodedb_generator.py

@@ -0,0 +1,364 @@
+"""Tests for the fake-NodeDB fixture pipeline (bin/gen-fake-nodedb-seed.py
++ bin/seed-json-to-proto.py + mcp-server fixtures.push_fake_nodedb).
+
+Lives under tests/unit/ because none of these touch real hardware — they
+shell out to the bin/ scripts and decode the resulting protobufs in-process.
+"""
+
+from __future__ import annotations
+
+import json
+import pathlib
+import subprocess
+import sys
+import time
+
+import pytest
+
+REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
+SEED_GEN = REPO_ROOT / "bin" / "gen-fake-nodedb-seed.py"
+COMPILE = REPO_ROOT / "bin" / "seed-json-to-proto.py"
+FIXTURES_DIR = REPO_ROOT / "test" / "fixtures" / "nodedb"
+
+# Ensure the locally-generated Python protobuf bindings are importable.
+# These live under `meshtastic_v25` (not `meshtastic`) so they don't shadow
+# the PyPI `meshtastic` package that the rest of the mcp-server depends on.
+_BINDINGS_DIR = REPO_ROOT / "bin" / "_generated"
+if _BINDINGS_DIR.is_dir() and str(_BINDINGS_DIR) not in sys.path:
+    sys.path.insert(0, str(_BINDINGS_DIR))
+
+try:
+    from meshtastic_v25.deviceonly_pb2 import (
+        NodeDatabase,  # type: ignore[import-not-found]
+    )
+except ImportError:
+    NodeDatabase = None  # type: ignore[assignment]
+
+
+def _require_v25_bindings() -> None:
+    if NodeDatabase is None:
+        pytest.skip(
+            "v25 Python protobuf bindings missing; run `./bin/regen-py-protos.sh`."
+        )
+    if "positions" not in NodeDatabase.DESCRIPTOR.fields_by_name:
+        pytest.skip(
+            "Loaded NodeDatabase predates v25 — run `./bin/regen-py-protos.sh`."
+        )
+
+
+def _run(cmd: list[str]) -> None:
+    subprocess.check_call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
+
+
+# ---------------------------------------------------------------------------
+# Seed generator: deterministic for given --seed (no wall-clock dependence).
+# ---------------------------------------------------------------------------
+def test_seed_generator_is_deterministic(tmp_path: pathlib.Path) -> None:
+    a = tmp_path / "a.jsonl"
+    b = tmp_path / "b.jsonl"
+    _run(
+        [
+            sys.executable,
+            str(SEED_GEN),
+            "--count",
+            "100",
+            "--seed",
+            "42",
+            "--out",
+            str(a),
+        ]
+    )
+    # Sleep so any sneaky wall-clock leak in the generator would surface as
+    # a byte diff between the two runs.
+    time.sleep(0.8)
+    _run(
+        [
+            sys.executable,
+            str(SEED_GEN),
+            "--count",
+            "100",
+            "--seed",
+            "42",
+            "--out",
+            str(b),
+        ]
+    )
+    assert a.read_bytes() == b.read_bytes()
+
+
+def test_seed_generator_meta_line(tmp_path: pathlib.Path) -> None:
+    out = tmp_path / "seed.jsonl"
+    _run(
+        [
+            sys.executable,
+            str(SEED_GEN),
+            "--count",
+            "50",
+            "--seed",
+            "1",
+            "--out",
+            str(out),
+        ]
+    )
+    lines = out.read_text(encoding="utf-8").splitlines()
+    assert len(lines) == 51  # 1 meta + 50 nodes
+    meta = json.loads(lines[0])
+    assert "_meta" in meta
+    assert meta["_meta"]["version"] == 25
+    assert meta["_meta"]["count"] == 50
+    assert meta["_meta"]["seed"] == 1
+
+
+def test_seed_only_uses_active_hardware_and_roles(tmp_path: pathlib.Path) -> None:
+    """Confirm no deprecated roles + no off-list HW models leak through."""
+    out = tmp_path / "seed.jsonl"
+    _run(
+        [
+            sys.executable,
+            str(SEED_GEN),
+            "--count",
+            "500",
+            "--seed",
+            "7",
+            "--out",
+            str(out),
+        ]
+    )
+    forbidden_roles = {"ROUTER_CLIENT", "REPEATER"}
+    forbidden_hw = {
+        "TLORA_V1",
+        "TLORA_V2",
+        "TLORA_V1_1P3",
+        "TLORA_V2_1_1P6",
+        "TLORA_V2_1_1P8",
+        "HELTEC_V1",
+        "HELTEC_V2_0",
+        "HELTEC_V2_1",
+        "TBEAM",
+        "TBEAM_V0P7",
+        "NANO_G1",
+        "NANO_G1_EXPLORER",
+        "NANO_G2_ULTRA",
+        "STATION_G1",
+        "STATION_G2",
+        "PORTDUINO",
+        "ANDROID_SIM",
+        "DIY_V1",
+        "LORA_RELAY_V1",
+        "NRF52840_PCA10059",
+        "NRF52_UNKNOWN",
+        "DR_DEV",
+        "GENIEBLOCKS",
+        "M5STACK",
+        "RP2040_LORA",
+        "PPR",
+    }
+    for raw in out.read_text(encoding="utf-8").splitlines()[1:]:
+        node = json.loads(raw)
+        assert node["role"] not in forbidden_roles, f"deprecated role: {node['role']}"
+        assert (
+            node["hw_model"] not in forbidden_hw
+        ), f"non-tier-1 HW: {node['hw_model']}"
+
+
+# ---------------------------------------------------------------------------
+# Compile step + committed seeds.
+# ---------------------------------------------------------------------------
+@pytest.mark.parametrize("size", [250, 500, 1000, 2000])
+def test_committed_seed_compiles_and_decodes(size: int, tmp_path: pathlib.Path) -> None:
+    _require_v25_bindings()
+    proto = tmp_path / "out.proto"
+    jsonl = FIXTURES_DIR / f"seed_v25_{size:04d}.jsonl"
+    if not jsonl.is_file():
+        pytest.skip(f"{jsonl} not present — run ./bin/regen-fake-nodedbs.sh")
+    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(proto)])
+
+    db = NodeDatabase()
+    db.ParseFromString(proto.read_bytes())
+    assert db.version == 25
+    assert len(db.nodes) == size
+    nums = {n.num for n in db.nodes}
+    assert len(nums) == size, "node numbers must be unique"
+    assert all(n.long_name and n.short_name for n in db.nodes)
+    assert all(len(n.long_name) <= 24 for n in db.nodes)  # max_size:25 - NUL
+
+    # Coverage sanity (±10pp tolerance for binomial fluctuation).
+    def in_range(actual: int, expected_ratio: float, tol_pp: float = 0.10) -> bool:
+        lo = max(0, int((expected_ratio - tol_pp) * size))
+        hi = min(size, int((expected_ratio + tol_pp) * size))
+        return lo <= actual <= hi
+
+    assert in_range(len(db.positions), 0.85)
+    assert in_range(len(db.telemetry), 0.70)
+    assert in_range(len(db.environment), 0.25)
+    assert in_range(len(db.status), 0.40)
+
+
+def test_compile_freshens_timestamps(tmp_path: pathlib.Path) -> None:
+    """Same JSONL compiled twice → identical structure, different timestamps."""
+    _require_v25_bindings()
+    jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
+    if not jsonl.is_file():
+        pytest.skip("250-node seed not present — run ./bin/regen-fake-nodedbs.sh")
+    a = tmp_path / "a.proto"
+    b = tmp_path / "b.proto"
+    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(a)])
+    time.sleep(1.2)
+    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(b)])
+
+    da = NodeDatabase()
+    db_ = NodeDatabase()
+    da.ParseFromString(a.read_bytes())
+    db_.ParseFromString(b.read_bytes())
+
+    # Zero out timestamp fields and confirm everything else is byte-identical.
+    for d in (da, db_):
+        for n in d.nodes:
+            n.last_heard = 0
+        for p in d.positions:
+            p.position.time = 0
+    assert da.SerializeToString() == db_.SerializeToString()
+
+    # Re-load fresh copies to confirm timestamps actually moved.
+    aa = NodeDatabase()
+    bb = NodeDatabase()
+    aa.ParseFromString(a.read_bytes())
+    bb.ParseFromString(b.read_bytes())
+    aa_max = max(n.last_heard for n in aa.nodes if n.last_heard)
+    bb_max = max(n.last_heard for n in bb.nodes if n.last_heard)
+    assert bb_max >= aa_max
+    assert bb_max - aa_max < 5  # within a few seconds
+
+
+def test_compile_pinned_now_epoch_is_byte_identical(tmp_path: pathlib.Path) -> None:
+    """With --now-epoch pinned, two compiles produce identical bytes."""
+    _require_v25_bindings()
+    jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
+    if not jsonl.is_file():
+        pytest.skip("250-node seed not present")
+    a = tmp_path / "a.proto"
+    b = tmp_path / "b.proto"
+    for o in (a, b):
+        _run(
+            [
+                sys.executable,
+                str(COMPILE),
+                "--in",
+                str(jsonl),
+                "--now-epoch",
+                "1700000000",
+                "--out",
+                str(o),
+            ]
+        )
+    assert a.read_bytes() == b.read_bytes()
+
+
+def test_compile_timestamps_are_recent(tmp_path: pathlib.Path) -> None:
+    _require_v25_bindings()
+    jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
+    if not jsonl.is_file():
+        pytest.skip("250-node seed not present")
+    out = tmp_path / "out.proto"
+    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(out)])
+    db = NodeDatabase()
+    db.ParseFromString(out.read_bytes())
+    now = int(time.time())
+    # No timestamp older than 7 days, none in the future.
+    for n in db.nodes:
+        if n.last_heard:
+            assert now - 7 * 86400 <= n.last_heard <= now
+    # At least half should be within the last hour
+    # (matches expovariate(mean=3600s)).
+    recent = sum(1 for n in db.nodes if n.last_heard and n.last_heard >= now - 3600)
+    assert recent >= 0.4 * len(db.nodes)
+
+
+def test_compile_hand_edit_round_trip(tmp_path: pathlib.Path) -> None:
+    """Edit one JSONL line, recompile, confirm edit appears in the proto."""
+    _require_v25_bindings()
+    src = FIXTURES_DIR / "seed_v25_0250.jsonl"
+    if not src.is_file():
+        pytest.skip("250-node seed not present")
+    dst = tmp_path / "edited.jsonl"
+    lines = src.read_text(encoding="utf-8").splitlines()
+
+    # Find a node that already has telemetry so the index relationship is
+    # easy to assert on the other side.
+    edit_idx = None
+    for i, raw in enumerate(lines[1:], start=1):
+        node = json.loads(raw)
+        if node.get("telemetry") is not None:
+            edit_idx = i
+            break
+    assert edit_idx is not None, "expected at least one node with telemetry"
+
+    node = json.loads(lines[edit_idx])
+    target_num = int(node["num"], 16)
+    node["long_name"] = "Hand Edited Node"
+    node["telemetry"] = {
+        "battery_level": 42,
+        "voltage": 3.71,
+        "channel_utilization": 0.0,
+        "air_util_tx": 0.0,
+        "uptime_seconds": 1,
+    }
+    lines[edit_idx] = json.dumps(node, ensure_ascii=False, sort_keys=True)
+    dst.write_text("\n".join(lines) + "\n", encoding="utf-8")
+
+    out = tmp_path / "out.proto"
+    _run([sys.executable, str(COMPILE), "--in", str(dst), "--out", str(out)])
+    db = NodeDatabase()
+    db.ParseFromString(out.read_bytes())
+    edited = next((n for n in db.nodes if n.num == target_num), None)
+    assert edited is not None
+    assert edited.long_name == "Hand Edited Node"
+    tel = next((t for t in db.telemetry if t.num == target_num), None)
+    assert tel is not None
+    assert tel.device_metrics.battery_level == 42
+
+
+# ---------------------------------------------------------------------------
+# Misc smoke checks on the module surface.
+# ---------------------------------------------------------------------------
+def test_crc16_ccitt_matches_known_vectors() -> None:
+    """Sanity-check the hand-rolled CRC16-CCITT matches well-known vectors.
+
+    Test vectors from the XModem-CRC spec (init=0, poly=0x1021):
+      crc16("123456789") = 0x31C3
+      crc16("")          = 0x0000
+    """
+    from meshtastic_mcp.fixtures import _crc16_ccitt
+
+    assert _crc16_ccitt(b"") == 0x0000
+    assert _crc16_ccitt(b"123456789") == 0x31C3
+
+
+def test_push_fake_nodedb_rejects_invalid_size() -> None:
+    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
+
+    with pytest.raises(FixtureError, match="size must be one of"):
+        push_fake_nodedb(size=999, target="portduino")  # type: ignore[arg-type]
+
+
+def test_push_fake_nodedb_hardware_requires_confirm() -> None:
+    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
+
+    with pytest.raises(FixtureError, match="confirm=True"):
+        push_fake_nodedb(size=250, target="hardware", port="/dev/cu.fake")
+
+
+def test_push_fake_nodedb_hardware_requires_port() -> None:
+    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
+
+    with pytest.raises(FixtureError, match="requires a port"):
+        push_fake_nodedb(size=250, target="hardware", confirm=True)
+
+
+def test_push_fake_nodedb_hardware_rejects_tcp_port() -> None:
+    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
+
+    with pytest.raises(FixtureError, match="not supported"):
+        push_fake_nodedb(
+            size=250, target="hardware", confirm=True, port="tcp://localhost:4403"
+        )

mcp-server/tests/unit/test_recorder.py

@@ -0,0 +1,548 @@
+"""Unit tests for the persistent device-log recorder.
+
+Hardware-free: drives the Recorder through its `_on_*` handlers with
+synthetic packet/line dicts, then queries via log_query. Validates
+prefix parsing, telemetry variant dispatch, marker round-trip, time
+window filtering, downsampling, slope estimation, and gzip rotation
++ archive pruning.
+"""
+
+from __future__ import annotations
+
+import gzip
+import json
+import logging
+import os
+import time
+from pathlib import Path
+
+import pubsub
+import pytest
+from meshtastic_mcp import log_query
+from meshtastic_mcp.recorder.parsers import (
+    extract_telemetry,
+    interface_label,
+    parse_log_line,
+    summarize_packet,
+)
+from meshtastic_mcp.recorder.recorder import Recorder
+from meshtastic_mcp.recorder.rotating import _RotatingJsonl
+
+# -- isolation: every test gets a fresh Recorder + tmp dir -----------
+
+
+@pytest.fixture
+def recorder(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Recorder:
+    # Redirect both the Recorder and the module-level singleton lookup
+    # to the same tmp dir so log_query queries the same files we write.
+    monkeypatch.setenv("MESHTASTIC_MCP_LOG_DIR", str(tmp_path))
+    monkeypatch.setattr(
+        "meshtastic_mcp.recorder.recorder._INSTANCE", None, raising=False
+    )
+    r = Recorder(base_dir=tmp_path)
+    r.start()
+    monkeypatch.setattr("meshtastic_mcp.recorder.recorder._INSTANCE", r, raising=False)
+    yield r
+    r.stop()
+
+
+class _FakeIface:
+    devPath = "/dev/cu.fake"
+
+
+# -- parsers ---------------------------------------------------------
+
+
+class TestParseLogLine:
+    def test_full_prefix(self) -> None:
+        out = parse_log_line("INFO  | 12:34:56 12345 [Main] Booting")
+        assert out["level"] == "INFO"
+        assert out["tag"] == "Main"
+        assert out["uptime_s"] == 12345
+        assert out["msg"] == "Booting"
+        assert out["clock"] == "12:34:56"
+
+    def test_invalid_clock(self) -> None:
+        out = parse_log_line("WARN  | ??:??:?? 7 [SerialConsole] Boot")
+        assert out["level"] == "WARN"
+        assert out["clock"] == "??:??:??"
+        assert out["uptime_s"] == 7
+
+    def test_no_thread_bracket(self) -> None:
+        out = parse_log_line("DEBUG | 00:00:00 0 raw message body")
+        assert out["level"] == "DEBUG"
+        assert out.get("tag") is None
+        assert out["msg"] == "raw message body"
+
+    def test_bare_message(self) -> None:
+        # LogRecord.message path — no level prefix at all.
+        out = parse_log_line("just a bare message")
+        assert "level" not in out or out.get("level") is None
+        assert out["line"] == "just a bare message"
+
+    def test_empty(self) -> None:
+        assert parse_log_line("") == {"line": ""}
+
+    def test_debug_heap_prefix_extracted(self) -> None:
+        out = parse_log_line("INFO  | 12:34:56 12345 [Main] [heap 92344] Booting")
+        assert out["level"] == "INFO"
+        assert out["tag"] == "Main"
+        assert out["heap_free"] == 92344
+        assert out["msg"] == "Booting"
+
+    def test_debug_heap_prefix_on_bare_line(self) -> None:
+        # LogRecord.message path: no level prefix but still has [heap N].
+        out = parse_log_line("[heap 12345] some message")
+        assert out["heap_free"] == 12345
+        assert out["msg"] == "some message"
+
+    def test_thread_leak_event(self) -> None:
+        out = parse_log_line(
+            "HEAP  | 00:00:01 100 [Power] [heap 90000] "
+            "------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------"
+        )
+        assert out["level"] == "HEAP"
+        assert out["heap_free"] == 90000
+        ev = out["heap_event"]
+        assert ev["kind"] == "leaked"
+        assert ev["thread"] == "MeshPacket"
+        assert ev["before"] == 92344
+        assert ev["after"] == 90000
+        assert ev["delta"] == -2344
+
+    def test_thread_freed_event(self) -> None:
+        out = parse_log_line(
+            "++++++ Thread Router freed heap 1000 -> 1500 (500) ++++++"
+        )
+        ev = out["heap_event"]
+        assert ev["kind"] == "freed"
+        assert ev["thread"] == "Router"
+        assert ev["delta"] == 500
+
+    def test_heap_status_periodic(self) -> None:
+        out = parse_log_line(
+            "HEAP  | 00:00:30 30 [Power] "
+            "Heap status: 92344/200000 bytes free (-128), running 8/12 threads"
+        )
+        assert out["heap_free"] == 92344
+        assert out["heap_total"] == 200000
+        assert out["heap_delta"] == -128
+
+
+class TestRecorderDebugHeapSynthesis:
+    def test_log_with_heap_writes_telemetry(self, recorder: "Recorder") -> None:
+        # When a log line carries [heap N], the recorder should also
+        # emit a synthesized telemetry row tagged source=debug_heap.
+        recorder._on_log_line(
+            "INFO  | 00:00:00 1 [Main] [heap 88888] hello",
+            _FakeIface(),
+        )
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
+        assert len(synth) == 1
+        assert synth[0]["fields"]["heap_free_bytes"] == 88888
+        assert synth[0]["variant"] == "local"
+
+    def test_heap_status_writes_total_too(self, recorder: "Recorder") -> None:
+        recorder._on_log_line(
+            "HEAP  | 00:00:30 30 [Power] "
+            "Heap status: 50000/200000 bytes free (-100), running 8/12 threads",
+            _FakeIface(),
+        )
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
+        assert synth[-1]["fields"]["heap_free_bytes"] == 50000
+        assert synth[-1]["fields"]["heap_total_bytes"] == 200000
+
+    def test_no_heap_no_synthesis(self, recorder: "Recorder") -> None:
+        # Plain log line (no [heap N], no Heap status) — telemetry.jsonl
+        # should NOT gain a synth row.
+        before = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
+        recorder._on_log_line("INFO  | 00:00:00 1 [Main] just a message", _FakeIface())
+        after = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
+        assert after == before
+
+    def test_thread_leak_event_persists_on_log_row(self, recorder: "Recorder") -> None:
+        recorder._on_log_line(
+            "HEAP  | 00:00:01 100 [Power] [heap 90000] "
+            "------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------",
+            _FakeIface(),
+        )
+        rows = [
+            json.loads(r)
+            for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+            if r
+        ]
+        evt_rows = [r for r in rows if r.get("heap_event")]
+        assert len(evt_rows) == 1
+        assert evt_rows[0]["heap_event"]["thread"] == "MeshPacket"
+        assert evt_rows[0]["heap_event"]["delta"] == -2344
+
+
+class TestSerialTap:
+    def test_serial_line_records_log_and_synthesizes_heap(
+        self, recorder: "Recorder"
+    ) -> None:
+        recorder._on_serial_line(
+            "INFO  | 00:00:00 5 [Main] [heap 88888] tap-line",
+            port="/dev/cu.tap",
+        )
+        logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        log_rows = [json.loads(r) for r in logs if r]
+        # Find the row from this call (port=/dev/cu.tap, role=serial_session)
+        tap_rows = [r for r in log_rows if r.get("port") == "/dev/cu.tap"]
+        assert len(tap_rows) == 1
+        assert tap_rows[0]["role"] == "serial_session"
+        assert tap_rows[0]["level"] == "INFO"
+        assert tap_rows[0]["tag"] == "Main"
+        assert tap_rows[0]["heap_free"] == 88888
+        synth = [json.loads(r) for r in telem if '"source":"debug_heap_serial"' in r]
+        assert len(synth) == 1
+        assert synth[0]["fields"]["heap_free_bytes"] == 88888
+        assert synth[0]["role"] == "serial_session"
+
+    def test_serial_line_thread_leak_event(self, recorder: "Recorder") -> None:
+        recorder._on_serial_line(
+            "HEAP  | 00:00:30 30 [Power] [heap 53484] "
+            "------ Thread Router leaked heap 53612 -> 53484 (-128) ------",
+            port="/dev/cu.tap",
+        )
+        rows = [
+            json.loads(r)
+            for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+            if r
+        ]
+        evt = [r for r in rows if r.get("heap_event")]
+        assert len(evt) == 1
+        assert evt[0]["heap_event"]["thread"] == "Router"
+        assert evt[0]["heap_event"]["delta"] == -128
+        # Heap also synthesized.
+        telem = (recorder.base_dir / "telemetry.jsonl").read_text()
+        assert '"source":"debug_heap_serial"' in telem
+
+    def test_serial_line_pause(self, recorder: "Recorder") -> None:
+        recorder.pause("baseline")
+        recorder._on_serial_line(
+            "INFO  | 00:00:00 1 [t] [heap 1000] dropped",
+            port="/dev/cu.tap",
+        )
+        # Only the pause event row should exist; no tap row.
+        logs = (recorder.base_dir / "logs.jsonl").read_text()
+        assert "dropped" not in logs
+
+    def test_serial_line_handler_swallows_exceptions(
+        self, recorder: "Recorder"
+    ) -> None:
+        # Hostile input — should not raise.
+        recorder._on_serial_line(None, port="/dev/cu.tap")  # type: ignore[arg-type]
+        recorder._on_serial_line(b"\x00\x01\x02\x03", port="/dev/cu.tap")  # type: ignore[arg-type]
+        # Survived.
+
+
+class TestExtractTelemetry:
+    def test_local_stats_camel(self) -> None:
+        pkt = {
+            "decoded": {
+                "telemetry": {
+                    "localStats": {"heap_total_bytes": 1000, "heap_free_bytes": 600}
+                }
+            }
+        }
+        out = extract_telemetry(pkt)
+        assert out is not None
+        assert out["variant"] == "local"
+        assert out["fields"]["heap_free_bytes"] == 600
+
+    def test_device_metrics_snake(self) -> None:
+        pkt = {
+            "decoded": {
+                "telemetry": {"device_metrics": {"battery_level": 88, "voltage": 4.1}}
+            }
+        }
+        out = extract_telemetry(pkt)
+        assert out is not None
+        assert out["variant"] == "device"
+        assert out["fields"]["battery_level"] == 88
+
+    def test_unknown_variant_returns_none(self) -> None:
+        assert extract_telemetry({"decoded": {"telemetry": {"weird": {}}}}) is None
+        assert extract_telemetry({}) is None
+        assert extract_telemetry({"decoded": "not-a-dict"}) is None
+
+
+class TestSummarizePacket:
+    def test_text_with_payload(self) -> None:
+        pkt = {
+            "fromId": "!abc",
+            "toId": "!def",
+            "decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hello"},
+            "hopLimit": 3,
+        }
+        out = summarize_packet(pkt)
+        assert out["from_node"] == "!abc"
+        assert out["portnum"] == "TEXT_MESSAGE_APP"
+        assert out["payload_size"] == 5
+        assert out["payload_hex_prefix"] == "68656c6c6f"
+
+    def test_no_decoded(self) -> None:
+        out = summarize_packet({"fromId": "!abc"})
+        assert out["from_node"] == "!abc"
+        assert out["portnum"] is None
+
+
+class TestInterfaceLabel:
+    def test_serial(self) -> None:
+        assert interface_label(_FakeIface()) == {
+            "port": "/dev/cu.fake",
+            "role": "serial",
+        }
+
+    def test_tcp(self) -> None:
+        class T:
+            hostname = "node.lan"
+            portNumber = 4403
+
+        assert interface_label(T()) == {"port": "tcp://node.lan:4403", "role": "tcp"}
+
+    def test_unknown(self) -> None:
+        assert interface_label(object()) == {"port": "object", "role": None}
+
+    def test_none(self) -> None:
+        assert interface_label(None) == {"port": None, "role": None}
+
+
+# -- recorder write side ---------------------------------------------
+
+
+class TestRecorderWrites:
+    def test_log_line_is_recorded(self, recorder: Recorder) -> None:
+        recorder._on_log_line("INFO  | 12:34:56 99 [T] hi", _FakeIface())
+        path = recorder.base_dir / "logs.jsonl"
+        rows = [json.loads(line) for line in path.read_text().splitlines() if line]
+        # First row is recorder_start_event mirror? No — that's events.jsonl only.
+        assert any(r.get("level") == "INFO" and r.get("tag") == "T" for r in rows)
+
+    def test_telemetry_recorded_and_packet_double(self, recorder: Recorder) -> None:
+        # _on_telemetry alone — only telemetry.jsonl
+        recorder._on_telemetry(
+            {
+                "fromId": "!abc",
+                "decoded": {"telemetry": {"localStats": {"heap_free_bytes": 600}}},
+            },
+            _FakeIface(),
+        )
+        telem_rows = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
+        assert any('"variant":"local"' in r for r in telem_rows)
+
+    def test_packets_summary(self, recorder: Recorder) -> None:
+        recorder._on_receive(
+            {
+                "fromId": "!abc",
+                "toId": "!def",
+                "decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hi"},
+            },
+            _FakeIface(),
+        )
+        rows = (recorder.base_dir / "packets.jsonl").read_text().splitlines()
+        assert any('"portnum":"TEXT_MESSAGE_APP"' in r for r in rows)
+
+    def test_mark_event_round_trip(self, recorder: Recorder) -> None:
+        out = recorder.mark_event("checkpoint", note="midpoint")
+        assert "ts" in out
+        events = (recorder.base_dir / "events.jsonl").read_text().splitlines()
+        logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
+        assert any('"label":"checkpoint"' in r and '"kind":"mark"' in r for r in events)
+        assert any('"level":"MARK"' in r and "checkpoint" in r for r in logs)
+
+    def test_pause_drops_writes(self, recorder: Recorder) -> None:
+        before = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
+        recorder.pause(reason="baseline")
+        recorder._on_log_line("INFO  | 00:00:00 1 [t] swallowed", _FakeIface())
+        after = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
+        assert after == before
+        recorder.resume()
+        recorder._on_log_line("INFO  | 00:00:00 2 [t] kept", _FakeIface())
+        post_resume = (recorder.base_dir / "logs.jsonl").read_text()
+        assert "kept" in post_resume
+
+    def test_pubsub_handler_swallows_exceptions(self, recorder: Recorder) -> None:
+        # If the writer dies, the pubsub callback must NOT raise — that
+        # would crash the meshtastic receive thread.
+        bad_packet = object()  # not a dict
+        recorder._on_receive(bad_packet, _FakeIface())  # type: ignore[arg-type]
+        recorder._on_telemetry(bad_packet, _FakeIface())  # type: ignore[arg-type]
+        recorder._on_log_line(None, _FakeIface())  # type: ignore[arg-type]
+        # No assertion needed — survival is the test.
+
+
+# -- log_query read side ---------------------------------------------
+
+
+class TestLogQuery:
+    def test_logs_window_grep_and_level(self, recorder: Recorder) -> None:
+        recorder._on_log_line("INFO  | 12:00:00 1 [A] alpha", _FakeIface())
+        recorder._on_log_line("WARN  | 12:00:01 2 [B] bravo failed", _FakeIface())
+        recorder._on_log_line("ERROR | 12:00:02 3 [C] charlie failed", _FakeIface())
+
+        out = log_query.logs_window(start="-1m", level="WARN|ERROR", max_lines=10)
+        assert out["total_matched"] == 2
+        levels = {r["level"] for r in out["lines"]}
+        assert levels == {"WARN", "ERROR"}
+
+        out2 = log_query.logs_window(start="-1m", grep=r"failed$", max_lines=10)
+        assert out2["total_matched"] == 2
+
+    def test_logs_window_invalid_regex(self, recorder: Recorder) -> None:
+        recorder._on_log_line("INFO  | 12:00:00 1 [A] alpha", _FakeIface())
+        with pytest.raises(ValueError, match="invalid grep regex"):
+            log_query.logs_window(start="-1m", grep="(")
+
+    def test_telemetry_timeline_slope_and_downsample(self, recorder: Recorder) -> None:
+        # Synthesize a downward leak: 100 points, free_heap drops 1 byte/sample.
+        base_ts = time.time() - 60
+        for i in range(100):
+            recorder._files["telemetry"].write(
+                {
+                    "ts": base_ts + i * 0.5,
+                    "port": "/dev/cu.fake",
+                    "role": "serial",
+                    "from_node": "!abc",
+                    "variant": "local",
+                    "fields": {"heap_free_bytes": 10000 - i},
+                }
+            )
+
+        out = log_query.telemetry_timeline(
+            window="2m", variant="local", field="free_heap", max_points=10
+        )
+        assert out["samples"] == 100
+        assert len(out["points"]) <= 10
+        # Negative slope (heap dropping). Magnitude: 1 byte every 0.5s = 120/min.
+        assert out["slope_per_min"] is not None
+        assert out["slope_per_min"] < -100
+
+    def test_export_bundles_slice(self, recorder: Recorder, tmp_path: Path) -> None:
+        recorder._on_log_line("INFO  | 00:00:00 1 [t] one", _FakeIface())
+        recorder._on_log_line("INFO  | 00:00:00 2 [t] two", _FakeIface())
+        dest = tmp_path / "bundle"
+        out = log_query.export(start="-1m", end="now", dest_dir=str(dest))
+        assert (dest / "logs.jsonl").exists()
+        assert "logs" in out["paths"]
+
+
+# -- time parser -----------------------------------------------------
+
+
+class TestParseTime:
+    def test_relative(self) -> None:
+        now = 1_000_000.0
+        assert log_query._parse_time("-15m", now=now) == now - 900
+        assert log_query._parse_time("-2h", now=now) == now - 7200
+        assert log_query._parse_time("-1d", now=now) == now - 86400
+
+    def test_now_and_epoch(self) -> None:
+        now = 1_000_000.0
+        assert log_query._parse_time("now", now=now) == now
+        assert log_query._parse_time(now) == now
+
+    def test_iso(self) -> None:
+        ts = log_query._parse_time("2026-01-01T00:00:00Z")
+        assert isinstance(ts, float) and ts > 1_700_000_000
+
+    def test_naive_iso_assumes_utc(self) -> None:
+        assert log_query._parse_time("2026-01-01T00:00:00") == log_query._parse_time(
+            "2026-01-01T00:00:00Z"
+        )
+
+    def test_invalid(self) -> None:
+        with pytest.raises(ValueError):
+            log_query._parse_time("not a time")
+
+
+# -- rotation --------------------------------------------------------
+
+
+class TestRotation:
+    def test_size_cap_rotates_and_gzips(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, max_bytes=512, keep_archives=5, check_every=1)
+        for i in range(100):
+            r.write({"ts": float(i), "i": i, "pad": "x" * 40})
+        r.close()
+        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
+        assert archives, "expected at least one rotation"
+        # Archive content is valid gzip + valid JSONL
+        with gzip.open(archives[0], "rt") as fh:
+            first = json.loads(fh.readline())
+            assert "ts" in first
+
+    def test_archive_pruning(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, max_bytes=200, keep_archives=2, check_every=1)
+        # Force several rotations.
+        for _ in range(8):
+            for i in range(20):
+                r.write({"ts": float(i), "pad": "x" * 30})
+            r.force_rotate()
+        r.close()
+        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
+        assert len(archives) <= 2, f"expected ≤2 kept archives, got {len(archives)}"
+
+    def test_archive_pruning_uses_filename_order(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, keep_archives=2)
+        old = tmp_path / "rot.20260101-000000-000000-00000.jsonl.gz"
+        mid = tmp_path / "rot.20260101-000001-000000-00000.jsonl.gz"
+        new = tmp_path / "rot.20260101-000002-000000-00000.jsonl.gz"
+        for archive in (old, mid, new):
+            with gzip.open(archive, "wt", encoding="utf-8") as fh:
+                fh.write('{"ts":1}\n')
+        # Deliberately scramble mtimes so lexicographic filename order is
+        # the only stable chronological signal.
+        os.utime(old, (300, 300))
+        os.utime(mid, (100, 100))
+        os.utime(new, (200, 200))
+
+        r._prune_archives()
+        r.close()
+
+        archives = sorted(p.name for p in tmp_path.glob("rot.*.jsonl.gz"))
+        assert archives == [mid.name, new.name]
+
+    def test_force_rotate_when_below_threshold(self, tmp_path: Path) -> None:
+        path = tmp_path / "rot.jsonl"
+        r = _RotatingJsonl(path, max_bytes=10_000_000, check_every=999_999)
+        r.write({"ts": 1.0, "msg": "tiny"})
+        r.force_rotate()
+        r.write({"ts": 2.0, "msg": "after-rotate"})
+        r.close()
+        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
+        assert len(archives) == 1
+        assert path.exists()
+        assert "after-rotate" in path.read_text()
+
+
+class TestRecorderLocks:
+    def test_force_rotate_all_returns_status(self, recorder: Recorder) -> None:
+        out = recorder.force_rotate_all()
+        assert out["running"] is True
+        assert out["files"]
+
+    def test_wire_pubsub_logs_subscription_failure(
+        self,
+        tmp_path: Path,
+        monkeypatch: pytest.MonkeyPatch,
+        caplog: pytest.LogCaptureFixture,
+    ) -> None:
+        class FailingPubSubMock:
+            def subscribe(self, callback: object, topic: str) -> None:
+                raise RuntimeError("boom")
+
+        monkeypatch.setattr(pubsub, "pub", FailingPubSubMock())
+        recorder = Recorder(base_dir=tmp_path)
+        with caplog.at_level(logging.WARNING):
+            recorder._wire_pubsub()
+        assert (
+            "Recorder failed to subscribe to meshtastic.log.line: boom" in caplog.text
+        )

platformio.ini

@@ -2,7 +2,7 @@
 ; https://docs.platformio.org/page/projectconf.html
 
 [platformio]
-default_envs = tbeam
+default_envs = heltec-v3
 
 extra_configs =
 	variants/*/*.ini
@@ -17,6 +17,7 @@ test_build_src = true
 extra_scripts =
 	pre:bin/platformio-pre.py
 	bin/platformio-custom.py
+	post:extra_scripts/nrf54l15_linker.py
 ; note: we add src to our include search path so that lmic_project_config can override
 ; note: TINYGPS_OPTION_NO_CUSTOM_FIELDS is VERY important.  We don't use custom fields and somewhere in that pile
 ; of code is a heap corruption bug!
@@ -49,6 +50,7 @@ build_flags = -Wno-missing-field-initializers
 	-DRADIOLIB_EXCLUDE_PAGER=1
 	-DRADIOLIB_EXCLUDE_FSK4=1
 	-DRADIOLIB_EXCLUDE_APRS=1
+	-DRADIOLIB_EXCLUDE_ADSB=1
 	-DRADIOLIB_EXCLUDE_LORAWAN=1
 	-DMESHTASTIC_EXCLUDE_DROPZONE=1
 	-DMESHTASTIC_EXCLUDE_REPLYBOT=1
@@ -137,7 +139,7 @@ lib_deps =
 	# renovate: datasource=github-tags depName=Adafruit GFX packageName=adafruit/Adafruit-GFX-Library
 	https://github.com/adafruit/Adafruit-GFX-Library/archive/refs/tags/1.12.6.zip
 	# renovate: datasource=github-tags depName=NeoPixel packageName=adafruit/Adafruit_NeoPixel
-	https://github.com/adafruit/Adafruit_NeoPixel/archive/refs/tags/1.15.4.zip
+	https://github.com/adafruit/Adafruit_NeoPixel/archive/1.15.5.zip
 	# renovate: datasource=github-tags depName=Adafruit SSD1306 packageName=adafruit/Adafruit_SSD1306
 	https://github.com/adafruit/Adafruit_SSD1306/archive/refs/tags/2.5.16.zip
 	# renovate: datasource=github-tags depName=Adafruit BMP280 packageName=adafruit/Adafruit_BMP280_Library

src/DebugConfiguration.cpp

@@ -151,7 +151,7 @@ inline bool Syslog::_sendLog(uint16_t pri, const char *appName, const char *mess
     if (!this->_enabled)
         return false;
 
-    if ((this->_server == NULL && this->_ip == INADDR_NONE) || this->_port == 0)
+    if ((this->_server == NULL && this->_ip == IPAddress(0, 0, 0, 0)) || this->_port == 0)
         return false;
 
     // Check priority against priMask values.

src/DebugConfiguration.h

@@ -13,6 +13,11 @@ extern MemGet memGet;
 #define LED_STATE_ON 1
 #endif
 
+// WIFI LED
+#ifndef WIFI_STATE_ON
+#define WIFI_STATE_ON 1
+#endif
+
 // -----------------------------------------------------------------------------
 // DEBUG
 // -----------------------------------------------------------------------------
@@ -147,13 +152,14 @@ extern "C" void logLegacy(const char *level, const char *fmt, ...);
 // Default Bluetooth PIN
 #define defaultBLEPin 123456
 
-#if HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && defined(USE_CH390D)
+#include <ESP32_CH390.h>
+#elif HAS_ETHERNET && !defined(USE_WS5500)
 #include <RAK13800_W5100S.h>
 #endif // HAS_ETHERNET
 
-#if HAS_ETHERNET && defined(USE_WS5500)
-#include <ETHClass2.h>
-#define ETH ETH2
+#if HAS_ETHERNET && defined(ARCH_ESP32)
+#include <ETH.h>
 #endif // HAS_ETHERNET
 
 #if HAS_WIFI

src/FSCommon.cpp

@@ -277,7 +277,7 @@ void fsInit()
  */
 void setupSDCard()
 {
-#if defined(HAS_SDCARD) && !defined(SDCARD_USE_SOFT_SPI)
+#if defined(HAS_SDCARD) && !defined(SDCARD_USE_SOFT_SPI) && !defined(HAS_SD_MMC)
     concurrency::LockGuard g(spiLock);
     SDHandler.begin(SPI_SCK, SPI_MISO, SPI_MOSI);
     if (!SD.begin(SDCARD_CS, SDHandler, SD_SPI_FREQUENCY)) {

src/FSCommon.h

@@ -48,6 +48,14 @@ using namespace STM32_LittleFS_Namespace;
 using namespace Adafruit_LittleFS_Namespace;
 #endif
 
+#if defined(ARCH_NRF54L15)
+// nRF54L15 — Zephyr LittleFS on 36 KB storage_partition (internal RRAM)
+#include "InternalFileSystem.h"
+#define FSCom InternalFS
+#define FSBegin() FSCom.begin()
+using namespace Adafruit_LittleFS_Namespace;
+#endif
+
 void fsInit();
 void fsListFiles();
 bool copyFile(const char *from, const char *to);

src/Power.cpp

@@ -24,6 +24,13 @@
 #include "meshUtils.h"
 #include "power/PowerHAL.h"
 #include "sleep.h"
+#ifdef ARCH_ESP32
+// #include <driver/adc.h>
+#include <esp_adc/adc_cali.h>
+#include <esp_adc/adc_cali_scheme.h>
+#include <esp_adc/adc_oneshot.h>
+#include <esp_err.h>
+#endif
 
 #if defined(ARCH_PORTDUINO)
 #include "api/WiFiServerAPI.h"
@@ -63,9 +70,8 @@
 #include <WiFi.h>
 #endif
 
-#if HAS_ETHERNET && defined(USE_WS5500)
-#include <ETHClass2.h>
-#define ETH ETH2
+#if HAS_ETHERNET && defined(ARCH_ESP32)
+#include <ETH.h>
 #endif // HAS_ETHERNET
 
 #endif
@@ -77,21 +83,86 @@
 #if defined(BATTERY_PIN) && defined(ARCH_ESP32)
 
 #ifndef BAT_MEASURE_ADC_UNIT // ADC1 is default
-static const adc1_channel_t adc_channel = ADC_CHANNEL;
+static const adc_channel_t adc_channel = ADC_CHANNEL;
 static const adc_unit_t unit = ADC_UNIT_1;
-#else // ADC2
-static const adc2_channel_t adc_channel = ADC_CHANNEL;
+#else  // ADC2
+static const adc_channel_t adc_channel = ADC_CHANNEL;
 static const adc_unit_t unit = ADC_UNIT_2;
-RTC_NOINIT_ATTR uint64_t RTC_reg_b;
-
 #endif // BAT_MEASURE_ADC_UNIT
 
-esp_adc_cal_characteristics_t *adc_characs = (esp_adc_cal_characteristics_t *)calloc(1, sizeof(esp_adc_cal_characteristics_t));
+static adc_oneshot_unit_handle_t adc_handle = nullptr;
+static adc_cali_handle_t adc_cali_handle = nullptr;
+static bool adc_calibrated = false;
 #ifndef ADC_ATTENUATION
 static const adc_atten_t atten = ADC_ATTEN_DB_12;
 #else
 static const adc_atten_t atten = ADC_ATTENUATION;
 #endif
+#ifdef ADC_BITWIDTH
+static const adc_bitwidth_t adc_width = ADC_BITWIDTH;
+#else
+static const adc_bitwidth_t adc_width = ADC_BITWIDTH_DEFAULT;
+#endif
+
+static int adcBitWidthToBits(adc_bitwidth_t width)
+{
+    switch (width) {
+    case ADC_BITWIDTH_9:
+        return 9;
+    case ADC_BITWIDTH_10:
+        return 10;
+    case ADC_BITWIDTH_11:
+        return 11;
+    case ADC_BITWIDTH_12:
+        return 12;
+#ifdef ADC_BITWIDTH_13
+    case ADC_BITWIDTH_13:
+        return 13;
+#endif
+    default:
+        return 12;
+    }
+}
+
+static bool initAdcCalibration()
+{
+#if ADC_CALI_SCHEME_CURVE_FITTING_SUPPORTED
+    adc_cali_curve_fitting_config_t cali_config = {
+        .unit_id = unit,
+        .atten = atten,
+        .bitwidth = adc_width,
+    };
+    esp_err_t ret = adc_cali_create_scheme_curve_fitting(&cali_config, &adc_cali_handle);
+    if (ret == ESP_OK) {
+        LOG_INFO("ADC calibration: curve fitting enabled");
+        return true;
+    }
+    if (ret != ESP_ERR_NOT_SUPPORTED) {
+        LOG_WARN("ADC calibration: curve fitting failed: %s", esp_err_to_name(ret));
+    }
+#endif
+
+#if ADC_CALI_SCHEME_LINE_FITTING_SUPPORTED
+    adc_cali_line_fitting_config_t cali_config = {
+        .unit_id = unit,
+        .atten = atten,
+        .bitwidth = adc_width,
+        .default_vref = DEFAULT_VREF,
+    };
+    esp_err_t ret = adc_cali_create_scheme_line_fitting(&cali_config, &adc_cali_handle);
+    if (ret == ESP_OK) {
+        LOG_INFO("ADC calibration: line fitting enabled");
+        return true;
+    }
+    if (ret != ESP_ERR_NOT_SUPPORTED) {
+        LOG_WARN("ADC calibration: line fitting failed: %s", esp_err_to_name(ret));
+    }
+#endif
+
+    LOG_INFO("ADC calibration not supported; using approximate scaling");
+    return false;
+}
+
 #endif // BATTERY_PIN && ARCH_ESP32
 
 #ifdef EXT_PWR_DETECT
@@ -367,8 +438,20 @@ class AnalogBatteryLevel : public HasBatteryLevel
             scaled *= operativeAdcMultiplier;
 #elif defined(ARCH_ESP32) // ADC block for espressif platforms
             raw = espAdcRead();
-            scaled = esp_adc_cal_raw_to_voltage(raw, adc_characs);
-            scaled *= operativeAdcMultiplier;
+            int voltage_mv = 0;
+            if (adc_calibrated && adc_cali_handle) {
+                if (adc_cali_raw_to_voltage(adc_cali_handle, raw, &voltage_mv) != ESP_OK) {
+                    LOG_WARN("ADC calibration read failed; using raw value");
+                    voltage_mv = 0;
+                }
+            }
+            if (voltage_mv == 0) {
+                // Fallback approximate conversion without calibration
+                const int bits = adcBitWidthToBits(adc_width);
+                const float max_code = powf(2.0f, bits) - 1.0f;
+                voltage_mv = (int)((raw / max_code) * DEFAULT_VREF);
+            }
+            scaled = voltage_mv * operativeAdcMultiplier;
 #else                     // block for all other platforms
 #ifdef ARCH_NRF52
             concurrency::LockGuard saadcGuard(concurrency::nrf52SaadcLock);
@@ -410,51 +493,22 @@ class AnalogBatteryLevel : public HasBatteryLevel
         uint32_t raw = 0;
         uint8_t raw_c = 0; // raw reading counter
 
-#ifndef BAT_MEASURE_ADC_UNIT // ADC1
+        if (!adc_handle) {
+            LOG_ERROR("ADC oneshot handle not initialized");
+            return 0;
+        }
+
         for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
-            int val_ = adc1_get_raw(adc_channel);
-            if (val_ >= 0) { // save only valid readings
-                raw += val_;
+            int val = 0;
+            esp_err_t err = adc_oneshot_read(adc_handle, adc_channel, &val);
+            if (err == ESP_OK) {
+                raw += val;
                 raw_c++;
-            }
-            // delayMicroseconds(100);
-        }
-#else                            // ADC2
-#ifdef CONFIG_IDF_TARGET_ESP32S3 // ESP32S3
-        // ADC2 wifi bug workaround not required, breaks compile
-        // On ESP32S3, ADC2 can take turns with Wifi (?)
-
-        int32_t adc_buf;
-        esp_err_t read_result;
-
-        // Multiple samples
-        for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
-            adc_buf = 0;
-            read_result = -1;
-
-            read_result = adc2_get_raw(adc_channel, ADC_WIDTH_BIT_12, &adc_buf);
-            if (read_result == ESP_OK) {
-                raw += adc_buf;
-                raw_c++; // Count valid samples
             } else {
-                LOG_DEBUG("An attempt to sample ADC2 failed");
+                LOG_DEBUG("ADC read failed: %s", esp_err_to_name(err));
             }
         }
 
-#else  // Other ESP32
-        int32_t adc_buf = 0;
-        for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
-            // ADC2 wifi bug workaround, see
-            // https://github.com/espressif/arduino-esp32/issues/102
-            WRITE_PERI_REG(SENS_SAR_READ_CTRL2_REG, RTC_reg_b);
-            SET_PERI_REG_MASK(SENS_SAR_READ_CTRL2_REG, SENS_SAR2_DATA_INV);
-            adc2_get_raw(adc_channel, ADC_WIDTH_BIT_12, &adc_buf);
-            raw += adc_buf;
-            raw_c++;
-        }
-#endif // BAT_MEASURE_ADC_UNIT
-
-#endif // End BAT_MEASURE_ADC_UNIT
         return (raw / (raw_c < 1 ? 1 : raw_c));
     }
 #endif
@@ -666,42 +720,31 @@ bool Power::analogInit()
 #ifdef ARCH_STM32WL
     analogReadResolution(BATTERY_SENSE_RESOLUTION_BITS);
 #elif defined(ARCH_ESP32) // ESP32 needs special analog stuff
+    adc_oneshot_unit_init_cfg_t init_config = {
+        .unit_id = unit,
+    };
 
-#ifndef ADC_WIDTH // max resolution by default
-    static const adc_bits_width_t width = ADC_WIDTH_BIT_12;
-#else
-    static const adc_bits_width_t width = ADC_WIDTH;
-#endif
-#ifndef BAT_MEASURE_ADC_UNIT // ADC1
-    adc1_config_width(width);
-    adc1_config_channel_atten(adc_channel, atten);
-#else                        // ADC2
-    adc2_config_channel_atten(adc_channel, atten);
-#ifndef CONFIG_IDF_TARGET_ESP32S3
-    // ADC2 wifi bug workaround
-    // Not required with ESP32S3, breaks compile
-    RTC_reg_b = READ_PERI_REG(SENS_SAR_READ_CTRL2_REG);
-#endif
-#endif
-    // calibrate ADC
-    esp_adc_cal_value_t val_type = esp_adc_cal_characterize(unit, atten, width, DEFAULT_VREF, adc_characs);
-    // show ADC characterization base
-    if (val_type == ESP_ADC_CAL_VAL_EFUSE_TP) {
-        LOG_INFO("ADC config based on Two Point values stored in eFuse");
-    } else if (val_type == ESP_ADC_CAL_VAL_EFUSE_VREF) {
-        LOG_INFO("ADC config based on reference voltage stored in eFuse");
+    if (!adc_handle) {
+        esp_err_t err = adc_oneshot_new_unit(&init_config, &adc_handle);
+        if (err != ESP_OK) {
+            LOG_ERROR("ADC oneshot init failed: %s", esp_err_to_name(err));
+            return false;
+        }
     }
-#ifdef CONFIG_IDF_TARGET_ESP32S3
-    // ESP32S3
-    else if (val_type == ESP_ADC_CAL_VAL_EFUSE_TP_FIT) {
-        LOG_INFO("ADC config based on Two Point values and fitting curve "
-                 "coefficients stored in eFuse");
+
+    adc_oneshot_chan_cfg_t chan_cfg = {
+        .atten = atten,
+        .bitwidth = adc_width,
+    };
+
+    esp_err_t err = adc_oneshot_config_channel(adc_handle, adc_channel, &chan_cfg);
+    if (err != ESP_OK) {
+        LOG_ERROR("ADC channel config failed: %s", esp_err_to_name(err));
+        return false;
     }
-#endif
-    else {
-        LOG_INFO("ADC config based on default reference voltage");
-    }
-#endif // ARCH_ESP32
+
+    adc_calibrated = initAdcCalibration();
+#endif                    // ARCH_ESP32
 
     // NRF52 ADC init moved to powerHAL_init in nrf52 platform
 
@@ -904,7 +947,16 @@ void Power::readPowerStatus()
 
     // Notify any status instances that are observing us
     const PowerStatus powerStatus2 = PowerStatus(hasBattery, usbPowered, isChargingNow, batteryVoltageMv, batteryChargePercent);
-    if (millis() > lastLogTime + 50 * 1000) {
+
+    // Log battery-presence transitions once; skip OptUnknown so we don't lie before the first probe.
+    static OptionalBool prevHasBattery = OptUnknown;
+    if (hasBattery != OptUnknown && hasBattery != prevHasBattery) {
+        LOG_INFO("Power: battery hardware %s", hasBattery == OptTrue ? "detected" : "absent (USB-only)");
+        prevHasBattery = hasBattery;
+    }
+
+    // Periodic telemetry only emits when a battery is actually present (otherwise values are constant -1/0).
+    if (hasBattery == OptTrue && !Throttle::isWithinTimespanMs(lastLogTime, 50 * 1000)) {
         LOG_DEBUG("Battery: usbPower=%d, isCharging=%d, batMv=%d, batPct=%d", powerStatus2.getHasUSB(),
                   powerStatus2.getIsCharging(), powerStatus2.getBatteryVoltageMv(), powerStatus2.getBatteryChargePercent());
         lastLogTime = millis();

src/RedirectablePrint.cpp

@@ -225,14 +225,16 @@ void RedirectablePrint::log_to_ble(const char *logLevel, const char *format, va_
         isBleConnected = nimbleBluetooth && nimbleBluetooth->isActive() && nimbleBluetooth->isConnected();
 #elif defined(ARCH_NRF52)
         isBleConnected = nrf52Bluetooth != nullptr && nrf52Bluetooth->isConnected();
+#elif defined(ARCH_NRF54L15)
+        isBleConnected = nrf54l15Bluetooth != nullptr && nrf54l15Bluetooth->isConnected();
 #endif
         if (isBleConnected) {
             auto thread = concurrency::OSThread::currentThread;
             meshtastic_LogRecord logRecord = meshtastic_LogRecord_init_zero;
             logRecord.level = getLogLevel(logLevel);
-            vsprintf(logRecord.message, format, arg);
+            vsnprintf(logRecord.message, sizeof(logRecord.message), format, arg);
             if (thread)
-                strcpy(logRecord.source, thread->ThreadName.c_str());
+                strlcpy(logRecord.source, thread->ThreadName.c_str(), sizeof(logRecord.source));
             logRecord.time = getValidTime(RTCQuality::RTCQualityDevice, true);
 
             auto buffer = std::unique_ptr<uint8_t[]>(new uint8_t[meshtastic_LogRecord_size]);
@@ -241,6 +243,8 @@ void RedirectablePrint::log_to_ble(const char *logLevel, const char *format, va_
             nimbleBluetooth->sendLog(buffer.get(), size);
 #elif defined(ARCH_NRF52)
             nrf52Bluetooth->sendLog(buffer.get(), size);
+#elif defined(ARCH_NRF54L15)
+            nrf54l15Bluetooth->sendLog(buffer.get(), size);
 #endif
         }
     }

src/configuration.h

@@ -561,5 +561,9 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #define HAS_SCREEN 0
 #endif
 
+#ifndef USE_ETHERNET_DEFAULT
+#define USE_ETHERNET_DEFAULT 0
+#endif
+
 #include "DebugConfiguration.h"
 #include "RF95Configuration.h"

src/gps/GPS.cpp

@@ -1304,7 +1304,7 @@ GnssModel_t GPS::probe(int serialSpeed)
             {"AG3335", "$PAIR021,AG3335", GNSS_MODEL_AG3335},
             {"AG3352", "$PAIR021,AG3352", GNSS_MODEL_AG3352},
             {"RYS3520", "$PAIR021,REYAX_RYS3520_V2", GNSS_MODEL_AG3352},
-            {"UC6580", "UC6580", GNSS_MODEL_UC6580},
+            {"UC6580", "UC6580", GNSS_MODEL_UC6580}
             // as L76K is sort of a last ditch effort, we won't attempt to detect it by startup messages for now.
             /*{"L76K", "SW=URANUS", GNSS_MODEL_MTK}*/};
         GnssModel_t detectedDriver = getProbeResponse(500, passive_detect, serialSpeed);
@@ -1331,8 +1331,10 @@ GnssModel_t GPS::probe(int serialSpeed)
     case 1: {
 
         // Unicore UFirebirdII Series: UC6580, UM620, UM621, UM670A, UM680A, or UM681A,or CM121
-        std::vector<ChipInfo> unicore = {
-            {"UC6580", "UC6580", GNSS_MODEL_UC6580}, {"UM600", "UM600", GNSS_MODEL_UC6580}, {"CM121", "CM121", GNSS_MODEL_CM121}};
+        std::vector<ChipInfo> unicore = {{"UC6580", "UC6580", GNSS_MODEL_UC6580},
+                                         {"UM600", "UM600", GNSS_MODEL_UC6580},
+                                         {"CM121", "CM121", GNSS_MODEL_CM121},
+                                         {"CC1167Q", "CC1167Q", GNSS_MODEL_CM121}};
         PROBE_FAMILY("Unicore Family", "$PDTINFO", unicore, 500);
         currentDelay = 20;
         currentStep = 2;

src/graphics/Screen.cpp

@@ -230,6 +230,7 @@ void Screen::showOverlayBanner(BannerOverlayOptions banner_overlay_options)
 #endif
     // Store the message and set the expiration timestamp
     strncpy(NotificationRenderer::alertBannerMessage, banner_overlay_options.message, 255);
+    NotificationRenderer::parseBannerMessageWithFonts(NotificationRenderer::alertBannerMessage);
     NotificationRenderer::alertBannerMessage[255] = '\0'; // Ensure null termination
     NotificationRenderer::alertBannerUntil =
         (banner_overlay_options.durationMs == 0) ? 0 : millis() + banner_overlay_options.durationMs;
@@ -239,9 +240,9 @@ void Screen::showOverlayBanner(BannerOverlayOptions banner_overlay_options)
     NotificationRenderer::alertBannerCallback = banner_overlay_options.bannerCallback;
     NotificationRenderer::curSelected = banner_overlay_options.InitialSelected;
     NotificationRenderer::pauseBanner = false;
-    NotificationRenderer::current_notification_type = notificationTypeEnum::selection_picker;
+    NotificationRenderer::current_notification_type = banner_overlay_options.notificationType;
     static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 2);
     ui->setTargetFPS(60);
     updateUiFrame(ui);
 }
@@ -263,7 +264,7 @@ void Screen::showNodePicker(const char *message, uint32_t durationMs, std::funct
     NotificationRenderer::current_notification_type = notificationTypeEnum::node_picker;
 
     static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 2);
     ui->setTargetFPS(60);
     updateUiFrame(ui);
 }
@@ -287,7 +288,7 @@ void Screen::showNumberPicker(const char *message, uint32_t durationMs, uint8_t
     NotificationRenderer::currentNumber = 0;
 
     static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 2);
     ui->setTargetFPS(60);
     updateUiFrame(ui);
 }
@@ -310,7 +311,7 @@ void Screen::showTextInput(const char *header, const char *initialText, uint32_t
 
     // Set the overlay using the same pattern as other notification types
     static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 2);
     ui->setTargetFPS(60);
     updateUiFrame(ui);
 }
@@ -424,6 +425,11 @@ Screen::Screen(ScanI2C::DeviceAddress address, meshtastic_Config_DisplayConfig_O
 #elif defined(USE_SSD1306)
     dispdev = new SSD1306Wire(address.address, -1, -1, geometry,
                               (address.port == ScanI2C::I2CPort::WIRE1) ? HW_I2C::I2C_TWO : HW_I2C::I2C_ONE);
+#if defined(OLED_Y_OFFSET_PAGES)
+    // Panels whose active window does not start at GDDRAM row 0 (e.g. 72x40
+    // modules on pages 3..7) need a fixed vertical page shift on every write.
+    static_cast<SSD1306Wire *>(dispdev)->setYOffset(OLED_Y_OFFSET_PAGES);
+#endif
 #elif defined(USE_SPISSD1306)
     dispdev = new SSD1306Spi(SSD1306_RESET, SSD1306_RS, SSD1306_NSS, GEOMETRY_64_48);
     if (!dispdev->init()) {
@@ -695,7 +701,7 @@ void Screen::setup()
     static OverlayCallback overlays[] = {
         graphics::UIRenderer::drawNavigationBar // Custom indicator icons for each frame
     };
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 1);
 
     // Enable UTF-8 to display mapping
     dispdev->setFontTableLookupFunction(customFontTableLookup);
@@ -909,7 +915,7 @@ int32_t Screen::runOnce()
 
 #ifndef DISABLE_WELCOME_UNSET
     if (!NotificationRenderer::isOverlayBannerShowing() && config.lora.region == meshtastic_Config_LoRaConfig_RegionCode_UNSET) {
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
         menuHandler::LoraRegionPicker();
 #else
         menuHandler::OnboardMessage();
@@ -1146,7 +1152,7 @@ void Screen::setFrames(FrameFocus focus)
 #if defined(DISPLAY_CLOCK_FRAME)
     if (!hiddenFrames.clock) {
         fsi.positions.clock = numframes;
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
         normalFrames[numframes++] = graphics::ClockRenderer::drawAnalogClockFrame;
 #else
         normalFrames[numframes++] = uiconfig.is_clockface_analog ? graphics::ClockRenderer::drawAnalogClockFrame
@@ -1312,7 +1318,7 @@ void Screen::setFrames(FrameFocus focus)
 
     // Add overlays: frame icons and alert banner)
     static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-    ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+    ui->setOverlays(overlays, 2);
 
     prevFrame = -1; // Force drawFavoriteNode to pick a new node (because our list just changed)
 
@@ -1605,7 +1611,7 @@ void Screen::showFrame(FrameDirection direction)
 
 void Screen::setFastFramerate()
 {
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
     dispdev->clear();
 #if GRAPHICS_TFT_COLORING_ENABLED
     prepareFrameColorRegions();
@@ -1687,7 +1693,7 @@ int Screen::handleInputEvent(const InputEvent *event)
     if (NotificationRenderer::current_notification_type == notificationTypeEnum::text_input) {
         NotificationRenderer::inEvent = *event;
         static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-        ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+        ui->setOverlays(overlays, 2);
         setFastFramerate(); // Draw ASAP
         updateUiFrame(ui);
         return 0;
@@ -1702,7 +1708,7 @@ int Screen::handleInputEvent(const InputEvent *event)
     if (NotificationRenderer::isOverlayBannerShowing()) {
         NotificationRenderer::inEvent = *event;
         static OverlayCallback overlays[] = {graphics::UIRenderer::drawNavigationBar, NotificationRenderer::drawBannercallback};
-        ui->setOverlays(overlays, sizeof(overlays) / sizeof(overlays[0]));
+        ui->setOverlays(overlays, 2);
         setFastFramerate(); // Draw ASAP
         updateUiFrame(ui);
 

src/graphics/ScreenFonts.h

@@ -88,6 +88,16 @@
 #endif
 #endif
 
+// nRF52 flash optimization: re-route FONT_LARGE_LOCAL to the 16pt glyph so
+// the display-tier dispatch below picks up 16pt everywhere it would have used
+// 24pt. Drops the ~9.6 KB ArialMT_Plain_24 table from the linked binary.
+// Set MESHTASTIC_LARGE_FONT_24PT=1 in build_flags to opt out per variant
+// (undefined or 0 keeps the optimization on).
+#if defined(ARCH_NRF52) && (!defined(MESHTASTIC_LARGE_FONT_24PT) || MESHTASTIC_LARGE_FONT_24PT == 0)
+#undef FONT_LARGE_LOCAL
+#define FONT_LARGE_LOCAL FONT_MEDIUM_LOCAL
+#endif
+
 #if (defined(USE_EINK) || defined(ILI9341_DRIVER) || defined(ILI9342_DRIVER) || defined(ST7701_CS) || defined(ST7735_CS) ||      \
      defined(ST7789_CS) || defined(USE_ST7789) || defined(HX8357_CS) || defined(ILI9488_CS) || defined(ST7796_CS) ||             \
      defined(USE_ST7796) || defined(HACKADAY_COMMUNICATOR)) &&                                                                   \
@@ -96,7 +106,7 @@
 #define FONT_SMALL FONT_MEDIUM_LOCAL // Height: 19
 #define FONT_MEDIUM FONT_LARGE_LOCAL // Height: 28
 #define FONT_LARGE FONT_LARGE_LOCAL  // Height: 28
-#elif defined(M5STACK_UNITC6L)
+#elif defined(OLED_TINY)
 #define FONT_SMALL FONT_SMALL_LOCAL  // Height: 13
 #define FONT_MEDIUM FONT_SMALL_LOCAL // Height: 13
 #define FONT_LARGE FONT_SMALL_LOCAL  // Height: 13

src/graphics/SharedUIDisplay.cpp

@@ -31,6 +31,12 @@ ScreenResolution determineScreenResolution(int16_t screenheight, int16_t screenw
         return ScreenResolution::UltraLow;
     }
 
+#ifdef DISPLAY_FORCE_SMALL_FONTS
+    if (screenwidth <= 160 && screenheight <= 80) {
+        return ScreenResolution::Low;
+    }
+#endif
+
     // Standard OLED screens
     if (screenwidth > 128 && screenheight <= 64) {
         return ScreenResolution::Low;
@@ -240,7 +246,7 @@ void drawCommonHeader(OLEDDisplay *display, int16_t x, int16_t y, const char *ti
 
     int batteryX = 1;
     int batteryY = HEADER_OFFSET_Y + 1;
-#if !defined(M5STACK_UNITC6L)
+#if !defined(OLED_TINY)
     // === Battery Icons ===
     if (usbPowered && !isCharging) { // This is a basic check to determine USB Powered is flagged but not charging
         batteryX += 1;

src/graphics/draw/DebugRenderer.cpp

@@ -460,7 +460,7 @@ void drawLoRaFocused(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x,
     nameX = (SCREEN_WIDTH - textWidth) / 2;
     display->drawString(nameX, getTextPositions(display)[line++], frequencyslot);
 
-#if !defined(M5STACK_UNITC6L)
+#if !defined(OLED_TINY)
     // === Fifth Row: Channel Utilization ===
     const char *chUtil = "ChUtil:";
     char chUtilPercentage[10];
@@ -592,7 +592,7 @@ void drawSystemScreen(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x
         // Label
         display->setTextAlignment(TEXT_ALIGN_LEFT);
         display->drawString(labelX, getTextPositions(display)[line], label);
-#if !defined(M5STACK_UNITC6L)
+#if !defined(OLED_TINY)
         // Bar
         int barY = getTextPositions(display)[line] + (FONT_HEIGHT_SMALL - barHeight) / 2;
         display->setColor(WHITE);

src/graphics/draw/MenuHandler.cpp

@@ -555,7 +555,7 @@ void menuHandler::TZPicker()
 
 void menuHandler::clockMenu()
 {
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
     static const char *optionsArray[] = {"Back", "Time Format", "Timezone"};
 #else
     static const char *optionsArray[] = {"Back", "Clock Face", "Time Format", "Timezone"};

src/graphics/draw/NodeListRenderer.cpp

@@ -26,7 +26,7 @@ extern bool haveGlyphs(const char *str);
 // Global screen instance
 extern graphics::Screen *screen;
 
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
 static uint32_t lastSwitchTime = 0;
 #endif
 namespace graphics
@@ -788,7 +788,7 @@ void drawDynamicListScreen_Nodes(OLEDDisplay *display, OLEDDisplayUiState *state
 
     unsigned long now = millis();
 
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
     display->clear();
     if (now - lastSwitchTime >= 3000) {
         display->display();
@@ -824,7 +824,7 @@ void drawDynamicListScreen_Location(OLEDDisplay *display, OLEDDisplayUiState *st
 
     unsigned long now = millis();
 
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
     display->clear();
     if (now - lastSwitchTime >= 3000) {
         display->display();
@@ -894,7 +894,7 @@ void drawNodeListWithCompasses(OLEDDisplay *display, OLEDDisplayUiState *state,
     double lat = DegD(ourSelfPos.latitude_i);
     double lon = DegD(ourSelfPos.longitude_i);
 
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
     display->clear();
     uint32_t now = millis();
     if (now - lastSwitchTime >= 2000) {

src/graphics/draw/NotificationRenderer.cpp

@@ -66,6 +66,110 @@ uint32_t pow_of_10(uint32_t n)
     return ret;
 }
 
+char graphics::NotificationRenderer::alertBannerLines[MAX_LINES + 1][64] = {};
+uint8_t graphics::NotificationRenderer::alertBannerLineCount = 0;
+graphics::NotificationRenderer::BannerFont graphics::NotificationRenderer::alertBannerLineFonts[MAX_LINES + 1] = {};
+
+static inline graphics::NotificationRenderer::BannerFont parseFontTagPrefix(const char *&p)
+{
+    // Tags must be at the start of the line:
+    // [S] small, [M] medium, [L] large
+    if (p && p[0] == '[' && p[2] == ']' && p[1] != '\0') {
+        char t = p[1];
+        if (t == 'S') {
+            p += 3;
+            return graphics::NotificationRenderer::BANNER_FONT_SMALL;
+        }
+        if (t == 'M') {
+            p += 3;
+            return graphics::NotificationRenderer::BANNER_FONT_MEDIUM;
+        }
+        if (t == 'L') {
+            p += 3;
+            return graphics::NotificationRenderer::BANNER_FONT_LARGE;
+        }
+    }
+    return graphics::NotificationRenderer::BANNER_FONT_DEFAULT;
+}
+
+static inline const uint8_t *fontForBannerLine(graphics::NotificationRenderer::BannerFont f)
+{
+    switch (f) {
+    case graphics::NotificationRenderer::BANNER_FONT_SMALL:
+        return FONT_SMALL;
+    case graphics::NotificationRenderer::BANNER_FONT_MEDIUM:
+        return FONT_MEDIUM;
+    case graphics::NotificationRenderer::BANNER_FONT_LARGE:
+        return FONT_LARGE;
+    case graphics::NotificationRenderer::BANNER_FONT_DEFAULT:
+    default:
+        return FONT_SMALL;
+    }
+}
+
+static inline uint8_t effectiveLineHeightForBannerLine(graphics::NotificationRenderer::BannerFont f)
+{
+    uint8_t height = FONT_HEIGHT_SMALL;
+    switch (f) {
+    case graphics::NotificationRenderer::BANNER_FONT_MEDIUM:
+        height = FONT_HEIGHT_MEDIUM;
+        break;
+    case graphics::NotificationRenderer::BANNER_FONT_LARGE:
+        height = FONT_HEIGHT_LARGE;
+        break;
+    case graphics::NotificationRenderer::BANNER_FONT_SMALL:
+    case graphics::NotificationRenderer::BANNER_FONT_DEFAULT:
+    default:
+        height = FONT_HEIGHT_SMALL;
+        break;
+    }
+    return (height > 3) ? (height - 3) : height;
+}
+
+void graphics::NotificationRenderer::parseBannerMessageWithFonts(const char *message)
+{
+    alertBannerLineCount = 0;
+    for (uint8_t i = 0; i < (MAX_LINES + 1); i++) {
+        alertBannerLines[i][0] = '\0';
+        alertBannerLineFonts[i] = BANNER_FONT_DEFAULT;
+    }
+
+    if (!message || !message[0]) {
+        return;
+    }
+
+    const char *p = message;
+
+    while (*p && alertBannerLineCount < (MAX_LINES + 1)) {
+        const char *lineStart = p;
+        while (*p && *p != '\n') {
+            p++;
+        }
+
+        char tmp[64] = {0};
+        size_t len = (size_t)(p - lineStart);
+        if (len > (sizeof(tmp) - 1)) {
+            len = sizeof(tmp) - 1;
+        }
+        memcpy(tmp, lineStart, len);
+        tmp[len] = '\0';
+
+        // Tag at start
+        const char *tp = tmp;
+        BannerFont f = parseFontTagPrefix(tp);
+        alertBannerLineFonts[alertBannerLineCount] = f;
+
+        // Store stripped text
+        strncpy(alertBannerLines[alertBannerLineCount], tp, sizeof(alertBannerLines[0]) - 1);
+        alertBannerLines[alertBannerLineCount][sizeof(alertBannerLines[0]) - 1] = '\0';
+        alertBannerLineCount++;
+
+        if (*p == '\n') {
+            p++;
+        }
+    }
+}
+
 // Used on boot when a certificate is being created
 void NotificationRenderer::drawSSLScreen(OLEDDisplay *display, OLEDDisplayUiState *state, int16_t x, int16_t y)
 {
@@ -375,23 +479,37 @@ void NotificationRenderer::drawAlertBannerOverlay(OLEDDisplay *display, OLEDDisp
     const char *lineStarts[MAX_LINES + 1] = {0};
     uint16_t lineCount = 0;
     char lineBuffer[40] = {0};
+    bool useTaggedTextBanner =
+        (current_notification_type == notificationTypeEnum::text_banner && alertBannerOptions == 0 && alertBannerLineCount > 0);
 
-    // Parse lines
-    char *alertEnd = alertBannerMessage + strnlen(alertBannerMessage, sizeof(alertBannerMessage));
-    lineStarts[lineCount] = alertBannerMessage;
+    if (useTaggedTextBanner) {
+        lineCount = std::min<uint8_t>(alertBannerLineCount, MAX_LINES);
+        for (uint16_t i = 0; i < lineCount; i++) {
+            lineStarts[i] = alertBannerLines[i];
+            lineLengths[i] = strlen(lineStarts[i]);
+            display->setFont(fontForBannerLine(alertBannerLineFonts[i]));
+            lineWidths[i] = display->getStringWidth(lineStarts[i], lineLengths[i], true);
+            if (lineWidths[i] > maxWidth)
+                maxWidth = lineWidths[i];
+        }
+    } else {
+        char *alertEnd = alertBannerMessage + strnlen(alertBannerMessage, sizeof(alertBannerMessage));
+        lineStarts[lineCount] = alertBannerMessage;
 
-    while ((lineCount < MAX_LINES) && (lineStarts[lineCount] < alertEnd)) {
-        lineStarts[lineCount + 1] = std::find((char *)lineStarts[lineCount], alertEnd, '\n');
-        lineLengths[lineCount] = lineStarts[lineCount + 1] - lineStarts[lineCount];
-        if (lineStarts[lineCount + 1][0] == '\n')
-            lineStarts[lineCount + 1] += 1;
-        lineWidths[lineCount] = display->getStringWidth(lineStarts[lineCount], lineLengths[lineCount], true);
-        if (lineWidths[lineCount] > maxWidth)
-            maxWidth = lineWidths[lineCount];
-        lineCount++;
+        while ((lineCount < MAX_LINES) && (lineStarts[lineCount] < alertEnd)) {
+            lineStarts[lineCount + 1] = std::find((char *)lineStarts[lineCount], alertEnd, '\n');
+            lineLengths[lineCount] = lineStarts[lineCount + 1] - lineStarts[lineCount];
+            if (lineStarts[lineCount + 1][0] == '\n')
+                lineStarts[lineCount + 1] += 1;
+            lineWidths[lineCount] = display->getStringWidth(lineStarts[lineCount], lineLengths[lineCount], true);
+            if (lineWidths[lineCount] > maxWidth)
+                maxWidth = lineWidths[lineCount];
+            lineCount++;
+        }
     }
 
     // Measure option widths
+    display->setFont(FONT_SMALL);
     for (int i = 0; i < alertBannerOptions; i++) {
         optionWidths[i] = display->getStringWidth(optionsArrayPtr[i], strlen(optionsArrayPtr[i]), true);
         if (optionWidths[i] > maxWidth)
@@ -505,6 +623,10 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
     bool needs_bell = false;
     uint16_t lineWidths[totalLines] = {0};
     uint16_t lineLengths[totalLines] = {0};
+    BannerFont lineFonts[totalLines] = {};
+    uint8_t lineEffectiveHeights[totalLines] = {0};
+    const char *renderLines[totalLines] = {0};
+    bool useTaggedBannerFonts = (current_notification_type == notificationTypeEnum::text_banner && alertBannerOptions == 0);
 
     if (maxWidth != 0)
         is_picker = true;
@@ -517,11 +639,22 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
     uint16_t widestLineWithBars = 0;
 
     while (lines[lineCount] != nullptr) {
-        auto newlinePointer = strchr(lines[lineCount], '\n');
+        const char *renderText = lines[lineCount];
+        BannerFont lineFont = BANNER_FONT_DEFAULT;
+        if (useTaggedBannerFonts && lineCount < alertBannerLineCount) {
+            renderText = alertBannerLines[lineCount];
+            lineFont = alertBannerLineFonts[lineCount];
+        }
+        renderLines[lineCount] = renderText;
+        lineFonts[lineCount] = lineFont;
+        lineEffectiveHeights[lineCount] = effectiveLineHeightForBannerLine(lineFont);
+        display->setFont(fontForBannerLine(lineFont));
+
+        auto newlinePointer = strchr(renderText, '\n');
         if (newlinePointer)
-            lineLengths[lineCount] = (newlinePointer - lines[lineCount]); // Check for newlines first
-        else // if the newline wasn't found, then pull string length from strlen
-            lineLengths[lineCount] = strlen(lines[lineCount]);
+            lineLengths[lineCount] = (newlinePointer - renderText);
+        else
+            lineLengths[lineCount] = strlen(renderText);
 
         if (current_notification_type == notificationTypeEnum::node_picker) {
             char measureBuffer[64] = {0};
@@ -533,7 +666,7 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
 
         // Consider extra width for signal bars on lines that contain "Signal:"
         uint16_t potentialWidth = lineWidths[lineCount];
-        if (graphics::bannerSignalBars >= 0 && strncmp(lines[lineCount], "Signal:", 7) == 0) {
+        if (graphics::bannerSignalBars >= 0 && strncmp(renderText, "Signal:", 7) == 0) {
             const int totalBars = 5;
             const int barWidth = 3;
             const int barSpacing = 2;
@@ -569,8 +702,21 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
 
     uint16_t screenHeight = display->height();
     uint8_t effectiveLineHeight = FONT_HEIGHT_SMALL - 3;
-    uint8_t visibleTotalLines = std::min<uint8_t>(lineCount, (screenHeight - vPadding * 2) / effectiveLineHeight);
-    uint16_t contentHeight = visibleTotalLines * effectiveLineHeight;
+    uint8_t visibleTotalLines = 0;
+    uint16_t contentHeight = 0;
+    const uint16_t availableHeight = (screenHeight > (vPadding * 2)) ? (screenHeight - vPadding * 2) : 0;
+    for (uint8_t i = 0; i < lineCount; i++) {
+        uint8_t thisLineHeight = lineEffectiveHeights[i] ? lineEffectiveHeights[i] : effectiveLineHeight;
+        if (contentHeight + thisLineHeight > availableHeight) {
+            break;
+        }
+        contentHeight += thisLineHeight;
+        visibleTotalLines++;
+    }
+    if (visibleTotalLines == 0 && lineCount > 0) {
+        visibleTotalLines = 1;
+        contentHeight = lineEffectiveHeights[0] ? lineEffectiveHeights[0] : effectiveLineHeight;
+    }
     uint16_t boxHeight = contentHeight + vPadding * 2;
     if (visibleTotalLines == 1) {
         boxHeight += (currentResolution == ScreenResolution::High) ? 4 : 3;
@@ -582,7 +728,7 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
     }
     int16_t boxTop = (display->height() / 2) - (boxHeight / 2);
     boxHeight += (currentResolution == ScreenResolution::High) ? 2 : 1;
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
     if (visibleTotalLines == 1) {
         boxTop += 25;
     }
@@ -616,15 +762,18 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
 
     // Draw Content
     int16_t lineY = boxTop + vPadding;
-    for (int i = 0; i < lineCount; i++) {
+    for (int i = 0; i < visibleTotalLines; i++) {
+        display->setFont(fontForBannerLine(lineFonts[i]));
+        int16_t thisLineHeight = lineEffectiveHeights[i] ? lineEffectiveHeights[i] : effectiveLineHeight;
         int16_t textX = boxLeft + (boxWidth - lineWidths[i]) / 2;
         if (needs_bell && i == 0) {
-            int bellY = lineY + (FONT_HEIGHT_SMALL - 8) / 2;
+            int fontHeight = thisLineHeight + 3;
+            int bellY = lineY + (fontHeight - 8) / 2;
             display->drawXbm(textX - 10, bellY, 8, 8, bell_alert);
             display->drawXbm(textX + lineWidths[i] + 2, bellY, 8, 8, bell_alert);
         }
         char lineBuffer[lineLengths[i] + 1];
-        strncpy(lineBuffer, lines[i], lineLengths[i]);
+        strncpy(lineBuffer, renderLines[i], lineLengths[i]);
         lineBuffer[lineLengths[i]] = '\0';
         // Determine if this is a pop-up or a pick list
         if (alertBannerOptions > 0 && i == 0) {
@@ -658,7 +807,7 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
                 display->drawString(textX, lineY - yOffset, lineBuffer);
             }
             display->setColor(WHITE);
-            lineY += (effectiveLineHeight - 2 - background_yOffset);
+            lineY += (thisLineHeight - 2 - background_yOffset);
         } else {
             // Pop-up
             // If this is the Signal line, center text + bars as one group
@@ -716,7 +865,7 @@ void NotificationRenderer::drawNotificationBox(OLEDDisplay *display, OLEDDisplay
                     display->drawString(textX, lineY, lineBuffer);
                 }
             }
-            lineY += (effectiveLineHeight);
+            lineY += thisLineHeight;
         }
     }
 

src/graphics/draw/NotificationRenderer.h

@@ -31,6 +31,12 @@ class NotificationRenderer
 
     static bool pauseBanner;
 
+    enum BannerFont : uint8_t { BANNER_FONT_DEFAULT = 0, BANNER_FONT_SMALL, BANNER_FONT_MEDIUM, BANNER_FONT_LARGE };
+
+    static char alertBannerLines[MAX_LINES + 1][64]; // parsed text per line
+    static uint8_t alertBannerLineCount;
+    static BannerFont alertBannerLineFonts[MAX_LINES + 1];
+    static void parseBannerMessageWithFonts(const char *message);
     static void resetBanner();
     static void showKeyboardMessagePopupWithTitle(const char *title, const char *content, uint32_t durationMs);
     static void drawBannercallback(OLEDDisplay *display, OLEDDisplayUiState *state);

src/graphics/draw/UIRenderer.cpp

@@ -25,7 +25,7 @@
 
 // External variables
 extern graphics::Screen *screen;
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
 static uint32_t lastSwitchTime = 0;
 #endif
 namespace graphics
@@ -753,7 +753,7 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
     if (!node || node->num == nodeDB->getNodeNum() || !nodeInfoLiteIsFavorite(node))
         return;
     display->clear();
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
     uint32_t now = millis();
     if (now - lastSwitchTime >= 10000) // 10000 ms = 10 秒
     {
@@ -959,7 +959,7 @@ void UIRenderer::drawFavoriteNode(OLEDDisplay *display, OLEDDisplayUiState *stat
     if (seenStr[0]) {
         display->drawString(x, getTextPositions(display)[line++], seenStr);
     }
-#if !defined(M5STACK_UNITC6L)
+#if !defined(OLED_TINY)
     // === 4. Uptime (only show if metric is present) ===
     char uptimeStr[32] = "";
     meshtastic_DeviceMetrics nodeMetrics;
@@ -1172,7 +1172,7 @@ void UIRenderer::drawDeviceFocused(OLEDDisplay *display, OLEDDisplayUiState *sta
     }
 #endif
 
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
     line += 1;
 
     // === Node Identity ===
@@ -1456,7 +1456,7 @@ void UIRenderer::drawIconScreen(const char *upperMsg, OLEDDisplay *display, OLED
     // needs to be drawn relative to x and y
 
     // draw centered icon left to right and centered above the one line of app text
-#if defined(M5STACK_UNITC6L)
+#if defined(OLED_TINY)
     display->drawXbm(x + (SCREEN_WIDTH - 50) / 2, y + (SCREEN_HEIGHT - 28) / 2, icon_width, icon_height, icon_bits);
     if (gBootSplashBoldPass) {
         display->drawXbm(x + (SCREEN_WIDTH - 50) / 2 + 1, y + (SCREEN_HEIGHT - 28) / 2, icon_width, icon_height, icon_bits);
@@ -1666,7 +1666,7 @@ void UIRenderer::drawCompassAndLocationScreen(OLEDDisplay *display, OLEDDisplayU
         }
         display->drawString(x, textPos[line++], altitudeLine);
     }
-#if !defined(M5STACK_UNITC6L)
+#if !defined(OLED_TINY)
     // === Draw Compass ===
     if (validHeading || statusLine1) {
         // --- Compass Rendering: landscape (wide) screens use original side-aligned logic ---

src/graphics/images.h

@@ -318,7 +318,7 @@ const uint8_t chirpy_small[] = {0x7f, 0x41, 0x55, 0x55, 0x55, 0x55, 0x41, 0x7f};
 #define connection_icon_height 5
 const uint8_t connection_icon[] = {0x36, 0x41, 0x5D, 0x41, 0x36};
 
-#ifdef M5STACK_UNITC6L
+#ifdef OLED_TINY
 #include "img/icon_small.xbm"
 #else
 #include "img/icon.xbm"

src/graphics/niche/InkHUD/MessageStore.cpp

@@ -53,10 +53,11 @@ void InkHUD::MessageStore::saveToFlash()
         f.write(reinterpret_cast<const uint8_t *>(&m.timestamp), sizeof(m.timestamp));       // Write timestamp. 4 bytes
         f.write(reinterpret_cast<const uint8_t *>(&m.sender), sizeof(m.sender));             // Write sender NodeId. 4 Bytes
         f.write(reinterpret_cast<const uint8_t *>(&m.channelIndex), sizeof(m.channelIndex)); // Write channel index. 1 Byte
-        f.write(reinterpret_cast<const uint8_t *>(m.text.c_str()), min(MAX_MESSAGE_SIZE, m.text.size())); // Write message text
-        f.write('\0');                                                                                    // Append null term
-        LOG_DEBUG("Wrote message %u, length %u, text \"%s\"", static_cast<uint32_t>(i), min(MAX_MESSAGE_SIZE, m.text.size()),
-                  m.text.c_str());
+        f.write(reinterpret_cast<const uint8_t *>(m.text.c_str()),
+                min((size_t)MAX_MESSAGE_SIZE, m.text.size())); // Write message text
+        f.write('\0');                                         // Append null term
+        LOG_DEBUG("Wrote message %u, length %u, text \"%s\"", static_cast<uint32_t>(i),
+                  min((size_t)MAX_MESSAGE_SIZE, m.text.size()), m.text.c_str());
     }
 
     // Release firmware's SPI lock, because SafeFile::close needs it

src/input/ExpressLRSFiveWay.h

@@ -13,6 +13,8 @@
 
 #ifdef INPUTBROKER_EXPRESSLRSFIVEWAY_TYPE
 
+// REVISIT esp_adc_cal.h
+// "legacy adc calibration driver is deprecated, please migrate to use esp_adc/adc_cali.h and esp_adc/adc_cali_scheme.h"
 #include <esp_adc_cal.h>
 #include <soc/adc_channel.h>
 

src/input/InputBroker.cpp

@@ -333,6 +333,12 @@ void InputBroker::Init()
             BaseType_t higherWake = 0;
             concurrency::mainDelay.interruptFromISR(&higherWake);
         };
+#if defined(ELECROW_ThinkNode_M7)
+        userConfigNoScreen.longLongPressTime = 15 * 1000;
+        userConfigNoScreen.longLongPress = INPUT_BROKER_FACTORY_RST;
+#else
+        userConfigNoScreen.longLongPress = INPUT_BROKER_SHUTDOWN;
+#endif
         userConfigNoScreen.singlePress = INPUT_BROKER_USER_PRESS;
         userConfigNoScreen.longPress = INPUT_BROKER_NONE;
         userConfigNoScreen.longPressTime = 500;

src/input/InputBroker.h

@@ -25,6 +25,7 @@ enum input_broker_event {
     INPUT_BROKER_USER_PRESS,
     INPUT_BROKER_ALT_PRESS,
     INPUT_BROKER_ALT_LONG,
+    INPUT_BROKER_FACTORY_RST = 0x9a,
     INPUT_BROKER_SHUTDOWN = 0x9b,
     INPUT_BROKER_GPS_TOGGLE = 0x9e,
     INPUT_BROKER_SEND_PING = 0xaf,

src/input/TLoraPagerKeyboard.cpp

@@ -68,7 +68,7 @@ TLoraPagerKeyboard::TLoraPagerKeyboard()
     : TCA8418KeyboardBase(_TCA8418_ROWS, _TCA8418_COLS), modifierFlag(0), last_modifier_time(0), last_key(UINT8_MAX),
       next_key(UINT8_MAX), last_tap(0L), char_idx(0), tap_interval(0)
 {
-#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 0, 0)
+#if ESP_ARDUINO_VERSION >= ESP_ARDUINO_VERSION_VAL(3, 0, 0)
     ledcAttach(KB_BL_PIN, LEDC_BACKLIGHT_FREQ, LEDC_BACKLIGHT_BIT_WIDTH);
 #else
     ledcSetup(LEDC_BACKLIGHT_CHANNEL, LEDC_BACKLIGHT_FREQ, LEDC_BACKLIGHT_BIT_WIDTH);
@@ -108,7 +108,7 @@ void TLoraPagerKeyboard::setBacklight(bool on)
     uint32_t _brightness = 0;
     if (on)
         _brightness = brightness;
-#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 0, 0)
+#if ESP_ARDUINO_VERSION >= ESP_ARDUINO_VERSION_VAL(3, 0, 0)
     ledcWrite(KB_BL_PIN, _brightness);
 #else
     ledcWrite(LEDC_BACKLIGHT_CHANNEL, _brightness);

src/main.cpp

@@ -2,6 +2,9 @@
 #if !MESHTASTIC_EXCLUDE_GPS
 #include "GPS.h"
 #endif
+#if !MESHTASTIC_EXCLUDE_INPUTBROKER
+#include "input/InputBroker.h"
+#endif
 #include "MeshRadio.h"
 #include "MeshService.h"
 #include "NodeDB.h"
@@ -59,12 +62,18 @@ NimbleBluetooth *nimbleBluetooth = nullptr;
 NRF52Bluetooth *nrf52Bluetooth = nullptr;
 #endif
 
-#if HAS_WIFI || defined(USE_WS5500)
+#ifdef ARCH_NRF54L15
+void nrf54l15Setup();
+void nrf54l15Loop();
+NRF54L15Bluetooth *nrf54l15Bluetooth = nullptr;
+#endif
+
+#if HAS_WIFI || defined(USE_WS5500) || defined(USE_CH390D)
 #include "mesh/api/WiFiServerAPI.h"
 #include "mesh/wifi/WiFiAPClient.h"
 #endif
 
-#if HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
 #include "mesh/api/ethServerAPI.h"
 #include "mesh/eth/ethClient.h"
 #endif
@@ -245,7 +254,7 @@ const char *getDeviceName()
 uint32_t timeLastPowered = 0;
 
 static OSThread *powerFSMthread;
-OSThread *ambientLightingThread;
+AmbientLightingThread *ambientLightingThread;
 
 RadioLibHal *RadioLibHAL = NULL;
 
@@ -335,7 +344,7 @@ void setup()
 
 #ifdef WIFI_LED
     pinMode(WIFI_LED, OUTPUT);
-    digitalWrite(WIFI_LED, LOW);
+    digitalWrite(WIFI_LED, HIGH ^ WIFI_STATE_ON);
 #endif
 
 #ifdef BLE_LED
@@ -696,6 +705,9 @@ void setup()
 #ifdef ARCH_NRF52
     nrf52Setup();
 #endif
+#ifdef ARCH_NRF54L15
+    nrf54l15Setup();
+#endif
 
 #ifdef ARCH_RP2040
     rp2040Setup();
@@ -741,6 +753,11 @@ void setup()
         }
     }
 #endif
+#ifdef OLED_GEOMETRY_OVERRIDE
+    // Per-variant geometry (e.g. 72x40 micro-OLEDs). Takes precedence over the
+    // default GEOMETRY_128_64 set at the top of setup().
+    screen_geometry = OLED_GEOMETRY_OVERRIDE;
+#endif
 #endif
 
 #if !MESHTASTIC_EXCLUDE_I2C
@@ -1126,6 +1143,9 @@ void loop()
 #endif
 #ifdef ARCH_NRF52
     nrf52Loop();
+#endif
+#ifdef ARCH_NRF54L15
+    nrf54l15Loop();
 #endif
     power->powerCommandsCheck();
 

src/main.h

@@ -19,6 +19,10 @@ extern NimbleBluetooth *nimbleBluetooth;
 #include "NRF52Bluetooth.h"
 extern NRF52Bluetooth *nrf52Bluetooth;
 #endif
+#ifdef ARCH_NRF54L15
+#include "NRF54L15Bluetooth.h"
+extern NRF54L15Bluetooth *nrf54l15Bluetooth;
+#endif
 #if !MESHTASTIC_EXCLUDE_I2C
 #include "detect/ScanI2CTwoWire.h"
 #endif

src/mesh/Channels.cpp

@@ -93,6 +93,23 @@ void Channels::initDefaultLoraConfig()
 #ifdef USERPREFS_LORACONFIG_CHANNEL_NUM
     loraConfig.channel_num = USERPREFS_LORACONFIG_CHANNEL_NUM;
 #endif
+
+    // Apply any hardcoded USERPREFS overrides for custom modem config (e.g. region-locked boards)
+#ifdef USERPREFS_LORACONFIG_USE_PRESET
+    loraConfig.use_preset = USERPREFS_LORACONFIG_USE_PRESET;
+#endif
+#ifdef USERPREFS_LORACONFIG_BANDWIDTH
+    loraConfig.bandwidth = USERPREFS_LORACONFIG_BANDWIDTH;
+#endif
+#ifdef USERPREFS_LORACONFIG_SPREAD_FACTOR
+    loraConfig.spread_factor = USERPREFS_LORACONFIG_SPREAD_FACTOR;
+#endif
+#ifdef USERPREFS_LORACONFIG_CODING_RATE
+    loraConfig.coding_rate = USERPREFS_LORACONFIG_CODING_RATE;
+#endif
+#ifdef USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY
+    loraConfig.override_frequency = USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY;
+#endif
 }
 
 bool Channels::ensureLicensedOperation()

src/mesh/InterfacesTemplates.cpp

@@ -32,7 +32,7 @@ template class LR20x0Interface<LR2021>;
 template class SX126xInterface<STM32WLx>;
 #endif
 
-#if HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
 #include "api/ethServerAPI.h"
 template class ServerAPI<EthernetClient>;
 template class APIServerPort<ethServerAPI, EthernetServer>;

src/mesh/MeshService.cpp

@@ -334,7 +334,9 @@ void MeshService::sendToPhone(meshtastic_MeshPacket *p)
 
     if (toPhoneQueue.enqueue(p, 0) == false) {
         LOG_CRIT("Failed to queue a packet into toPhoneQueue!");
-        abort();
+        releaseToPool(p);
+        fromNum++; // notify observers so phone can resync
+        return;
     }
     fromNum++;
 }
@@ -351,7 +353,8 @@ void MeshService::sendMqttMessageToClientProxy(meshtastic_MqttClientProxyMessage
 
     if (toPhoneMqttProxyQueue.enqueue(m, 0) == false) {
         LOG_CRIT("Failed to queue a packet into toPhoneMqttProxyQueue!");
-        abort();
+        releaseMqttClientProxyMessageToPool(m);
+        return;
     }
     fromNum++;
 }
@@ -383,7 +386,8 @@ void MeshService::sendClientNotification(meshtastic_ClientNotification *n)
 
     if (toPhoneClientNotificationQueue.enqueue(n, 0) == false) {
         LOG_CRIT("Failed to queue a notification into toPhoneClientNotificationQueue!");
-        abort();
+        releaseClientNotificationToPool(n);
+        return;
     }
     fromNum++;
 }

src/mesh/NodeDB.cpp

@@ -25,6 +25,7 @@
 #include "mesh/generated/meshtastic/deviceonly_legacy.pb.h"
 #include "meshUtils.h"
 #include "modules/NeighborInfoModule.h"
+#include "xmodem.h"
 #include <ErriezCRC32.h>
 #include <algorithm>
 #include <pb_decode.h>
@@ -81,6 +82,14 @@ static unsigned char userprefs_admin_key_1[] = USERPREFS_USE_ADMIN_KEY_1;
 static unsigned char userprefs_admin_key_2[] = USERPREFS_USE_ADMIN_KEY_2;
 #endif
 
+// Weak empty variant initialization function.
+// May be redefined by variant files.
+void variantDefaultConfig() __attribute__((weak));
+void variantDefaultConfig() {}
+
+void variantDefaultModuleConfig() __attribute__((weak));
+void variantDefaultModuleConfig() {}
+
 #ifdef HELTEC_MESH_NODE_T114
 
 uint32_t read8(uint8_t bits, uint8_t dummy, uint8_t cs, uint8_t sck, uint8_t mosi, uint8_t dc, uint8_t rst)
@@ -152,6 +161,25 @@ uint32_t get_st7789_id(uint8_t cs, uint8_t sck, uint8_t mosi, uint8_t dc, uint8_
 
 #endif
 
+// When armed by loadFromDisk, the decode callback writes satellite entries
+// straight into these maps instead of the temp vectors. Nullptr = legacy
+// push_back-to-vector path for backup/restore and other decoders.
+namespace
+{
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+std::map<NodeNum, meshtastic_PositionLite> *s_decodePositionsTarget = nullptr;
+#endif
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+std::map<NodeNum, meshtastic_DeviceMetrics> *s_decodeTelemetryTarget = nullptr;
+#endif
+#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+std::map<NodeNum, meshtastic_EnvironmentMetrics> *s_decodeEnvironmentTarget = nullptr;
+#endif
+#if !MESHTASTIC_EXCLUDE_STATUSDB
+std::map<NodeNum, meshtastic_StatusMessage> *s_decodeStatusTarget = nullptr;
+#endif
+} // namespace
+
 bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostream, const pb_field_t *field)
 {
     const auto *iter = reinterpret_cast<const pb_field_iter_t *>(field);
@@ -166,10 +194,10 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
                     return false;
             }
         }
-        if (istream) {
-            meshtastic_NodeInfoLite node;
+        if (istream && istream->bytes_left) {
+            meshtastic_NodeInfoLite node = meshtastic_NodeInfoLite_init_zero;
             auto *vec = static_cast<std::vector<meshtastic_NodeInfoLite> *>(iter->pData);
-            if (istream->bytes_left && pb_decode(istream, meshtastic_NodeInfoLite_fields, &node))
+            if (pb_decode(istream, meshtastic_NodeInfoLite_fields, &node))
                 vec->push_back(node);
         }
         return true;
@@ -184,11 +212,19 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
                     return false;
             }
         }
-        if (istream) {
-            meshtastic_NodePositionEntry entry;
-            auto *vec = static_cast<std::vector<meshtastic_NodePositionEntry> *>(iter->pData);
-            if (istream->bytes_left && pb_decode(istream, meshtastic_NodePositionEntry_fields, &entry))
+        if (istream && istream->bytes_left) {
+            meshtastic_NodePositionEntry entry = meshtastic_NodePositionEntry_init_zero;
+            if (pb_decode(istream, meshtastic_NodePositionEntry_fields, &entry)) {
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+                if (s_decodePositionsTarget) {
+                    if (entry.has_position)
+                        (*s_decodePositionsTarget)[entry.num] = entry.position;
+                    return true;
+                }
+#endif
+                auto *vec = static_cast<std::vector<meshtastic_NodePositionEntry> *>(iter->pData);
                 vec->push_back(entry);
+            }
         }
         return true;
     }
@@ -202,11 +238,19 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
                     return false;
             }
         }
-        if (istream) {
-            meshtastic_NodeTelemetryEntry entry;
-            auto *vec = static_cast<std::vector<meshtastic_NodeTelemetryEntry> *>(iter->pData);
-            if (istream->bytes_left && pb_decode(istream, meshtastic_NodeTelemetryEntry_fields, &entry))
+        if (istream && istream->bytes_left) {
+            meshtastic_NodeTelemetryEntry entry = meshtastic_NodeTelemetryEntry_init_zero;
+            if (pb_decode(istream, meshtastic_NodeTelemetryEntry_fields, &entry)) {
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+                if (s_decodeTelemetryTarget) {
+                    if (entry.has_device_metrics)
+                        (*s_decodeTelemetryTarget)[entry.num] = entry.device_metrics;
+                    return true;
+                }
+#endif
+                auto *vec = static_cast<std::vector<meshtastic_NodeTelemetryEntry> *>(iter->pData);
                 vec->push_back(entry);
+            }
         }
         return true;
     }
@@ -220,11 +264,19 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
                     return false;
             }
         }
-        if (istream) {
-            meshtastic_NodeStatusEntry entry;
-            auto *vec = static_cast<std::vector<meshtastic_NodeStatusEntry> *>(iter->pData);
-            if (istream->bytes_left && pb_decode(istream, meshtastic_NodeStatusEntry_fields, &entry))
+        if (istream && istream->bytes_left) {
+            meshtastic_NodeStatusEntry entry = meshtastic_NodeStatusEntry_init_zero;
+            if (pb_decode(istream, meshtastic_NodeStatusEntry_fields, &entry)) {
+#if !MESHTASTIC_EXCLUDE_STATUSDB
+                if (s_decodeStatusTarget) {
+                    if (entry.has_status)
+                        (*s_decodeStatusTarget)[entry.num] = entry.status;
+                    return true;
+                }
+#endif
+                auto *vec = static_cast<std::vector<meshtastic_NodeStatusEntry> *>(iter->pData);
                 vec->push_back(entry);
+            }
         }
         return true;
     }
@@ -238,11 +290,19 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
                     return false;
             }
         }
-        if (istream) {
-            meshtastic_NodeEnvironmentEntry entry;
-            auto *vec = static_cast<std::vector<meshtastic_NodeEnvironmentEntry> *>(iter->pData);
-            if (istream->bytes_left && pb_decode(istream, meshtastic_NodeEnvironmentEntry_fields, &entry))
+        if (istream && istream->bytes_left) {
+            meshtastic_NodeEnvironmentEntry entry = meshtastic_NodeEnvironmentEntry_init_zero;
+            if (pb_decode(istream, meshtastic_NodeEnvironmentEntry_fields, &entry)) {
+#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+                if (s_decodeEnvironmentTarget) {
+                    if (entry.has_environment_metrics)
+                        (*s_decodeEnvironmentTarget)[entry.num] = entry.environment_metrics;
+                    return true;
+                }
+#endif
+                auto *vec = static_cast<std::vector<meshtastic_NodeEnvironmentEntry> *>(iter->pData);
                 vec->push_back(entry);
+            }
         }
         return true;
     }
@@ -251,6 +311,42 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
     }
 }
 
+void NodeDB::armNodeDatabaseDecodeTargets()
+{
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+    nodePositions.clear();
+    s_decodePositionsTarget = &nodePositions;
+#endif
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+    nodeTelemetry.clear();
+    s_decodeTelemetryTarget = &nodeTelemetry;
+#endif
+#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+    nodeEnvironment.clear();
+    s_decodeEnvironmentTarget = &nodeEnvironment;
+#endif
+#if !MESHTASTIC_EXCLUDE_STATUSDB
+    nodeStatus.clear();
+    s_decodeStatusTarget = &nodeStatus;
+#endif
+}
+
+void NodeDB::disarmNodeDatabaseDecodeTargets()
+{
+#if !MESHTASTIC_EXCLUDE_POSITIONDB
+    s_decodePositionsTarget = nullptr;
+#endif
+#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
+    s_decodeTelemetryTarget = nullptr;
+#endif
+#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
+    s_decodeEnvironmentTarget = nullptr;
+#endif
+#if !MESHTASTIC_EXCLUDE_STATUSDB
+    s_decodeStatusTarget = nullptr;
+#endif
+}
+
 /** The current change # for radio settings.  Starts at 0 on boot and any time the radio settings
  * might have changed is incremented.  Allows others to detect they might now be on a new channel.
  */
@@ -297,6 +393,13 @@ NodeDB::NodeDB()
     } else {
         LOG_WARN("Failed to read unique id from efuse");
     }
+#elif defined(ARCH_NRF54L15)
+    // nRF54L15: DEVICEID is under FICR->INFO sub-struct (not top-level as on nRF52)
+    uint64_t device_id_start = ((uint64_t)NRF_FICR->INFO.DEVICEID[1] << 32) | NRF_FICR->INFO.DEVICEID[0];
+    uint64_t device_id_end = ((uint64_t)NRF_FICR->DEVICEADDR[1] << 32) | NRF_FICR->DEVICEADDR[0];
+    memcpy(myNodeInfo.device_id.bytes, &device_id_start, sizeof(device_id_start));
+    memcpy(myNodeInfo.device_id.bytes + sizeof(device_id_start), &device_id_end, sizeof(device_id_end));
+    myNodeInfo.device_id.size = 16;
 #elif defined(ARCH_NRF52)
     // Nordic applies a FIPS compliant Random ID to each chip at the factory
     // We concatenate the device address to the Random ID to create a unique ID for now
@@ -730,6 +833,23 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
     config.lora.modem_preset = USERPREFS_LORACONFIG_MODEM_PRESET;
 #else
     config.lora.modem_preset = meshtastic_Config_LoRaConfig_ModemPreset_LONG_FAST;
+#endif
+#ifdef USERPREFS_LORACONFIG_USE_PRESET
+    config.lora.use_preset = USERPREFS_LORACONFIG_USE_PRESET;
+#else
+    config.lora.use_preset = true;
+#endif
+#ifdef USERPREFS_LORACONFIG_BANDWIDTH
+    config.lora.bandwidth = USERPREFS_LORACONFIG_BANDWIDTH;
+#endif
+#ifdef USERPREFS_LORACONFIG_SPREAD_FACTOR
+    config.lora.spread_factor = USERPREFS_LORACONFIG_SPREAD_FACTOR;
+#endif
+#ifdef USERPREFS_LORACONFIG_CODING_RATE
+    config.lora.coding_rate = USERPREFS_LORACONFIG_CODING_RATE;
+#endif
+#ifdef USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY
+    config.lora.override_frequency = USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY;
 #endif
     config.lora.hop_limit = HOP_RELIABLE;
 #ifdef USERPREFS_CONFIG_LORA_IGNORE_MQTT
@@ -807,7 +927,7 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
     strncpy(config.network.ntp_server, "meshtastic.pool.ntp.org", 32);
 
 #if (defined(T_DECK) || defined(T_WATCH_S3) || defined(UNPHONE) || defined(PICOMPUTER_S3) || defined(SENSECAP_INDICATOR) ||      \
-     defined(ELECROW_PANEL) || defined(HELTEC_V4_TFT) || defined(HELTEC_V4_R8_TFT)) &&                                           \
+     defined(ELECROW_PANEL) || defined(HELTEC_V4_TFT) || defined(HELTEC_V4_R8_TFT) || defined(ELECROW_ThinkNode_M9)) &&          \
     HAS_TFT
     // switch BT off by default; use TFT programming mode or hotkey to enable
     config.bluetooth.enabled = false;
@@ -865,6 +985,10 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
     config.network.wifi_enabled = USERPREFS_NETWORK_WIFI_ENABLED;
 #endif
 
+#if USE_ETHERNET_DEFAULT
+    config.network.eth_enabled = true;
+#endif
+
 #ifdef USERPREFS_NETWORK_WIFI_SSID
     strncpy(config.network.wifi_ssid, USERPREFS_NETWORK_WIFI_SSID, sizeof(config.network.wifi_ssid));
 #endif
@@ -904,6 +1028,8 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
 #endif
 
     initConfigIntervals();
+    variantDefaultConfig();
+    variantDefaultModuleConfig();
 }
 
 void NodeDB::initConfigIntervals()
@@ -945,7 +1071,8 @@ void NodeDB::installDefaultModuleConfig()
     moduleConfig.has_store_forward = true;
     moduleConfig.has_telemetry = true;
     moduleConfig.has_external_notification = true;
-#if defined(PIN_BUZZER) || defined(PIN_VIBRATION) || defined(LED_NOTIFICATION) || defined(PCA_LED_NOTIFICATION)
+#if defined(PIN_BUZZER) || defined(PIN_VIBRATION) || defined(LED_NOTIFICATION) || defined(PCA_LED_NOTIFICATION) ||               \
+    defined(NEOPIXEL_STATUS_NOTIFICATION_PIN)
     moduleConfig.external_notification.enabled = true;
 #endif
 #if defined(PIN_BUZZER)
@@ -966,7 +1093,7 @@ void NodeDB::installDefaultModuleConfig()
 #endif
 #if defined(PIN_VIBRATION)
     moduleConfig.external_notification.nag_timeout = 2;
-#elif defined(PIN_BUZZER) || defined(LED_NOTIFICATION)
+#elif defined(PIN_BUZZER) || defined(LED_NOTIFICATION) || defined(NEOPIXEL_STATUS_NOTIFICATION_PIN)
     moduleConfig.external_notification.nag_timeout = default_ringtone_nag_secs;
 #endif
 
@@ -1119,7 +1246,14 @@ void NodeDB::initModuleConfigIntervals()
 #else
     moduleConfig.telemetry.device_update_interval = MAX_INTERVAL;
 #endif
+#ifdef USERPREFS_CONFIG_ENV_TELEM_UPDATE_INTERVAL
+    moduleConfig.telemetry.environment_update_interval = USERPREFS_CONFIG_ENV_TELEM_UPDATE_INTERVAL;
+#else
     moduleConfig.telemetry.environment_update_interval = 0;
+#endif
+#ifdef USERPREFS_CONFIG_ENVIRONMENT_MEASUREMENT_ENABLED
+    moduleConfig.telemetry.environment_measurement_enabled = USERPREFS_CONFIG_ENVIRONMENT_MEASUREMENT_ENABLED;
+#endif
     moduleConfig.telemetry.air_quality_interval = 0;
     moduleConfig.telemetry.power_update_interval = 0;
     moduleConfig.telemetry.health_update_interval = 0;
@@ -1508,6 +1642,19 @@ void NodeDB::loadFromDisk()
     }
 
 #endif
+    // Arm the direct-into-map decode so satellite entries skip the temp vectors.
+    {
+        concurrency::LockGuard guard(&satelliteMutex);
+        armNodeDatabaseDecodeTargets();
+    }
+    struct Disarm {
+        NodeDB &self;
+        ~Disarm() { self.disarmNodeDatabaseDecodeTargets(); }
+    } disarm{*this};
+
+    // Avoid push_back's power-of-2 capacity growth wasting RAM at small N.
+    nodeDatabase.nodes.reserve(MAX_NUM_NODES);
+
     auto state = loadProto(nodeDatabaseFileName, getMaxNodesAllocatedSize(), sizeof(meshtastic_NodeDatabase),
                            &meshtastic_NodeDatabase_msg, &nodeDatabase);
     if (nodeDatabase.version < DEVICESTATE_MIN_VER) {
@@ -1521,50 +1668,33 @@ void NodeDB::loadFromDisk()
     } else {
         meshNodes = &nodeDatabase.nodes;
         numMeshNodes = nodeDatabase.nodes.size();
-        // Hydrate the satellite maps; the on-disk vectors stay empty in steady
-        // state and are repopulated only at save time.
-        concurrency::LockGuard guard(&satelliteMutex);
+        // Counts computed outside LOG_INFO() so cppcheck doesn't choke on #if in macro args.
+        const unsigned posCount =
 #if !MESHTASTIC_EXCLUDE_POSITIONDB
-        nodePositions.clear();
-        nodePositions.reserve(nodeDatabase.positions.size());
-        for (const auto &entry : nodeDatabase.positions) {
-            if (entry.has_position)
-                nodePositions[entry.num] = entry.position;
-        }
-        nodeDatabase.positions.clear();
-        nodeDatabase.positions.shrink_to_fit();
+            (unsigned)nodePositions.size();
+#else
+            0u;
 #endif
+        const unsigned telCount =
 #if !MESHTASTIC_EXCLUDE_TELEMETRYDB
-        nodeTelemetry.clear();
-        nodeTelemetry.reserve(nodeDatabase.telemetry.size());
-        for (const auto &entry : nodeDatabase.telemetry) {
-            if (entry.has_device_metrics)
-                nodeTelemetry[entry.num] = entry.device_metrics;
-        }
-        nodeDatabase.telemetry.clear();
-        nodeDatabase.telemetry.shrink_to_fit();
+            (unsigned)nodeTelemetry.size();
+#else
+            0u;
 #endif
+        const unsigned envCount =
 #if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
-        nodeEnvironment.clear();
-        nodeEnvironment.reserve(nodeDatabase.environment.size());
-        for (const auto &entry : nodeDatabase.environment) {
-            if (entry.has_environment_metrics)
-                nodeEnvironment[entry.num] = entry.environment_metrics;
-        }
-        nodeDatabase.environment.clear();
-        nodeDatabase.environment.shrink_to_fit();
+            (unsigned)nodeEnvironment.size();
+#else
+            0u;
 #endif
+        const unsigned statusCount =
 #if !MESHTASTIC_EXCLUDE_STATUSDB
-        nodeStatus.clear();
-        nodeStatus.reserve(nodeDatabase.status.size());
-        for (const auto &entry : nodeDatabase.status) {
-            if (entry.has_status)
-                nodeStatus[entry.num] = entry.status;
-        }
-        nodeDatabase.status.clear();
-        nodeDatabase.status.shrink_to_fit();
+            (unsigned)nodeStatus.size();
+#else
+            0u;
 #endif
-        LOG_INFO("Loaded saved nodedatabase version %d, with nodes count: %d", nodeDatabase.version, nodeDatabase.nodes.size());
+        LOG_INFO("Loaded saved nodedatabase v%d: %d nodes, %u pos, %u tel, %u env, %u status", nodeDatabase.version,
+                 nodeDatabase.nodes.size(), posCount, telCount, envCount, statusCount);
     }
 
     if (numMeshNodes > MAX_NUM_NODES) {
@@ -1632,6 +1762,28 @@ void NodeDB::loadFromDisk()
     config.lora.tx_enabled = false;
 #endif
 
+    // Always-apply LoRa overrides: applied after loading saved config so they
+    // take effect even when NVS already has a valid config (e.g. region-locked
+    // dev boards with no BLE/serial to set the region at runtime).
+#ifdef USERPREFS_CONFIG_LORA_REGION
+    config.lora.region = USERPREFS_CONFIG_LORA_REGION;
+#endif
+#ifdef USERPREFS_LORACONFIG_USE_PRESET
+    config.lora.use_preset = USERPREFS_LORACONFIG_USE_PRESET;
+#endif
+#ifdef USERPREFS_LORACONFIG_BANDWIDTH
+    config.lora.bandwidth = USERPREFS_LORACONFIG_BANDWIDTH;
+#endif
+#ifdef USERPREFS_LORACONFIG_SPREAD_FACTOR
+    config.lora.spread_factor = USERPREFS_LORACONFIG_SPREAD_FACTOR;
+#endif
+#ifdef USERPREFS_LORACONFIG_CODING_RATE
+    config.lora.coding_rate = USERPREFS_LORACONFIG_CODING_RATE;
+#endif
+#ifdef USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY
+    config.lora.override_frequency = USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY;
+#endif
+
     if (backupSecurity.private_key.size > 0) {
         LOG_DEBUG("Restoring backup of security config");
         config.security = backupSecurity;
@@ -1842,6 +1994,15 @@ bool NodeDB::saveNodeDatabaseToDisk()
         return false;
     }
 
+    // Defer (don't fail) while xmodem holds the prefs file handle. Returning false
+    // would propagate through saveToDisk() and trigger fsFormat() mid-transfer.
+#ifdef FSCom
+    if (xModem.isBusy()) {
+        LOG_DEBUG("Deferring NodeDB save: xmodem transfer in progress");
+        return true;
+    }
+#endif
+
 #ifdef FSCom
     spiLock->lock();
     FSCom.mkdir("/prefs");

src/mesh/NodeDB.h

@@ -4,9 +4,9 @@
 #include <Arduino.h>
 #include <algorithm>
 #include <assert.h>
+#include <map>
 #include <pb_encode.h>
 #include <string>
-#include <unordered_map>
 #include <vector>
 
 #include "MeshTypes.h"
@@ -170,19 +170,19 @@ class NodeDB
     Observable<const meshtastic::NodeStatus *> newStatus;
     pb_size_t numMeshNodes;
 
-    // Satellite per-NodeNum maps for data we used to inline into NodeInfoLite,
-    // gated by MESHTASTIC_EXCLUDE_*DB so STM32WL can omit them.
+    // Satellite per-NodeNum maps. std::map avoids unordered_map's bucket-array
+    // preallocation; O(log N) lookup is fine at these sizes.
 #if !MESHTASTIC_EXCLUDE_POSITIONDB
-    std::unordered_map<NodeNum, meshtastic_PositionLite> nodePositions;
+    std::map<NodeNum, meshtastic_PositionLite> nodePositions;
 #endif
 #if !MESHTASTIC_EXCLUDE_TELEMETRYDB
-    std::unordered_map<NodeNum, meshtastic_DeviceMetrics> nodeTelemetry;
+    std::map<NodeNum, meshtastic_DeviceMetrics> nodeTelemetry;
 #endif
 #if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
-    std::unordered_map<NodeNum, meshtastic_EnvironmentMetrics> nodeEnvironment;
+    std::map<NodeNum, meshtastic_EnvironmentMetrics> nodeEnvironment;
 #endif
 #if !MESHTASTIC_EXCLUDE_STATUSDB
-    std::unordered_map<NodeNum, meshtastic_StatusMessage> nodeStatus;
+    std::map<NodeNum, meshtastic_StatusMessage> nodeStatus;
 #endif
 
     bool keyIsLowEntropy = false;
@@ -429,6 +429,11 @@ class NodeDB
     // the legacy descriptor and copies entries into the v25 layout. Caller
     // is responsible for save / install-default on the result.
     bool migrateLegacyNodeDatabase();
+
+    // Route satellite-store decode entries straight into our maps instead of
+    // temp vectors. Must be paired — disarm before any other NodeDatabase decode.
+    void armNodeDatabaseDecodeTargets();
+    void disarmNodeDatabaseDecodeTargets();
 };
 
 extern NodeDB *nodeDB;

src/mesh/PacketHistory.cpp

@@ -9,8 +9,8 @@
 #include "Throttle.h"
 
 #define PACKETHISTORY_MAX                                                                                                        \
-    max((u_int32_t)(MAX_NUM_NODES * 2.0),                                                                                        \
-        (u_int32_t)100) // x2..3  Should suffice. Empirical setup. 16B per record malloc'ed, but no less than 100
+    max((uint32_t)(MAX_NUM_NODES * 2.0),                                                                                         \
+        (uint32_t)100) // x2..3  Should suffice. Empirical setup. 16B per record malloc'ed, but no less than 100
 
 #define RECENT_WARN_AGE (10 * 60 * 1000L) // Warn if the packet that gets removed was more recent than 10 min
 

src/mesh/PhoneAPI.cpp

@@ -62,7 +62,7 @@ void PhoneAPI::handleStartConfig()
     onConfigStart();
 
     // even if we were already connected - restart our state machine
-    if (config_nonce == SPECIAL_NONCE_ONLY_NODES || config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES) {
+    if (config_nonce == SPECIAL_NONCE_ONLY_NODES) {
         // If client only wants node info, jump directly to sending nodes
         state = STATE_SEND_OWN_NODEINFO;
         LOG_INFO("Client only wants node info, skipping other config");
@@ -71,7 +71,15 @@ void PhoneAPI::handleStartConfig()
     }
     pauseBluetoothLogging = true;
     spiLock->lock();
+#if defined(MESHTASTIC_EXCLUDE_FILES_MANIFEST)
+    // Skip the recursive FS walk. Used by platforms whose Zephyr LittleFS
+    // backend can't safely traverse a deep tree (e.g. nRF54L15) and platforms
+    // that don't support OTA browsing — the manifest is only consumed by
+    // companion apps for those flows.
+    filesManifest.clear();
+#else
     filesManifest = getFiles("/", 10);
+#endif
     spiLock->unlock();
     LOG_DEBUG("Got %d files in manifest", filesManifest.size());
 
@@ -138,6 +146,7 @@ void PhoneAPI::close()
             replayTelemetryIndex = 0;
             replayEnvironmentIndex = 0;
             replayStatusIndex = 0;
+            replayPhase = REPLAY_PHASE_IDLE;
         }
         packetForPhone = NULL;
         filesManifest.clear();
@@ -320,7 +329,7 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
                 nodeInfoForPhone.num = 0;
             }
         }
-        if (config_nonce == SPECIAL_NONCE_ONLY_NODES || config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES) {
+        if (config_nonce == SPECIAL_NONCE_ONLY_NODES) {
             // If client only wants node info, jump directly to sending nodes
             state = STATE_SEND_OTHER_NODEINFOS;
             onNowHasData(0);
@@ -535,135 +544,16 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
             // Just in case we stored a different user.id in the past, but should never happen going forward
             sprintf(infoToSend.user.id, "!%08x", infoToSend.num);
 
-            // Logging this really slows down sending nodes on initial connection because the serial console is so slow, so only
-            // uncomment if you really need to:
-            // LOG_INFO("nodeinfo: num=0x%x, lastseen=%u, id=%s, name=%s", nodeInfoForPhone.num, nodeInfoForPhone.last_heard,
-            // nodeInfoForPhone.user.id, nodeInfoForPhone.user.long_name);
-
             fromRadioScratch.which_payload_variant = meshtastic_FromRadio_node_info_tag;
             fromRadioScratch.node_info = infoToSend;
             prefetchNodeInfos();
         } else {
             LOG_DEBUG("Done sending %d of %d nodeinfos millis=%u", readIndex, nodeDB->getNumMeshNodes(), millis());
-            concurrency::LockGuard guard(&nodeInfoMutex);
+            nodeInfoMutex.lock();
             nodeInfoQueue.clear();
-            // Replay states no-op for legacy clients / excluded DBs.
-            state = STATE_REPLAY_POSITIONS;
-            return getFromRadio(buf);
-        }
-        break;
-    }
-
-    case STATE_REPLAY_POSITIONS: {
-        if (replayPositionOrder.empty() && replayPositionIndex == 0)
-            beginReplayPositions();
-        prefetchReplayPositions();
-
-        meshtastic_MeshPacket pkt = {};
-        bool havePkt = false;
-        {
-            concurrency::LockGuard guard(&nodeInfoMutex);
-            if (!replayQueue.empty()) {
-                pkt = replayQueue.front();
-                replayQueue.pop_front();
-                havePkt = true;
-            }
-        }
-
-        if (havePkt) {
-            fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
-            fromRadioScratch.packet = pkt;
-        } else {
-            LOG_DEBUG("Done replaying positions count=%u millis=%u", (unsigned)replayPositionIndex, millis());
-            state = STATE_REPLAY_TELEMETRY;
-            return getFromRadio(buf);
-        }
-        break;
-    }
-
-    case STATE_REPLAY_TELEMETRY: {
-        if (replayTelemetryOrder.empty() && replayTelemetryIndex == 0)
-            beginReplayTelemetry();
-        prefetchReplayTelemetry();
-
-        meshtastic_MeshPacket pkt = {};
-        bool havePkt = false;
-        {
-            concurrency::LockGuard guard(&nodeInfoMutex);
-            if (!replayQueue.empty()) {
-                pkt = replayQueue.front();
-                replayQueue.pop_front();
-                havePkt = true;
-            }
-        }
-
-        if (havePkt) {
-            fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
-            fromRadioScratch.packet = pkt;
-        } else {
-            LOG_DEBUG("Done replaying telemetry count=%u millis=%u", (unsigned)replayTelemetryIndex, millis());
-            state = STATE_REPLAY_ENVIRONMENT;
-            return getFromRadio(buf);
-        }
-        break;
-    }
-
-    case STATE_REPLAY_ENVIRONMENT: {
-        if (replayEnvironmentOrder.empty() && replayEnvironmentIndex == 0)
-            beginReplayEnvironment();
-        prefetchReplayEnvironment();
-
-        meshtastic_MeshPacket pkt = {};
-        bool havePkt = false;
-        {
-            concurrency::LockGuard guard(&nodeInfoMutex);
-            if (!replayQueue.empty()) {
-                pkt = replayQueue.front();
-                replayQueue.pop_front();
-                havePkt = true;
-            }
-        }
-
-        if (havePkt) {
-            fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
-            fromRadioScratch.packet = pkt;
-        } else {
-            LOG_DEBUG("Done replaying environment count=%u millis=%u", (unsigned)replayEnvironmentIndex, millis());
-            state = STATE_REPLAY_STATUS;
-            return getFromRadio(buf);
-        }
-        break;
-    }
-
-    case STATE_REPLAY_STATUS: {
-        if (replayStatusOrder.empty() && replayStatusIndex == 0)
-            beginReplayStatus();
-        prefetchReplayStatus();
-
-        meshtastic_MeshPacket pkt = {};
-        bool havePkt = false;
-        {
-            concurrency::LockGuard guard(&nodeInfoMutex);
-            if (!replayQueue.empty()) {
-                pkt = replayQueue.front();
-                replayQueue.pop_front();
-                havePkt = true;
-            }
-        }
-
-        if (havePkt) {
-            fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
-            fromRadioScratch.packet = pkt;
-        } else {
-            LOG_DEBUG("Done replaying status count=%u millis=%u", (unsigned)replayStatusIndex, millis());
-            replayPositionOrder.clear();
-            replayPositionOrder.shrink_to_fit();
-            replayTelemetryOrder.clear();
-            replayTelemetryOrder.shrink_to_fit();
-            replayEnvironmentOrder.clear();
-            replayEnvironmentOrder.shrink_to_fit();
-            replayStatusOrder.clear();
-            replayStatusOrder.shrink_to_fit();
+            nodeInfoMutex.unlock();
+            // Satellite-DB replay (positions/telemetry/environment/status) now happens
+            // *after* config_complete_id, interleaved with live traffic in STATE_SEND_PACKETS.
             state = STATE_SEND_FILEMANIFEST;
             return getFromRadio(buf);
         }
@@ -673,8 +563,7 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
     case STATE_SEND_FILEMANIFEST: {
         LOG_DEBUG("FromRadio=STATE_SEND_FILEMANIFEST");
         // ONLY_NODES variants skip the manifest.
-        if (config_state == filesManifest.size() || config_nonce == SPECIAL_NONCE_ONLY_NODES ||
-            config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES) {
+        if (config_state == filesManifest.size() || config_nonce == SPECIAL_NONCE_ONLY_NODES) {
             config_state = 0;
             filesManifest.clear();
             // Skip to complete packet
@@ -719,6 +608,16 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
             fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
             fromRadioScratch.packet = *packetForPhone;
             releasePhonePacket();
+        } else if (replayPending()) {
+            // No live packet pending — feed the phone one cached satellite-DB packet.
+            // popReplayPacket advances through positions->telemetry->environment->status,
+            // and flips replayPhase back to IDLE when everything has been drained.
+            meshtastic_MeshPacket replayPkt;
+            if (popReplayPacket(replayPkt)) {
+                printPacket("replay packet to phone", &replayPkt);
+                fromRadioScratch.which_payload_variant = meshtastic_FromRadio_packet_tag;
+                fromRadioScratch.packet = replayPkt;
+            }
         }
         break;
 
@@ -743,10 +642,19 @@ size_t PhoneAPI::getFromRadio(uint8_t *buf)
 void PhoneAPI::sendConfigComplete()
 {
     LOG_INFO("Config Send Complete millis=%u", millis());
+    const bool shouldReplaySatellites = (config_nonce != SPECIAL_NONCE_ONLY_CONFIG);
+    // The phone sees config_complete_id first (treats sync as done), then the cached
+    // satellite-DB packets (positions / telemetry / environment / status) trickle in
+    // afterward as ordinary mesh packets (except SPECIAL_NONCE_ONLY_CONFIG, which
+    // skips node/satellite sync entirely). Any client that handles live POSITION_APP /
+    // TELEMETRY_APP / NODE_STATUS_APP packets handles these identically. STM32WL and
+    // other builds that compile the satellite DBs out produce no replay packets and
+    // the phase advances to IDLE in microseconds.
     fromRadioScratch.which_payload_variant = meshtastic_FromRadio_config_complete_id_tag;
     fromRadioScratch.config_complete_id = config_nonce;
     config_nonce = 0;
     state = STATE_SEND_PACKETS;
+    replayPhase = shouldReplaySatellites ? REPLAY_PHASE_POSITIONS : REPLAY_PHASE_IDLE;
     if (api_type == TYPE_BLE) {
         service->api_state = service->STATE_BLE;
     } else if (api_type == TYPE_WIFI) {
@@ -787,7 +695,8 @@ void PhoneAPI::prefetchNodeInfos()
 {
     bool added = false;
     bool wasEmpty = false;
-    const bool gradient = clientWantsGradientSync();
+    // Other-node NodeInfos always go out thin (no bundled position/device_metrics).
+    // The post-config_complete_id replay drain delivers those as ordinary mesh packets.
     // Keep the queue topped up so BLE reads stay responsive even if DB fetches take a moment.
     {
         concurrency::LockGuard guard(&nodeInfoMutex);
@@ -797,8 +706,7 @@ void PhoneAPI::prefetchNodeInfos()
             if (!nextNode)
                 break;
 
-            auto info =
-                gradient ? TypeConversions::ConvertToNodeInfoThin(nextNode) : TypeConversions::ConvertToNodeInfo(nextNode);
+            auto info = TypeConversions::ConvertToNodeInfoThin(nextNode);
             bool isUs = info.num == nodeDB->getNodeNum();
             info.hops_away = isUs ? 0 : info.hops_away;
             info.last_heard = isUs ? getValidTime(RTCQualityFromNet) : info.last_heard;
@@ -820,11 +728,20 @@ void PhoneAPI::prefetchNodeInfos()
 
 meshtastic_MeshPacket PhoneAPI::makeReplayPositionPacket(NodeNum num, const meshtastic_PositionLite &pos)
 {
+    // Shape this exactly like a fresh live broadcast Position from the peer so the
+    // phone runs it through its normal "live position broadcast" handler path.
+    // to=ourNum would read as a DM-from-peer and never lands in node detail UI.
     meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
     pkt.from = num;
-    pkt.to = nodeDB->getNodeNum();
+    pkt.to = NODENUM_BROADCAST;
     pkt.id = generatePacketId();
     pkt.rx_time = pos.time;
+    pkt.channel = 0;
+    pkt.hop_limit = Default::getConfiguredOrDefaultHopLimit(config.lora.hop_limit);
+    pkt.hop_start = pkt.hop_limit;
+    pkt.priority = meshtastic_MeshPacket_Priority_BACKGROUND;
+    // Mark as if heard over the air, not internally generated
+    pkt.transport_mechanism = meshtastic_MeshPacket_TransportMechanism_TRANSPORT_LORA;
     pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
     pkt.decoded.portnum = meshtastic_PortNum_POSITION_APP;
     meshtastic_Position fullPos = TypeConversions::ConvertToPosition(pos);
@@ -838,11 +755,18 @@ meshtastic_MeshPacket PhoneAPI::makeReplayTelemetryPacket(NodeNum num, const mes
 {
     meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
     pkt.from = num;
-    pkt.to = nodeDB->getNodeNum();
+    pkt.to = NODENUM_BROADCAST;
     pkt.id = generatePacketId();
     // No native timestamp on telemetry packets here; use last_heard.
     const meshtastic_NodeInfoLite *header = nodeDB->getMeshNode(num);
     pkt.rx_time = header ? header->last_heard : 0;
+    pkt.channel = 0;
+    pkt.hop_limit = Default::getConfiguredOrDefaultHopLimit(config.lora.hop_limit);
+    pkt.hop_start = pkt.hop_limit;
+    pkt.priority = meshtastic_MeshPacket_Priority_BACKGROUND;
+    // Mark as if heard over the air, not internally generated — iOS client filters
+    // TRANSPORT_INTERNAL packets out of broadcast peer state updates.
+    pkt.transport_mechanism = meshtastic_MeshPacket_TransportMechanism_TRANSPORT_LORA;
     pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
     pkt.decoded.portnum = meshtastic_PortNum_TELEMETRY_APP;
     meshtastic_Telemetry fullTel = meshtastic_Telemetry_init_default;
@@ -858,16 +782,12 @@ meshtastic_MeshPacket PhoneAPI::makeReplayTelemetryPacket(NodeNum num, const mes
 void PhoneAPI::beginReplayPositions()
 {
 #if MESHTASTIC_EXCLUDE_POSITIONDB
-    // Build excluded entirely - leave the order list empty so the state arm
+    // Build excluded entirely - leave the order list empty so the phase
     // immediately drains and advances.
     replayPositionOrder.clear();
     replayPositionIndex = 0;
 #else
-    if (!clientWantsGradientSync()) {
-        replayPositionOrder.clear();
-        replayPositionIndex = 0;
-        return;
-    }
+    // Caller (popReplayPacket) only invokes us when replayPhase is armed.
     // Snapshot the keyset at phase start so concurrent inserts/erases on the
     // map don't invalidate iteration. Skip our own node - the phone already
     // got our position bundled in STATE_SEND_OWN_NODEINFO.
@@ -882,8 +802,6 @@ void PhoneAPI::prefetchReplayPositions()
 #if MESHTASTIC_EXCLUDE_POSITIONDB
     return;
 #else
-    if (!clientWantsGradientSync())
-        return;
     bool added = false;
     bool wasEmpty = false;
     {
@@ -909,11 +827,6 @@ void PhoneAPI::beginReplayTelemetry()
     replayTelemetryOrder.clear();
     replayTelemetryIndex = 0;
 #else
-    if (!clientWantsGradientSync()) {
-        replayTelemetryOrder.clear();
-        replayTelemetryIndex = 0;
-        return;
-    }
     replayTelemetryOrder = nodeDB->snapshotTelemetryNodeNums(nodeDB->getNodeNum());
     replayTelemetryIndex = 0;
     LOG_INFO("Begin telemetry replay: %u entries millis=%u", (unsigned)replayTelemetryOrder.size(), millis());
@@ -925,8 +838,6 @@ void PhoneAPI::prefetchReplayTelemetry()
 #if MESHTASTIC_EXCLUDE_TELEMETRYDB
     return;
 #else
-    if (!clientWantsGradientSync())
-        return;
     bool added = false;
     bool wasEmpty = false;
     {
@@ -950,10 +861,17 @@ meshtastic_MeshPacket PhoneAPI::makeReplayEnvironmentPacket(uint32_t num, const
 {
     meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
     pkt.from = num;
-    pkt.to = nodeDB->getNodeNum();
+    pkt.to = NODENUM_BROADCAST;
     pkt.id = generatePacketId();
     const meshtastic_NodeInfoLite *header = nodeDB->getMeshNode(num);
     pkt.rx_time = header ? header->last_heard : 0;
+    pkt.channel = 0;
+    pkt.hop_limit = Default::getConfiguredOrDefaultHopLimit(config.lora.hop_limit);
+    pkt.hop_start = pkt.hop_limit;
+    pkt.priority = meshtastic_MeshPacket_Priority_BACKGROUND;
+    // Mark as if heard over the air, not internally generated — iOS client filters
+    // TRANSPORT_INTERNAL packets out of broadcast peer state updates.
+    pkt.transport_mechanism = meshtastic_MeshPacket_TransportMechanism_TRANSPORT_LORA;
     pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
     pkt.decoded.portnum = meshtastic_PortNum_TELEMETRY_APP;
     meshtastic_Telemetry fullTel = meshtastic_Telemetry_init_default;
@@ -972,11 +890,6 @@ void PhoneAPI::beginReplayEnvironment()
     replayEnvironmentOrder.clear();
     replayEnvironmentIndex = 0;
 #else
-    if (!clientWantsGradientSync()) {
-        replayEnvironmentOrder.clear();
-        replayEnvironmentIndex = 0;
-        return;
-    }
     replayEnvironmentOrder = nodeDB->snapshotEnvironmentNodeNums(nodeDB->getNodeNum());
     replayEnvironmentIndex = 0;
     LOG_INFO("Begin environment replay: %u entries millis=%u", (unsigned)replayEnvironmentOrder.size(), millis());
@@ -988,8 +901,6 @@ void PhoneAPI::prefetchReplayEnvironment()
 #if MESHTASTIC_EXCLUDE_ENVIRONMENTDB
     return;
 #else
-    if (!clientWantsGradientSync())
-        return;
     bool added = false;
     bool wasEmpty = false;
     {
@@ -1013,11 +924,17 @@ meshtastic_MeshPacket PhoneAPI::makeReplayStatusPacket(uint32_t num, const mesht
 {
     meshtastic_MeshPacket pkt = meshtastic_MeshPacket_init_default;
     pkt.from = num;
-    pkt.to = nodeDB->getNodeNum();
+    pkt.to = NODENUM_BROADCAST;
     pkt.id = generatePacketId();
     // StatusMessage has no native timestamp; use last_heard.
     const meshtastic_NodeInfoLite *header = nodeDB->getMeshNode(num);
     pkt.rx_time = header ? header->last_heard : 0;
+    pkt.channel = 0;
+    pkt.hop_limit = Default::getConfiguredOrDefaultHopLimit(config.lora.hop_limit);
+    pkt.hop_start = pkt.hop_limit;
+    pkt.priority = meshtastic_MeshPacket_Priority_BACKGROUND;
+    // Mark as if heard over the air, not internally generated — client filters
+    pkt.transport_mechanism = meshtastic_MeshPacket_TransportMechanism_TRANSPORT_LORA;
     pkt.which_payload_variant = meshtastic_MeshPacket_decoded_tag;
     pkt.decoded.portnum = meshtastic_PortNum_NODE_STATUS_APP;
     size_t len =
@@ -1032,11 +949,6 @@ void PhoneAPI::beginReplayStatus()
     replayStatusOrder.clear();
     replayStatusIndex = 0;
 #else
-    if (!clientWantsGradientSync()) {
-        replayStatusOrder.clear();
-        replayStatusIndex = 0;
-        return;
-    }
     replayStatusOrder = nodeDB->snapshotStatusNodeNums(nodeDB->getNodeNum());
     replayStatusIndex = 0;
     LOG_INFO("Begin status replay: %u entries millis=%u", (unsigned)replayStatusOrder.size(), millis());
@@ -1048,8 +960,6 @@ void PhoneAPI::prefetchReplayStatus()
 #if MESHTASTIC_EXCLUDE_STATUSDB
     return;
 #else
-    if (!clientWantsGradientSync())
-        return;
     bool added = false;
     bool wasEmpty = false;
     {
@@ -1069,6 +979,94 @@ void PhoneAPI::prefetchReplayStatus()
 #endif
 }
 
+// Pop one cached satellite-DB packet from the active replay phase.
+// Phases drain in order: positions -> telemetry -> environment -> status.
+// When the current phase's cursor is exhausted (queue empty AND no more entries
+// to snapshot), advance to the next phase. When all four phases are done,
+// flip replayPhase back to IDLE and release the snapshot vectors.
+//
+// Returns true if a packet was placed in `out`; false if everything is drained.
+bool PhoneAPI::popReplayPacket(meshtastic_MeshPacket &out)
+{
+    while (replayPhase != REPLAY_PHASE_IDLE) {
+        // Prime the active phase: seed the snapshot vector on first entry,
+        // top up replayQueue from the snapshot up to kReplayPrefetchDepth.
+        switch (replayPhase) {
+        case REPLAY_PHASE_POSITIONS:
+            if (replayPositionOrder.empty() && replayPositionIndex == 0)
+                beginReplayPositions();
+            prefetchReplayPositions();
+            break;
+        case REPLAY_PHASE_TELEMETRY:
+            if (replayTelemetryOrder.empty() && replayTelemetryIndex == 0)
+                beginReplayTelemetry();
+            prefetchReplayTelemetry();
+            break;
+        case REPLAY_PHASE_ENVIRONMENT:
+            if (replayEnvironmentOrder.empty() && replayEnvironmentIndex == 0)
+                beginReplayEnvironment();
+            prefetchReplayEnvironment();
+            break;
+        case REPLAY_PHASE_STATUS:
+            if (replayStatusOrder.empty() && replayStatusIndex == 0)
+                beginReplayStatus();
+            prefetchReplayStatus();
+            break;
+        default:
+            break;
+        }
+
+        {
+            concurrency::LockGuard guard(&nodeInfoMutex);
+            if (!replayQueue.empty()) {
+                out = replayQueue.front();
+                replayQueue.pop_front();
+                return true;
+            }
+        }
+
+        // Queue empty AND no more entries to feed it — phase is exhausted.
+        advanceReplayPhase();
+    }
+    return false;
+}
+
+void PhoneAPI::advanceReplayPhase()
+{
+    switch (replayPhase) {
+    case REPLAY_PHASE_POSITIONS:
+        LOG_DEBUG("Replay drain: positions done (count=%u) millis=%u", (unsigned)replayPositionIndex, millis());
+        replayPhase = REPLAY_PHASE_TELEMETRY;
+        break;
+    case REPLAY_PHASE_TELEMETRY:
+        LOG_DEBUG("Replay drain: telemetry done (count=%u) millis=%u", (unsigned)replayTelemetryIndex, millis());
+        replayPhase = REPLAY_PHASE_ENVIRONMENT;
+        break;
+    case REPLAY_PHASE_ENVIRONMENT:
+        LOG_DEBUG("Replay drain: environment done (count=%u) millis=%u", (unsigned)replayEnvironmentIndex, millis());
+        replayPhase = REPLAY_PHASE_STATUS;
+        break;
+    case REPLAY_PHASE_STATUS:
+        LOG_INFO("Replay drain complete (status count=%u) millis=%u", (unsigned)replayStatusIndex, millis());
+        replayPositionOrder.clear();
+        replayPositionOrder.shrink_to_fit();
+        replayTelemetryOrder.clear();
+        replayTelemetryOrder.shrink_to_fit();
+        replayEnvironmentOrder.clear();
+        replayEnvironmentOrder.shrink_to_fit();
+        replayStatusOrder.clear();
+        replayStatusOrder.shrink_to_fit();
+        replayPositionIndex = 0;
+        replayTelemetryIndex = 0;
+        replayEnvironmentIndex = 0;
+        replayStatusIndex = 0;
+        replayPhase = REPLAY_PHASE_IDLE;
+        break;
+    default:
+        break;
+    }
+}
+
 void PhoneAPI::releaseMqttClientProxyPhonePacket()
 {
     if (mqttClientProxyMessageForPhone) {
@@ -1115,31 +1113,6 @@ bool PhoneAPI::available()
     PREFETCH_NODEINFO:
         prefetchNodeInfos();
         return true;
-    case STATE_REPLAY_POSITIONS: {
-        // Prime the iterator if we haven't yet, then top up the queue.
-        if (replayPositionOrder.empty() && replayPositionIndex == 0)
-            beginReplayPositions();
-        prefetchReplayPositions();
-        return true; // Always advance state machine; arm itself transitions when drained
-    }
-    case STATE_REPLAY_TELEMETRY: {
-        if (replayTelemetryOrder.empty() && replayTelemetryIndex == 0)
-            beginReplayTelemetry();
-        prefetchReplayTelemetry();
-        return true;
-    }
-    case STATE_REPLAY_ENVIRONMENT: {
-        if (replayEnvironmentOrder.empty() && replayEnvironmentIndex == 0)
-            beginReplayEnvironment();
-        prefetchReplayEnvironment();
-        return true;
-    }
-    case STATE_REPLAY_STATUS: {
-        if (replayStatusOrder.empty() && replayStatusIndex == 0)
-            beginReplayStatus();
-        prefetchReplayStatus();
-        return true;
-    }
     case STATE_SEND_PACKETS: {
         if (!queueStatusPacketForPhone)
             queueStatusPacketForPhone = service->getQueueStatusForPhone();
@@ -1171,7 +1144,11 @@ bool PhoneAPI::available()
         if (!packetForPhone)
             packetForPhone = service->getForPhone();
         hasPacket = !!packetForPhone;
-        return hasPacket;
+        if (hasPacket)
+            return true;
+        // Trailing replay drain — feeds cached satellite-DB packets alongside
+        // (lower priority than) live traffic.
+        return replayPending();
     }
     default:
         LOG_ERROR("PhoneAPI::available unexpected state %d", state);

src/mesh/PhoneAPI.h

@@ -26,9 +26,6 @@
 
 #define SPECIAL_NONCE_ONLY_CONFIG 69420
 #define SPECIAL_NONCE_ONLY_NODES 69421 // ( ͡° ͜ʖ ͡°)
-// Gradient sync: phone sends one of these to opt into thin-header + replay.
-#define SPECIAL_NONCE_GRADIENT_SYNC 69422
-#define SPECIAL_NONCE_GRADIENT_ONLY_NODES 69423
 
 /**
  * Provides our protobuf based API which phone/PC clients can use to talk to our device
@@ -52,15 +49,22 @@ class PhoneAPI
         STATE_SEND_CONFIG,          // Replacement for the old Radioconfig
         STATE_SEND_MODULECONFIG,    // Send Module specific config
         STATE_SEND_OTHER_NODEINFOS, // states progress in this order as the device sends to to the client
-        // Drain satellite DBs as synthetic POSITION_APP / TELEMETRY_APP /
-        // NODE_STATUS_APP packets when the phone opted into gradient sync.
-        STATE_REPLAY_POSITIONS,
-        STATE_REPLAY_TELEMETRY,
-        STATE_REPLAY_ENVIRONMENT,
-        STATE_REPLAY_STATUS,
-        STATE_SEND_FILEMANIFEST, // Send file manifest
+        STATE_SEND_FILEMANIFEST,    // Send file manifest
         STATE_SEND_COMPLETE_ID,
-        STATE_SEND_PACKETS // send packets or debug strings
+        STATE_SEND_PACKETS // live mesh packets + any cached satellite-DB replay that trails sync completion
+    };
+
+    // Satellite-DB replay (positions / telemetry / environment / status) used to live
+    // as four top-level states between STATE_SEND_OTHER_NODEINFOS and STATE_SEND_FILEMANIFEST.
+    // It now drains *after* config_complete_id has been emitted: the phone considers the
+    // initial sync done as soon as headers + manifest are delivered, and the cached
+    // position/telemetry/etc. trickle in alongside live mesh traffic inside STATE_SEND_PACKETS.
+    enum ReplayPhase : uint8_t {
+        REPLAY_PHASE_IDLE = 0, // not replaying (legacy clients, no-op DBs, or replay finished)
+        REPLAY_PHASE_POSITIONS,
+        REPLAY_PHASE_TELEMETRY,
+        REPLAY_PHASE_ENVIRONMENT,
+        REPLAY_PHASE_STATUS,
     };
 
     State state = STATE_SEND_NOTHING;
@@ -114,6 +118,7 @@ class PhoneAPI
     size_t replayTelemetryIndex = 0;
     size_t replayEnvironmentIndex = 0;
     size_t replayStatusIndex = 0;
+    ReplayPhase replayPhase = REPLAY_PHASE_IDLE; // armed by sendConfigComplete() for full/default sync
 
     meshtastic_ToRadio toRadioScratch = {
         0}; // this is a static scratch object, any data must be copied elsewhere before returning
@@ -164,10 +169,6 @@ class PhoneAPI
 
     bool isConnected() { return state != STATE_SEND_NOTHING; }
     bool isSendingPackets() { return state == STATE_SEND_PACKETS; }
-    bool clientWantsGradientSync() const
-    {
-        return config_nonce == SPECIAL_NONCE_GRADIENT_SYNC || config_nonce == SPECIAL_NONCE_GRADIENT_ONLY_NODES;
-    }
 
   protected:
     /// Our fromradio packet while it is being assembled
@@ -229,6 +230,12 @@ class PhoneAPI
     meshtastic_MeshPacket makeReplayEnvironmentPacket(uint32_t num, const meshtastic_EnvironmentMetrics &env);
     meshtastic_MeshPacket makeReplayStatusPacket(uint32_t num, const meshtastic_StatusMessage &status);
 
+    // Post-sync replay drain: pop one cached packet from the active phase, advancing
+    // through positions -> telemetry -> environment -> status until everything is drained.
+    bool popReplayPacket(meshtastic_MeshPacket &out);
+    void advanceReplayPhase();
+    bool replayPending() const { return replayPhase != REPLAY_PHASE_IDLE; }
+
     void releaseMqttClientProxyPhonePacket();
 
     void releaseClientNotification();

src/mesh/PositionPrecision.cpp

@@ -0,0 +1,75 @@
+#include "PositionPrecision.h"
+#include "Channels.h"
+#include "mesh-pb-constants.h"
+
+#include <Arduino.h>
+
+uint32_t getPositionPrecisionForChannel(uint8_t channelIndex)
+{
+    const meshtastic_Channel &channel = channels.getByIndex(channelIndex);
+
+    if (channel.settings.has_module_settings) {
+        return channel.settings.module_settings.position_precision;
+    } else if (channel.role == meshtastic_Channel_Role_PRIMARY) {
+        return 32;
+    } else {
+        return 0;
+    }
+}
+
+static int32_t truncateCoordinate(int32_t coordinate, uint32_t precision)
+{
+    uint32_t coordinateBits = static_cast<uint32_t>(coordinate);
+    uint32_t truncated = coordinateBits & (UINT32_MAX << (32 - precision));
+
+    // Use the middle of the possible location, not the low edge of the bucket.
+    truncated += (1UL << (31 - precision));
+
+    return static_cast<int32_t>(truncated);
+}
+
+void applyPositionPrecision(meshtastic_Position &position, uint32_t precision)
+{
+    if (precision == 0) {
+        uint32_t time = position.time;
+        position = meshtastic_Position_init_default;
+        position.time = time;
+        return;
+    }
+
+    uint32_t effectivePrecision = precision > 32 ? 32 : precision;
+    position.precision_bits = effectivePrecision;
+
+    if (effectivePrecision < 32) {
+        position.latitude_i = truncateCoordinate(position.latitude_i, effectivePrecision);
+        position.longitude_i = truncateCoordinate(position.longitude_i, effectivePrecision);
+    }
+}
+
+bool applyPositionPrecision(meshtastic_MeshPacket &packet, uint32_t precision)
+{
+    if (packet.which_payload_variant != meshtastic_MeshPacket_decoded_tag ||
+        packet.decoded.portnum != meshtastic_PortNum_POSITION_APP) {
+        return true;
+    }
+
+    meshtastic_Position position = meshtastic_Position_init_default;
+    if (!pb_decode_from_bytes(packet.decoded.payload.bytes, packet.decoded.payload.size, &meshtastic_Position_msg, &position)) {
+        return false;
+    }
+
+    applyPositionPrecision(position, precision);
+    packet.decoded.payload.size = pb_encode_to_bytes(packet.decoded.payload.bytes, sizeof(packet.decoded.payload.bytes),
+                                                     &meshtastic_Position_msg, &position);
+    return true;
+}
+
+bool applyPositionPrecisionForChannel(meshtastic_MeshPacket &packet, uint8_t channelIndex)
+{
+    if (packet.which_payload_variant != meshtastic_MeshPacket_decoded_tag ||
+        packet.decoded.portnum != meshtastic_PortNum_POSITION_APP) {
+        return true;
+    }
+
+    return applyPositionPrecision(packet, getPositionPrecisionForChannel(channelIndex));
+}

src/mesh/PositionPrecision.h

@@ -0,0 +1,9 @@
+#pragma once
+
+#include "meshtastic/mesh.pb.h"
+#include <stdint.h>
+
+uint32_t getPositionPrecisionForChannel(uint8_t channelIndex);
+void applyPositionPrecision(meshtastic_Position &position, uint32_t precision);
+bool applyPositionPrecision(meshtastic_MeshPacket &packet, uint32_t precision);
+bool applyPositionPrecisionForChannel(meshtastic_MeshPacket &packet, uint8_t channelIndex);

src/mesh/RadioLibInterface.cpp

@@ -46,16 +46,6 @@ RadioLibInterface::RadioLibInterface(LockingArduinoHal *hal, RADIOLIB_PIN_TYPE c
 #endif
 }
 
-RadioLibInterface::~RadioLibInterface()
-{
-    // If the static `instance` pointer still references us, clear it.
-    // A later successful init() may have replaced `instance` with a newer
-    // interface — don't clobber that case.
-    if (instance == this) {
-        instance = nullptr;
-    }
-}
-
 #ifdef ARCH_ESP32
 // ESP32 doesn't use that flag
 #define YIELD_FROM_ISR(x) portYIELD_FROM_ISR()
@@ -434,6 +424,9 @@ void RadioLibInterface::completeSending()
     // that can take a long time
     auto p = sendingPacket;
     sendingPacket = NULL;
+#ifdef LED_LORA
+    digitalWrite(LED_LORA, LED_STATE_OFF);
+#endif
 
     if (p) {
         // Packet has been sent, count it toward our TX airtime utilization.
@@ -555,6 +548,9 @@ void RadioLibInterface::pollMissedIrqs()
     if (isReceiving) {
         checkRxDoneIrqFlag();
     }
+    if (sendingPacket) {
+        checkTxDoneIrqFlag();
+    }
 }
 
 void RadioLibInterface::resetAGC()
@@ -570,6 +566,14 @@ void RadioLibInterface::checkRxDoneIrqFlag()
     }
 }
 
+void RadioLibInterface::checkTxDoneIrqFlag()
+{
+    if (iface->checkIrq(RADIOLIB_IRQ_TX_DONE)) {
+        LOG_WARN("caught missed TX_DONE");
+        notify(ISR_TX, true);
+    }
+}
+
 void RadioLibInterface::configHardwareForSend()
 {
     powerMon->setState(meshtastic_PowerMon_State_Lora_TXOn);
@@ -611,6 +615,9 @@ bool RadioLibInterface::startSend(meshtastic_MeshPacket *txp)
             enableInterrupt(isrTxLevel0);
             lastTxStart = millis();
             printPacket("Started Tx", txp);
+#ifdef LED_LORA
+            digitalWrite(LED_LORA, LED_STATE_ON);
+#endif
         }
 
         return res == RADIOLIB_ERR_NONE;

src/mesh/RadioLibInterface.h

@@ -104,6 +104,13 @@ class RadioLibInterface : public RadioInterface, protected concurrency::Notified
      */
     static RadioLibInterface *instance;
 
+    /** Clear instance on destruction so stale pointer checks in loop() are safe */
+    virtual ~RadioLibInterface()
+    {
+        if (instance == this)
+            instance = nullptr;
+    }
+
     /**
      * Glue functions called from ISR land
      */
@@ -136,13 +143,6 @@ class RadioLibInterface : public RadioInterface, protected concurrency::Notified
     RadioLibInterface(LockingArduinoHal *hal, RADIOLIB_PIN_TYPE cs, RADIOLIB_PIN_TYPE irq, RADIOLIB_PIN_TYPE rst,
                       RADIOLIB_PIN_TYPE busy, PhysicalLayer *iface = NULL);
 
-    /**
-     * Clear the static `instance` pointer if it still points at us, so callers
-     * that check `RadioLibInterface::instance != nullptr` don't dereference a
-     * freed object after a failed init() + unique_ptr reset.
-     */
-    virtual ~RadioLibInterface();
-
     virtual ErrorCode send(meshtastic_MeshPacket *p) override;
 
     /**
@@ -293,4 +293,5 @@ class RadioLibInterface : public RadioInterface, protected concurrency::Notified
     bool removePendingTXPacket(NodeNum from, PacketId id, uint32_t hop_limit_lt) override;
 
     void checkRxDoneIrqFlag();
+    void checkTxDoneIrqFlag();
 };

src/mesh/Router.cpp

@@ -4,6 +4,7 @@
 #include "MeshRadio.h"
 #include "MeshService.h"
 #include "NodeDB.h"
+#include "PositionPrecision.h"
 #include "RTC.h"
 
 #include "configuration.h"
@@ -367,6 +368,11 @@ ErrorCode Router::send(meshtastic_MeshPacket *p)
     }
 
     fixPriority(p); // Before encryption, fix the priority if it's unset
+    if (!applyPositionPrecisionForChannel(*p, p->channel)) {
+        LOG_ERROR("Dropping malformed position packet before send");
+        packetPool.release(p);
+        return meshtastic_Routing_Error_BAD_REQUEST;
+    }
 
     // If the packet is not yet encrypted, do so now
     if (p->which_payload_variant == meshtastic_MeshPacket_decoded_tag) {

src/mesh/TypeConversions.cpp

@@ -37,6 +37,7 @@ meshtastic_NodeInfo TypeConversions::ConvertToNodeInfo(const meshtastic_NodeInfo
         info.position.altitude = position->altitude;
         info.position.location_source = position->location_source;
         info.position.time = position->time;
+        info.position.precision_bits = position->precision_bits;
     }
     if (nodeInfoLiteHasUser(lite)) {
         info.has_user = true;
@@ -71,6 +72,7 @@ meshtastic_PositionLite TypeConversions::ConvertToPositionLite(meshtastic_Positi
     lite.altitude = position.altitude;
     lite.location_source = position.location_source;
     lite.time = position.time;
+    lite.precision_bits = position.precision_bits;
 
     return lite;
 }
@@ -89,6 +91,11 @@ meshtastic_Position TypeConversions::ConvertToPosition(meshtastic_PositionLite l
     position.altitude = lite.altitude;
     position.location_source = lite.location_source;
     position.time = lite.time;
+    // Preserve the peer's broadcast precision; falls back to 0 for entries cached
+    // before the precision_bits field existed in PositionLite (pre-migration data).
+    // iOS treats 0 as "unspecified precision" and won't render the pin — so for
+    // unset values, declare full precision so the stored lat/lon renders as a point.
+    position.precision_bits = lite.precision_bits == 0 ? 32 : lite.precision_bits;
 
     return position;
 }

src/mesh/api/WiFiServerAPI.h

@@ -3,9 +3,8 @@
 #include "ServerAPI.h"
 #include <WiFi.h>
 
-#if HAS_ETHERNET && defined(USE_WS5500)
-#include <ETHClass2.h>
-#define ETH ETH2
+#if HAS_ETHERNET && defined(ARCH_ESP32)
+#include <ETH.h>
 #endif // HAS_ETHERNET
 
 /**

src/mesh/api/ethServerAPI.cpp

@@ -1,7 +1,7 @@
 #include "configuration.h"
 #include <Arduino.h>
 
-#if HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && !defined(USE_WS5500) && !defined(USE_CH390D)
 
 #include "ethServerAPI.h"
 

src/mesh/api/ethServerAPI.h

@@ -1,7 +1,7 @@
 #pragma once
 
 #include "ServerAPI.h"
-#ifndef USE_WS5500
+#if !defined(USE_WS5500) && !defined(USE_CH390D)
 #include <RAK13800_W5100S.h>
 
 /**

src/mesh/eth/ethClient.cpp

@@ -9,7 +9,7 @@
 #include <RAK13800_W5100S.h>
 #include <SPI.h>
 
-#if HAS_NETWORKING
+#if HAS_NETWORKING && !defined(USE_WS5500) && !defined(USE_CH390D)
 
 #ifndef DISABLE_NTP
 #include <NTPClient.h>