Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

.claude/commands/diagnose.md

@@ -49,17 +49,11 @@ Call the meshtastic MCP tool bundle and format a structured health report for on
    - Do the LoRa configs match? (region, channel_num, modem_preset should all agree; mismatch = no mesh)
    - Do the primary channel NAMES match? Mismatch = different PSK = no decode.
 
-7. **Recorder slice (cheap, always available).** The mcp-server runs an autouse log recorder that's been collecting from every connected device. Pull two short slices to surface anything weird that's already happened:
-   - `mcp__meshtastic__logs_window(start="-2m", level="WARN|ERROR|CRIT", max_lines=20)` — recent firmware errors. If empty, say "no recent errors"; don't manufacture concern.
-   - `mcp__meshtastic__telemetry_timeline(window="1h", field="free_heap", max_points=60)` — heap trend. If `slope_per_min < -50`, flag it and recommend `/leakhunt window=6h` for a deeper read; otherwise just note the current free heap.
-   - If `recorder_status` shows `running:false` or `files.telemetry.last_ts` is null, note "recorder has no telemetry yet — enable `set_debug_log_api(True)` to populate" and skip this step gracefully.
-
-8. **Suggest next actions only for specific, recognisable failure modes**:
+7. **Suggest next actions only for specific, recognisable failure modes**:
    - Stale PKI pubkey one-way → "run `/test tests/mesh/test_direct_with_ack.py` — the retry + nodeinfo-ping heals this in the test path."
    - Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
    - Device unreachable, reachable via DFU → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds AND the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
    - CP2102-wedged-driver on macOS → see the note in `run-tests.sh`.
-   - Heap slope strongly negative → "run `/leakhunt window=6h` for a full timeline + classification."
 
 ## What NOT to do
 

.claude/commands/leakhunt.md

@@ -1,103 +0,0 @@
----
-description: Hunt for memory leaks (and other slow degradations) by reading the persistent recorder's heap timeline + log slice over a window
-argument-hint: [window=1h] [field=free_heap] [variant=local]
----
-
-<!-- markdownlint-disable MD029 -->
-
-# `/leakhunt` — read the recorder, classify a memory leak
-
-Use the always-on recorder (`mcp-server/.mtlog/`) to read a heap timeline plus the matching log slice and produce a one-page verdict: **steady / slow leak / fragmentation / OOM-imminent**. No firmware changes, no special build flags — the LocalStats telemetry packet that the firmware already broadcasts every ~60 s carries `heap_free_bytes` and `heap_total_bytes`.
-
-## Two signal paths — pick the right one
-
-| Path                  | Build flag       | Cadence        | Per-thread attribution | Cost                      |
-| --------------------- | ---------------- | -------------- | ---------------------- | ------------------------- |
-| LocalStats packet     | (default)        | ~60 s          | No                     | Free — always on          |
-| `[heap N]` log prefix | `-DDEBUG_HEAP=1` | every log line | Yes (Thread X leaked)  | Bigger flash + log volume |
-
-Both feed the same `telemetry_timeline(field="free_heap")` query — when DEBUG_HEAP is on, the recorder synthesizes telemetry rows from log prefixes (tagged `source: debug_heap`), so a single timeline call gets whichever signal is available. **For a slow leak diagnosis, the default path is plenty** (60 s cadence over 6 h = 360 points; linear regression over that nails sub-100-byte/min slopes). **DEBUG_HEAP is for attribution** — when the slope is real and you need to know which thread is leaking.
-
-## What to do
-
-1. **Parse `$ARGUMENTS`**: optional `window` (default `1h`, accepts `30m`/`6h`/`-3d`/etc.), optional `field` (default `free_heap`; alternates: `total_heap`, `battery_level`, anything in the LocalStats variant), optional `variant` (default `local`; alternates: `device`, `environment`, `power`, `airQuality`, `health`).
-
-2. **Verify the recorder is alive** — call `mcp__meshtastic__recorder_status`. Check:
-   - `running == True`
-   - `files.telemetry.lines > 0` (at least one telemetry packet recorded — if zero, the device hasn't broadcast LocalStats yet OR `set_debug_log_api` has never been on; tell the operator to run `mcp__meshtastic__set_debug_log_api(enabled=True)` and wait one device-update interval)
-   - `files.telemetry.last_ts` within the last 5 minutes (if older, the device is silent — log that, not "leak detected")
-
-3. **Detect whether DEBUG_HEAP is active** — `mcp__meshtastic__logs_window(start="-2m", grep=r"\\[heap \\d+\\]", max_lines=3)`. If any line matches, the firmware has the prefix → DEBUG_HEAP is on, expect higher-cadence data and `heap_event` rows. If zero matches over the last 2 minutes, you're on the LocalStats-only path.
-
-4. **Pull the timeline** — `mcp__meshtastic__telemetry_timeline(window=$window, variant=$variant, field=$field, max_points=200)`. Read:
-   - `samples` — how many raw points contributed
-   - `min`, `max` — total swing
-   - `slope_per_min` — units per minute (linear regression over the whole window)
-
-5. **Pull the log context for the same window** — `mcp__meshtastic__logs_window(start="-${window}", grep="Heap status|leaked heap|freed heap|out of memory|Alloc an err|panic|abort", max_lines=200)`. These are the strings the firmware emits when something memory-related happens (`DEBUG_HEAP` builds emit `"Heap status:"` and `"leaked heap"` lines; production builds emit `"Alloc an err"` on failure and `"out of memory"` on OOM).
-
-6. **Pull marker events** so we know if the operator labeled phases — `mcp__meshtastic__events_window(start="-${window}", kind="mark|connection_lost|connection_established")`. If a `connection_lost` overlaps a sharp drop, that's not a leak; that's a reboot.
-
-6a. **(DEBUG_HEAP only) Per-thread attribution** — `mcp__meshtastic__logs_window(start="-${window}", grep="leaked heap", max_lines=200)`. Each row has a structured `heap_event` field with `{kind, thread, before, after, delta}`. Aggregate by thread: sum the `delta` over the window per thread name. The thread with the largest cumulative negative delta is your suspect. Note the count too — a thread with 50× small leaks is different from 1× big leak.
-
-7. **Classify** based on what the data says, NOT on what you wish it said. Use these rules in order:
-   - **Insufficient data** (< 5 samples): say so. Suggest a longer window or longer wait. Stop.
-   - **Reboot mid-window**: if any `connection_lost` event is present AND `free_heap` jumped UP at that timestamp, the device rebooted. Note it; pre-reboot trend may be a leak but you only have part of the curve.
-   - **OOM-imminent**: any `Alloc an err=` or `out of memory` line in the log slice. This trumps everything; flag urgently.
-   - **Slow leak**: `slope_per_min < -50` AND `max - min > 1000` AND no reboot. The heap is monotonically (or near-monotonically) declining. Estimate time-to-zero: `min / -slope_per_min` minutes. Surface it.
-   - **Fragmentation suspect**: `slope_per_min` close to zero (|x| < 50) BUT min trends down across the window AND the log slice shows `Alloc an err` warnings WITHOUT total OOM. Means free total is OK but largest contiguous block is shrinking. Recommend a `DEBUG_HEAP` build to confirm.
-   - **Steady**: |slope_per_min| < 50, no error lines. Heap is fine.
-   - **Recovery curve**: slope is POSITIVE — heap recovered. Either a workload completed or GC fired. Note it; not a leak.
-
-8. **Report**:
-
-   ```text
-   /leakhunt window=6h field=free_heap variant=local
-   ────────────────────────────────────────────────────
-   recorder        : running, telem last_ts 8s ago
-   build           : DEBUG_HEAP=ON (per-line prefix detected)
-   samples         : 14,200 over 6h (cadence ~1.5s, log-line synth)
-   free_heap       : min 92,344 / max 124,008 / range 31,664
-   slope           : -82 bytes/min (negative — heap declining)
-   reboots         : none in window
-   OOM events      : none
-   error lines     : 3× "Alloc an err=ESP_ERR_NO_MEM" at +4h12m, +5h08m, +5h44m
-   thread leaks    : (DEBUG_HEAP) MeshPacket -3,124 B over 18 events
-                                  Router       -1,408 B over 4 events
-                                  others       -240 B
-   verdict         : SLOW LEAK — primary suspect MeshPacket thread
-   est. time-to-OOM: ~1,127 min (~18.8 h) at current slope
-   evidence        : (3 log line citations with uptimes)
-   ```
-
-   Then: **what to do next.**
-   - SLOW LEAK, **DEBUG_HEAP off** → recommend rebuilding with the flag and re-running this skill. Concrete one-liner the operator can copy:
-     ```text
-     mcp__meshtastic__build(env="<env>", build_flags={"DEBUG_HEAP": 1})
-     mcp__meshtastic__pio_flash(env="<env>", port="<port>", confirm=True)
-     ```
-     After flash, set debug_log_api back on and wait one window; re-run `/leakhunt`.
-   - SLOW LEAK, **DEBUG_HEAP on** → cite the top-leaking thread name from step 6a. Point at the corresponding source file (`grep -rn "ThreadName(\"<name>\")" src/`); the operator decides what to fix.
-   - FRAGMENTATION SUSPECT → propose pre-allocating any per-packet buffers; or rebuilding with `CONFIG_HEAP_TASK_TRACKING=y` on ESP32 to see who's holding the largest blocks.
-   - OOM-IMMINENT → flag for immediate attention; don't wait for the next telemetry interval.
-   - STEADY → say so; stop. Don't invent problems.
-
-## What NOT to do
-
-- Don't assume a leak from a single dip. LocalStats fires every ~60 s and the firmware naturally allocates+frees on each broadcast cycle; one packet sees the trough. Look at the slope, not the deltas.
-- Don't recommend code changes. This skill diagnoses; the operator decides what to fix.
-- Don't enable `set_debug_log_api` automatically — if it's off, telemetry isn't reaching pubsub anyway, and the recorder will be empty. Tell the operator to flip it on and wait, then re-run.
-- Don't run heavy workloads to "trigger the leak." The recorder is passive; we read what's there.
-
-## Companion: `mark_event` for stress runs
-
-If the operator wants to test under stimulus (e.g. blast 50 broadcasts and see what the heap does), they can frame the experiment with markers:
-
-```text
-mark_event("burst-start")
-… run the workload …
-mark_event("burst-end")
-/leakhunt window=15m
-```
-
-The markers land in both `events.jsonl` and `logs.jsonl`, so the report can show "free_heap dipped 8 KB during the burst window, recovered to baseline within 2 LocalStats cycles" → not a leak.

.claude/commands/repro.md

@@ -3,8 +3,6 @@ description: Re-run a specific test N times in isolation to triage flakes, diff
 argument-hint: <test-node-id> [count=5]
 ---
 
-<!-- markdownlint-disable MD029 -->
-
 # `/repro` — flakiness triage for one test
 
 Re-run a single pytest node ID N times in isolation, track pass rate, and surface what's _different_ in the firmware logs between the passing attempts and the failing ones. Turns "it's flaky, I guess" into "it fails when X, passes when Y."
@@ -42,8 +40,6 @@ Re-run a single pytest node ID N times in isolation, track pass rate, and surfac
 
    Surface the top 3 differences as a "passes when / fails when" table. Don't dump full logs — pull specific lines with uptime timestamps.
 
-5a. **Archive recorder slices per attempt** (no extra device interaction; the recorder runs autouse). Right after each attempt finishes, capture its `(start_ts, end_ts)` and call `mcp__meshtastic__recorder_export(start=<start>, end=<end>, dest_dir="mcp-server/tests/repro_artifacts/<safe-test-id>/attempt_<n>/")`. This drops a `logs.jsonl`, `telemetry.jsonl`, `packets.jsonl`, and `events.jsonl` snapshot scoped to the attempt window. Use these for cross-attempt diffs in step 5: `jq '.line' logs.jsonl` is faster than re-running the test, and the telemetry slice lets you compare heap behavior across attempts.
-
 6. **Classify the flake** into one of:
    - **LoRa airtime collision** → pass rate improves with fewer concurrent transmitters; propose a `time.sleep` gap or retry bump in the test body.
    - **PKI key staleness** → fails on first attempt, passes after self-heal; existing retry loop in `test_direct_with_ack.py` handles this.

.clusterfuzzlite/Dockerfile

@@ -18,7 +18,7 @@ ENV PIP_ROOT_USER_ACTION=ignore
 # trunk-ignore(hadolint/DL3008): apt packages are not pinned.
 # trunk-ignore(terrascan/AC_DOCKER_0002): apt packages are not pinned.
 RUN apt-get update && apt-get install --no-install-recommends -y \
-        cmake git zip libgpiod-dev libjsoncpp-dev libbluetooth-dev libi2c-dev \
+        cmake git zip libgpiod-dev libbluetooth-dev libi2c-dev \
         libunistring-dev libmicrohttpd-dev libgnutls28-dev libgcrypt20-dev \
         libusb-1.0-0-dev libssl-dev pkg-config libsqlite3-dev libsdl2-dev && \
     apt-get clean && rm -rf /var/lib/apt/lists/* && \

.clusterfuzzlite/build.sh

@@ -31,7 +31,7 @@ cmake --install "$WORK/ulfius/$SANITIZER" --prefix /usr
 cd "$SRC/firmware"
 
 PLATFORMIO_EXTRA_SCRIPTS=$(echo -e "pre:.clusterfuzzlite/platformio-clusterfuzzlite-pre.py\npost:.clusterfuzzlite/platformio-clusterfuzzlite-post.py")
-STATIC_LIBS=$(pkg-config --libs --static libulfius openssl libgpiod yaml-cpp jsoncpp bluez --silence-errors)
+STATIC_LIBS=$(pkg-config --libs --static libulfius openssl libgpiod yaml-cpp bluez --silence-errors)
 export PLATFORMIO_EXTRA_SCRIPTS
 export STATIC_LIBS
 export PLATFORMIO_WORKSPACE_DIR="$WORK/pio/$SANITIZER"

.devcontainer/Dockerfile

@@ -16,7 +16,6 @@ RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
     libssl-dev \
     libulfius-dev \
     libyaml-cpp-dev \
-    libjsoncpp-dev \
     pipx \
     pkg-config \
     python3 \

.devcontainer/devcontainer.json

@@ -8,21 +8,17 @@
   "features": {
     "ghcr.io/devcontainers/features/python:1": {
       "installTools": true,
-      "version": "3.13"
+      "version": "3.14"
     }
   },
   "customizations": {
     "vscode": {
       "extensions": [
         "ms-vscode.cpptools",
-        "Jason2866.esp-decoder",
-        "pioarduino.pioarduino-ide",
+        "platformio.platformio-ide",
         "Trunk.io"
       ],
-      "unwantedRecommendations": [
-        "ms-azuretools.vscode-docker",
-        "platformio.platformio-ide"
-      ],
+      "unwantedRecommendations": ["ms-azuretools.vscode-docker"],
       "settings": {
         "extensions.ignoreRecommendations": true
       }

.github/actions/setup-base/action.yml

@@ -13,7 +13,7 @@ runs:
       shell: bash
       run: |
         sudo apt-get -y update --fix-missing
-        sudo apt-get install -y cppcheck libbluetooth-dev libgpiod-dev libyaml-cpp-dev libjsoncpp-dev lsb-release
+        sudo apt-get install -y cppcheck libbluetooth-dev libgpiod-dev libyaml-cpp-dev lsb-release
 
     - name: Setup Python
       uses: actions/setup-python@v6

.github/actions/setup-native/action.yml

@@ -11,4 +11,4 @@ runs:
     - name: Install libs needed for native build
       shell: bash
       run: |
-        sudo apt-get install -y libbluetooth-dev libgpiod-dev libyaml-cpp-dev libjsoncpp-dev openssl libssl-dev libulfius-dev liborcania-dev libusb-1.0-0-dev libi2c-dev libuv1-dev
+        sudo apt-get install -y libbluetooth-dev libgpiod-dev libyaml-cpp-dev openssl libssl-dev libulfius-dev liborcania-dev libusb-1.0-0-dev libi2c-dev libuv1-dev

.github/copilot-instructions.md

@@ -193,26 +193,22 @@ Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which d
 
 Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly.
 
-### Sync flow: thin NodeInfo + post-COMPLETE_ID replay (no opt-in)
+### Gradient sync (opt-in via special nonces)
 
-There is no capability flag and no special "gradient" nonce. The **default** sync flow is:
+`client_capabilities` is **not** a thing in this branch. Phone clients opt into the new sync flow by sending one of two values in the `ToRadio.want_config_id`:
 
-1. Config / module-config / channel / metadata segments (same as before).
-2. `STATE_SEND_OWN_NODEINFO` — **our own** NodeInfo, still bundled with our position and device_metrics (because the replay snapshot excludes our own NodeNum). Emitted via `ConvertToNodeInfo(lite)`.
-3. `STATE_SEND_OTHER_NODEINFOS` — every other peer's NodeInfo, **always thin** (no `position`, no `device_metrics`). Emitted via `ConvertToNodeInfoThin(lite)`.
-4. `STATE_SEND_FILEMANIFEST` → `STATE_SEND_COMPLETE_ID` — the phone sees `config_complete_id` and treats sync as done.
-5. `STATE_SEND_PACKETS` — live mesh packets, with a trailing replay drain interleaved. The replay drain walks four cached satellite stores in order (positions → telemetry → environment → status) and emits each cached entry as an ordinary `MeshPacket` on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` device + environment variants, `NODE_STATUS_APP`). These are indistinguishable on the wire from live mesh traffic, so clients need no special handling — any code that already updates UI on `POSITION_APP` etc. works.
+- `SPECIAL_NONCE_GRADIENT_SYNC` (69422) — full config + thin NodeInfo + replay phases.
+- `SPECIAL_NONCE_GRADIENT_ONLY_NODES` (69423) — skip config segments, NodeInfo + replay only.
 
-`PhoneAPI::sendConfigComplete()` arms `replayPhase = REPLAY_PHASE_POSITIONS` for default/full sync and `SPECIAL_NONCE_ONLY_NODES`, while `SPECIAL_NONCE_ONLY_CONFIG` skips replay. The drain runs inside `STATE_SEND_PACKETS` via `popReplayPacket()`, lower priority than live traffic. When all four phases drain, `replayPhase` flips back to `REPLAY_PHASE_IDLE` and the snapshot vectors get `shrink_to_fit`ed.
+`PhoneAPI::clientWantsGradientSync()` is the single switch. When true, `STATE_SEND_OTHER_NODEINFOS` is followed by:
 
-STM32WL and any other build with all four `MESHTASTIC_EXCLUDE_*DB` flags set produces zero replay packets — `popReplayPacket` advances through each phase in microseconds without emitting anything.
+```text
+STATE_REPLAY_POSITIONS → STATE_REPLAY_TELEMETRY → STATE_REPLAY_ENVIRONMENT → STATE_REPLAY_STATUS
+```
 
-Special nonces that still mean something:
+Each replay phase walks the corresponding satellite map and emits synthetic `MeshPacket`s on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` for both device + environment variants, `STATUS_MESSAGE_APP`). Legacy clients (no special nonce) get the bundled-NodeInfo path with position/device_metrics joined back in by `ConvertToNodeInfo(lite, pos*, dm*)` — wire bytes are byte-identical to pre-v25 for them.
 
-- `SPECIAL_NONCE_ONLY_CONFIG` (69420) — skip node sync entirely, just config.
-- `SPECIAL_NONCE_ONLY_NODES` (69421) — skip config segments, jump straight to `STATE_SEND_OWN_NODEINFO`. Still gets the post-COMPLETE_ID replay drain.
-
-There are no other reserved nonces; everything else is a fresh random `want_config_id` from the client.
+`ConvertToNodeInfoThin(lite)` is the gradient-sync emitter (no position/telemetry).
 
 ### v24 → v25 migration
 
@@ -289,8 +285,6 @@ firmware/
 - Prefer `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR` for logging
 - Use `assert()` for invariants that should never fail
 - C++17 features are available (`std::optional`, structured bindings, `if constexpr`, etc.)
-- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
-- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
 
 ### Naming Conventions
 
@@ -376,14 +370,14 @@ Multiple display driver families in `src/graphics/`:
 
 - **OLED**: SSD1306, SH1106, ST7567
 - **TFT**: TFTDisplay (LovyanGFX-based)
-- **E-Ink**: `src/graphics/BaseUIEInkDisplay.*` is the OLEDDisplay-compatible adapter (peer of `TFTDisplay`). The hardware layer it drives lives in `src/graphics/eink/` — chipset drivers in `Drivers/`, panel profiles in `Panels/`, optional `Backlight/`. Shared by both BaseUI-on-eink and InkHUD builds.
+- **E-Ink**: EInkDisplay2, EInkDynamicDisplay, EInkParallelDisplay
 
-**InkHUD** (`src/graphics/niche/`) is an event-driven e-ink UI framework that sits on top of the `graphics/eink/` layer:
+**InkHUD** (`src/graphics/niche/InkHUD/`) is an event-driven e-ink UI framework:
 
 - Applet-based architecture — modular display tiles
 - Read-only, static display optimized for minimal refreshes and low power
 - Configured per-variant via `nicheGraphics.h`
-- Build helpers in top-level `platformio.ini` — `[niche]` pulls `graphics/eink/` only (BaseUI path), `[inkhud]` extends it with `graphics/niche/`
+- Separate PlatformIO config: `src/graphics/niche/InkHUD/PlatformioConfig.ini`
 
 ### Input System
 
@@ -609,35 +603,20 @@ Most workflows can be triggered manually via `workflow_dispatch` for testing.
 
 ### Native unit tests (C++)
 
-Unit tests in `test/` directory with 17 test suites:
+Unit tests in `test/` directory with 12 test suites:
 
-- `test_admin_radio/` - LoRa region/config validation and AdminModule dispatch
-- `test_atak/` - ATAK integration
 - `test_crypto/` - Cryptography
-- `test_default/` - Default configuration
-- `test_http_content_handler/` - HTTP handling
-- `test_mac_from_string/` - MAC address parsing
+- `test_mqtt/` - MQTT integration
+- `test_radio/` - Radio interface
 - `test_mesh_module/` - Module framework
 - `test_meshpacket_serializer/` - Packet serialization
-- `test_mqtt/` - MQTT integration
-- `test_packet_history/` - Packet history tracking
-- `test_position_precision/` - Position precision helpers
-- `test_radio/` - Radio interface
-- `test_serial/` - Serial communication
-- `test_traffic_management/` - Traffic management
 - `test_transmit_history/` - Retransmission tracking
-- `test_type_conversions/` - NodeDB v25 type conversion (bitfield round-trips, NodeInfoLite)
-- `test_utf8/` - UTF-8 utilities
+- `test_atak/` - ATAK integration
+- `test_default/` - Default configuration
+- `test_http_content_handler/` - HTTP handling
+- `test_serial/` - Serial communication
 
-Run command (preferred — avoids pipe-buffering and the Ubuntu externally-managed-environment error):
-
-```bash
-~/.platformio/penv/bin/python -m platformio test -e native -f test_your_suite > /tmp/test_out.txt 2>&1
-grep -E 'error:|PASS|FAIL|succeeded|failed' /tmp/test_out.txt
-tail -15 /tmp/test_out.txt
-```
-
-Do **not** use `pio test … | tail -N` — it discards build errors and shows stale cached results. Do **not** use `pio test … | grep` — line-buffering makes the terminal appear hung until the process exits. Redirect to a file first, then grep.
+Run with: `pio test -e native`
 
 Simulation testing: `bin/test-simulator.sh`
 

.github/prompts/new-variant.prompt.md

@@ -132,7 +132,7 @@ For e-ink display variants using the InkHUD framework, add `nicheGraphics.h`:
 // Configure display, applets, and refresh behavior per device
 ```
 
-InkHUD and the shared E-Ink layer are wired up via the top-level `platformio.ini` `[niche]` (BaseUI + driver/panel layer in `src/graphics/eink/`) and `[inkhud]` (adds the InkHUD UI in `src/graphics/niche/`). Variants opt in with `extends = ..., niche` or `extends = ..., inkhud`.
+InkHUD has its own PlatformIO config: `src/graphics/niche/InkHUD/PlatformioConfig.ini`
 
 ## I2C Device Detection
 

.github/workflows/firmware-size-comment.yml

@@ -1,62 +0,0 @@
-name: Post Firmware Size Comment
-
-on:
-  workflow_run:
-    workflows: [CI]
-    types: [completed]
-
-permissions:
-  pull-requests: write
-  actions: read
-
-jobs:
-  post-size-comment:
-    if: >
-      github.event.workflow_run.event == 'pull_request' &&
-      github.event.workflow_run.conclusion != 'cancelled' &&
-      github.repository == 'meshtastic/firmware'
-    continue-on-error: true
-    runs-on: ubuntu-latest
-    steps:
-      - name: Download size report
-        id: download
-        uses: actions/download-artifact@v8
-        continue-on-error: true
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          run-id: ${{ github.event.workflow_run.id }}
-          name: size-report
-          path: ./
-
-      - name: Post or update PR comment
-        if: steps.download.outcome == 'success'
-        uses: actions/github-script@v8
-        with:
-          script: |
-            const fs = require('fs');
-            const marker = '<!-- firmware-size-report -->';
-            const body = fs.readFileSync('./size-report.md', 'utf8');
-            const prNumber = parseInt(fs.readFileSync('./pr-number.txt', 'utf8').trim(), 10);
-
-            const { data: comments } = await github.rest.issues.listComments({
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-              issue_number: prNumber,
-            });
-
-            const existing = comments.find(c => c.body.includes(marker));
-            if (existing) {
-              await github.rest.issues.updateComment({
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                comment_id: existing.id,
-                body,
-              });
-            } else {
-              await github.rest.issues.createComment({
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                issue_number: prNumber,
-                body,
-              });
-            }

.github/workflows/main_matrix.yml

@@ -245,126 +245,47 @@ jobs:
           path: ./*.elf
           retention-days: 30
 
-  firmware-size-report:
+  shame:
     if: github.repository == 'meshtastic/firmware'
     continue-on-error: true
-    permissions:
-      contents: read
-      actions: read
     runs-on: ubuntu-latest
     needs: [build]
     steps:
       - uses: actions/checkout@v6
-
-      - name: Download current manifests
+        if: github.event_name == 'pull_request'
+        with:
+          filter: blob:none # means we download all the git history but none of the commit (except ones with checkout like the head)
+          fetch-depth: 0
+      - name: Download the current manifests
         uses: actions/download-artifact@v8
         with:
-          path: ./manifests/
+          path: ./manifests-new/
           pattern: manifest-*
           merge-multiple: true
-
-      - name: Collect current firmware sizes
-        run: python3 bin/collect_sizes.py ./manifests/ ./current-sizes.json
-
-      - name: Upload size report artifact
+      - name: Upload combined manifests for later commit and global stats crunching.
         uses: actions/upload-artifact@v7
+        id: upload-manifest
         with:
-          name: firmware-sizes-${{ github.sha }}
+          name: manifests-${{ github.sha }}
           overwrite: true
-          path: ./current-sizes.json
-          retention-days: 90
-
-      - name: Download baseline sizes from develop
+          path: manifests-new/*.mt.json
+      - name: Find the merge base
         if: github.event_name == 'pull_request'
-        continue-on-error: true
-        id: baseline-develop
+        run: echo "MERGE_BASE=$(git merge-base "origin/$base" "$head")" >> $GITHUB_ENV
         env:
-          GH_TOKEN: ${{ github.token }}
-        run: |
-          RUN_ID=$(gh run list -R "${{ github.repository }}" \
-            --workflow CI --branch develop --status success \
-            --limit 1 --json databaseId --jq '.[0].databaseId // empty')
-          if [ -n "$RUN_ID" ]; then
-            ARTIFACT_NAME=$(gh api "repos/${{ github.repository }}/actions/runs/${RUN_ID}/artifacts" \
-              --jq '.artifacts[] | select(.name | startswith("firmware-sizes-")) | .name' | head -1)
-            if [ -n "$ARTIFACT_NAME" ]; then
-              gh run download "$RUN_ID" -R "${{ github.repository }}" \
-                --name "$ARTIFACT_NAME" --dir ./baseline-develop/
-              cp "./baseline-develop/${ARTIFACT_NAME}/current-sizes.json" ./develop-sizes.json
-              echo "found=true" >> "$GITHUB_OUTPUT"
-            else
-              echo "found=false" >> "$GITHUB_OUTPUT"
-            fi
-          else
-            echo "found=false" >> "$GITHUB_OUTPUT"
-          fi
-
-      - name: Download baseline sizes from master
-        if: github.event_name == 'pull_request'
-        continue-on-error: true
-        id: baseline-master
-        env:
-          GH_TOKEN: ${{ github.token }}
-        run: |
-          RUN_ID=$(gh run list -R "${{ github.repository }}" \
-            --workflow CI --branch master --status success \
-            --limit 1 --json databaseId --jq '.[0].databaseId // empty')
-          if [ -n "$RUN_ID" ]; then
-            ARTIFACT_NAME=$(gh api "repos/${{ github.repository }}/actions/runs/${RUN_ID}/artifacts" \
-              --jq '.artifacts[] | select(.name | startswith("firmware-sizes-")) | .name' | head -1)
-            if [ -n "$ARTIFACT_NAME" ]; then
-              gh run download "$RUN_ID" -R "${{ github.repository }}" \
-                --name "$ARTIFACT_NAME" --dir ./baseline-master/
-              cp "./baseline-master/${ARTIFACT_NAME}/current-sizes.json" ./master-sizes.json
-              echo "found=true" >> "$GITHUB_OUTPUT"
-            else
-              echo "found=false" >> "$GITHUB_OUTPUT"
-            fi
-          else
-            echo "found=false" >> "$GITHUB_OUTPUT"
-          fi
-
-      - name: Generate size comparison report
-        if: github.event_name == 'pull_request'
-        id: report
-        run: |
-          ARGS="./current-sizes.json"
-          if [ -f ./develop-sizes.json ]; then
-            ARGS="$ARGS --baseline develop:./develop-sizes.json"
-          fi
-          if [ -f ./master-sizes.json ]; then
-            ARGS="$ARGS --baseline master:./master-sizes.json"
-          fi
-          REPORT=$(python3 bin/size_report.py $ARGS)
-          if [ -z "$REPORT" ]; then
-            echo "has_report=false" >> "$GITHUB_OUTPUT"
-          else
-            echo "has_report=true" >> "$GITHUB_OUTPUT"
-            {
-              echo '<!-- firmware-size-report -->'
-              echo '# Firmware Size Report'
-              echo ''
-              echo "$REPORT"
-              echo ''
-              echo '---'
-              echo "*Updated for ${{ github.sha }}*"
-            } > ./size-report.md
-            cat ./size-report.md >> "$GITHUB_STEP_SUMMARY"
-          fi
-
-      - name: Save PR number
-        if: github.event_name == 'pull_request' && steps.report.outputs.has_report == 'true'
-        run: echo "${{ github.event.pull_request.number }}" > ./pr-number.txt
-
-      - name: Upload size report
-        if: github.event_name == 'pull_request' && steps.report.outputs.has_report == 'true'
-        uses: actions/upload-artifact@v7
-        with:
-          name: size-report
-          path: |
-            ./size-report.md
-            ./pr-number.txt
-          retention-days: 5
+          base: ${{ github.base_ref }}
+          head: ${{ github.sha }}
+      # Currently broken (for-loop through EVERY artifact -- rate limiting)
+      # - name: Download the old manifests
+      #   if: github.event_name == 'pull_request'
+      #   run: gh run download -R "$repo" --name "manifests-$merge_base" --dir manifest-old/
+      #   env:
+      #     GH_TOKEN: ${{ github.token }}
+      #     merge_base: ${{ env.MERGE_BASE }}
+      #     repo: ${{ github.repository }}
+      # - name: Do scan and post comment
+      #   if: github.event_name == 'pull_request'
+      #   run: python3 bin/shame.py ${{ github.event.pull_request.number }} manifests-old/ manifests-new/
 
   release-artifacts:
     permissions: # Needed for 'gh release upload'.
@@ -412,7 +333,6 @@ jobs:
           prerelease: true
           name: Meshtastic Firmware ${{ needs.version.outputs.long }} Alpha
           tag_name: v${{ needs.version.outputs.long }}
-          target_commitish: ${{ github.sha }}
           body: ${{ steps.release_notes.outputs.notes }}
 
       - name: Download source deb

.gitignore

@@ -47,10 +47,6 @@ data/boot/logo.*
 managed_components/*
 arduino-lib-builder*
 dependencies.lock
-
-# JLink / RTT debug artifacts (nRF SoCs)
-flash.jlink
-rtt_*.txt
 idf_component.yml
 CMakeLists.txt
 /sdkconfig.*
@@ -60,9 +56,3 @@ CMakeLists.txt
 .python3
 .claude/scheduled_tasks.lock
 userPrefs.jsonc.mcp-session-bak
-
-# Fake-NodeDB fixture pipeline (bin/regen-fake-nodedbs.sh)
-# JSONL seeds are committed (test/fixtures/nodedb/seed_v25_*.jsonl);
-# compiled .proto outputs are ephemeral build artifacts.
-build/fixtures/
-bin/_generated/

.trunk/trunk.yaml

@@ -4,19 +4,19 @@ cli:
 plugins:
   sources:
     - id: trunk
-      ref: v1.10.0
+      ref: v1.8.0
       uri: https://github.com/trunk-io/plugins
 lint:
   enabled:
-    - checkov@3.2.529
+    - checkov@3.2.526
     - renovate@43.150.0
     - prettier@3.8.3
-    - trufflehog@3.95.3
+    - trufflehog@3.95.2
     - yamllint@1.38.0
     - bandit@1.9.4
     - trivy@0.70.0
     - taplo@0.10.0
-    - ruff@0.15.13
+    - ruff@0.15.12
     - isort@8.0.1
     - markdownlint@0.48.0
     - oxipng@10.1.1
@@ -26,7 +26,7 @@ lint:
     - hadolint@2.14.0
     - shfmt@3.6.0
     - shellcheck@0.11.0
-    - black@26.5.1
+    - black@26.3.1
     - git-diff-check
     - gitleaks@8.30.1
     - clang-format@16.0.3
@@ -34,13 +34,6 @@ lint:
     - linters: [ALL]
       paths:
         - bin/**
-    # Fake-NodeDB fixture JSONL files contain deterministic synthetic
-    # public_key_hex (64-char hex) values that gitleaks misidentifies as
-    # generic-api-key. These are not secrets — they're test fixtures
-    # produced by bin/gen-fake-nodedb-seed.py with a fixed RNG seed.
-    - linters: [gitleaks]
-      paths:
-        - test/fixtures/nodedb/seed_v25_*.jsonl
 runtimes:
   enabled:
     - python@3.14.4

.vscode/extensions.json

@@ -1,10 +1,10 @@
 {
+    // See http://go.microsoft.com/fwlink/?LinkId=827846
+    // for the documentation about the extensions.json format
     "recommendations": [
-        "Jason2866.esp-decoder",
-        "pioarduino.pioarduino-ide"
+        "platformio.platformio-ide"
     ],
     "unwantedRecommendations": [
-        "ms-vscode.cpptools-extension-pack",
-        "platformio.platformio-ide"
+        "ms-vscode.cpptools-extension-pack"
     ]
 }

AGENTS.md

@@ -10,18 +10,18 @@ This file (`AGENTS.md`) is a short pointer + quick reference for agents that don
 
 ## Quick command reference
 
-| Action                           | Command                                                                                                                                                                                                        |
-| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Build a firmware variant         | `pio run -e <env>` (e.g. `pio run -e rak4631`, `pio run -e heltec-v3`)                                                                                                                                         |
-| Build native macOS host binary   | `pio run -e native-macos` (Homebrew prereqs + CH341 LoRa setup in `variants/native/portduino/platformio.ini`)                                                                                                  |
-| Clean + rebuild                  | `pio run -e <env> -t clean && pio run -e <env>`                                                                                                                                                                |
-| Flash a device                   | `pio run -e <env> -t upload --upload-port <port>` (or use the `pio_flash` MCP tool)                                                                                                                            |
-| Run firmware unit tests (native) | `~/.platformio/penv/bin/python -m platformio test -e native > /tmp/test_out.txt 2>&1` then `grep -E 'error:\|PASS\|FAIL\|succeeded\|failed' /tmp/test_out.txt` (redirect first — piping causes line-buffering) |
-| Run MCP hardware tests           | `./mcp-server/run-tests.sh`                                                                                                                                                                                    |
-| Live TUI test runner             | `mcp-server/.venv/bin/meshtastic-mcp-test-tui`                                                                                                                                                                 |
-| Format before commit             | `trunk fmt`                                                                                                                                                                                                    |
-| Regenerate protobuf bindings     | `bin/regen-protos.sh`                                                                                                                                                                                          |
-| Generate CI matrix               | `./bin/generate_ci_matrix.py all [--level pr]`                                                                                                                                                                 |
+| Action                           | Command                                                                                                       |
+| -------------------------------- | ------------------------------------------------------------------------------------------------------------- |
+| Build a firmware variant         | `pio run -e <env>` (e.g. `pio run -e rak4631`, `pio run -e heltec-v3`)                                        |
+| Build native macOS host binary   | `pio run -e native-macos` (Homebrew prereqs + CH341 LoRa setup in `variants/native/portduino/platformio.ini`) |
+| Clean + rebuild                  | `pio run -e <env> -t clean && pio run -e <env>`                                                               |
+| Flash a device                   | `pio run -e <env> -t upload --upload-port <port>` (or use the `pio_flash` MCP tool)                           |
+| Run firmware unit tests (native) | `pio test -e native`                                                                                          |
+| Run MCP hardware tests           | `./mcp-server/run-tests.sh`                                                                                   |
+| Live TUI test runner             | `mcp-server/.venv/bin/meshtastic-mcp-test-tui`                                                                |
+| Format before commit             | `trunk fmt`                                                                                                   |
+| Regenerate protobuf bindings     | `bin/regen-protos.sh`                                                                                         |
+| Generate CI matrix               | `./bin/generate_ci_matrix.py all [--level pr]`                                                                |
 
 ## MCP server (device + test automation)
 
@@ -66,8 +66,6 @@ Key rotation to never trigger casually: only the **full** factory reset (`factor
 - **Don't speculate about firmware root causes.** When evidence doesn't support a classification, say "unknown" and list what would disambiguate.
 - **Run `trunk fmt` before proposing a commit.** The `trunk_check` CI gate will reject unformatted code.
 - **`confirm=True` on destructive MCP tools is a real gate, not a formality.** Don't bypass it via auto-approve settings.
-- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
-- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
 
 ## Typical agent workflows
 
@@ -108,7 +106,7 @@ Sequence these; don't parallelize on the same port.
 | `src/modules/`                    | Feature modules; `Telemetry/Sensor/` has 50+ I2C sensor drivers                                                          |
 | `variants/`                       | 200+ hardware variant definitions (`variant.h` + `platformio.ini` per board)                                             |
 | `protobufs/`                      | `.proto` definitions; regenerate with `bin/regen-protos.sh`                                                              |
-| `test/`                           | Firmware unit tests (17 suites; `pio test -e native`)                                                                    |
+| `test/`                           | Firmware unit tests (12 suites; `pio test -e native`)                                                                    |
 | `mcp-server/`                     | Python MCP server + pytest hardware integration tests                                                                    |
 | `mcp-server/tests/`               | Tiered pytest suite: `unit/`, `mesh/`, `telemetry/`, `monitor/`, `recovery/`, `ui/`, `fleet/`, `admin/`, `provisioning/` |
 | `.claude/commands/`               | Claude Code slash command bodies                                                                                         |

Dockerfile

@@ -14,7 +14,7 @@ ENV PIP_BREAK_SYSTEM_PACKAGES=1
 RUN apt-get update && apt-get install --no-install-recommends -y \
         curl wget g++ zip git ca-certificates pkg-config \
         python3-pip python3-grpc-tools \
-        libgpiod-dev libyaml-cpp-dev libjsoncpp-dev libbluetooth-dev libi2c-dev libuv1-dev \
+        libgpiod-dev libyaml-cpp-dev libbluetooth-dev libi2c-dev libuv1-dev \
         libusb-1.0-0-dev libulfius-dev liborcania-dev libssl-dev \
         libx11-dev libinput-dev libxkbcommon-x11-dev libsqlite3-dev libsdl2-dev \
     && apt-get clean && rm -rf /var/lib/apt/lists/* \
@@ -53,7 +53,7 @@ ENV TZ=Etc/UTC
 USER root
 
 RUN apt-get update && apt-get --no-install-recommends -y install \
-        libc-bin libc6 libgpiod3 libyaml-cpp0.8 libjsoncpp26 libi2c0 libuv1t64 libusb-1.0-0-dev \
+        libc-bin libc6 libgpiod3 libyaml-cpp0.8 libi2c0 libuv1t64 libusb-1.0-0-dev \
         liborcania2.3 libulfius2.7t64 libssl3t64 \
         libx11-6 libinput10 libxkbcommon-x11-0 libsdl2-2.0-0 \
     && apt-get clean && rm -rf /var/lib/apt/lists/* \

Dockerfile.test

@@ -7,7 +7,7 @@ ENV PIP_ROOT_USER_ACTION=ignore
 # hadolint ignore=DL3008
 RUN apt-get update && apt-get install --no-install-recommends -y \
         g++ git ca-certificates pkg-config \
-        libgpiod-dev libyaml-cpp-dev libjsoncpp-dev libbluetooth-dev libi2c-dev libuv1-dev \
+        libgpiod-dev libyaml-cpp-dev libbluetooth-dev libi2c-dev libuv1-dev \
         libusb-1.0-0-dev libulfius-dev liborcania-dev libssl-dev \
         libx11-dev libinput-dev libxkbcommon-x11-dev libsqlite3-dev libsdl2-dev \
     && apt-get clean && rm -rf /var/lib/apt/lists/* \

alpine.Dockerfile

@@ -16,7 +16,7 @@ ENV PIP_BREAK_SYSTEM_PACKAGES=1
 RUN apk --no-cache add \
         bash g++ libstdc++-dev linux-headers zip git ca-certificates libbsd-dev \
         py3-pip py3-grpcio-tools \
-        libgpiod-dev yaml-cpp-dev jsoncpp-dev bluez-dev \
+        libgpiod-dev yaml-cpp-dev bluez-dev \
         libusb-dev i2c-tools-dev libuv-dev openssl-dev pkgconf argp-standalone \
         libx11-dev libinput-dev libxkbcommon-dev sqlite-dev sdl2-dev \
     && rm -rf /var/cache/apk/* \
@@ -48,7 +48,7 @@ LABEL org.opencontainers.image.title="Meshtastic" \
 USER root
 
 RUN apk --no-cache add \
-        shadow libstdc++ libbsd libgpiod yaml-cpp jsoncpp libusb \
+        shadow libstdc++ libbsd libgpiod yaml-cpp libusb \
         i2c-tools libuv libx11 libinput libxkbcommon sdl2 \
     && rm -rf /var/cache/apk/* \
     && mkdir -p /var/lib/meshtasticd \

bin/_rewrite_proto_namespace.py

@@ -1,64 +0,0 @@
-#!/usr/bin/env python3
-"""Post-process protoc-generated Python files to live under a local namespace.
-
-Called by bin/regen-py-protos.sh. Walks the generated *_pb2.py files in the
-target directory and rewrites every `meshtastic` reference (imports, dotted
-attribute access) to use the new namespace (e.g., `meshtastic_v25`).
-
-Why: the .proto files declare `package meshtastic;`, so protoc emits
-`from meshtastic import mesh_pb2 as ...` lines. That would shadow the PyPI
-`meshtastic` package which other parts of the mcp-server depend on. Renaming
-to a local namespace keeps both available.
-
-Usage:
-    _rewrite_proto_namespace.py <generated_dir> <new_namespace>
-"""
-
-from __future__ import annotations
-
-import pathlib
-import re
-import sys
-
-
-def rewrite(dir_path: pathlib.Path, new_ns: str) -> int:
-    # Standard protoc import forms:
-    #   from meshtastic.X_pb2 import ...     (rare, for direct symbol pulls)
-    #   from meshtastic import X_pb2 as ...  (common, the cross-file ref)
-    #   import meshtastic.X_pb2              (also possible)
-    pattern_dotted_from = re.compile(r"^from meshtastic\.", re.MULTILINE)
-    pattern_bare_from = re.compile(r"^from meshtastic import ", re.MULTILINE)
-    pattern_dotted_import = re.compile(r"^import meshtastic\.", re.MULTILINE)
-
-    count = 0
-    for p in dir_path.glob("*.py"):
-        text = p.read_text(encoding="utf-8")
-        new = pattern_dotted_from.sub(f"from {new_ns}.", text)
-        new = pattern_bare_from.sub(f"from {new_ns} import ", new)
-        new = pattern_dotted_import.sub(f"import {new_ns}.", new)
-        # NOTE: we deliberately leave `meshtastic/X.proto` source-filename
-        # references inside descriptor strings alone. The descriptor pool is
-        # keyed by source filename (independent of Python package layout), so
-        # those don't collide with the PyPI package's descriptors.
-        if new != text:
-            p.write_text(new, encoding="utf-8")
-            count += 1
-    return count
-
-
-def main(argv: list[str]) -> int:
-    if len(argv) != 2:
-        print("usage: _rewrite_proto_namespace.py <generated_dir> <new_namespace>", file=sys.stderr)
-        return 2
-    dir_path = pathlib.Path(argv[0])
-    new_ns = argv[1]
-    if not dir_path.is_dir():
-        print(f"directory not found: {dir_path}", file=sys.stderr)
-        return 2
-    n = rewrite(dir_path, new_ns)
-    print(f"rewrote {n} file(s) in {dir_path} → namespace {new_ns}", file=sys.stderr)
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main(sys.argv[1:]))

bin/build-esp32.sh

@@ -38,4 +38,4 @@ cp bin/device-install.* $OUTDIR/
 cp bin/device-update.* $OUTDIR/
 
 echo "Copying manifest"
-cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json
+cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json || true

bin/collect_sizes.py

@@ -1,53 +0,0 @@
-#!/usr/bin/env python3
-
-"""Collect firmware binary sizes from manifest (.mt.json) files into a single report."""
-
-import json
-import os
-import sys
-
-
-def collect_sizes(manifest_dir):
-    """Scan manifest_dir for .mt.json files and return {board: size_bytes} dict."""
-    sizes = {}
-    for fname in sorted(os.listdir(manifest_dir)):
-        if not fname.endswith(".mt.json"):
-            continue
-        path = os.path.join(manifest_dir, fname)
-        with open(path) as f:
-            data = json.load(f)
-        board = data.get("platformioTarget", fname.replace(".mt.json", ""))
-        # Find the main firmware .bin size (largest .bin, excluding OTA/littlefs/bleota)
-        bin_size = None
-        for entry in data.get("files", []):
-            name = entry.get("name", "")
-            if name.startswith("firmware-") and name.endswith(".bin"):
-                bin_size = entry["bytes"]
-                break
-        # Fallback: any .bin that isn't ota/littlefs/bleota
-        if bin_size is None:
-            for entry in data.get("files", []):
-                name = entry.get("name", "")
-                if name.endswith(".bin") and not any(
-                    x in name for x in ["littlefs", "bleota", "ota"]
-                ):
-                    bin_size = entry["bytes"]
-                    break
-        if bin_size is not None:
-            sizes[board] = bin_size
-    return sizes
-
-
-if __name__ == "__main__":
-    if len(sys.argv) != 3:
-        print(f"Usage: {sys.argv[0]} <manifest_dir> <output.json>", file=sys.stderr)
-        sys.exit(1)
-
-    manifest_dir = sys.argv[1]
-    output_path = sys.argv[2]
-
-    sizes = collect_sizes(manifest_dir)
-    with open(output_path, "w") as f:
-        json.dump(sizes, f, indent=2, sort_keys=True)
-
-    print(f"Collected sizes for {len(sizes)} targets -> {output_path}")

bin/config.d/lora-RAK6421-13300-slot2.yaml

@@ -5,9 +5,7 @@ Meta:
     - raspberry-pi
 
 Lora:
-  
-  ### RAK13300 in Slot 2
-  Module: sx1262
+  ### RAK13300 in Slot 2 pins
   IRQ: 18 #IO6
   Reset: 24 # IO4
   Busy: 19 # IO5
@@ -15,7 +13,5 @@ Lora:
   Enable_Pins:
     - 26
     - 23
-  DIO3_TCXO_VOLTAGE: true
-  DIO2_AS_RF_SWITCH: true
   spidev: spidev0.1
   # CS: 7

bin/config.d/lora-RAK6421-13302-slot2.yaml

@@ -5,18 +5,14 @@ Meta:
     - raspberry-pi
 
 Lora:
-
-  ### RAK13302 in Slot 2
-  Module: sx1262
+  ### RAK13302 in Slot 2 pins
   IRQ: 18 #IO6
   Reset: 24 # IO4
   Busy: 19 # IO5
-  # Ant_sw: 23 # IO3
+  # Ant_sw: 23 # IO3 
   Enable_Pins:
     - 26
     - 23
-  DIO3_TCXO_VOLTAGE: true
-  DIO2_AS_RF_SWITCH: true
   spidev: spidev0.1
   # CS: 7
   TX_GAIN_LORA: [9, 9, 10, 11, 9, 8, 9, 10, 10, 10, 11, 12, 12, 12, 12, 12, 12, 12, 12, 10, 9, 8]

bin/config.d/lora-station-g3.yaml

@@ -1,18 +0,0 @@
-# Station G3 motherboard with a Raspberry Pi Zero 2W as the MCU daughterboard.
-# Verify spidev / I2C device paths for your OS — they may differ.
-Meta:
-  name: Station G3
-  support: community
-  compatible:
-    - raspberry-pi
-
-Lora:
-  Module: sx1262
-  IRQ: 22                       # BCM pin — wiki spec
-  Reset: 16                     # BCM pin — wiki spec
-  Busy: 24                      # BCM pin — wiki spec
-  # CS: 8                       # BCM 8 = SPI0 CE0 (default); uncomment only to override
-  DIO2_AS_RF_SWITCH: true
-  DIO3_TCXO_VOLTAGE: true
-  spidev: spidev0.0
-  # SX126X_MAX_POWER: 19        # matches Station G2 firmware cap; raise carefully per PA jumper mode

bin/gen-fake-nodedb-seed.py

@@ -1,439 +0,0 @@
-#!/usr/bin/env python3
-"""Deterministic seed-data generator for the fake NodeDB fixture pipeline.
-
-Writes a JSONL file describing N fake-but-realistic Meshtastic peers.
-The output is hand-editable and committed; a sibling compile step
-(bin/seed-json-to-proto.py) turns it into a binary `meshtastic_NodeDatabase`
-v25 protobuf with fresh "now-relative" timestamps.
-
-Determinism contract:
-  Same --seed -> byte-identical JSONL output, regardless of wall clock.
-  All timestamps are stored as `*_offset_sec` (seconds before "now"); the
-  compile step resolves them to absolute epochs at compile time.
-
-Structural fields covered:
-  * NodeInfoLite header: num, long_name, short_name, hw_model, role,
-    public_key, snr, channel, hops_away, next_hop, bitfield flags
-  * PositionLite: lat/long Gaussian around --centroid, altitude, source
-  * DeviceMetrics: battery/voltage/util/uptime
-  * EnvironmentMetrics: temp/humidity/pressure/iaq
-  * StatusMessage: error_code (usually zero)
-
-Active-board allow-list:
-  hw_model values are restricted to the intersection of
-    (a) variants with `custom_meshtastic_support_level = 1` in
-        variants/*/*/platformio.ini, AND
-    (b) values present in the `HardwareModel` enum in mesh.proto.
-  See HW_MODEL_WEIGHTS below. Deprecated boards (legacy TLORA / Heltec V1-2 /
-  classic TBEAM / TBEAM_V0P7 / Nano G1 / etc.) and fuzzer-only sentinels
-  (PORTDUINO, ANDROID_SIM, DIY_V1, ...) are excluded.
-
-Active-role allow-list:
-  Excludes ROUTER_CLIENT (deprecated v2.3.15) and REPEATER (deprecated v2.7.11).
-"""
-
-from __future__ import annotations
-
-import argparse
-import datetime as _dt
-import json
-import math
-import pathlib
-import random
-import sys
-
-# --------------------------------------------------------------------------
-# Active-board allow-list (intersection of tier-1 variants + HardwareModel enum).
-# Refresh by running:
-#   for f in $(find variants -name 'platformio.ini' | xargs grep -lE 'custom_meshtastic_support_level = 1'); do
-#       grep custom_meshtastic_hw_model_slug $f | awk -F= '{print $2}' | tr -d ' ';
-#   done | sort -u | comm -12 - <(python3 -c "from meshtastic.protobuf.mesh_pb2 import HardwareModel; print('\\n'.join(HardwareModel.keys()))" | sort)
-# --------------------------------------------------------------------------
-HW_MODEL_WEIGHTS: dict[str, float] = {
-    "HELTEC_V3": 14.0,
-    "T_DECK": 9.0,
-    "HELTEC_V4": 8.0,
-    "RAK4631": 8.0,
-    "HELTEC_MESH_POCKET": 6.0,
-    "TRACKER_T1000_E": 5.0,
-    "HELTEC_MESH_NODE_T114": 5.0,
-    "T_DECK_PRO": 5.0,
-    "LILYGO_TBEAM_S3_CORE": 4.0,
-    "HELTEC_WIRELESS_PAPER": 4.0,
-    "HELTEC_WSL_V3": 3.0,
-    "T_ECHO": 3.0,
-    "HELTEC_WIRELESS_TRACKER": 3.0,
-    "HELTEC_WIRELESS_TRACKER_V2": 2.0,
-    "HELTEC_VISION_MASTER_E290": 2.0,
-    "HELTEC_MESH_SOLAR": 2.0,
-    "SEEED_WIO_TRACKER_L1": 2.0,
-    "T_LORA_PAGER": 1.5,
-    "HELTEC_VISION_MASTER_E213": 1.5,
-    "T_ECHO_PLUS": 1.0,
-    "MUZI_BASE": 1.0,
-    "WISMESH_TAP_V2": 1.0,
-    "THINKNODE_M2": 1.0,
-    "THINKNODE_M5": 1.0,
-    "TLORA_T3_S3": 1.0,
-    # Long tail (uniform low weight across remaining tier-1 boards):
-    "HELTEC_V4_R8": 0.3,
-    "HELTEC_VISION_MASTER_T190": 0.3,
-    "HELTEC_HT62": 0.3,
-    "HELTEC_MESH_NODE_T096": 0.3,
-    "M5STACK_C6L": 0.3,
-    "MINI_EPAPER_S3": 0.3,
-    "MUZI_R1_NEO": 0.3,
-    "NOMADSTAR_METEOR_PRO": 0.3,
-    "RAK3312": 0.3,
-    "RAK3401": 0.3,
-    "SEEED_SOLAR_NODE": 0.3,
-    "SEEED_WIO_TRACKER_L1_EINK": 0.3,
-    "SENSECAP_INDICATOR": 0.3,
-    "TBEAM_1_WATT": 0.3,
-    "THINKNODE_M1": 0.3,
-    "THINKNODE_M3": 0.3,
-    "THINKNODE_M6": 0.3,
-    "T_ECHO_LITE": 0.3,
-    "WISMESH_TAG": 0.3,
-    "WISMESH_TAP": 0.3,
-    "XIAO_NRF52_KIT": 0.3,
-    "CROWPANEL": 0.3,
-}
-
-# Non-deprecated roles only.
-ROLE_WEIGHTS: dict[str, float] = {
-    "CLIENT": 75.0,
-    "CLIENT_MUTE": 5.0,
-    "ROUTER": 7.0,
-    "TRACKER": 3.0,
-    "SENSOR": 2.0,
-    "CLIENT_HIDDEN": 2.0,
-    "ROUTER_LATE": 2.0,
-    "CLIENT_BASE": 2.0,
-    "TAK": 1.0,
-    "TAK_TRACKER": 0.5,
-    "LOST_AND_FOUND": 0.5,
-}
-
-# Name pools — 60 firsts × 60 lasts = 3600 combinations.
-FIRSTS = [
-    "Quick", "Brave", "Silent", "Wild", "Lone", "Bright", "Red", "Blue",
-    "Green", "Black", "White", "Iron", "Steel", "Copper", "Silver", "Gold",
-    "Stone", "River", "Forest", "Mountain", "Canyon", "Desert", "Storm", "Sky",
-    "Solar", "Lunar", "Dawn", "Dusk", "Misty", "Frosty", "Sunny", "Shady",
-    "Happy", "Sleepy", "Drowsy", "Sneaky", "Sharp", "Smooth", "Rough", "Loud",
-    "Soft", "Slow", "Fast", "Tall", "Short", "Old", "New", "Tiny",
-    "Giant", "Hidden", "Lost", "Found", "Wandering", "Roving", "Drifting", "Floating",
-    "Burning", "Frozen", "Whispering", "Howling",
-]
-LASTS = [
-    "Phoenix", "Lion", "Bear", "Wolf", "Hawk", "Eagle", "Fox", "Lynx",
-    "Cougar", "Coyote", "Raven", "Owl", "Crow", "Falcon", "Heron", "Crane",
-    "Otter", "Badger", "Bison", "Elk", "Moose", "Stag", "Doe", "Hare",
-    "Marmot", "Mole", "Beaver", "Squirrel", "Mustang", "Bronco", "Pony", "Colt",
-    "Cobra", "Viper", "Mamba", "Adder", "Gecko", "Iguana", "Tortoise", "Turtle",
-    "Salmon", "Trout", "Bass", "Pike", "Shark", "Whale", "Dolphin", "Seal",
-    "Cactus", "Yucca", "Sage", "Juniper", "Pine", "Cedar", "Aspen", "Oak",
-    "Bluff", "Mesa", "Arroyo", "Ridge",
-]
-
-# Brief callsign pool for licensed-looking suffixes.
-CALLSIGN_PREFIXES = ["KX", "WD", "N5", "KE", "AB", "W5", "K1", "KQ", "AE", "NM"]
-
-# Only emojis that fit in 4 UTF-8 bytes (no variation selectors). short_name's
-# nanopb max_size:5 (incl. NUL) limits content to 4 bytes. ❄️ / ☀️ would be
-# 6 bytes due to U+FE0F variation selector — explicitly excluded.
-EMOJI_SHORTNAMES = ["🦊", "🐺", "🦅", "🐢", "🌵", "🔥", "🌙",
-                    "🌊", "🗻", "🌲", "🦌", "🐝", "🦂", "🦉",
-                    "🦇", "🦋"]
-
-# --------------------------------------------------------------------------
-# Helpers
-# --------------------------------------------------------------------------
-
-NUM_RESERVED = 4         # firmware reserves 0..3 (per NodeDB constants)
-NUM_MAX_EXCLUSIVE = 0x80000000  # restrict to positive int32 range for readability
-
-
-def _weighted_choice(rng: random.Random, weights: dict[str, float]) -> str:
-    """Deterministic weighted pick. Uses sorted keys so dict order is fixed."""
-    keys = sorted(weights.keys())
-    totals = [weights[k] for k in keys]
-    return rng.choices(keys, weights=totals, k=1)[0]
-
-
-def _gen_long_name(rng: random.Random, is_licensed: bool) -> str:
-    base = f"{rng.choice(FIRSTS)} {rng.choice(LASTS)}"
-    if is_licensed:
-        prefix = rng.choice(CALLSIGN_PREFIXES)
-        # Two trailing alpha chars after the digit; keep within 25 - len(base) - 1
-        suffix = f" {prefix}{rng.randint(0,9)}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}"
-        # nanopb max_size:25 means C string fits 24 bytes + NUL.
-        if len(base) + len(suffix) <= 24:
-            base = base + suffix
-    # Hard cap to 24 chars (nanopb max_size:25 minus NUL).
-    return base[:24]
-
-
-def _gen_short_name(rng: random.Random, long_name: str) -> str:
-    # 10% emoji-only short_name
-    if rng.random() < 0.10:
-        return rng.choice(EMOJI_SHORTNAMES)
-    first_char = long_name[0].upper() if long_name else "X"
-    alphanums = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
-    return first_char + "".join(rng.choices(alphanums, k=3))
-
-
-def _gen_hops_away(rng: random.Random) -> int:
-    # Geometric-ish: 0→55%, 1→25%, 2→12%, 3→5%, 4→2%, 5+→1%
-    r = rng.random()
-    if r < 0.55:
-        return 0
-    if r < 0.80:
-        return 1
-    if r < 0.92:
-        return 2
-    if r < 0.97:
-        return 3
-    if r < 0.99:
-        return 4
-    return rng.randint(5, 7)
-
-
-def _gen_position(
-    rng: random.Random,
-    centroid_lat: float,
-    centroid_lon: float,
-    spread_km: float,
-    last_heard_offset_sec: int,
-) -> dict:
-    # 1 deg ≈ 111 km at the equator; we use this as a flat approximation.
-    lat = centroid_lat + rng.gauss(0.0, spread_km / 111.0)
-    lon = centroid_lon + rng.gauss(0.0, spread_km / 111.0)
-    altitude = max(0, round(rng.gauss(1376.0, 250.0)))  # T or C valley floor + relief
-    # Position was reported up to 300s before last_heard.
-    time_offset_sec = last_heard_offset_sec + rng.randint(0, 300)
-    return {
-        "latitude": round(lat, 6),
-        "longitude": round(lon, 6),
-        "altitude": altitude,
-        "time_offset_sec": time_offset_sec,
-        "location_source": "LOC_INTERNAL",
-    }
-
-
-def _gen_telemetry(rng: random.Random) -> dict:
-    # 5% plugged-in (battery_level == 101); rest uniform [10..100].
-    if rng.random() < 0.05:
-        battery_level = 101
-        voltage = 4.20
-    else:
-        battery_level = rng.randint(10, 100)
-        voltage = round(3.3 + (battery_level / 100.0) * 0.9, 3)
-    # Beta distributions for low/right-skewed metrics; randomly draw via gammavariate.
-    def _beta(a: float, b: float) -> float:
-        x = rng.gammavariate(a, 1.0)
-        y = rng.gammavariate(b, 1.0)
-        return x / (x + y)
-    channel_utilization = round(_beta(2.0, 15.0) * 100.0, 2)
-    air_util_tx = round(_beta(1.5, 20.0) * 10.0, 3)
-    uptime_seconds = int(rng.expovariate(1.0 / 86400.0))
-    return {
-        "battery_level": battery_level,
-        "voltage": voltage,
-        "channel_utilization": channel_utilization,
-        "air_util_tx": air_util_tx,
-        "uptime_seconds": uptime_seconds,
-    }
-
-
-def _gen_environment(rng: random.Random) -> dict:
-    return {
-        "temperature": round(rng.gauss(22.0, 8.0), 2),
-        "relative_humidity": round(min(100.0, max(0.0, rng.gauss(55.0, 20.0))), 2),
-        "barometric_pressure": round(rng.gauss(1013.0, 8.0), 2),
-        "iaq": int(min(500, max(0, round(rng.gauss(50.0, 30.0))))),
-    }
-
-
-def _gen_status(rng: random.Random) -> dict:
-    # `StatusMessage` (mesh.proto:1445) has a single free-form `string status`.
-    # Most peers report a healthy short status; occasional alert string.
-    healthy = ["OK", "online", "active", "running", "ready", "nominal"]
-    alert = ["low-batt", "no-gps", "weak-signal", "rebooted", "offline-soon"]
-    if rng.random() < 0.92:
-        return {"status": rng.choice(healthy)}
-    return {"status": rng.choice(alert)}
-
-
-def _gen_node(
-    rng: random.Random,
-    num: int,
-    centroid_lat: float,
-    centroid_lon: float,
-    spread_km: float,
-    coverage: dict[str, float],
-    last_heard_mean_sec: int,
-    last_heard_max_sec: int,
-) -> dict:
-    is_licensed = rng.random() < 0.05
-    long_name = _gen_long_name(rng, is_licensed)
-    short_name = _gen_short_name(rng, long_name)
-    hw_model = _weighted_choice(rng, HW_MODEL_WEIGHTS)
-    role = _weighted_choice(rng, ROLE_WEIGHTS)
-    has_public_key = rng.random() < 0.92
-    public_key_hex = (
-        "".join(f"{rng.randint(0,255):02x}" for _ in range(32)) if has_public_key else ""
-    )
-    snr = round(max(-20.0, min(12.0, rng.gauss(6.0, 4.0))), 2)
-    channel = 0 if rng.random() < 0.90 else rng.randint(1, 7)
-    hops_away = _gen_hops_away(rng)
-    next_hop = rng.randint(0, 255) if hops_away > 0 else 0
-    last_heard_offset_sec = int(min(rng.expovariate(1.0 / last_heard_mean_sec), last_heard_max_sec))
-
-    bitfield = {
-        "has_user": True,
-        "is_favorite": rng.random() < 0.08,
-        "is_muted": rng.random() < 0.03,
-        "via_mqtt": rng.random() < 0.12,
-        "is_ignored": rng.random() < 0.01,
-        "is_licensed": is_licensed,
-        "has_is_unmessagable": True,
-        "is_unmessagable": rng.random() < 0.02,
-        "is_key_manually_verified": rng.random() < 0.04,
-    }
-
-    node: dict = {
-        "num": f"0x{num:08x}",
-        "long_name": long_name,
-        "short_name": short_name,
-        "hw_model": hw_model,
-        "role": role,
-        "public_key_hex": public_key_hex,
-        "snr": snr,
-        "channel": channel,
-        "hops_away": hops_away,
-        "next_hop": next_hop,
-        "last_heard_offset_sec": last_heard_offset_sec,
-        "bitfield": bitfield,
-        "position": (
-            _gen_position(rng, centroid_lat, centroid_lon, spread_km, last_heard_offset_sec)
-            if rng.random() < coverage["position"]
-            else None
-        ),
-        "telemetry": _gen_telemetry(rng) if rng.random() < coverage["telemetry"] else None,
-        "environment": _gen_environment(rng) if rng.random() < coverage["environment"] else None,
-        "status": _gen_status(rng) if rng.random() < coverage["status"] else None,
-    }
-    return node
-
-
-def _parse_my_node_num(s: str | None) -> int | None:
-    if s is None:
-        return None
-    s = s.strip()
-    if s.startswith("0x") or s.startswith("0X"):
-        return int(s, 16)
-    return int(s)
-
-
-def main(argv: list[str]) -> int:
-    p = argparse.ArgumentParser(
-        description="Deterministic JSONL seed for the fake NodeDB fixture.",
-        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
-    )
-    p.add_argument("--count", type=int, required=True, help="Number of fake nodes to emit.")
-    p.add_argument("--seed", type=int, required=True, help="Deterministic seed.")
-    p.add_argument("--out", required=True, help="Output JSONL path.")
-    p.add_argument(
-        "--centroid",
-        default="33.1284,-107.2528",
-        help="LAT,LON centroid (default: Truth or Consequences, NM).",
-    )
-    p.add_argument("--spread-km", type=float, default=60.0, help="Gaussian std-dev in km.")
-    p.add_argument("--position-coverage", type=float, default=0.85)
-    p.add_argument("--telemetry-coverage", type=float, default=0.70)
-    p.add_argument("--environment-coverage", type=float, default=0.25)
-    p.add_argument("--status-coverage", type=float, default=0.40)
-    p.add_argument("--my-node-num", default=None, help="Exclude this NodeNum from generated set (hex or dec).")
-    p.add_argument("--last-heard-mean-sec", type=int, default=3600)
-    p.add_argument("--last-heard-max-sec", type=int, default=7 * 86400)
-    args = p.parse_args(argv)
-
-    if args.count <= 0:
-        print("--count must be positive", file=sys.stderr)
-        return 2
-
-    try:
-        centroid_lat, centroid_lon = (float(s) for s in args.centroid.split(","))
-    except ValueError:
-        print(f"--centroid must be LAT,LON; got {args.centroid!r}", file=sys.stderr)
-        return 2
-
-    my_node_num = _parse_my_node_num(args.my_node_num)
-
-    rng = random.Random(args.seed)
-
-    # 1) Generate a unique deterministic set of NodeNums.
-    nums: set[int] = set()
-    while len(nums) < args.count:
-        n = rng.randrange(NUM_RESERVED, NUM_MAX_EXCLUSIVE)
-        if my_node_num is not None and n == my_node_num:
-            continue
-        nums.add(n)
-    ordered_nums = sorted(nums)  # sort to fix output order independent of set hash
-
-    # 2) Per-node generation (in num order, single RNG continues).
-    coverage = {
-        "position": args.position_coverage,
-        "telemetry": args.telemetry_coverage,
-        "environment": args.environment_coverage,
-        "status": args.status_coverage,
-    }
-    nodes = [
-        _gen_node(
-            rng,
-            n,
-            centroid_lat,
-            centroid_lon,
-            args.spread_km,
-            coverage,
-            args.last_heard_mean_sec,
-            args.last_heard_max_sec,
-        )
-        for n in ordered_nums
-    ]
-
-    # 3) Write JSONL.
-    out_path = pathlib.Path(args.out)
-    out_path.parent.mkdir(parents=True, exist_ok=True)
-    # `generated_at_iso` is informational; it does NOT affect determinism because
-    # we derive it from the seed, not from wall clock. (Same seed -> same string.)
-    generated_at = _dt.datetime.fromtimestamp(args.seed, tz=_dt.timezone.utc).isoformat().replace("+00:00", "Z")
-    meta = {
-        "_meta": {
-            "version": 25,
-            "seed": args.seed,
-            "count": args.count,
-            "centroid": [centroid_lat, centroid_lon],
-            "spread_km": args.spread_km,
-            "generated_at_iso": generated_at,
-            "my_node_num_excluded": (None if my_node_num is None else f"0x{my_node_num:08x}"),
-            "coverage": coverage,
-            "last_heard_mean_sec": args.last_heard_mean_sec,
-            "last_heard_max_sec": args.last_heard_max_sec,
-        }
-    }
-    with out_path.open("w", encoding="utf-8") as f:
-        # `ensure_ascii=False` so emoji short_names survive. `sort_keys=True` for
-        # determinism (insertion order varies by Python version otherwise).
-        f.write(json.dumps(meta, ensure_ascii=False, sort_keys=True) + "\n")
-        for node in nodes:
-            f.write(json.dumps(node, ensure_ascii=False, sort_keys=True) + "\n")
-
-    print(f"wrote {args.count} nodes to {out_path} ({out_path.stat().st_size} bytes)", file=sys.stderr)
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main(sys.argv[1:]))

bin/platformio-custom.py

@@ -293,12 +293,9 @@ if ("HAS_TFT", 1) in env.get("CPPDEFINES", []):
 board_arch = infer_architecture(env.BoardConfig())
 should_skip_manifest = board_arch is None
 
-# Most platforms can generate the manifest as part of the default 'buildprog' target.
-# Typically this passes success/failure properly.
-mtjson_deps = ["buildprog"]
-if platform.name == "espressif32":
-    # On ESP32, we need to explicitly depend upon the binary to prevent fake-success upon failure.
-    mtjson_deps = ["$BUILD_DIR/${PROGNAME}.bin"]
+# For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
+mtjson_deps = [] if should_skip_manifest else ["buildprog"]
+if not should_skip_manifest and platform.name == "espressif32":
     # Build littlefs image as part of mtjson target
     # Equivalent to `pio run -t buildfs`
     target_lfs = env.DataToBin(
@@ -312,8 +309,7 @@ if should_skip_manifest:
 
     env.AddCustomTarget(
         name="mtjson",
-        # For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
-        dependencies=[],
+        dependencies=mtjson_deps,
         actions=[skip_manifest],
         title="Meshtastic Manifest (skipped)",
         description="mtjson generation is skipped for native environments",

bin/regen-fake-nodedbs.sh

@@ -1,73 +0,0 @@
-#!/usr/bin/env bash
-# Regenerate the fake-NodeDB fixtures: produces 250 / 500 / 1000 / 2000-node
-# JSONL seed files + their compiled v25 protobufs.
-#
-# Layout:
-#   test/fixtures/nodedb/seed_v25_<N>.jsonl   — COMMITTED, hand-editable.
-#   build/fixtures/nodedb/nodes_v25_<N>.proto — .gitignored, build artifact.
-#                                               Drop into /prefs/nodes.proto.
-#
-# Daily use:  ./bin/regen-fake-nodedbs.sh
-#   - Recompiles protos from committed seeds (fresh wall-clock timestamps).
-# Intentional seed bump:  REGEN_SEEDS=yes ./bin/regen-fake-nodedbs.sh
-#   - Overwrites the committed JSONL files with freshly-seeded data.
-
-set -euo pipefail
-cd "$(dirname "$0")/.."
-
-# 1) Make sure the Python protobuf bindings exist (in-tree generation; .gitignored).
-if [[ ! -d bin/_generated/meshtastic ]]; then
-    echo "regenerating Python protobuf bindings (one-time)..."
-    ./bin/regen-py-protos.sh
-fi
-
-# 2) Pick a Python interpreter that has the meshtastic deps installed.
-#    Prefer the mcp-server venv (most likely to be set up by the operator).
-PY="python3"
-for cand in mcp-server/.venv/bin/python3 .venv/bin/python3; do
-    if [[ -x "$cand" ]]; then
-        PY="$cand"
-        break
-    fi
-done
-
-# 3) Pinned seeds per size — bump only when you intentionally want different
-#    structural data committed. Parallel arrays so the script works on
-#    macOS bash 3.2 (no `declare -A`).
-SIZES=(250 500 1000 2000)
-SEEDS=(20260511 20260512 20260513 20260514)
-
-REGEN_SEEDS="${REGEN_SEEDS:-no}"
-
-mkdir -p build/fixtures/nodedb test/fixtures/nodedb
-
-for i in 0 1 2 3; do
-    n="${SIZES[$i]}"
-    seed="${SEEDS[$i]}"
-    jsonl=$(printf "test/fixtures/nodedb/seed_v25_%04d.jsonl" "$n")
-    proto=$(printf "build/fixtures/nodedb/nodes_v25_%04d.proto" "$n")
-
-    if [[ "$REGEN_SEEDS" == "yes" || ! -f "$jsonl" ]]; then
-        $PY bin/gen-fake-nodedb-seed.py \
-            --count "$n" \
-            --seed "$seed" \
-            --out "$jsonl" \
-            --centroid 33.1284,-107.2528 \
-            --spread-km 60 \
-            --position-coverage 0.85 \
-            --telemetry-coverage 0.70 \
-            --environment-coverage 0.25 \
-            --status-coverage 0.40
-        echo "  seed:   $jsonl ($(wc -c < "$jsonl") bytes)"
-    fi
-
-    $PY bin/seed-json-to-proto.py --in "$jsonl" --out "$proto"
-    echo "  proto:  $proto ($(wc -c < "$proto") bytes)"
-done
-
-echo ""
-echo "Done. To load on Portduino native:"
-echo "  cp build/fixtures/nodedb/nodes_v25_1000.proto ~/.portduino/default/prefs/nodes.proto"
-echo ""
-echo "To push to a hardware device:"
-echo "  Use the mcp-server tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"

bin/regen-py-protos.sh

@@ -1,51 +0,0 @@
-#!/usr/bin/env bash
-# Regenerate Python protobuf bindings from the in-tree `protobufs/` submodule
-# into `bin/_generated/`. Called by bin/regen-fake-nodedbs.sh; also useful as
-# a standalone refresh after any change to a .proto file.
-#
-# Output is .gitignored — bindings are a build artifact.
-#
-# Namespace rewrite:
-# The .proto files declare `package meshtastic;`, which makes protoc emit
-# imports like `from meshtastic import mesh_pb2`. That conflicts with the
-# PyPI `meshtastic` package (which the mcp-server relies on for its
-# SerialInterface/BLEInterface transport). We post-process the generated
-# files to live under `meshtastic_v25` instead — both the directory layout
-# and all internal imports — so they coexist cleanly with the PyPI package.
-
-set -euo pipefail
-cd "$(dirname "$0")/.."
-
-if ! command -v protoc >/dev/null 2>&1; then
-    echo "ERROR: protoc not found in PATH." >&2
-    echo "  macOS: brew install protobuf" >&2
-    echo "  Ubuntu/Debian: apt install protobuf-compiler" >&2
-    exit 1
-fi
-
-OUT=bin/_generated
-LOCAL_NS=meshtastic_v25
-
-rm -rf "$OUT"
-mkdir -p "$OUT"
-
-# 1) Generate from the in-tree protos. nanopb.proto first so its descriptor
-#    is available for the [(nanopb).*] options on other messages.
-protoc \
-    --proto_path=protobufs \
-    --python_out="$OUT" \
-    protobufs/nanopb.proto \
-    protobufs/meshtastic/*.proto
-
-# 2) Move the generated `meshtastic/` directory to `meshtastic_v25/`.
-mv "$OUT/meshtastic" "$OUT/$LOCAL_NS"
-
-# 3) Rewrite internal imports: any reference to `meshtastic.X_pb2` or
-#    `from meshtastic import X_pb2` becomes `meshtastic_v25.*`.
-python3 bin/_rewrite_proto_namespace.py "$OUT/$LOCAL_NS" "$LOCAL_NS"
-
-# 4) Make the package importable.
-touch "$OUT/__init__.py"
-touch "$OUT/$LOCAL_NS/__init__.py"
-
-echo "regenerated Python protobuf bindings -> $OUT/$LOCAL_NS/ (namespace: $LOCAL_NS)" >&2

bin/seed-json-to-proto.py

@@ -1,342 +0,0 @@
-#!/usr/bin/env python3
-"""Compile a committed seed JSONL into a binary meshtastic_NodeDatabase v25 proto.
-
-The input is produced by `bin/gen-fake-nodedb-seed.py`. Timestamps in the JSONL
-are stored as `*_offset_sec` (seconds before "now"); this script resolves them
-to absolute epochs using `--now-epoch` (default: current wall clock).
-
-Output is a raw `pb_encode`-compatible binary that can be dropped at
-`/prefs/nodes.proto` on the device (Portduino prefs dir or hardware via
-XModem) and loaded by `NodeDB::loadFromDisk` at boot.
-
-Wire format reference:
-  protobufs/meshtastic/deviceonly.proto  (NodeDatabase, NodeInfoLite, sat entries)
-  src/mesh/NodeDB.h:467-484              (bitfield bit positions)
-  src/mesh/NodeDB.cpp:1523-1524          (pb_decode entry point)
-"""
-
-from __future__ import annotations
-
-import argparse
-import json
-import pathlib
-import sys
-import time
-from typing import Any
-
-# Prefer the in-tree generated Python protobuf bindings (bin/_generated/meshtastic_v25/)
-# because the firmware branch's protos (v25 NodeDatabase satellite arrays, slim
-# NodeInfoLite) are typically newer than what the PyPI `meshtastic` package
-# ships. Run `bin/regen-py-protos.sh` to (re)generate.
-#
-# Namespace note: the local bindings live under `meshtastic_v25` (NOT `meshtastic`)
-# to avoid shadowing the PyPI `meshtastic` package — bin/regen-py-protos.sh
-# post-processes the protoc output to rename the package.
-_HERE = pathlib.Path(__file__).resolve().parent
-_LOCAL_PROTO_DIR = _HERE / "_generated"
-if _LOCAL_PROTO_DIR.is_dir():
-    sys.path.insert(0, str(_LOCAL_PROTO_DIR))
-
-try:
-    from meshtastic_v25.deviceonly_pb2 import (  # type: ignore[import-not-found]
-        NodeDatabase,
-        NodeInfoLite,
-        NodePositionEntry,
-        NodeTelemetryEntry,
-        NodeEnvironmentEntry,
-        NodeStatusEntry,
-        PositionLite,
-    )
-    from meshtastic_v25.mesh_pb2 import HardwareModel, Position, StatusMessage  # type: ignore[import-not-found]
-    from meshtastic_v25.config_pb2 import Config  # type: ignore[import-not-found]
-    from meshtastic_v25.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics  # type: ignore[import-not-found]
-except ImportError as local_err:
-    # Fall back to the PyPI package if in-tree bindings haven't been generated.
-    # Will fail the v25 assertion below if the PyPI package predates the
-    # satellite-DB schema, but at least gives a clear "run regen-py-protos.sh"
-    # error message instead of an opaque ImportError.
-    try:
-        from meshtastic.protobuf.deviceonly_pb2 import (
-            NodeDatabase,
-            NodeInfoLite,
-            NodePositionEntry,
-            NodeTelemetryEntry,
-            NodeEnvironmentEntry,
-            NodeStatusEntry,
-            PositionLite,
-        )
-        from meshtastic.protobuf.mesh_pb2 import HardwareModel, Position, StatusMessage
-        from meshtastic.protobuf.config_pb2 import Config
-        from meshtastic.protobuf.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics
-    except ImportError as pypi_err:
-        print(
-            "ERROR: could not import meshtastic protobuf bindings.\n"
-            "  In-tree generation: run `bin/regen-py-protos.sh` (requires protoc).\n"
-            "  PyPI fallback: `pip install meshtastic` (may lag firmware branch).\n"
-            f"  local error (meshtastic_v25): {local_err}\n"
-            f"  pypi error  (meshtastic.protobuf): {pypi_err}",
-            file=sys.stderr,
-        )
-        sys.exit(1)
-
-# Fail loudly if bindings predate v25 (no satellite arrays).
-assert (
-    hasattr(NodeDatabase, "DESCRIPTOR")
-    and "positions" in NodeDatabase.DESCRIPTOR.fields_by_name
-), (
-    "Loaded meshtastic bindings are older than v25 (NodeDatabase.positions missing). "
-    "Run `bin/regen-py-protos.sh` against the in-tree protobufs/ submodule."
-)
-
-# ---------------------------------------------------------------------------
-# Bitfield bit positions (mirror src/mesh/NodeDB.h:467-484).
-# ---------------------------------------------------------------------------
-BIT_IS_KEY_MANUALLY_VERIFIED = 0
-BIT_IS_MUTED = 1
-BIT_VIA_MQTT = 2
-BIT_IS_FAVORITE = 3
-BIT_IS_IGNORED = 4
-BIT_HAS_USER = 5
-BIT_IS_LICENSED = 6
-BIT_IS_UNMESSAGABLE = 7
-BIT_HAS_IS_UNMESSAGABLE = 8
-
-BITFIELD_LAYOUT = (
-    # JSON key            bit position
-    ("is_key_manually_verified", BIT_IS_KEY_MANUALLY_VERIFIED),
-    ("is_muted", BIT_IS_MUTED),
-    ("via_mqtt", BIT_VIA_MQTT),
-    ("is_favorite", BIT_IS_FAVORITE),
-    ("is_ignored", BIT_IS_IGNORED),
-    ("has_user", BIT_HAS_USER),
-    ("is_licensed", BIT_IS_LICENSED),
-    ("is_unmessagable", BIT_IS_UNMESSAGABLE),
-    ("has_is_unmessagable", BIT_HAS_IS_UNMESSAGABLE),
-)
-
-
-def _pack_bitfield(bf: dict[str, bool]) -> int:
-    out = 0
-    for key, shift in BITFIELD_LAYOUT:
-        if bf.get(key, False):
-            out |= (1 << shift)
-    return out
-
-
-def _validate_node(node: dict[str, Any]) -> None:
-    """Friendly errors so hand-editors get clear feedback."""
-    if "num" not in node or not isinstance(node["num"], str):
-        raise ValueError(f"node missing/invalid 'num' (must be hex string): {node!r}")
-    if "long_name" not in node:
-        raise ValueError(f"node {node['num']}: missing 'long_name'")
-    if len(node["long_name"]) > 24:
-        raise ValueError(
-            f"node {node['num']}: long_name {node['long_name']!r} is "
-            f"{len(node['long_name'])} chars; max 24 (nanopb max_size:25 minus NUL)"
-        )
-    if "short_name" in node:
-        # short_name max_size:5 (incl. NUL) → 4 bytes of content.
-        # Char count is irrelevant — emojis with variation selectors (e.g. ❄️ = 6 B)
-        # would slip past a `len(str) > 4` check. Always measure bytes.
-        b = node["short_name"].encode("utf-8")
-        if len(b) > 4:
-            raise ValueError(
-                f"node {node['num']}: short_name {node['short_name']!r} is "
-                f"{len(b)} bytes UTF-8; max 4 (nanopb max_size:5 minus NUL)"
-            )
-    pk = node.get("public_key_hex", "")
-    if pk and len(pk) != 64:
-        raise ValueError(
-            f"node {node['num']}: public_key_hex must be 64 hex chars or empty; "
-            f"got {len(pk)} chars"
-        )
-    if pk:
-        try:
-            bytes.fromhex(pk)
-        except ValueError as e:
-            raise ValueError(f"node {node['num']}: public_key_hex is not valid hex: {e}")
-
-
-def _resolve_time(
-    node: dict[str, Any],
-    field_absolute: str,
-    field_offset: str,
-    now_epoch: int,
-) -> int:
-    """If `field_absolute` is set, use it; else compute `now_epoch - offset`."""
-    if field_absolute in node and node[field_absolute] is not None:
-        return int(node[field_absolute])
-    offset = node.get(field_offset, 0)
-    return max(0, int(now_epoch) - int(offset))
-
-
-def _build_node_info_lite(node: dict[str, Any], now_epoch: int) -> NodeInfoLite:
-    _validate_node(node)
-    info = NodeInfoLite()
-    info.num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
-    info.long_name = node.get("long_name", "")
-    info.short_name = node.get("short_name", "")
-    # Enum lookups will raise ValueError on unknown names — that's exactly what we want.
-    info.hw_model = HardwareModel.Value(node.get("hw_model", "UNSET"))
-    info.role = Config.DeviceConfig.Role.Value(node.get("role", "CLIENT"))
-    pk_hex = node.get("public_key_hex", "")
-    if pk_hex:
-        info.public_key = bytes.fromhex(pk_hex)
-    info.snr = float(node.get("snr", 0.0))
-    info.channel = int(node.get("channel", 0))
-    if "hops_away" in node:
-        # `optional uint32 hops_away = 9;` — in Python protobuf, assigning the
-        # field implicitly sets HasField("hops_away") to True. No has_hops_away
-        # setter exists (unlike the C++ nanopb-generated header).
-        info.hops_away = int(node["hops_away"])
-    info.next_hop = int(node.get("next_hop", 0))
-    info.last_heard = _resolve_time(node, "last_heard", "last_heard_offset_sec", now_epoch)
-    info.bitfield = _pack_bitfield(node.get("bitfield", {}))
-    return info
-
-
-def _build_position_entry(num: int, pos: dict[str, Any], now_epoch: int) -> NodePositionEntry:
-    entry = NodePositionEntry()
-    entry.num = num
-    pl = PositionLite()
-    # Firmware stores lat/long as int32 in 1e-7 degrees.
-    pl.latitude_i = int(round(float(pos["latitude"]) * 1e7))
-    pl.longitude_i = int(round(float(pos["longitude"]) * 1e7))
-    pl.altitude = int(pos.get("altitude", 0))
-    pl.time = _resolve_time(pos, "time", "time_offset_sec", now_epoch)
-    pl.location_source = Position.LocSource.Value(pos.get("location_source", "LOC_UNSET"))
-    entry.position.CopyFrom(pl)
-    return entry
-
-
-def _build_telemetry_entry(num: int, tel: dict[str, Any]) -> NodeTelemetryEntry:
-    entry = NodeTelemetryEntry()
-    entry.num = num
-    dm = DeviceMetrics()
-    if "battery_level" in tel:
-        dm.battery_level = int(tel["battery_level"])
-    if "voltage" in tel:
-        dm.voltage = float(tel["voltage"])
-    if "channel_utilization" in tel:
-        dm.channel_utilization = float(tel["channel_utilization"])
-    if "air_util_tx" in tel:
-        dm.air_util_tx = float(tel["air_util_tx"])
-    if "uptime_seconds" in tel:
-        dm.uptime_seconds = int(tel["uptime_seconds"])
-    entry.device_metrics.CopyFrom(dm)
-    return entry
-
-
-def _build_environment_entry(num: int, env: dict[str, Any]) -> NodeEnvironmentEntry:
-    entry = NodeEnvironmentEntry()
-    entry.num = num
-    em = EnvironmentMetrics()
-    if "temperature" in env:
-        em.temperature = float(env["temperature"])
-    if "relative_humidity" in env:
-        em.relative_humidity = float(env["relative_humidity"])
-    if "barometric_pressure" in env:
-        em.barometric_pressure = float(env["barometric_pressure"])
-    if "iaq" in env:
-        em.iaq = int(env["iaq"])
-    entry.environment_metrics.CopyFrom(em)
-    return entry
-
-
-def _build_status_entry(num: int, status: dict[str, Any]) -> NodeStatusEntry:
-    # `StatusMessage` (mesh.proto:1445) has a single `string status` field.
-    entry = NodeStatusEntry()
-    entry.num = num
-    sm = StatusMessage()
-    if "status" in status:
-        sm.status = str(status["status"])
-    entry.status.CopyFrom(sm)
-    return entry
-
-
-def compile_jsonl_to_proto(jsonl_path: pathlib.Path, now_epoch: int) -> bytes:
-    """Read a seed JSONL and return the encoded NodeDatabase bytes."""
-    lines = jsonl_path.read_text(encoding="utf-8").splitlines()
-    if not lines:
-        raise ValueError(f"{jsonl_path} is empty")
-    meta_line = lines[0]
-    meta_obj = json.loads(meta_line)
-    meta = meta_obj.get("_meta", {})
-    version = meta.get("version")
-    if version != 25:
-        raise ValueError(
-            f"{jsonl_path}: meta version is {version!r}; this compiler "
-            f"requires version=25. Regenerate the seed with the matching tooling."
-        )
-
-    db = NodeDatabase()
-    db.version = 25
-
-    for ln, raw in enumerate(lines[1:], start=2):
-        raw = raw.strip()
-        if not raw:
-            continue
-        try:
-            node = json.loads(raw)
-        except json.JSONDecodeError as e:
-            raise ValueError(f"{jsonl_path}:{ln} JSON parse error: {e}")
-
-        num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
-
-        # Header
-        info = _build_node_info_lite(node, now_epoch)
-        db.nodes.append(info)
-
-        # Satellites (nullable)
-        if node.get("position"):
-            db.positions.append(_build_position_entry(num, node["position"], now_epoch))
-        if node.get("telemetry"):
-            db.telemetry.append(_build_telemetry_entry(num, node["telemetry"]))
-        if node.get("environment"):
-            db.environment.append(_build_environment_entry(num, node["environment"]))
-        if node.get("status"):
-            db.status.append(_build_status_entry(num, node["status"]))
-
-    return db.SerializeToString()
-
-
-def main(argv: list[str]) -> int:
-    p = argparse.ArgumentParser(
-        description="Compile a seed JSONL into a binary v25 NodeDatabase proto.",
-        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
-    )
-    p.add_argument("--in", dest="in_path", required=True, help="Input seed JSONL.")
-    p.add_argument("--out", required=True, help="Output binary .proto path.")
-    p.add_argument(
-        "--now-epoch",
-        type=int,
-        default=None,
-        help="Pin 'now' to this Unix epoch (for byte-identical CI). Default: time.time().",
-    )
-    args = p.parse_args(argv)
-
-    in_path = pathlib.Path(args.in_path)
-    if not in_path.is_file():
-        print(f"input not found: {in_path}", file=sys.stderr)
-        return 2
-
-    now_epoch = args.now_epoch if args.now_epoch is not None else int(time.time())
-
-    try:
-        encoded = compile_jsonl_to_proto(in_path, now_epoch)
-    except ValueError as e:
-        print(f"ERROR: {e}", file=sys.stderr)
-        return 3
-
-    out_path = pathlib.Path(args.out)
-    out_path.parent.mkdir(parents=True, exist_ok=True)
-    out_path.write_bytes(encoded)
-    print(
-        f"compiled {in_path} -> {out_path} ({len(encoded)} bytes, now_epoch={now_epoch})",
-        file=sys.stderr,
-    )
-    return 0
-
-
-if __name__ == "__main__":
-    sys.exit(main(sys.argv[1:]))

bin/shame.py

@@ -0,0 +1,95 @@
+import sys
+import os
+import json
+from github import Github
+
+def parseFile(path):
+    with open(path, "r") as f:
+        data = json.loads(f)
+    for file in data["files"]:
+        if file["name"].endswith(".bin"):
+            return file["name"], file["bytes"]
+
+if len(sys.argv) != 4:
+    print(f"expected usage: {sys.argv[0]} <PR number> <path to old-manifests> <path to new-manifests>")
+    sys.exit(1)
+
+pr_number = int(sys.argv[1])
+
+token = os.getenv("GITHUB_TOKEN")
+if not token:
+    raise EnvironmentError("GITHUB_TOKEN not found in environment.")
+
+repo_name = os.getenv("GITHUB_REPOSITORY")  # "owner/repo"
+if not repo_name:
+    raise EnvironmentError("GITHUB_REPOSITORY not found in environment.")
+
+oldFiles = sys.argv[2]
+old = set(os.path.join(oldFiles, f) for f in os.listdir(oldFiles) if os.path.isfile(f))
+newFiles = sys.argv[3]
+new = set(os.path.join(newFiles, f) for f in os.listdir(newFiles) if os.path.isfile(f))
+
+startMarkdown = "# Target Size Changes\n\n"
+markdown = ""
+
+newlyIntroduced = new - old
+if len(newlyIntroduced) > 0:
+    markdown += "## Newly Introduced Targets\n\n"
+    # create a table
+    markdown += "| File | Size |\n"
+    markdown += "| ---- | ---- |\n"
+    for f in newlyIntroduced:
+        name, size = parseFile(f)
+        markdown += f"| `{name}` | {size}b |\n"
+
+# do not log removed targets
+# PRs only run a small subset of builds, so removed targets are not meaningful
+# since they are very likely to just be not ran in PR CI
+
+both = old & new
+degradations = []
+improvements = []
+for f in both:
+    oldName, oldSize = parseFile(f)
+    _, newSize = parseFile(f)
+    if oldSize != newSize:
+        if newSize < oldSize:
+            improvements.append((oldName, oldSize, newSize))
+        else:
+            degradations.append((oldName, oldSize, newSize))
+
+if len(degradations) > 0:
+    markdown += "\n## Degradation\n\n"
+    # create a table
+    markdown += "| File | Difference | Old Size | New Size |\n"
+    markdown += "| ---- | ---------- | -------- | -------- |\n"
+    for oldName, oldSize, newSize in degradations:
+        markdown += f"| `{oldName}` | **{oldSize - newSize}b** | {oldSize}b | {newSize}b |\n"
+
+if len(improvements) > 0:
+    markdown += "\n## Improvement\n\n"
+    # create a table
+    markdown += "| File | Difference | Old Size | New Size |\n"
+    markdown += "| ---- | ---------- | -------- | -------- |\n"
+    for oldName, oldSize, newSize in improvements:
+        markdown += f"| `{oldName}` | **{oldSize - newSize}b** | {oldSize}b | {newSize}b |\n"
+
+if len(markdown) == 0:
+    markdown = "No changes in target sizes detected."
+
+g = Github(token)
+repo = g.get_repo(repo_name)
+pr = repo.get_pull(pr_number)
+
+existing_comment = None
+for comment in pr.get_issue_comments():
+    if comment.body.startswith(startMarkdown):
+        existing_comment = comment
+        break
+
+final_markdown = startMarkdown + markdown
+
+if existing_comment:
+    existing_comment.edit(body=final_markdown)
+else:
+    pr.create_issue_comment(body=final_markdown)

bin/size_report.py

@@ -1,171 +0,0 @@
-#!/usr/bin/env python3
-
-"""Compare firmware size reports and generate a markdown summary.
-
-Usage:
-    size_report.py <new_sizes.json> [--baseline <label>:<old_sizes.json>]...
-
-Examples:
-    # Compare PR against develop and master baselines
-    size_report.py pr.json --baseline develop:develop.json --baseline master:master.json
-
-    # Single baseline comparison
-    size_report.py pr.json --baseline develop:develop.json
-
-    # No baselines — shows sizes with blank delta columns
-    size_report.py pr.json
-"""
-
-import argparse
-import json
-import sys
-
-
-def load_sizes(path):
-    with open(path) as f:
-        return json.load(f)
-
-
-def format_delta(n):
-    """Format byte delta with sign and human-friendly suffix."""
-    sign = "+" if n > 0 else ""
-    if abs(n) >= 1024:
-        return f"{sign}{n:,} ({sign}{n / 1024:.1f} KB)"
-    return f"{sign}{n:,}"
-
-
-def generate_markdown(new_sizes, baselines, top_n=5):
-    """Generate a single table with current size and delta columns per baseline.
-
-    baselines: list of (label, old_sizes_dict), may be empty
-    """
-    labels = [label for label, _ in baselines]
-
-    # Build rows: (board, current_size, [(delta, abs_delta) per baseline])
-    rows = []
-    for board in sorted(new_sizes):
-        current = new_sizes[board]
-        deltas = []
-        for _, old_sizes in baselines:
-            old = old_sizes.get(board)
-            if old is not None:
-                d = current - old
-                deltas.append((d, abs(d)))
-            else:
-                deltas.append((None, 0))
-        # Sort key: max abs delta across baselines (biggest changes first)
-        max_abs = max((ad for _, ad in deltas), default=0)
-        rows.append((board, current, deltas, max_abs))
-
-    rows.sort(key=lambda r: r[3], reverse=True)
-
-    # Summary line
-    sections = []
-    summary_parts = [f"{len(new_sizes)} targets"]
-    for i, (label, old_sizes) in enumerate(baselines):
-        increases = sum(
-            1 for _, _, deltas, _ in rows if deltas[i][0] is not None and deltas[i][0] > 0
-        )
-        decreases = sum(
-            1 for _, _, deltas, _ in rows if deltas[i][0] is not None and deltas[i][0] < 0
-        )
-        net = sum(
-            deltas[i][0] for _, _, deltas, _ in rows if deltas[i][0] is not None
-        )
-        parts = []
-        if increases:
-            parts.append(f"{increases} increased")
-        if decreases:
-            parts.append(f"{decreases} decreased")
-        if parts:
-            parts.append(f"net {format_delta(net)}")
-            summary_parts.append(f"vs `{label}`: {', '.join(parts)}")
-        else:
-            summary_parts.append(f"vs `{label}`: no changes")
-
-    if not baselines:
-        summary_parts.append("no baseline available yet")
-
-    sections.append(f"**{' | '.join(summary_parts)}**\n")
-
-    # Table header
-    header = "| Target | Size |"
-    separator = "|--------|-----:|"
-    for label in labels:
-        header += f" vs `{label}` |"
-        separator += "----------:|"
-    sections.append(header)
-    sections.append(separator)
-
-    def format_row(board, current, deltas):
-        row = f"| `{board}` | {current:,} |"
-        for d, _ in deltas:
-            if d is None:
-                row += " |"
-            elif d == 0:
-                row += " 0 |"
-            else:
-                icon = "📈" if d > 0 else "📉"
-                row += f" {icon} {format_delta(d)} |"
-        return row
-
-    # Top N rows always visible
-    top = rows[:top_n]
-    for board, current, deltas, _ in top:
-        sections.append(format_row(board, current, deltas))
-
-    # Remaining rows in expandable section
-    rest = rows[top_n:]
-    if rest:
-        sections.append("")
-        sections.append(
-            f"<details><summary>Show {len(rest)} more target(s)</summary>\n"
-        )
-        sections.append(header)
-        sections.append(separator)
-        for board, current, deltas, _ in rest:
-            sections.append(format_row(board, current, deltas))
-        sections.append("\n</details>")
-
-    sections.append("")
-    return "\n".join(sections)
-
-
-def main():
-    parser = argparse.ArgumentParser(description="Compare firmware size reports")
-    parser.add_argument("new_sizes", help="Path to new sizes JSON")
-    parser.add_argument(
-        "--baseline",
-        action="append",
-        default=[],
-        metavar="LABEL:PATH",
-        help="Baseline to compare against (e.g. develop:develop.json)",
-    )
-    parser.add_argument(
-        "--top",
-        type=int,
-        default=5,
-        help="Number of top changes to show before collapsing (default: 5)",
-    )
-    args = parser.parse_args()
-
-    new_sizes = load_sizes(args.new_sizes)
-
-    # Silence output when no targets were built — repo maintainer choice
-    if not new_sizes:
-        return
-
-    baselines = []
-    for b in args.baseline:
-        if ":" not in b:
-            print(f"Error: baseline must be LABEL:PATH, got '{b}'", file=sys.stderr)
-            sys.exit(1)
-        label, path = b.split(":", 1)
-        baselines.append((label, load_sizes(path)))
-
-    md = generate_markdown(new_sizes, baselines, top_n=args.top)
-    print(md)
-
-
-if __name__ == "__main__":
-    main()

bin/test_size_scripts.py

@@ -1,262 +0,0 @@
-#!/usr/bin/env python3
-
-"""Tests for bin/collect_sizes.py and bin/size_report.py."""
-
-import json
-import os
-import subprocess
-import sys
-import tempfile
-
-SCRIPTS_DIR = os.path.join(os.path.dirname(__file__), "..", "bin")
-
-
-def make_manifest(target, firmware_bytes, extra_files=None):
-    """Create a minimal .mt.json manifest dict."""
-    files = [{"name": f"firmware-{target}-2.6.0.bin", "bytes": firmware_bytes}]
-    if extra_files:
-        files.extend(extra_files)
-    return {
-        "platformioTarget": target,
-        "version": "2.6.0.test",
-        "files": files,
-    }
-
-
-def write_manifests(tmpdir, manifests):
-    """Write manifest dicts as .mt.json files into tmpdir."""
-    for target, data in manifests.items():
-        path = os.path.join(tmpdir, f"firmware-{target}.mt.json")
-        with open(path, "w") as f:
-            json.dump(data, f)
-
-
-def run_script(script, args):
-    """Run a Python script and return (returncode, stdout, stderr)."""
-    result = subprocess.run(
-        [sys.executable, os.path.join(SCRIPTS_DIR, script)] + args,
-        capture_output=True,
-        text=True,
-    )
-    return result.returncode, result.stdout, result.stderr
-
-
-def test_collect_sizes_basic():
-    """collect_sizes picks up firmware-*.bin entries from manifests."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        outfile = os.path.join(tmpdir, "sizes.json")
-        manifests = {
-            "heltec-v3": make_manifest("heltec-v3", 1048576),
-            "rak4631": make_manifest("rak4631", 524288),
-            "tbeam": make_manifest("tbeam", 786432),
-        }
-        write_manifests(tmpdir, manifests)
-
-        rc, stdout, stderr = run_script("collect_sizes.py", [tmpdir, outfile])
-        assert rc == 0, f"collect_sizes failed: {stderr}"
-        assert "3 targets" in stdout
-
-        with open(outfile) as f:
-            sizes = json.load(f)
-        assert sizes == {"heltec-v3": 1048576, "rak4631": 524288, "tbeam": 786432}
-
-
-def test_collect_sizes_fallback_bin():
-    """collect_sizes falls back to non-firmware-prefixed .bin if no firmware-*.bin."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        outfile = os.path.join(tmpdir, "sizes.json")
-        # Manifest with only a generic .bin (no firmware- prefix)
-        data = {
-            "platformioTarget": "custom-board",
-            "files": [
-                {"name": "littlefs-custom-board.bin", "bytes": 100000},
-                {"name": "custom-board.bin", "bytes": 500000},
-            ],
-        }
-        path = os.path.join(tmpdir, "firmware-custom-board.mt.json")
-        with open(path, "w") as f:
-            json.dump(data, f)
-
-        rc, stdout, stderr = run_script("collect_sizes.py", [tmpdir, outfile])
-        assert rc == 0, f"collect_sizes failed: {stderr}"
-
-        with open(outfile) as f:
-            sizes = json.load(f)
-        assert sizes == {"custom-board": 500000}
-
-
-def test_collect_sizes_skips_ota_littlefs():
-    """collect_sizes ignores ota/littlefs/bleota .bin files in fallback."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        outfile = os.path.join(tmpdir, "sizes.json")
-        data = {
-            "platformioTarget": "board-x",
-            "files": [
-                {"name": "littlefs-board-x.bin", "bytes": 100000},
-                {"name": "bleota-board-x.bin", "bytes": 50000},
-                {"name": "mt-board-x-ota.bin", "bytes": 60000},
-            ],
-        }
-        path = os.path.join(tmpdir, "firmware-board-x.mt.json")
-        with open(path, "w") as f:
-            json.dump(data, f)
-
-        rc, stdout, stderr = run_script("collect_sizes.py", [tmpdir, outfile])
-        assert rc == 0
-        with open(outfile) as f:
-            sizes = json.load(f)
-        # No valid firmware .bin found, board should be absent
-        assert sizes == {}
-
-
-def test_collect_sizes_ignores_non_mt_json():
-    """collect_sizes skips non .mt.json files."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        outfile = os.path.join(tmpdir, "sizes.json")
-        # Write a valid manifest
-        manifests = {"rak4631": make_manifest("rak4631", 500000)}
-        write_manifests(tmpdir, manifests)
-        # Write a decoy file
-        with open(os.path.join(tmpdir, "readme.txt"), "w") as f:
-            f.write("not a manifest")
-
-        rc, stdout, stderr = run_script("collect_sizes.py", [tmpdir, outfile])
-        assert rc == 0
-        with open(outfile) as f:
-            sizes = json.load(f)
-        assert list(sizes.keys()) == ["rak4631"]
-
-
-def test_size_report_no_baseline():
-    """size_report with no baselines shows sizes only."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        sizes_file = os.path.join(tmpdir, "new.json")
-        with open(sizes_file, "w") as f:
-            json.dump({"heltec-v3": 1000000, "rak4631": 500000}, f)
-
-        rc, stdout, stderr = run_script("size_report.py", [sizes_file])
-        assert rc == 0, f"size_report failed: {stderr}"
-        assert "2 targets" in stdout
-        assert "no baseline available yet" in stdout
-        assert "`heltec-v3`" in stdout
-        assert "`rak4631`" in stdout
-
-
-def test_size_report_with_baseline():
-    """size_report shows deltas against a baseline."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        new_file = os.path.join(tmpdir, "new.json")
-        old_file = os.path.join(tmpdir, "old.json")
-        with open(new_file, "w") as f:
-            json.dump({"heltec-v3": 1050000, "rak4631": 500000, "tbeam": 800000}, f)
-        with open(old_file, "w") as f:
-            json.dump({"heltec-v3": 1000000, "rak4631": 500000, "tbeam": 810000}, f)
-
-        rc, stdout, stderr = run_script(
-            "size_report.py", [new_file, "--baseline", f"develop:{old_file}"]
-        )
-        assert rc == 0, f"size_report failed: {stderr}"
-        assert "3 targets" in stdout
-        assert "1 increased" in stdout
-        assert "1 decreased" in stdout
-        # heltec-v3 grew by 50000
-        assert "📈" in stdout
-        # tbeam shrank by 10000
-        assert "📉" in stdout
-        # rak4631 unchanged
-        assert "vs `develop`" in stdout
-
-
-def test_size_report_multiple_baselines():
-    """size_report handles multiple baselines."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        new_file = os.path.join(tmpdir, "new.json")
-        dev_file = os.path.join(tmpdir, "develop.json")
-        master_file = os.path.join(tmpdir, "master.json")
-        with open(new_file, "w") as f:
-            json.dump({"board-a": 100000}, f)
-        with open(dev_file, "w") as f:
-            json.dump({"board-a": 95000}, f)
-        with open(master_file, "w") as f:
-            json.dump({"board-a": 90000}, f)
-
-        rc, stdout, stderr = run_script(
-            "size_report.py",
-            [new_file, "--baseline", f"develop:{dev_file}", "--baseline", f"master:{master_file}"],
-        )
-        assert rc == 0, f"size_report failed: {stderr}"
-        assert "vs `develop`" in stdout
-        assert "vs `master`" in stdout
-
-
-def test_size_report_new_target_no_baseline_entry():
-    """size_report handles targets not present in baseline (new boards)."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        new_file = os.path.join(tmpdir, "new.json")
-        old_file = os.path.join(tmpdir, "old.json")
-        with open(new_file, "w") as f:
-            json.dump({"new-board": 300000, "existing": 500000}, f)
-        with open(old_file, "w") as f:
-            json.dump({"existing": 500000}, f)
-
-        rc, stdout, stderr = run_script(
-            "size_report.py", [new_file, "--baseline", f"develop:{old_file}"]
-        )
-        assert rc == 0, f"size_report failed: {stderr}"
-        assert "`new-board`" in stdout
-        assert "no changes" in stdout  # only existing is compared, delta=0
-
-
-def test_size_report_all_unchanged():
-    """size_report shows 'no changes' when all sizes match."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        sizes_file = os.path.join(tmpdir, "sizes.json")
-        with open(sizes_file, "w") as f:
-            json.dump({"board-a": 100000, "board-b": 200000}, f)
-
-        rc, stdout, stderr = run_script(
-            "size_report.py", [sizes_file, "--baseline", f"develop:{sizes_file}"]
-        )
-        assert rc == 0, f"size_report failed: {stderr}"
-        assert "no changes" in stdout
-
-
-def test_collect_sizes_bad_args():
-    """collect_sizes exits with error on wrong arg count."""
-    rc, stdout, stderr = run_script("collect_sizes.py", [])
-    assert rc == 1
-    assert "Usage" in stderr
-
-
-def test_size_report_bad_baseline_format():
-    """size_report exits with error on malformed --baseline."""
-    with tempfile.TemporaryDirectory() as tmpdir:
-        sizes_file = os.path.join(tmpdir, "sizes.json")
-        with open(sizes_file, "w") as f:
-            json.dump({"x": 1}, f)
-
-        rc, stdout, stderr = run_script(
-            "size_report.py", [sizes_file, "--baseline", "no-colon-here"]
-        )
-        assert rc == 1
-        assert "LABEL:PATH" in stderr
-
-
-if __name__ == "__main__":
-    tests = [v for k, v in globals().items() if k.startswith("test_")]
-    passed = 0
-    failed = 0
-    for test in tests:
-        try:
-            test()
-            print(f"  PASS: {test.__name__}")
-            passed += 1
-        except AssertionError as e:
-            print(f"  FAIL: {test.__name__}: {e}")
-            failed += 1
-        except Exception as e:
-            print(f"  ERROR: {test.__name__}: {type(e).__name__}: {e}")
-            failed += 1
-
-    print(f"\n{passed} passed, {failed} failed out of {passed + failed}")
-    sys.exit(1 if failed else 0)

boards/CDEBYTE_EoRa-S3.json

@@ -7,7 +7,7 @@
     "extra_flags": [
       "-D CDEBYTE_EORA_S3",
       "-D ARDUINO_USB_CDC_ON_BOOT=1",
-      "-D ARDUINO_USB_MODE=1",
+      "-D ARDUINO_USB_MODE=0",
       "-D ARDUINO_RUNNING_CORE=1",
       "-D ARDUINO_EVENT_RUNNING_CORE=1",
       "-D BOARD_HAS_PSRAM"

boards/ThinkNode-M7.json

@@ -1,42 +0,0 @@
-{
-  "build": {
-    "arduino": {
-      "ldscript": "esp32s3_out.ld",
-      "memory_type": "qio_opi"
-    },
-    "core": "esp32",
-    "extra_flags": [
-      "-D BOARD_HAS_PSRAM",
-      "-D ARDUINO_USB_CDC_ON_BOOT=0",
-      "-D ARDUINO_USB_MODE=0",
-      "-D ARDUINO_RUNNING_CORE=1",
-      "-D ARDUINO_EVENT_RUNNING_CORE=0"
-    ],
-    "f_cpu": "240000000L",
-    "f_flash": "80000000L",
-    "flash_mode": "qio",
-    "psram_type": "qio_opi",
-    "hwids": [["0x303A", "0x1001"]],
-    "mcu": "esp32s3",
-    "variant": "ELECROW-ThinkNode-M7"
-  },
-  "connectivity": ["wifi", "bluetooth", "lora"],
-  "debug": {
-    "default_tool": "esp-builtin",
-    "onboard_tools": ["esp-builtin"],
-    "openocd_target": "esp32s3.cfg"
-  },
-  "frameworks": ["arduino", "espidf"],
-  "name": "ELECROW ThinkNode M7",
-  "upload": {
-    "flash_size": "8MB",
-    "maximum_ram_size": 524288,
-    "maximum_size": 8388608,
-    "use_1200bps_touch": true,
-    "wait_for_upload_port": true,
-    "require_upload_port": true,
-    "speed": 921600
-  },
-  "url": "https://www.elecrow.com",
-  "vendor": "ELECROW"
-}

boards/bpi_picow_esp32_s3.json

@@ -6,7 +6,7 @@
     "core": "esp32",
     "extra_flags": [
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1",
       "-DBOARD_HAS_PSRAM"

boards/hackaday-communicator.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_mesh_node_t1.json

@@ -1,54 +0,0 @@
-{
-  "build": {
-    "arduino": {
-      "ldscript": "nrf52840_s140_v6.ld"
-    },
-    "core": "nRF5",
-    "cpu": "cortex-m4",
-    "extra_flags": "-DNRF52840_XXAA",
-    "f_cpu": "64000000L",
-    "hwids": [
-      ["0x239A", "0x4405"],
-      ["0x239A", "0x0029"],
-      ["0x239A", "0x002A"],
-      ["0x2886", "0x1667"]
-    ],
-    "usb_product": "HT-n5262",
-    "mcu": "nrf52840",
-    "variant": "heltec_mesh_node_t1",
-    "variants_dir": "variants",
-    "bsp": {
-      "name": "adafruit"
-    },
-    "softdevice": {
-      "sd_flags": "-DS140",
-      "sd_name": "s140",
-      "sd_version": "6.1.1",
-      "sd_fwid": "0x00B6"
-    },
-    "bootloader": {
-      "settings_addr": "0xFF000"
-    }
-  },
-  "connectivity": ["bluetooth"],
-  "debug": {
-    "jlink_device": "nRF52840_xxAA",
-    "onboard_tools": ["jlink"],
-    "svd_path": "nrf52840.svd",
-    "openocd_target": "nrf52840-mdk-rs"
-  },
-  "frameworks": ["arduino"],
-  "name": "Heltec Mesh Node T1",
-  "upload": {
-    "maximum_ram_size": 248832,
-    "maximum_size": 815104,
-    "speed": 115200,
-    "protocol": "nrfutil",
-    "protocols": ["jlink", "nrfjprog", "nrfutil", "stlink"],
-    "use_1200bps_touch": true,
-    "require_upload_port": true,
-    "wait_for_upload_port": true
-  },
-  "url": "https://heltec.org",
-  "vendor": "Heltec"
-}

boards/heltec_vision_master_e213.json

@@ -9,7 +9,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_vision_master_e290.json

@@ -9,7 +9,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_vision_master_t190.json

@@ -9,7 +9,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_wireless_tracker.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DHELTEC_WIRELESS_TRACKER",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/heltec_wireless_tracker_v2.json

@@ -7,7 +7,7 @@
     "core": "esp32",
     "extra_flags": [
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/nrf54l15dk.json

@@ -1,26 +0,0 @@
-{
-  "build": {
-    "cpu": "cortex-m33",
-    "f_cpu": "128000000L",
-    "mcu": "nrf54l15",
-    "zephyr": {
-      "variant": "nrf54l15dk/nrf54l15/cpuapp"
-    }
-  },
-  "connectivity": ["bluetooth"],
-  "debug": {
-    "default_tools": ["jlink"],
-    "jlink_device": "nRF54L15_M33",
-    "svd_path": "nrf54l15.svd"
-  },
-  "frameworks": ["zephyr"],
-  "name": "Nordic nRF54L15-DK (PCA10156)",
-  "upload": {
-    "maximum_ram_size": 262144,
-    "maximum_size": 1572864,
-    "protocol": "jlink",
-    "protocols": ["jlink"]
-  },
-  "url": "https://www.nordicsemi.com/Products/nRF54L15",
-  "vendor": "Nordic Semiconductor"
-}

boards/seeed-xiao-s3.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=0"
     ],

boards/station-g3.json

@@ -1,41 +0,0 @@
-{
-  "build": {
-    "arduino": {
-      "ldscript": "esp32s3_out.ld",
-      "memory_type": "qio_opi"
-    },
-    "core": "esp32",
-    "extra_flags": [
-      "-DBOARD_HAS_PSRAM",
-      "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
-      "-DARDUINO_RUNNING_CORE=1",
-      "-DARDUINO_EVENT_RUNNING_CORE=0"
-    ],
-    "f_cpu": "240000000L",
-    "f_flash": "80000000L",
-    "flash_mode": "qio",
-    "hwids": [["0x303A", "0x1001"]],
-    "mcu": "esp32s3",
-    "variant": "station-g3"
-  },
-  "connectivity": ["wifi", "bluetooth", "lora"],
-  "debug": {
-    "default_tool": "esp-builtin",
-    "onboard_tools": ["esp-builtin"],
-    "openocd_target": "esp32s3.cfg"
-  },
-  "frameworks": ["arduino", "espidf"],
-  "name": "BQ Station G3",
-  "upload": {
-    "flash_size": "16MB",
-    "maximum_ram_size": 327680,
-    "maximum_size": 16777216,
-    "use_1200bps_touch": true,
-    "wait_for_upload_port": true,
-    "require_upload_port": true,
-    "speed": 921600
-  },
-  "url": "",
-  "vendor": "BQ Consulting"
-}

boards/t-beam-1w.json

@@ -9,7 +9,7 @@
       "-DBOARD_HAS_PSRAM",
       "-DLILYGO_TBEAM_1W",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/t-deck.json

@@ -8,7 +8,7 @@
     "extra_flags": [
       "-DBOARD_HAS_PSRAM",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/tbeam-s3-core.json

@@ -8,7 +8,7 @@
       "-DBOARD_HAS_PSRAM",
       "-DLILYGO_TBEAM_S3_CORE",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

boards/tlora-t3s3-v1.json

@@ -7,7 +7,7 @@
     "extra_flags": [
       "-DLILYGO_T3S3_V1",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1",
       "-DBOARD_HAS_PSRAM"

boards/unphone.json

@@ -10,7 +10,7 @@
       "-DBOARD_HAS_PSRAM",
       "-DUNPHONE_SPIN=9",
       "-DARDUINO_USB_CDC_ON_BOOT=1",
-      "-DARDUINO_USB_MODE=1",
+      "-DARDUINO_USB_MODE=0",
       "-DARDUINO_RUNNING_CORE=1",
       "-DARDUINO_EVENT_RUNNING_CORE=1"
     ],

debian/control

@@ -14,7 +14,6 @@ Build-Depends: debhelper-compat (= 13),
                g++,
                pkg-config,
                libyaml-cpp-dev,
-               libjsoncpp-dev,
                libgpiod-dev,
                libbluetooth-dev,
                libusb-1.0-0-dev,

default_16MB.csv

@@ -1,7 +0,0 @@
-# Name,   Type, SubType, Offset,  Size, Flags
-nvs,      data, nvs,     0x9000,  0x5000,
-otadata,  data, ota,     0xe000,  0x2000,
-app0,     app,  ota_0,   0x10000, 0x640000,
-app1,     app,  ota_1,   0x650000,0x640000,
-spiffs,   data, spiffs,  0xc90000,0x360000,
-coredump, data, coredump,0xFF0000,0x10000,

default_8MB.csv

@@ -1,7 +0,0 @@
-# Name,   Type, SubType, Offset,  Size, Flags
-nvs,      data, nvs,     0x9000,  0x5000,
-otadata,  data, ota,     0xe000,  0x2000,
-app0,     app,  ota_0,   0x10000, 0x330000,
-app1,     app,  ota_1,   0x340000,0x330000,
-spiffs,   data, spiffs,  0x670000,0x180000,
-coredump, data, coredump,0x7F0000,0x10000,

extra_scripts/esp32_extra.py

@@ -70,6 +70,17 @@ def esp32_create_combined_bin(source, target, env):
 
 env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_combined_bin)
 
-# Enable Newlib Nano formatting to save space
-# ...but allow printf float support (compromise)
-env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])
+esp32_kind = env.GetProjectOption("custom_esp32_kind")
+if esp32_kind == "esp32":
+    # Free up some IRAM by removing auxiliary SPI flash chip drivers.
+    # Wrapped stub symbols are defined in src/platform/esp32/iram-quirk.c.
+    env.Append(
+        LINKFLAGS=[
+            "-Wl,--wrap=esp_flash_chip_gd",
+            "-Wl,--wrap=esp_flash_chip_issi",
+            "-Wl,--wrap=esp_flash_chip_winbond",
+        ]
+    )
+else:
+    # For newer ESP32 targets, using newlib nano works better.
+    env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])

extra_scripts/ld_response_file.py

@@ -1,23 +0,0 @@
-#!/usr/bin/env python3
-# trunk-ignore-all(ruff/F821)
-# trunk-ignore-all(flake8/F821): For SConstruct imports
-
-# force linker response file instead of command line arguments
-
-Import("env")
-
-
-def wrap_with_tempfile(command_key):
-    command = env.get(command_key)
-    if not command or not isinstance(command, str):
-        return
-    if "TEMPFILE(" in command:
-        return
-    env.Replace(**{command_key: "${TEMPFILE('%s')}" % command})
-
-
-# Force SCons to spill long commands into response files on this target.
-env.Replace(MAXLINELENGTH=8192)
-
-for key in ("LINKCOM", "CXXLINKCOM", "SHLINKCOM", "SHCXXLINKCOM"):
-    wrap_with_tempfile(key)

extra_scripts/nrf54l15_linker.py

@@ -1,140 +0,0 @@
-#!/usr/bin/env python3
-# trunk-ignore-all(ruff/F821)
-# trunk-ignore-all(flake8/F821): For SConstruct imports
-#
-# post:extra_scripts/nrf54l15_linker.py
-#
-# Fix for Zephyr two-pass link on nRF54L15:
-# platformio-build.py registers env.Depends("$PROG_PATH", final_ld_script) but
-# the SCons dependency chain is broken (final_ld_script Command never runs).
-# This script adds a PreAction on the final firmware binary that runs the gcc
-# preprocessing command directly (extracted from build.ninja) to generate
-# zephyr/linker.cmd before the link step.
-#
-# PlatformIO bundles an old Ninja that can't handle multi-output depslog rules,
-# so we parse the COMMAND line from build.ninja and run just the gcc -E part,
-# skipping the cmake_transform_depfile step (only needed for Ninja deps tracking).
-
-import os
-import re
-import subprocess
-
-Import("env")
-
-if env.get("PIOENV") != "nrf54l15dk":
-    pass  # Only for the nrf54l15dk environment
-else:
-
-    def _extract_gcc_command(ninja_build):
-        """Parse build.ninja to find the gcc -E command that generates linker.cmd.
-
-        The rule format depends on the host:
-          Windows (CMake's RunCMake wraps every command):
-            COMMAND = cmd.exe /C "cd /D DIR && arm-none-eabi-gcc.exe ... -o linker.cmd && cmake.exe -E cmake_transform_depfile ..."
-          POSIX (Linux/macOS — no wrapper):
-            COMMAND = cd DIR && arm-none-eabi-gcc ... -o linker.cmd && cmake -E cmake_transform_depfile ...
-
-        Returns (gcc_cmd_string, cwd_path) or raises RuntimeError.
-        """
-        in_rule = False
-        with open(ninja_build, "r", encoding="utf-8", errors="replace") as f:
-            for line in f:
-                # Detect start of the linker.cmd custom command rule
-                if not in_rule:
-                    if "build zephyr/linker.cmd" in line and "CUSTOM_COMMAND" in line:
-                        in_rule = True
-                    continue
-
-                stripped = line.strip()
-                if not stripped.startswith("COMMAND = "):
-                    continue
-
-                command_val = stripped[len("COMMAND = ") :]
-
-                # On Windows the value is wrapped in `cmd.exe /C "..."` — strip
-                # the wrapper. On POSIX hosts the inner sequence is the value
-                # itself (no quoting layer).
-                m = re.search(r'/C\s+"(.*)"\s*$', command_val)
-                inner = m.group(1) if m else command_val
-                parts = inner.split(" && ")
-
-                cwd = None
-                gcc_cmd = None
-                for part in parts:
-                    part = part.strip()
-                    if part.startswith("cd /D "):  # Windows form
-                        cwd = part[len("cd /D ") :]
-                    elif part.startswith("cd "):  # POSIX form
-                        cwd = part[len("cd ") :]
-                    elif "arm-none-eabi-gcc" in part:
-                        gcc_cmd = part
-
-                if not gcc_cmd:
-                    raise RuntimeError(
-                        "nRF54L15 linker fix: arm-none-eabi-gcc command not found in:\n%s"
-                        % inner[:400]
-                    )
-
-                return gcc_cmd, cwd
-
-        raise RuntimeError(
-            "nRF54L15 linker fix: 'build zephyr/linker.cmd' rule not found in build.ninja"
-        )
-
-    def _generate_linker_cmd(target, source, env):
-        """Generate zephyr/linker.cmd via direct gcc invocation before the final link."""
-        build_dir = env.subst("$BUILD_DIR")
-        zephyr_dir = os.path.join(build_dir, "zephyr")
-        linker_cmd = os.path.join(zephyr_dir, "linker.cmd")
-
-        if os.path.exists(linker_cmd):
-            return  # Already present — nothing to do
-
-        ninja_build = os.path.join(build_dir, "build.ninja")
-        if not os.path.exists(ninja_build):
-            raise RuntimeError(
-                "nRF54L15 linker fix: build.ninja not found at %s\n"
-                "Run a full build first so CMake generates the Ninja files."
-                % ninja_build
-            )
-
-        gcc_cmd, cwd = _extract_gcc_command(ninja_build)
-        run_cwd = cwd if cwd else zephyr_dir
-
-        print(
-            "==> nRF54L15: Generating zephyr/linker.cmd (LINKER_ZEPHYR_FINAL) via GCC"
-        )
-        # gcc_cmd comes verbatim from our own build.ninja (never user input) and
-        # contains Windows-style paths with spaces that cannot be safely argv-split
-        # with shlex, so we run it via the platform shell. nosec/nosemgrep below
-        # acknowledge this deliberate, scoped use of shell=True.
-        result = subprocess.run(  # nosec B602
-            gcc_cmd,
-            shell=True,  # nosemgrep: python.lang.security.audit.subprocess-shell-true.subprocess-shell-true
-            cwd=run_cwd,
-            capture_output=True,
-            text=True,
-        )
-        if result.returncode != 0:
-            print("GCC stdout:", result.stdout[:2000])
-            print("GCC stderr:", result.stderr[:2000])
-            raise RuntimeError(
-                "nRF54L15 linker fix: GCC failed to generate linker.cmd (rc=%d)"
-                % result.returncode
-            )
-        if not os.path.exists(linker_cmd):
-            raise RuntimeError(
-                "nRF54L15 linker fix: GCC returned 0 but linker.cmd was not created at %s"
-                % linker_cmd
-            )
-        print("==> linker.cmd generated successfully")
-
-    # Use PIOMAINPROG (set by ZephyrBuildProgram) to get the exact SCons node
-    prog = env.get("PIOMAINPROG")
-    if prog:
-        env.AddPreAction(prog, _generate_linker_cmd)
-    else:
-        print(
-            "[nrf54l15_linker] WARNING: PIOMAINPROG not set, falling back to $PROG_PATH"
-        )
-        env.AddPreAction(env.subst("$PROG_PATH"), _generate_linker_cmd)

mcp-server/.gitignore

@@ -7,12 +7,6 @@ __pycache__/
 dist/
 build/
 
-# Persistent device-log capture (recorder + Datadog cursor).
-# Cross-session JSONL streams written by the autouse Recorder singleton
-# (see src/meshtastic_mcp/recorder/). Lives outside tests/ so the pytest
-# fixture truncate doesn't touch it.
-.mtlog/
-
 # Test harness artifacts
 tests/report.html
 tests/junit.xml

mcp-server/run-tests.sh

@@ -191,12 +191,10 @@ echo
 # PASS/FAIL — every hardware test would SKIP with "role not present". We
 # narrow to tests/unit explicitly so the summary reads as "no hardware,
 # unit suite only" instead of "big skip count looks suspicious".
-# Keep terminal output condensed (`-q -r fE`) so skip-heavy runs do not print
-# each skipped test in full; skip counts still appear in pytest's summary.
 if [[ -z $DETECTED && $# -eq 0 ]]; then
 	echo "[pre-flight] no supported devices detected; running unit tier only."
 	echo
-	exec "$VENV_PY" -m pytest tests/unit -q -r fE --report-log=tests/reportlog.jsonl
+	exec "$VENV_PY" -m pytest tests/unit -v --report-log=tests/reportlog.jsonl
 fi
 
 # Default pytest args when the user passed none. Power users can invoke
@@ -212,13 +210,11 @@ fi
 # skipping half the hardware tests with "not baked with session profile"
 # errors. Power users who know their hardware is current and want to shave
 # those seconds can pass `--assume-baked` explicitly.
-# Defaults also use condensed reporting (`-q -r fE`) to avoid listing every
-# skipped test verbatim while still surfacing failures/errors and summary data.
 if [[ $# -eq 0 ]]; then
 	set -- tests/ \
 		--html=tests/report.html --self-contained-html \
 		--junitxml=tests/junit.xml \
-		-q -r fE --tb=short
+		-v --tb=short
 fi
 
 # UI tier requires opencv-python-headless (and ideally easyocr). If it's

mcp-server/scripts/datadog-dashboard.json

@@ -1,217 +0,0 @@
-{
-  "title": "Meshtastic Firmware — Recorder Stream",
-  "description": "Live view of `.mtlog/` streams shipped by `mtlog_to_datadog.py`. Heap, packet volume, log levels, errors. One row per port.",
-  "widgets": [
-    {
-      "definition": {
-        "title": "Free heap (bytes)",
-        "type": "timeseries",
-        "show_legend": true,
-        "requests": [
-          {
-            "queries": [
-              {
-                "name": "free_heap",
-                "data_source": "metrics",
-                "query": "avg:mesh.local.heap_free_bytes{service:meshtastic-firmware} by {port}"
-              }
-            ],
-            "response_format": "timeseries",
-            "display_type": "line"
-          }
-        ],
-        "yaxis": { "label": "bytes" }
-      }
-    },
-    {
-      "definition": {
-        "title": "Heap slope (bytes/min) — last 1h",
-        "type": "query_value",
-        "precision": 0,
-        "requests": [
-          {
-            "queries": [
-              {
-                "name": "slope",
-                "data_source": "metrics",
-                "query": "derivative(avg:mesh.local.heap_free_bytes{service:meshtastic-firmware})",
-                "aggregator": "avg"
-              }
-            ],
-            "response_format": "scalar"
-          }
-        ],
-        "conditional_formats": [
-          { "comparator": "<", "value": -100, "palette": "white_on_red" },
-          { "comparator": "<", "value": 0, "palette": "white_on_yellow" },
-          { "comparator": ">=", "value": 0, "palette": "white_on_green" }
-        ]
-      }
-    },
-    {
-      "definition": {
-        "title": "Total heap (bytes)",
-        "type": "timeseries",
-        "requests": [
-          {
-            "queries": [
-              {
-                "name": "total_heap",
-                "data_source": "metrics",
-                "query": "avg:mesh.local.heap_total_bytes{service:meshtastic-firmware} by {port}"
-              }
-            ],
-            "response_format": "timeseries",
-            "display_type": "line"
-          }
-        ]
-      }
-    },
-    {
-      "definition": {
-        "title": "Battery level (%)",
-        "type": "timeseries",
-        "requests": [
-          {
-            "queries": [
-              {
-                "name": "battery",
-                "data_source": "metrics",
-                "query": "avg:mesh.device.battery_level{service:meshtastic-firmware} by {port}"
-              }
-            ],
-            "response_format": "timeseries",
-            "display_type": "line"
-          }
-        ],
-        "yaxis": { "min": "0", "max": "105" }
-      }
-    },
-    {
-      "definition": {
-        "title": "Air utilization (TX %)",
-        "type": "timeseries",
-        "requests": [
-          {
-            "queries": [
-              {
-                "name": "airutil",
-                "data_source": "metrics",
-                "query": "avg:mesh.device.air_util_tx{service:meshtastic-firmware} by {port}"
-              }
-            ],
-            "response_format": "timeseries",
-            "display_type": "line"
-          }
-        ]
-      }
-    },
-    {
-      "definition": {
-        "title": "Channel utilization (%)",
-        "type": "timeseries",
-        "requests": [
-          {
-            "queries": [
-              {
-                "name": "chutil",
-                "data_source": "metrics",
-                "query": "avg:mesh.device.channel_utilization{service:meshtastic-firmware} by {port}"
-              }
-            ],
-            "response_format": "timeseries",
-            "display_type": "line"
-          }
-        ]
-      }
-    },
-    {
-      "definition": {
-        "title": "Log volume by level",
-        "type": "timeseries",
-        "show_legend": true,
-        "requests": [
-          {
-            "response_format": "timeseries",
-            "display_type": "bars",
-            "queries": [
-              {
-                "name": "log_count",
-                "data_source": "logs",
-                "indexes": ["*"],
-                "compute": { "aggregation": "count" },
-                "search": { "query": "service:meshtastic-firmware" },
-                "group_by": [
-                  {
-                    "facet": "@level",
-                    "limit": 10,
-                    "sort": { "order": "desc", "aggregation": "count" }
-                  }
-                ]
-              }
-            ]
-          }
-        ]
-      }
-    },
-    {
-      "definition": {
-        "title": "Recent ERROR / CRIT firmware logs",
-        "type": "list_stream",
-        "requests": [
-          {
-            "response_format": "event_list",
-            "query": {
-              "data_source": "logs_stream",
-              "query_string": "service:meshtastic-firmware (status:error OR @level:ERROR OR @level:CRIT)",
-              "indexes": [],
-              "sort": { "column": "timestamp", "order": "desc" }
-            },
-            "columns": [
-              { "field": "timestamp", "width": "auto" },
-              { "field": "host", "width": "auto" },
-              { "field": "@port", "width": "auto" },
-              { "field": "@level", "width": "auto" },
-              { "field": "@thread", "width": "auto" },
-              { "field": "message", "width": "stretch" }
-            ]
-          }
-        ]
-      }
-    },
-    {
-      "definition": {
-        "title": "Recorder marker events",
-        "type": "list_stream",
-        "requests": [
-          {
-            "response_format": "event_list",
-            "query": {
-              "data_source": "logs_stream",
-              "query_string": "service:meshtastic-firmware @level:MARK",
-              "indexes": [],
-              "sort": { "column": "timestamp", "order": "desc" }
-            },
-            "columns": [
-              { "field": "timestamp", "width": "auto" },
-              { "field": "host", "width": "auto" },
-              { "field": "message", "width": "stretch" }
-            ]
-          }
-        ]
-      }
-    }
-  ],
-  "template_variables": [
-    {
-      "name": "port",
-      "prefix": "port",
-      "available_values": [],
-      "default": "*"
-    },
-    { "name": "host", "prefix": "host", "available_values": [], "default": "*" }
-  ],
-  "layout_type": "ordered",
-  "notify_list": [],
-  "reflow_type": "auto"
-}

mcp-server/scripts/mtlog_to_datadog.py

@@ -1,389 +0,0 @@
-#!/usr/bin/env python3
-"""Forward selected recorder JSONL streams to Datadog.
-
-Reads `.mtlog/logs.jsonl` and `.mtlog/telemetry.jsonl`, ships logs to the
-Logs Intake API and telemetry numerics to the Metrics v2 series API.
-Resumes from `.mtlog/.dd-cursor.json` so a daemon restart doesn't
-duplicate rows already shipped from the current live files.
-
-This forwarder does not currently backfill rotated `.jsonl.gz` archives.
-If the recorder rotates before this process drains the live file, or the
-forwarder is down across a rotation, those older rows are skipped.
-
-Usage:
-    DD_API_KEY=... ./scripts/mtlog_to_datadog.py --tail
-    ./scripts/mtlog_to_datadog.py --once          # catch up + exit
-    ./scripts/mtlog_to_datadog.py --since 3600   # backfill last hour from start
-
-Default `DD_SITE` is `us5.datadoghq.com` — the team's Datadog instance.
-Override via `DD_SITE=...` env var or `--site` flag for one-offs.
-
-The forwarder is a separate process by design — a Datadog outage or
-auth failure must not backpressure the recorder. We exit non-zero on
-fatal config errors (missing API key) and keep retrying on transient
-network/HTTP errors.
-"""
-
-from __future__ import annotations
-
-import argparse
-import json
-import os
-import socket
-import sys
-import time
-from pathlib import Path
-from typing import Any, Iterator
-
-try:
-    import requests
-except ImportError:
-    print(
-        "requests is required. Install it in the mcp-server venv: "
-        "uv pip install requests",
-        file=sys.stderr,
-    )
-    sys.exit(2)
-
-
-_DEFAULT_LOG_DIR = Path(__file__).resolve().parents[1] / ".mtlog"
-_LOG_INTAKE_TPL = "https://http-intake.logs.{site}/api/v2/logs"
-_METRICS_TPL = "https://api.{site}/api/v2/series"
-_LOG_BATCH = 50
-_METRICS_BATCH = 100
-_MAX_RETRIES = 5
-_RETRY_BASE_S = 1.5
-
-
-# --- streaming JSONL with byte-position cursor -------------------------
-
-
-class _StreamReader:
-    """Reads a single rotating JSONL with cursor-based resume.
-
-    This tails only the live `.jsonl` file. The recorder rotates files
-    (live `.jsonl` → `.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`), which means
-    the live file shrinks abruptly. We detect that via inode change OR live
-    size < cursor position, and reset the live-file cursor to 0.
-    """
-
-    def __init__(self, path: Path, cursor: dict[str, Any]):
-        self.path = path
-        self.cursor = cursor
-
-    def _state(self) -> tuple[int, int]:
-        """Return (inode, size) for the live file. (0, 0) if missing."""
-        try:
-            st = self.path.stat()
-            return (st.st_ino, st.st_size)
-        except FileNotFoundError:
-            return (0, 0)
-
-    def iter_new_records(self) -> Iterator[dict[str, Any]]:
-        ino, size = self._state()
-        last_ino = self.cursor.get("ino")
-        last_pos = int(self.cursor.get("pos") or 0)
-        if ino == 0:
-            return
-        if last_ino is not None and last_ino != ino:
-            # Rotation happened. Start over.
-            last_pos = 0
-        if last_pos > size:
-            # Live file truncated/shrunk under us — recorder rotated.
-            last_pos = 0
-        try:
-            with self.path.open("r", encoding="utf-8") as fh:
-                fh.seek(last_pos)
-                for line in fh:
-                    line = line.rstrip("\n")
-                    if not line:
-                        continue
-                    try:
-                        yield json.loads(line)
-                    except json.JSONDecodeError:
-                        continue
-                    last_pos = fh.tell()
-        except FileNotFoundError:
-            return
-        self.cursor["ino"] = ino
-        self.cursor["pos"] = last_pos
-
-
-def _load_cursor(path: Path) -> dict[str, Any]:
-    if not path.exists():
-        return {}
-    try:
-        return json.loads(path.read_text())
-    except (OSError, json.JSONDecodeError):
-        return {}
-
-
-def _save_cursor(path: Path, data: dict[str, Any]) -> None:
-    tmp = path.with_suffix(".json.tmp")
-    tmp.write_text(json.dumps(data, separators=(",", ":")))
-    tmp.replace(path)
-
-
-# --- Datadog clients ---------------------------------------------------
-
-
-class _DDSession:
-    """Pool one HTTPS session, share retry logic."""
-
-    def __init__(self, api_key: str, site: str, hostname: str) -> None:
-        self.api_key = api_key
-        self.site = site
-        self.hostname = hostname
-        self.session = requests.Session()
-        self.session.headers.update(
-            {
-                "DD-API-KEY": api_key,
-                "Content-Type": "application/json",
-            }
-        )
-
-    def _post(self, url: str, payload: Any) -> bool:
-        for attempt in range(_MAX_RETRIES):
-            try:
-                resp = self.session.post(url, json=payload, timeout=30)
-            except requests.RequestException as e:
-                _wait_retry(attempt, f"network error: {e}")
-                continue
-            if 200 <= resp.status_code < 300:
-                return True
-            if resp.status_code in (408, 429, 500, 502, 503, 504):
-                _wait_retry(
-                    attempt,
-                    f"HTTP {resp.status_code} retrying",
-                )
-                continue
-            print(
-                f"datadog refused: {resp.status_code} {resp.text[:200]}",
-                file=sys.stderr,
-            )
-            return False
-        return False
-
-    def send_logs(self, records: list[dict[str, Any]]) -> int:
-        if not records:
-            return 0
-        url = _LOG_INTAKE_TPL.format(site=self.site)
-        sent = 0
-        for i in range(0, len(records), _LOG_BATCH):
-            batch = records[i : i + _LOG_BATCH]
-            if self._post(url, batch):
-                sent += len(batch)
-        return sent
-
-    def send_metrics(self, series: list[dict[str, Any]]) -> int:
-        if not series:
-            return 0
-        url = _METRICS_TPL.format(site=self.site)
-        sent = 0
-        for i in range(0, len(series), _METRICS_BATCH):
-            batch = series[i : i + _METRICS_BATCH]
-            if self._post(url, {"series": batch}):
-                sent += len(batch)
-        return sent
-
-
-def _wait_retry(attempt: int, reason: str) -> None:
-    wait = _RETRY_BASE_S * (2**attempt)
-    print(
-        f"  retry {attempt + 1}/{_MAX_RETRIES} in {wait:.1f}s ({reason})",
-        file=sys.stderr,
-    )
-    time.sleep(wait)
-
-
-# --- record → datadog payload ------------------------------------------
-
-
-def _log_record_to_dd(rec: dict[str, Any], host: str) -> dict[str, Any]:
-    line = rec.get("line") or ""
-    tags = [
-        f"role:{rec.get('role')}",
-        f"port:{rec.get('port')}",
-    ]
-    level = rec.get("level")
-    if level:
-        tags.append(f"level:{level}")
-    tag = rec.get("tag")
-    if tag:
-        tags.append(f"thread:{tag}")
-    return {
-        "ddsource": "meshtastic-firmware",
-        "service": "meshtastic-firmware",
-        "hostname": host,
-        "message": line,
-        "ddtags": ",".join(t for t in tags if t and "None" not in t),
-        "timestamp": int((rec.get("ts") or time.time()) * 1000),
-        "level": level,
-    }
-
-
-def _telemetry_record_to_metrics(
-    rec: dict[str, Any], host: str
-) -> list[dict[str, Any]]:
-    fields = rec.get("fields") or {}
-    if not isinstance(fields, dict):
-        return []
-    variant = rec.get("variant") or "unknown"
-    ts = int(rec.get("ts") or time.time())
-    out: list[dict[str, Any]] = []
-    tags = []
-    if rec.get("port"):
-        tags.append(f"port:{rec['port']}")
-    if rec.get("role"):
-        tags.append(f"role:{rec['role']}")
-    if rec.get("from_node"):
-        tags.append(f"from_node:{rec['from_node']}")
-    tags.append(f"variant:{variant}")
-    for field, value in fields.items():
-        if not isinstance(value, (int, float)) or isinstance(value, bool):
-            continue
-        metric = f"mesh.{variant}.{_metric_safe(field)}"
-        out.append(
-            {
-                "metric": metric,
-                "type": 3,  # GAUGE
-                "points": [{"timestamp": ts, "value": float(value)}],
-                "tags": tags,
-                "resources": [{"type": "host", "name": host}],
-            }
-        )
-    return out
-
-
-def _metric_safe(name: str) -> str:
-    # Lowercase, replace non-alnum with underscore for safe metric names.
-    return "".join(c.lower() if c.isalnum() else "_" for c in name)
-
-
-# --- main loop ---------------------------------------------------------
-
-
-def run(
-    log_dir: Path,
-    *,
-    once: bool,
-    since_seconds: float | None,
-    poll_interval: float,
-    dd: _DDSession,
-) -> int:
-    cursor_path = log_dir / ".dd-cursor.json"
-    cursors = _load_cursor(cursor_path)
-
-    # `--since` overrides cursor: rewind to (now-since) timestamp.
-    # We can't seek by timestamp directly (cursor is byte position), so
-    # we just reset cursors to 0 and let the time filter in iter_new
-    # drop older records.
-    cutoff_ts: float | None = None
-    if since_seconds is not None:
-        cursors = {}
-        cutoff_ts = time.time() - since_seconds
-
-    sent_total = {"logs": 0, "telemetry": 0}
-
-    while True:
-        # logs.jsonl → DD logs
-        log_cursor = cursors.setdefault("logs", {})
-        log_batch: list[dict[str, Any]] = []
-        for rec in _StreamReader(log_dir / "logs.jsonl", log_cursor).iter_new_records():
-            if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
-                continue
-            log_batch.append(_log_record_to_dd(rec, dd.hostname))
-        if log_batch:
-            n = dd.send_logs(log_batch)
-            sent_total["logs"] += n
-            print(f"logs: sent {n}/{len(log_batch)}")
-
-        # telemetry.jsonl → DD metrics
-        telem_cursor = cursors.setdefault("telemetry", {})
-        metric_series: list[dict[str, Any]] = []
-        for rec in _StreamReader(
-            log_dir / "telemetry.jsonl", telem_cursor
-        ).iter_new_records():
-            if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
-                continue
-            metric_series.extend(_telemetry_record_to_metrics(rec, dd.hostname))
-        if metric_series:
-            n = dd.send_metrics(metric_series)
-            sent_total["telemetry"] += n
-            print(f"telemetry: sent {n}/{len(metric_series)} metric points")
-
-        _save_cursor(cursor_path, cursors)
-
-        if once:
-            print(f"done. logs={sent_total['logs']} metrics={sent_total['telemetry']}")
-            return 0
-        time.sleep(poll_interval)
-
-
-def main(argv: list[str] | None = None) -> int:
-    parser = argparse.ArgumentParser(description=__doc__)
-    parser.add_argument(
-        "--log-dir",
-        default=str(_DEFAULT_LOG_DIR),
-        help="Path to .mtlog/ (default: mcp-server/.mtlog)",
-    )
-    mode = parser.add_mutually_exclusive_group()
-    mode.add_argument("--once", action="store_true", help="Catch up then exit")
-    mode.add_argument(
-        "--tail",
-        action="store_true",
-        help="Daemon: poll forever (default)",
-    )
-    parser.add_argument(
-        "--since",
-        type=float,
-        default=None,
-        help="Backfill last N seconds. Resets cursor.",
-    )
-    parser.add_argument(
-        "--poll-interval",
-        type=float,
-        default=5.0,
-        help="Seconds between tail polls (default 5)",
-    )
-    parser.add_argument(
-        "--site",
-        default=os.environ.get("DD_SITE", "us5.datadoghq.com"),
-        help=(
-            "Datadog site. Default is the team's instance (us5.datadoghq.com). "
-            "Override via DD_SITE env var or this flag."
-        ),
-    )
-    parser.add_argument(
-        "--host",
-        default=socket.gethostname(),
-        help="Hostname tag (default: socket.gethostname())",
-    )
-    args = parser.parse_args(argv)
-
-    api_key = os.environ.get("DD_API_KEY")
-    if not api_key:
-        print("DD_API_KEY env var required.", file=sys.stderr)
-        return 2
-
-    log_dir = Path(args.log_dir)
-    if not log_dir.exists():
-        print(
-            f"log dir {log_dir} does not exist — start the mcp-server first.",
-            file=sys.stderr,
-        )
-        return 2
-
-    dd = _DDSession(api_key=api_key, site=args.site, hostname=args.host)
-    once = args.once and not args.tail
-    return run(
-        log_dir,
-        once=once,
-        since_seconds=args.since,
-        poll_interval=args.poll_interval,
-        dd=dd,
-    )
-
-
-if __name__ == "__main__":
-    sys.exit(main())

mcp-server/src/meshtastic_mcp/fixtures.py

@@ -1,382 +0,0 @@
-"""Fake NodeDB fixture push — Portduino file copy + hardware XModem upload.
-
-The fixture pipeline is two-stage:
-  1. `bin/gen-fake-nodedb-seed.py` produces a deterministic JSONL describing N
-     fake-but-realistic peers. Committed under `test/fixtures/nodedb/`.
-  2. `bin/seed-json-to-proto.py` compiles JSONL → binary v25 NodeDatabase
-     protobuf with fresh wall-clock timestamps.
-
-This module exposes `push_fake_nodedb(...)`, the MCP tool that:
-  - target="portduino": compiles the JSONL into the device's prefs dir on
-    the local filesystem (`~/.portduino/<config>/prefs/nodes.proto`).
-  - target="hardware":  compiles to a temp file, then streams it over the
-    XModem protocol (via the meshtastic SerialInterface/BLEInterface +
-    `meshtastic.xmodempacket` pubsub topic) to `/prefs/nodes.proto` on the
-    device. Triggers a reboot so the firmware loads the new state on next
-    boot.
-
-XModem wire details (mirrors firmware impl at src/xmodem.cpp:115-260):
-  * 128-byte chunks; final chunk padded to 128 B with 0x1A (SUB) bytes.
-  * CRC16-CCITT (poly 0x1021, init 0x0000).
-  * SOH/seq=0 carries the destination filename in `buffer.bytes`. ACK if
-    `FSCom.open(filename, FILE_O_WRITE)` succeeds; NAK otherwise.
-  * SOH/seq≥1 carries a 128-byte chunk. ACK = advance; NAK = retransmit.
-  * EOT after the last chunk flushes + closes the file on-device.
-
-Hardware push requires `confirm=True` (mirrors factory_reset / erase_and_flash
-in the .github/copilot-instructions.md "never do these without asking" list).
-"""
-
-from __future__ import annotations
-
-import dataclasses
-import hashlib
-import pathlib
-import queue
-import shutil
-import subprocess
-import sys
-import tempfile
-import time
-from typing import Any, Literal
-
-from .connection import connect, is_tcp_port
-
-# Resolve repo root so the tool works regardless of mcp-server cwd.
-_REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
-_SEED_DIR = _REPO_ROOT / "test" / "fixtures" / "nodedb"
-_COMPILE_SCRIPT = _REPO_ROOT / "bin" / "seed-json-to-proto.py"
-
-_DEFAULT_NODES_FILENAME = "/prefs/nodes.proto"
-_XMODEM_CHUNK = 128
-_XMODEM_SUB = 0x1A
-_ACK_TIMEOUT_INIT_S = 5.0
-_ACK_TIMEOUT_CHUNK_S = 2.0
-_MAX_CHUNK_RETRIES = 5
-
-_VALID_SIZES = (250, 500, 1000, 2000)
-
-
-class FixtureError(RuntimeError):
-    """Raised for any fixture-push failure (compile, transport, ack timeout, …)."""
-
-
-# ---------------------------------------------------------------------------
-# CRC16-CCITT (poly 0x1021, init 0x0000). Matches the firmware's `crc16_ccitt`.
-# Hand-rolled to avoid the optional `crcmod` dep.
-# ---------------------------------------------------------------------------
-def _crc16_ccitt(data: bytes, *, init: int = 0x0000) -> int:
-    crc = init
-    for b in data:
-        crc ^= b << 8
-        for _ in range(8):
-            if crc & 0x8000:
-                crc = ((crc << 1) ^ 0x1021) & 0xFFFF
-            else:
-                crc = (crc << 1) & 0xFFFF
-    return crc
-
-
-# ---------------------------------------------------------------------------
-# Compile step — shells out to bin/seed-json-to-proto.py so the MCP module
-# doesn't have to duplicate the proto-encoding logic.
-# ---------------------------------------------------------------------------
-def _compile_proto(jsonl_path: pathlib.Path, out_path: pathlib.Path) -> None:
-    if not _COMPILE_SCRIPT.is_file():
-        raise FixtureError(f"compile script missing at {_COMPILE_SCRIPT}")
-    cmd = [
-        sys.executable,
-        str(_COMPILE_SCRIPT),
-        "--in",
-        str(jsonl_path),
-        "--out",
-        str(out_path),
-    ]
-    try:
-        subprocess.run(cmd, check=True, capture_output=True, text=True)
-    except subprocess.CalledProcessError as exc:
-        raise FixtureError(
-            f"seed-json-to-proto.py failed (exit {exc.returncode}):\n"
-            f"  stdout: {exc.stdout}\n  stderr: {exc.stderr}"
-        ) from exc
-
-
-def _resolve_seed_jsonl(size: int, custom: str | None) -> pathlib.Path:
-    if custom is not None:
-        p = pathlib.Path(custom).expanduser().resolve()
-        if not p.is_file():
-            raise FixtureError(f"custom_seed_jsonl not found: {p}")
-        return p
-    p = _SEED_DIR / f"seed_v25_{size:04d}.jsonl"
-    if not p.is_file():
-        raise FixtureError(
-            f"missing committed seed at {p}. "
-            f"Run `./bin/regen-fake-nodedbs.sh` to generate it."
-        )
-    return p
-
-
-# ---------------------------------------------------------------------------
-# Portduino push — file copy into ~/.portduino/<config>/prefs/
-# ---------------------------------------------------------------------------
-def _portduino_prefs_dir(config_name: str) -> pathlib.Path:
-    home = pathlib.Path.home()
-    return home / ".portduino" / config_name / "prefs"
-
-
-def _push_portduino(
-    size: int,
-    jsonl: pathlib.Path,
-    portduino_config: str,
-    backup_existing: bool,
-) -> dict[str, Any]:
-    prefs = _portduino_prefs_dir(portduino_config)
-    prefs.mkdir(parents=True, exist_ok=True)
-    target = prefs / "nodes.proto"
-    backed_up_to: str | None = None
-    if backup_existing and target.is_file():
-        ts = int(time.time())
-        backup = prefs / f"nodes.proto.bak.{ts}"
-        shutil.move(str(target), str(backup))
-        backed_up_to = str(backup)
-    _compile_proto(jsonl, target)
-    raw = target.read_bytes()
-    return {
-        "transport": "portduino",
-        "path": str(target),
-        "bytes": len(raw),
-        "sha256": hashlib.sha256(raw).hexdigest(),
-        "jsonl_source": str(jsonl),
-        "backed_up_to": backed_up_to,
-    }
-
-
-# ---------------------------------------------------------------------------
-# Hardware push — XModem over BLE/serial via the meshtastic Python interface.
-# ---------------------------------------------------------------------------
-@dataclasses.dataclass
-class _AckEvent:
-    control: int
-    seq: int
-
-
-def _wait_for_response(q: "queue.Queue[_AckEvent]", timeout_s: float) -> _AckEvent:
-    try:
-        return q.get(timeout=timeout_s)
-    except queue.Empty as exc:
-        raise FixtureError(
-            f"XModem response timeout after {timeout_s:.1f}s — device not responding"
-        ) from exc
-
-
-def _push_hardware(
-    size: int,
-    jsonl: pathlib.Path,
-    port: str | None,
-    reboot_after: bool,
-) -> dict[str, Any]:
-    # Lazy imports so the module loads even when the meshtastic deps aren't
-    # available (e.g. CI in a Python env without the package installed).
-    try:
-        from meshtastic.protobuf import mesh_pb2, xmodem_pb2
-        from pubsub import pub
-    except ImportError as exc:  # pragma: no cover — dep missing
-        raise FixtureError(
-            f"hardware push requires the meshtastic + pypubsub packages: {exc}"
-        ) from exc
-
-    if is_tcp_port(port):
-        raise FixtureError(
-            "hardware push over TCP/portduino is not supported — use "
-            "target='portduino' to drop the fixture directly into the prefs dir."
-        )
-
-    # Compile the fixture to a temp file with fresh timestamps.
-    with tempfile.NamedTemporaryFile(suffix=".proto", delete=False) as tf:
-        proto_path = pathlib.Path(tf.name)
-    try:
-        _compile_proto(jsonl, proto_path)
-        payload = proto_path.read_bytes()
-    finally:
-        proto_path.unlink(missing_ok=True)
-
-    sha256 = hashlib.sha256(payload).hexdigest()
-    total_bytes = len(payload)
-
-    # Subscribe to XModem responses BEFORE we open the interface, so we don't
-    # race the first ACK that arrives during the SOH/seq=0 handshake.
-    #
-    # NB: the signature MUST declare every kwarg pypubsub will see for this
-    # topic, or pubsub locks the topic spec to a smaller set (whichever
-    # subscribe arrives first) and then *rejects* the meshtastic library's
-    # publish call with `SenderUnknownMsgDataError: unknown ... interface`.
-    # The meshtastic lib publishes both `packet=` and `interface=`
-    # (mesh_interface.py:1389-1395), so both must appear here.
-    response_q: "queue.Queue[_AckEvent]" = queue.Queue()
-
-    def _on_xmodem(packet: Any = None, interface: Any = None, **_kw: Any) -> None:
-        if packet is None:
-            return
-        response_q.put(_AckEvent(control=int(packet.control), seq=int(packet.seq)))
-
-    pub.subscribe(_on_xmodem, "meshtastic.xmodempacket")
-
-    chunks_sent = 0
-    retried = 0
-    rebooted = False
-
-    XMC = xmodem_pb2.XModem.Control
-    try:
-        with connect(port=port) as iface:
-            # 1) Send the filename (SOH, seq=0).
-            init_pkt = xmodem_pb2.XModem(
-                control=XMC.Value("SOH"),
-                seq=0,
-                buffer=_DEFAULT_NODES_FILENAME.encode("utf-8"),
-            )
-            iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=init_pkt))
-            ack = _wait_for_response(response_q, _ACK_TIMEOUT_INIT_S)
-            if ack.control != XMC.Value("ACK"):
-                raise FixtureError(
-                    f"device refused filename {_DEFAULT_NODES_FILENAME!r} "
-                    f"(got control={ack.control}, expected ACK). "
-                    f"Filesystem full or permissions issue?"
-                )
-
-            # 2) Stream the payload in 128 B chunks.
-            for offset in range(0, total_bytes, _XMODEM_CHUNK):
-                chunk = payload[offset : offset + _XMODEM_CHUNK]
-                if len(chunk) < _XMODEM_CHUNK:
-                    # Pad final chunk to 128 B with SUB. The trailing 0x1A bytes
-                    # become part of the file on-device, but nanopb ignores
-                    # bytes past the end of the top-level message.
-                    chunk = chunk + bytes([_XMODEM_SUB] * (_XMODEM_CHUNK - len(chunk)))
-                seq = ((offset // _XMODEM_CHUNK) + 1) % 256
-                # Retry loop on NAK / timeout.
-                attempts = 0
-                while True:
-                    pkt = xmodem_pb2.XModem(
-                        control=XMC.Value("SOH"),
-                        seq=seq,
-                        buffer=chunk,
-                        crc16=_crc16_ccitt(chunk),
-                    )
-                    iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=pkt))
-                    ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
-                    if ack.control == XMC.Value("ACK"):
-                        chunks_sent += 1
-                        break
-                    if ack.control == XMC.Value("NAK"):
-                        attempts += 1
-                        retried += 1
-                        if attempts >= _MAX_CHUNK_RETRIES:
-                            # Abort: send CAN so the firmware removes the half-
-                            # written file via FSCom.remove(filename).
-                            iface._sendToRadio(
-                                mesh_pb2.ToRadio(
-                                    xmodemPacket=xmodem_pb2.XModem(
-                                        control=XMC.Value("CAN")
-                                    )
-                                )
-                            )
-                            raise FixtureError(
-                                f"chunk seq={seq} NAK'd {attempts} times; "
-                                f"aborted transfer (file removed on-device)."
-                            )
-                        continue  # retry the same chunk
-                    raise FixtureError(
-                        f"unexpected XModem control={ack.control} on seq={seq}"
-                    )
-
-            # 3) Tell the device we're done.
-            iface._sendToRadio(
-                mesh_pb2.ToRadio(
-                    xmodemPacket=xmodem_pb2.XModem(control=XMC.Value("EOT"))
-                )
-            )
-            ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
-            if ack.control != XMC.Value("ACK"):
-                raise FixtureError(f"EOT not ACKed (got control={ack.control})")
-
-            # 4) Reboot so loadFromDisk picks up the new file.
-            if reboot_after:
-                iface.localNode.reboot(secs=1)
-                rebooted = True
-    finally:
-        try:
-            pub.unsubscribe(_on_xmodem, "meshtastic.xmodempacket")
-        except Exception:
-            pass
-
-    return {
-        "transport": "hardware",
-        "port": port,
-        "filename_on_device": _DEFAULT_NODES_FILENAME,
-        "bytes": total_bytes,
-        "chunks_sent": chunks_sent,
-        "retried": retried,
-        "sha256": sha256,
-        "jsonl_source": str(jsonl),
-        "rebooted": rebooted,
-    }
-
-
-# ---------------------------------------------------------------------------
-# Public entry point — registered as an MCP tool in server.py.
-# ---------------------------------------------------------------------------
-def push_fake_nodedb(
-    size: int,
-    target: Literal["portduino", "hardware"] = "portduino",
-    *,
-    port: str | None = None,
-    portduino_config: str = "default",
-    backup_existing: bool = True,
-    confirm: bool = False,
-    reboot_after: bool = True,
-    custom_seed_jsonl: str | None = None,
-) -> dict[str, Any]:
-    """Compile a fresh-timestamp NodeDatabase fixture and push it to a device.
-
-    Args:
-      size: 250, 500, 1000, or 2000 — selects which committed seed JSONL to use.
-      target: "portduino" (file copy to ~/.portduino/<config>/prefs/) or
-              "hardware" (XModem upload to /prefs/nodes.proto + reboot).
-      port: required for target="hardware". Serial path (e.g. /dev/cu.usbmodemXXXX)
-            or BLE identifier. TCP endpoints are rejected — use target="portduino"
-            instead.
-      portduino_config: which Portduino instance dir under ~/.portduino/. Default "default".
-      backup_existing: portduino only. Move nodes.proto -> nodes.proto.bak.<ts>
-                        if present, so you can roll back.
-      confirm: required True for target="hardware" (writes flash + reboots).
-      reboot_after: hardware only. If True, send a 1-second reboot after the
-                    final ACK so loadFromDisk picks up the new file at next boot.
-      custom_seed_jsonl: override the committed JSONL. Use to push a hand-edited
-                         test scenario.
-
-    Returns:
-        dict with transport, bytes, sha256, etc. — depends on target.
-
-    """
-    if size not in _VALID_SIZES:
-        raise FixtureError(
-            f"size must be one of {_VALID_SIZES}; got {size!r}. "
-            f"Add a new committed seed if you need a different cardinality."
-        )
-
-    jsonl = _resolve_seed_jsonl(size, custom_seed_jsonl)
-
-    if target == "portduino":
-        return _push_portduino(size, jsonl, portduino_config, backup_existing)
-
-    if target == "hardware":
-        if not confirm:
-            raise FixtureError(
-                "hardware push writes flash and triggers a reboot — pass confirm=True."
-            )
-        if not port:
-            raise FixtureError(
-                "target='hardware' requires a port (e.g. /dev/cu.usbmodemXXXX)."
-            )
-        return _push_hardware(size, jsonl, port, reboot_after)
-
-    raise FixtureError(f"unknown target {target!r}; expected 'portduino' or 'hardware'")

mcp-server/src/meshtastic_mcp/flash.py

@@ -108,33 +108,18 @@ def build(
     env: str,
     with_manifest: bool = True,
     userprefs_overrides: dict[str, Any] | None = None,
-    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Run `pio run -e <env>` and return artifact paths.
 
     `userprefs_overrides` (optional): dict of `USERPREFS_<KEY>: value` to inject
     into userPrefs.jsonc for this build only. File is restored byte-for-byte
     on exit. Use `userprefs_set()` for persistent changes.
-
-    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros to set for
-    this build only via `PLATFORMIO_BUILD_FLAGS`. Common useful flag:
-    `{"DEBUG_HEAP": 1}` enables per-thread leak detection + `[heap N]`
-    prefix on every log line. Combines with the recorder so heap shows
-    up at log cadence (much higher resolution than the ~60 s LocalStats
-    packet) — see `recorder/parsers.py:_HEAP_PREFIX_RE`. Bool values
-    expand to bare `-D<NAME>` (presence-only flags).
     """
     args = ["run", "-e", env]
     if with_manifest:
         args.extend(["-t", "mtjson"])
-    extra_env = _build_flags_env(build_flags) if build_flags else None
     with userprefs.temporary_overrides(userprefs_overrides) as effective:
-        result = pio.run(
-            args,
-            timeout=pio.TIMEOUT_BUILD,
-            check=False,
-            extra_env=extra_env,
-        )
+        result = pio.run(args, timeout=pio.TIMEOUT_BUILD, check=False)
     return {
         "exit_code": result.returncode,
         "artifacts": [str(p) for p in _artifacts_for(env)],
@@ -142,27 +127,9 @@ def build(
         "stderr_tail": pio.tail_lines(result.stderr, 200),
         "duration_s": round(result.duration_s, 2),
         "userprefs": _userprefs_summary(effective),
-        "build_flags": dict(build_flags) if build_flags else None,
     }
 
 
-def _build_flags_env(build_flags: dict[str, Any]) -> dict[str, str]:
-    """Translate `{"DEBUG_HEAP": 1, "FOO": "bar"}` → `{"PLATFORMIO_BUILD_FLAGS":
-    "-DDEBUG_HEAP=1 -DFOO=bar"}`. Bool True → bare `-D<NAME>`; False/None drop
-    the flag entirely. Other types stringify."""
-    parts: list[str] = []
-    for key, value in build_flags.items():
-        if value is False or value is None:
-            continue
-        if value is True:
-            parts.append(f"-D{key}")
-        else:
-            parts.append(f"-D{key}={value}")
-    if not parts:
-        return {}
-    return {"PLATFORMIO_BUILD_FLAGS": " ".join(parts)}
-
-
 def clean(env: str) -> dict[str, Any]:
     """Run `pio run -e <env> -t clean`."""
     result = pio.run(["run", "-e", env, "-t", "clean"], timeout=120, check=False)
@@ -179,29 +146,20 @@ def flash(
     port: str,
     confirm: bool = False,
     userprefs_overrides: dict[str, Any] | None = None,
-    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """`pio run -e <env> -t upload --upload-port <port>`. All architectures.
 
     `userprefs_overrides` (optional): see `build()` — the rebuild-before-upload
     that pio performs will pick up the injected values.
-
-    `build_flags` (optional): same shape as `build()` — `PLATFORMIO_BUILD_FLAGS`
-    is exported for the rebuild-before-upload, so the uploaded firmware
-    actually carries the flags. Without this propagation, `pio run -t upload`
-    would relink without the env var and silently drop them. Common use:
-    `build_flags={"DEBUG_HEAP": 1}` for the leak-hunt path.
     """
     _require_confirm(confirm, "flash")
     _reject_native_env(env, "flash")
     connection.reject_if_tcp(port, "flash")
-    extra_env = _build_flags_env(build_flags) if build_flags else None
     with userprefs.temporary_overrides(userprefs_overrides) as effective:
         result = pio.run(
             ["run", "-e", env, "-t", "upload", "--upload-port", port],
             timeout=pio.TIMEOUT_UPLOAD,
             check=False,
-            extra_env=extra_env,
         )
     return {
         "exit_code": result.returncode,
@@ -209,7 +167,6 @@ def flash(
         "stderr_tail": pio.tail_lines(result.stderr, 200),
         "duration_s": round(result.duration_s, 2),
         "userprefs": _userprefs_summary(effective),
-        "build_flags": dict(build_flags) if build_flags else None,
     }
 
 

mcp-server/src/meshtastic_mcp/log_query.py

@@ -1,410 +0,0 @@
-"""Read-side queries over the recorder's JSONL streams.
-
-Pure functions over `mcp-server/.mtlog/`. Streaming JSONL reader: never
-loads a whole file. Time-bound queries short-circuit as soon as `ts`
-exceeds the requested end. The recorder writes monotonically, so a
-forward scan is cheap; we don't need an index.
-
-All time arguments accept:
-  - epoch seconds (int/float)
-  - relative strings: "-15m", "-2h", "-3d", "now"
-  - ISO-ish absolute strings: "2026-05-07T14:30:00" (naive timestamps are
-    treated as UTC)
-
-Tools that return data ALWAYS cap their output (max_lines / max_points
-/ max), and report whether more matched than was returned.
-"""
-
-from __future__ import annotations
-
-import gzip
-import json
-import re
-import statistics
-import time
-from datetime import datetime, timezone
-from pathlib import Path
-from typing import Any, Iterator
-
-from .recorder.recorder import get_recorder
-
-_REL_RE = re.compile(r"^\s*-\s*(\d+(?:\.\d+)?)\s*([smhd])\s*$")
-_REGEX_PREVIEW_MAX = 100
-_REGEX_PREVIEW_TRUNCATE = 97
-
-
-def _parse_time(value: Any, *, now: float | None = None) -> float:
-    """Coerce to epoch seconds. Defaults `now` to `time.time()`."""
-    if value is None:
-        return time.time()
-    if isinstance(value, (int, float)):
-        return float(value)
-    if not isinstance(value, str):
-        raise ValueError(f"invalid time: {value!r}")
-    s = value.strip().lower()
-    if s in ("", "now"):
-        return time.time() if now is None else now
-    m = _REL_RE.match(s)
-    if m:
-        n = float(m.group(1))
-        unit = m.group(2)
-        secs = n * {"s": 1, "m": 60, "h": 3600, "d": 86400}[unit]
-        base = time.time() if now is None else now
-        return base - secs
-    # Try ISO 8601. Accept naive (assume UTC) and Z-suffixed.
-    try:
-        if s.endswith("z"):
-            s = s[:-1] + "+00:00"
-        dt = datetime.fromisoformat(s)
-        if dt.tzinfo is None:
-            dt = dt.replace(tzinfo=timezone.utc)
-        return dt.timestamp()
-    except ValueError as e:
-        raise ValueError(f"unparseable time: {value!r}") from e
-
-
-def _iter_jsonl(path: Path, *, since: float, until: float) -> Iterator[dict[str, Any]]:
-    """Stream records in chronological order: rotated archives first
-    (oldest → newest by lex sort, which is chronological for our
-    `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` archive naming), then the live file
-    last. The "keep last N" pop-front logic in the window queries
-    relies on records arriving in time order across files.
-    """
-    files: list[Path] = []
-    # Gzipped archives are named "<stem>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz".
-    for archive in sorted(path.parent.glob(f"{path.stem}.*.jsonl.gz")):
-        files.append(archive)
-    if path.exists():
-        files.append(path)
-    for f in files:
-        opener = gzip.open if f.suffix == ".gz" else open
-        try:
-            with opener(f, "rt", encoding="utf-8") as fh:  # type: ignore[arg-type]
-                for line in fh:
-                    line = line.strip()
-                    if not line:
-                        continue
-                    try:
-                        rec = json.loads(line)
-                    except json.JSONDecodeError:
-                        continue
-                    ts = rec.get("ts")
-                    if not isinstance(ts, (int, float)):
-                        continue
-                    if ts < since:
-                        continue
-                    if ts > until:
-                        # Records are append-monotonic within a file, so
-                        # the rest of this file is also past `until`.
-                        # Archives can still overlap each other, so only
-                        # short-circuit this file, not the whole scan.
-                        break
-                    yield rec
-        except (FileNotFoundError, OSError):
-            continue
-
-
-# -- queries ------------------------------------------------------------
-
-
-def logs_window(
-    start: Any = "-15m",
-    end: Any = "now",
-    *,
-    grep: str | None = None,
-    level: str | None = None,
-    tag: str | None = None,
-    port: str | None = None,
-    max_lines: int = 200,
-) -> dict[str, Any]:
-    """Recent firmware log lines, filtered.
-
-    `level` accepts a single level name or pipe-separated set
-    ("WARN|ERROR|CRIT"). `grep` is a regex (Python re) over the raw
-    `line` field. Returns the last `max_lines` matches.
-    """
-    s = _parse_time(start)
-    e = _parse_time(end)
-    levels = _split_set(level)
-    if grep:
-        try:
-            grep_re = re.compile(grep)
-        except re.error as exc:
-            preview = (
-                grep
-                if len(grep) <= _REGEX_PREVIEW_MAX
-                else f"{grep[:_REGEX_PREVIEW_TRUNCATE]}..."
-            )
-            raise ValueError(f"invalid grep regex {preview!r}: {exc}") from exc
-    else:
-        grep_re = None
-
-    base = get_recorder().base_dir
-    matched = 0
-    out: list[dict[str, Any]] = []
-    for rec in _iter_jsonl(base / "logs.jsonl", since=s, until=e):
-        if levels and rec.get("level") not in levels:
-            continue
-        if tag and rec.get("tag") != tag:
-            continue
-        if port and rec.get("port") != port:
-            continue
-        if grep_re and not grep_re.search(rec.get("line") or ""):
-            continue
-        matched += 1
-        out.append(rec)
-        if len(out) > max_lines:
-            out.pop(0)  # keep the most recent N
-    return {
-        "lines": out,
-        "total_matched": matched,
-        "dropped": max(0, matched - max_lines),
-        "window": {"start": s, "end": e},
-    }
-
-
-def telemetry_timeline(
-    window: Any = "1h",
-    *,
-    variant: str = "local",
-    field: str = "free_heap",
-    port: str | None = None,
-    max_points: int = 200,
-) -> dict[str, Any]:
-    """Timeseries of one telemetry field, downsampled.
-
-    `field` matches both the protobuf snake_case name (`free_heap`,
-    `heap_free_bytes`, `battery_level`) and camelCase (`freeHeap`).
-    Server-side bucket-mean downsamples to ≤ `max_points`. Returns
-    `slope_per_min` (linear regression slope, units/min) so a leak
-    detector can read one number.
-    """
-    end = time.time()
-    if isinstance(window, (int, float)):
-        # Numeric `window` is a duration in seconds — "last N seconds".
-        # Without this branch, `_parse_time(-N)` would treat -N as an
-        # absolute epoch timestamp (i.e., Jan 1 1970 minus N seconds),
-        # producing a wildly negative `start` and matching nothing.
-        start = end - float(window)
-    elif isinstance(window, str) and not window.startswith("-"):
-        # Bare string like "1h" is sugar for "-1h".
-        start = _parse_time(f"-{window}", now=end)
-    else:
-        start = _parse_time(window, now=end)
-
-    base = get_recorder().base_dir
-    raw: list[tuple[float, float]] = []
-    field_aliases = _field_aliases(field)
-    for rec in _iter_jsonl(base / "telemetry.jsonl", since=start, until=end):
-        if rec.get("variant") != variant:
-            continue
-        if port and rec.get("port") != port:
-            continue
-        fields = rec.get("fields") or {}
-        value: Any = None
-        for alias in field_aliases:
-            if alias in fields:
-                value = fields[alias]
-                break
-        if not isinstance(value, (int, float)):
-            continue
-        raw.append((float(rec["ts"]), float(value)))
-
-    if not raw:
-        return {
-            "points": [],
-            "samples": 0,
-            "min": None,
-            "max": None,
-            "slope_per_min": None,
-            "window": {"start": start, "end": end, "variant": variant, "field": field},
-        }
-
-    points = _downsample(raw, max_points=max_points)
-    values = [v for _, v in raw]
-    return {
-        "points": [{"ts": ts, "value": v} for ts, v in points],
-        "samples": len(raw),
-        "min": min(values),
-        "max": max(values),
-        "slope_per_min": _slope_per_min(raw),
-        "window": {"start": start, "end": end, "variant": variant, "field": field},
-    }
-
-
-def packets_window(
-    start: Any = "-5m",
-    end: Any = "now",
-    *,
-    portnum: str | None = None,
-    from_node: str | None = None,
-    to_node: str | None = None,
-    max: int = 200,
-) -> dict[str, Any]:
-    s = _parse_time(start)
-    e = _parse_time(end)
-    portnums = _split_set(portnum)
-    base = get_recorder().base_dir
-    matched = 0
-    out: list[dict[str, Any]] = []
-    for rec in _iter_jsonl(base / "packets.jsonl", since=s, until=e):
-        if portnums and rec.get("portnum") not in portnums:
-            continue
-        if from_node and str(rec.get("from_node")) != str(from_node):
-            continue
-        if to_node and str(rec.get("to_node")) != str(to_node):
-            continue
-        matched += 1
-        out.append(rec)
-        if len(out) > max:
-            out.pop(0)
-    return {
-        "packets": out,
-        "total_matched": matched,
-        "dropped": matched - max if matched > max else 0,
-        "window": {"start": s, "end": e},
-    }
-
-
-def events_window(
-    start: Any = "-1h",
-    end: Any = "now",
-    *,
-    kind: str | None = None,
-    max: int = 200,
-) -> dict[str, Any]:
-    s = _parse_time(start)
-    e = _parse_time(end)
-    kinds = _split_set(kind)
-    base = get_recorder().base_dir
-    matched = 0
-    out: list[dict[str, Any]] = []
-    for rec in _iter_jsonl(base / "events.jsonl", since=s, until=e):
-        if kinds and rec.get("kind") not in kinds:
-            continue
-        matched += 1
-        out.append(rec)
-        if len(out) > max:
-            out.pop(0)
-    return {
-        "events": out,
-        "total_matched": matched,
-        "dropped": matched - max if matched > max else 0,
-        "window": {"start": s, "end": e},
-    }
-
-
-def export(
-    start: Any,
-    end: Any,
-    dest_dir: str,
-    *,
-    streams: list[str] | None = None,
-) -> dict[str, Any]:
-    """Bundle a slice of each requested stream into `dest_dir`.
-
-    For a notebook, a bug report, or a Datadog backfill. Output files
-    are uncompressed JSONL (callers gzip themselves if they want to).
-    """
-    s = _parse_time(start)
-    e = _parse_time(end)
-    selected = streams or ["logs", "telemetry", "packets", "events"]
-    dest = Path(dest_dir)
-    dest.mkdir(parents=True, exist_ok=True)
-
-    base = get_recorder().base_dir
-    paths: dict[str, str] = {}
-    for stream in selected:
-        src = base / f"{stream}.jsonl"
-        if not src.exists() and not list(base.glob(f"{stream}.*.jsonl.gz")):
-            continue
-        out_path = dest / f"{stream}.jsonl"
-        n = 0
-        with out_path.open("w", encoding="utf-8") as fh:
-            for rec in _iter_jsonl(src, since=s, until=e):
-                fh.write(json.dumps(rec, separators=(",", ":")) + "\n")
-                n += 1
-        paths[stream] = str(out_path)
-        paths[f"{stream}_count"] = str(n)
-    return {"dest_dir": str(dest), "paths": paths, "window": {"start": s, "end": e}}
-
-
-# -- helpers ------------------------------------------------------------
-
-
-def _split_set(value: str | None) -> set[str] | None:
-    if not value:
-        return None
-    return {v.strip() for v in value.split("|") if v.strip()}
-
-
-def _field_aliases(field: str) -> list[str]:
-    """Accept snake_case OR camelCase, plus a few legacy aliases."""
-    snake = field
-    camel = _snake_to_camel(field)
-    aliases = {snake, camel}
-    # Old protobuf fields (pre-LocalStats) used different names
-    legacy = {
-        "free_heap": ["free_heap", "freeHeap", "heap_free_bytes", "heapFreeBytes"],
-        "heap_free_bytes": [
-            "heap_free_bytes",
-            "heapFreeBytes",
-            "free_heap",
-            "freeHeap",
-        ],
-        "total_heap": ["total_heap", "totalHeap", "heap_total_bytes", "heapTotalBytes"],
-        "heap_total_bytes": [
-            "heap_total_bytes",
-            "heapTotalBytes",
-            "total_heap",
-            "totalHeap",
-        ],
-    }
-    if field in legacy:
-        aliases.update(legacy[field])
-    return list(aliases)
-
-
-def _snake_to_camel(name: str) -> str:
-    parts = name.split("_")
-    return parts[0] + "".join(p.title() for p in parts[1:])
-
-
-def _downsample(
-    points: list[tuple[float, float]], *, max_points: int
-) -> list[tuple[float, float]]:
-    if len(points) <= max_points:
-        return points
-    # Even-bucket mean. Preserves shape better than nth-sample picking.
-    n = len(points)
-    bucket = n / max_points
-    out: list[tuple[float, float]] = []
-    i = 0
-    for k in range(max_points):
-        end = int((k + 1) * bucket)
-        end = min(end, n)
-        if end <= i:
-            continue
-        chunk = points[i:end]
-        ts = chunk[len(chunk) // 2][0]
-        val = statistics.fmean(v for _, v in chunk)
-        out.append((ts, val))
-        i = end
-    return out
-
-
-def _slope_per_min(points: list[tuple[float, float]]) -> float | None:
-    """Least-squares slope (units per minute). None if too few points."""
-    if len(points) < 2:
-        return None
-    xs = [t for t, _ in points]
-    ys = [v for _, v in points]
-    n = len(xs)
-    mean_x = sum(xs) / n
-    mean_y = sum(ys) / n
-    num = sum((xs[i] - mean_x) * (ys[i] - mean_y) for i in range(n))
-    den = sum((x - mean_x) ** 2 for x in xs)
-    if den == 0:
-        return None
-    slope_per_sec = num / den
-    return slope_per_sec * 60.0

mcp-server/src/meshtastic_mcp/pio.py

@@ -92,7 +92,6 @@ def _run_capturing(
     cwd: Path | None = None,
     timeout: float | None = None,
     tee_header: str | None = None,
-    extra_env: dict[str, str] | None = None,
 ) -> tuple[int, str, str, float]:
     """Run a subprocess, capture stdout+stderr, optionally tee to the flash log.
 
@@ -100,9 +99,6 @@ def _run_capturing(
     `subprocess.TimeoutExpired` on timeout (callers map this to their own
     domain-specific error).
 
-    `extra_env` merges into the subprocess environment (parent env stays
-    intact). Used for `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar.
-
     Fast path: `subprocess.run(capture_output=True)` when no flash log is
     configured (unchanged behavior).
 
@@ -114,9 +110,6 @@ def _run_capturing(
     """
     log_path = _flash_log_path()
     t0 = time.monotonic()
-    env = None
-    if extra_env:
-        env = {**os.environ, **extra_env}
 
     if log_path is None:
         # Fast path — unchanged.
@@ -126,7 +119,6 @@ def _run_capturing(
             capture_output=True,
             text=True,
             timeout=timeout,
-            env=env,
         )
         return (
             proc.returncode,
@@ -153,7 +145,6 @@ def _run_capturing(
         stderr=subprocess.PIPE,
         text=True,
         bufsize=1,  # line-buffered
-        env=env,
     )
     stdout_chunks: list[str] = []
     stderr_chunks: list[str] = []
@@ -241,17 +232,12 @@ def run(
     cwd: Path | None = None,
     timeout: float | None = TIMEOUT_DEFAULT,
     check: bool = True,
-    extra_env: dict[str, str] | None = None,
 ) -> PioResult:
     """Invoke `pio <args>` and return captured output.
 
     `cwd` defaults to the firmware root. `check=True` raises `PioError` on
     non-zero exit; set `check=False` to inspect `returncode` manually.
 
-    `extra_env` merges into the subprocess environment — used for
-    `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar build-time
-    toggles that can't be expressed as command-line args.
-
     If `MESHTASTIC_MCP_FLASH_LOG` is set, output is also tee'd to that file
     line-by-line as it arrives (for live flash progress in the TUI).
     """
@@ -264,7 +250,6 @@ def run(
             cwd=work_dir,
             timeout=timeout,
             tee_header=f"pio {' '.join(args)}",
-            extra_env=extra_env,
         )
     except subprocess.TimeoutExpired as exc:
         raise PioTimeout(f"pio {' '.join(args)} timed out after {timeout}s") from exc

mcp-server/src/meshtastic_mcp/recorder/__init__.py

@@ -1,19 +0,0 @@
-"""Persistent device-log capture.
-
-Singleton `Recorder` subscribes once to the meshtastic pubsub fan-out
-(`meshtastic.log.line`, `meshtastic.receive.*`, `meshtastic.connection.*`)
-and appends to four JSONL files under `mcp-server/.mtlog/`. Pubsub is
-process-global so a single subscription captures every active interface
-(serial / TCP / BLE) without any per-connection bookkeeping.
-
-The recorder is opt-in-by-import: importing this package is a no-op; call
-`get_recorder().start()` (which `server.py` does at FastMCP app init) to
-begin writing. `pause()` / `resume()` exist for the rare case the user
-wants a clean stretch of file (e.g. capturing a known-good baseline).
-"""
-
-from __future__ import annotations
-
-from .recorder import Recorder, get_recorder
-
-__all__ = ["Recorder", "get_recorder"]

mcp-server/src/meshtastic_mcp/recorder/parsers.py

@@ -1,309 +0,0 @@
-"""Best-effort parsers for log lines and telemetry packets.
-
-Two flavors of log line cross our pubsub subscription:
-  1. Text-mode path (debug_log_api disabled): the meshtastic Python lib
-     accumulates bytes between protobuf frames and emits the full
-     firmware-formatted line, e.g.
-         "INFO  | 12:34:56 12345 [Main] Booting"
-     — level, HH:MM:SS, uptime seconds, thread bracket, then message.
-  2. LogRecord protobuf path (debug_log_api enabled): the lib calls
-     `_handleLogLine(record.message)` with ONLY the message body. The
-     level/source/time fields on the LogRecord are dropped before
-     pubsub fan-out. We get e.g. just "Booting".
-
-Both arrive on `meshtastic.log.line`. The parser tries to recover a
-level + thread when the prefix is present and falls back to level=None
-otherwise. Consumers who want level filtering on protobuf-mode hosts
-should grep the raw `line` field instead.
-
-Telemetry: `meshtastic.receive.telemetry` packets carry one of several
-metric variants in `packet["decoded"]["telemetry"]`. We flatten the
-chosen variant into a {field: value} dict so callers don't have to
-know the protobuf shape.
-"""
-
-from __future__ import annotations
-
-import re
-from typing import Any
-
-# Match: LEVEL | HH:MM:SS UPTIME [Thread] message
-# HH:MM:SS may be ??:??:?? when RTC isn't valid. The level alternation
-# below is the canonical list — DebugConfiguration.h's MESHTASTIC_LOG_LEVEL_*
-# macros must stay in sync with these strings.
-_LINE_RE = re.compile(
-    r"""
-    ^
-    (?P<level>DEBUG|INFO\ |WARN\ |ERROR|CRIT\ |TRACE|HEAP\ )
-    \s*\|\s*
-    (?P<clock>(?:\d{2}:\d{2}:\d{2})|(?:\?{2}:\?{2}:\?{2}))
-    \s+
-    (?P<uptime>\d+)
-    \s+
-    (?:\[(?P<thread>[^\]]+)\]\s+)?
-    (?P<msg>.*)
-    $
-    """,
-    re.VERBOSE,
-)
-
-# DEBUG_HEAP build prepends `[heap N] ` to every message body, AFTER the
-# thread bracket. See src/RedirectablePrint.cpp:175.
-_HEAP_PREFIX_RE = re.compile(r"^\[heap\s+(?P<heap>\d+)\]\s+(?P<rest>.*)$")
-
-# OSThread leak/free detection. See src/concurrency/OSThread.cpp:89-91.
-# Format: "------ Thread NAME leaked heap A -> B (delta) ------"
-#         "++++++ Thread NAME freed heap A -> B (delta) ++++++"
-_THREAD_HEAP_RE = re.compile(
-    r"""
-    ^[\-+]+\s*
-    Thread\s+(?P<thread>\S+)\s+
-    (?P<kind>leaked|freed)\s+heap\s+
-    (?P<before>-?\d+)\s*->\s*(?P<after>-?\d+)\s+
-    \((?P<delta>-?\d+)\)
-    """,
-    re.VERBOSE,
-)
-
-# Power.cpp:908 periodic heap status (DEBUG_HEAP only).
-# Format: "Heap status: FREE/TOTAL bytes free (DELTA), running R/N threads"
-_HEAP_STATUS_RE = re.compile(
-    r"""
-    Heap\s+status:\s+
-    (?P<free>\d+)\s*/\s*(?P<total>\d+)\s+bytes\s+free
-    (?:\s+\((?P<delta>-?\d+)\))?
-    """,
-    re.VERBOSE,
-)
-
-
-_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
-_HEAP_BRACKET_RE = re.compile(r"^heap\s+(?P<heap>\d+)$")
-
-
-def parse_log_line(line: str) -> dict[str, Any]:
-    """Best-effort decompose a raw firmware log line.
-
-    Returns a dict with at least `line` (the original, unmodified — ANSI
-    codes preserved for fidelity). Adds `level`, `tag`, `clock`,
-    `uptime_s`, and `msg` when the full prefix is present.
-
-    Handles two firmware quirks:
-      - LogRecord.message can carry ANSI color escapes from RedirectablePrint
-        (the BLE/StreamAPI path inherited the colored body in some builds).
-        We strip ANSI before regex matching so the prefix survives.
-      - DEBUG_HEAP injects `[heap N]` after the thread bracket. When NO
-        thread name is set, the heap takes the thread bracket position —
-        looks like `[heap 12345] msg`. We detect that shape and move it
-        out of `tag` and into `heap_free`.
-
-    DEBUG_HEAP-build extras (when `[heap N]` is injected): `heap_free`
-    (bytes), and when a `Thread X leaked|freed heap` line is recognized,
-    `heap_event` = {kind, thread, before, after, delta}.
-
-    Never raises.
-    """
-    out: dict[str, Any] = {"line": line}
-    if not line:
-        return out
-
-    # Strip ANSI escapes BEFORE any regex matching. The original `line`
-    # stays in `out["line"]` for fidelity / future grep.
-    clean = _ANSI_RE.sub("", line)
-
-    m = _LINE_RE.match(clean)
-    msg: str | None = None
-    if m:
-        level = m.group("level").rstrip()
-        out["level"] = level
-        out["clock"] = m.group("clock")
-        try:
-            out["uptime_s"] = int(m.group("uptime"))
-        except (TypeError, ValueError):
-            out["uptime_s"] = None
-        thread = m.group("thread")
-        if thread:
-            # If "thread" is actually the heap prefix taking the bracket
-            # position (DEBUG_HEAP build, no thread set), capture heap
-            # and leave tag unset.
-            hb = _HEAP_BRACKET_RE.match(thread.strip())
-            if hb:
-                try:
-                    out["heap_free"] = int(hb.group("heap"))
-                except (TypeError, ValueError):
-                    pass
-            else:
-                out["tag"] = thread
-        msg = m.group("msg")
-        out["msg"] = msg
-    else:
-        # No prefix — bare LogRecord.message body. Inspect the whole
-        # line for DEBUG_HEAP-style content; the heap-prefix and
-        # thread-leak patterns can survive on either path.
-        msg = clean
-
-    # DEBUG_HEAP per-line heap prefix: `[heap 92344] message`.
-    # Sits AFTER the thread bracket and BEFORE the message body, but
-    # for bare LogRecord lines it's at the start. Match it at the
-    # head of `msg`.
-    if msg:
-        hp = _HEAP_PREFIX_RE.match(msg)
-        if hp:
-            try:
-                out["heap_free"] = int(hp.group("heap"))
-            except (TypeError, ValueError):
-                pass
-            else:
-                # Strip the prefix from `msg` so a grep on the message
-                # body doesn't have to know about it.
-                out["msg"] = hp.group("rest")
-                msg = hp.group("rest")
-
-        # Thread-level leak/free detection.
-        thr = _THREAD_HEAP_RE.search(msg)
-        if thr:
-            try:
-                out["heap_event"] = {
-                    "kind": thr.group("kind"),
-                    "thread": thr.group("thread"),
-                    "before": int(thr.group("before")),
-                    "after": int(thr.group("after")),
-                    "delta": int(thr.group("delta")),
-                }
-            except (TypeError, ValueError):
-                pass
-
-        # Power.cpp periodic "Heap status: F/T bytes free (D), running ..."
-        hs = _HEAP_STATUS_RE.search(msg)
-        if hs:
-            try:
-                out["heap_free"] = int(hs.group("free"))
-                out["heap_total"] = int(hs.group("total"))
-                if hs.group("delta") is not None:
-                    out["heap_delta"] = int(hs.group("delta"))
-            except (TypeError, ValueError):
-                pass
-
-    return out
-
-
-# -- Telemetry ----------------------------------------------------------
-
-# Order matters: meshtastic-python decoded packets use the protobuf
-# `oneof variant` field name (snake_case) as the dict key.
-_TELEMETRY_VARIANTS = (
-    ("device_metrics", "device"),
-    ("local_stats", "local"),
-    ("environment_metrics", "environment"),
-    ("power_metrics", "power"),
-    ("air_quality_metrics", "airQuality"),
-    ("health_metrics", "health"),
-    ("host_metrics", "host"),
-)
-
-
-def extract_telemetry(packet: dict[str, Any]) -> dict[str, Any] | None:
-    """Pull the telemetry variant + flat fields out of a `meshtastic.receive.telemetry`
-    packet. Returns None when the shape isn't what we expect — so the
-    caller can fall back to a generic packets.jsonl row.
-    """
-    if not isinstance(packet, dict):
-        return None
-    decoded = packet.get("decoded")
-    if not isinstance(decoded, dict):
-        return None
-    telem = decoded.get("telemetry")
-    if not isinstance(telem, dict):
-        return None
-    # The Python lib produces dict-of-camelCase keys via MessageToDict.
-    # Try both camelCase and snake_case to be robust to lib version drift.
-    for snake, label in _TELEMETRY_VARIANTS:
-        camel = _snake_to_camel(snake)
-        for key in (snake, camel):
-            value = telem.get(key)
-            if isinstance(value, dict):
-                return {
-                    "variant": label,
-                    "fields": {k: _scalarize(v) for k, v in value.items()},
-                    "time": telem.get("time"),
-                }
-    return None
-
-
-def _snake_to_camel(name: str) -> str:
-    parts = name.split("_")
-    return parts[0] + "".join(p.title() for p in parts[1:])
-
-
-def _scalarize(value: Any) -> Any:
-    """Keep telemetry fields JSON-friendly. Lists/dicts pass through
-    untouched; bytes -> hex string; protobuf enums occasionally arrive
-    as ints (fine) or strings (also fine)."""
-    if isinstance(value, (bytes, bytearray, memoryview)):
-        return bytes(value).hex()
-    return value
-
-
-# -- Generic packet summary ---------------------------------------------
-
-
-def summarize_packet(
-    packet: dict[str, Any], *, payload_hex_len: int = 64
-) -> dict[str, Any]:
-    """Reduce a packet dict to a stable, queryable summary. Drops the
-    full payload bytes — the recorder records summaries, not pcaps.
-    """
-    if not isinstance(packet, dict):
-        return {"raw_type": type(packet).__name__}
-    decoded = packet.get("decoded") if isinstance(packet.get("decoded"), dict) else {}
-    portnum = decoded.get("portnum") if isinstance(decoded, dict) else None
-    payload = decoded.get("payload") if isinstance(decoded, dict) else None
-    payload_hex = None
-    payload_size = None
-    if isinstance(payload, (bytes, bytearray, memoryview)):
-        b = bytes(payload)
-        payload_size = len(b)
-        payload_hex = b[:payload_hex_len].hex() if b else ""
-    elif isinstance(payload, str):
-        # Some decoded payloads (text messages) come as decoded strings.
-        payload_size = len(payload)
-        payload_hex = None  # not bytes
-    return {
-        "from_node": packet.get("fromId") or packet.get("from"),
-        "to_node": packet.get("toId") or packet.get("to"),
-        "portnum": portnum,
-        "hop_limit": packet.get("hopLimit"),
-        "want_ack": packet.get("wantAck"),
-        "rx_rssi": packet.get("rxRssi"),
-        "rx_snr": packet.get("rxSnr"),
-        "channel": packet.get("channel"),
-        "id": packet.get("id"),
-        "payload_size": payload_size,
-        "payload_hex_prefix": payload_hex,
-    }
-
-
-# -- Interface identification ------------------------------------------
-
-
-def interface_label(interface: Any) -> dict[str, Any]:
-    """Stable identifier for the meshtastic interface that emitted an event.
-
-    Used as the `port`/`role` tag on every recorded row. SerialInterface
-    has `devPath`; TCPInterface has `hostname`+`portNumber`; BLEInterface
-    has `address`. Falls back to the class name when none of those exist.
-    """
-    if interface is None:
-        return {"port": None, "role": None}
-    dev_path = getattr(interface, "devPath", None)
-    if dev_path:
-        return {"port": str(dev_path), "role": "serial"}
-    hostname = getattr(interface, "hostname", None)
-    if hostname:
-        port_num = getattr(interface, "portNumber", None)
-        endpoint = f"tcp://{hostname}:{port_num}" if port_num else f"tcp://{hostname}"
-        return {"port": endpoint, "role": "tcp"}
-    address = getattr(interface, "address", None)
-    if address:
-        return {"port": str(address), "role": "ble"}
-    return {"port": type(interface).__name__, "role": None}

mcp-server/src/meshtastic_mcp/recorder/recorder.py

@@ -1,467 +0,0 @@
-"""Process-global recorder singleton.
-
-Subscribes once to the meshtastic pubsub fan-out and writes four append-only
-JSONL streams under `mcp-server/.mtlog/`. The pubsub fan-out is
-process-global — a single subscription captures every active interface
-without per-connection bookkeeping.
-
-Files:
-  logs.jsonl      — every `meshtastic.log.line` event (best-effort prefix
-                    parsed for level/tag/uptime; raw `line` always preserved)
-  telemetry.jsonl — `meshtastic.receive.telemetry` packets, flattened by
-                    variant (device / local / environment / power / etc.)
-  packets.jsonl   — every other `meshtastic.receive.*` packet, summarized
-                    (portnum, hops, RSSI/SNR, payload size + 64-byte hex)
-  events.jsonl    — connection lifecycle, node-DB updates, and manual
-                    `mark_event` rows. Lower volume; useful for aligning
-                    timelines.
-
-Pause/resume: `pause()` flips a flag; subscriptions stay registered. The
-write methods short-circuit when paused, so we don't lose ordering when
-resumed (we just have a gap). No queueing.
-"""
-
-from __future__ import annotations
-
-import logging
-import os
-import threading
-import time
-from pathlib import Path
-from typing import Any
-
-from . import parsers
-from .rotating import _RotatingJsonl
-
-_DEFAULT_DIR = Path(__file__).resolve().parents[3] / ".mtlog"
-log = logging.getLogger(__name__)
-
-
-class Recorder:
-    """Singleton write-side of the persistent log capture system."""
-
-    def __init__(self, base_dir: Path | None = None) -> None:
-        self.base_dir = Path(base_dir) if base_dir else _DEFAULT_DIR
-        self._lock = threading.RLock()
-        self._started = False
-        self._paused = False
-        self._pause_reason: str | None = None
-        self._started_at: float | None = None
-        self._handlers: list[tuple[str, Any]] = []
-        self._files: dict[str, _RotatingJsonl] = {}
-
-    # -- lifecycle ----------------------------------------------------
-
-    def start(self) -> None:
-        """Idempotent. Safe to call from FastMCP app startup."""
-        with self._lock:
-            if self._started:
-                return
-            self.base_dir.mkdir(parents=True, exist_ok=True)
-            self._files = {
-                "logs": _RotatingJsonl(self.base_dir / "logs.jsonl"),
-                "telemetry": _RotatingJsonl(self.base_dir / "telemetry.jsonl"),
-                "packets": _RotatingJsonl(self.base_dir / "packets.jsonl"),
-                "events": _RotatingJsonl(self.base_dir / "events.jsonl"),
-            }
-            self._wire_pubsub()
-            self._started = True
-            self._started_at = time.time()
-        # Write the recorder_start marker after the initialization block.
-        # `_write_event()` re-checks recorder state via `_files_snapshot()`,
-        # so keeping this out of the setup block avoids nested lifecycle work.
-        self._write_event(kind="recorder_start", label="recorder_started")
-
-    def stop(self) -> None:
-        with self._lock:
-            if not self._started:
-                return
-            self._unwire_pubsub()
-            for f in self._files.values():
-                f.close()
-            self._files = {}
-            self._started = False
-
-    def pause(self, reason: str | None = None) -> None:
-        # Write the pause marker BEFORE flipping the flag — `_write_event`
-        # short-circuits when paused, so the order matters for this event
-        # to actually land in events.jsonl.
-        self._write_event(
-            kind="recorder_pause",
-            label="paused",
-            note=reason,
-        )
-        with self._lock:
-            self._paused = True
-            self._pause_reason = reason
-
-    def resume(self) -> None:
-        # Mirror of `pause()`: clear the flag first, then write the marker
-        # so it isn't suppressed by the still-paused short-circuit.
-        with self._lock:
-            self._paused = False
-            self._pause_reason = None
-        self._write_event(kind="recorder_resume", label="resumed")
-
-    # -- pubsub wiring ------------------------------------------------
-
-    def _wire_pubsub(self) -> None:
-        from pubsub import pub  # type: ignore[import-untyped]
-
-        # Subscribers — one per topic. Each pubsub publisher sends
-        # keyword args matching its handler's signature; pubsub
-        # introspects the function signature to route args.
-        bindings = [
-            ("meshtastic.log.line", self._on_log_line),
-            ("meshtastic.serial.line", self._on_serial_line),
-            ("meshtastic.receive", self._on_receive),
-            ("meshtastic.receive.telemetry", self._on_telemetry),
-            ("meshtastic.connection.established", self._on_connection_established),
-            ("meshtastic.connection.lost", self._on_connection_lost),
-            ("meshtastic.node.updated", self._on_node_updated),
-        ]
-        for topic, handler in bindings:
-            try:
-                pub.subscribe(handler, topic)
-                self._handlers.append((topic, handler))
-            except Exception as exc:
-                # If pubsub refuses one binding (signature mismatch on
-                # an old lib version), log it and keep the rest.
-                log.warning("Recorder failed to subscribe to %s: %s", topic, exc)
-
-    def _unwire_pubsub(self) -> None:
-        from pubsub import pub  # type: ignore[import-untyped]
-
-        for topic, handler in self._handlers:
-            try:
-                pub.unsubscribe(handler, topic)
-            except Exception:
-                pass
-        self._handlers.clear()
-
-    # -- handlers -----------------------------------------------------
-    #
-    # Pubsub callbacks must never raise. Every handler is wrapped in a
-    # try/except that swallows so a bug here can't take down the
-    # SerialInterface receive thread.
-    #
-    # Threading: handlers fire on whatever thread the meshtastic library
-    # dispatches from (varies by interface), while `stop()` clears
-    # `self._files` under `self._lock`. We snapshot `_files` under the
-    # lock at the top of each handler so a concurrent stop can't
-    # KeyError us mid-write. The actual file write goes through
-    # `_RotatingJsonl` which has its own lock.
-
-    def _files_snapshot(self) -> dict[str, _RotatingJsonl] | None:
-        """Atomic-ish view of `self._files`. Returns None when the recorder
-        is paused or stopped, so handlers can early-exit cleanly without
-        racing `stop()`'s clear."""
-        with self._lock:
-            if not self._started or self._paused:
-                return None
-            return dict(self._files)
-
-    def _on_log_line(self, line: str, interface: Any = None) -> None:
-        files = self._files_snapshot()
-        if files is None:
-            return
-        try:
-            tags = parsers.interface_label(interface)
-            parsed = parsers.parse_log_line(str(line))
-            ts = time.time()
-            record: dict[str, Any] = {
-                "ts": ts,
-                "port": tags["port"],
-                "role": tags["role"],
-                "level": parsed.get("level"),
-                "tag": parsed.get("tag"),
-                "uptime_s": parsed.get("uptime_s"),
-                "line": parsed["line"],
-            }
-            # DEBUG_HEAP enrichments (only present when the firmware
-            # was built with -DDEBUG_HEAP=1). Surface as first-class
-            # fields so logs_window can grep/filter on them and so
-            # heap_free synthesizes a telemetry point below.
-            if "heap_free" in parsed:
-                record["heap_free"] = parsed["heap_free"]
-            if "heap_total" in parsed:
-                record["heap_total"] = parsed["heap_total"]
-            if "heap_delta" in parsed:
-                record["heap_delta"] = parsed["heap_delta"]
-            heap_event = parsed.get("heap_event")
-            if heap_event:
-                record["heap_event"] = heap_event
-            files["logs"].write(record)
-
-            # If the line carried a heap snapshot, also write it as a
-            # synthesized LocalStats-shaped row so telemetry_timeline
-            # picks it up at log cadence (much higher resolution than
-            # the ~60 s LocalStats packet). Tagged source=debug_heap so
-            # consumers can filter if mixing scales is unwanted.
-            heap_free = parsed.get("heap_free")
-            if isinstance(heap_free, int):
-                fields: dict[str, Any] = {"heap_free_bytes": heap_free}
-                heap_total = parsed.get("heap_total")
-                if isinstance(heap_total, int):
-                    fields["heap_total_bytes"] = heap_total
-                files["telemetry"].write(
-                    {
-                        "ts": ts,
-                        "port": tags["port"],
-                        "role": tags["role"],
-                        "from_node": None,
-                        "variant": "local",
-                        "fields": fields,
-                        "source": "debug_heap",
-                    }
-                )
-        except Exception:
-            pass
-
-    def _on_serial_line(self, line: str, port: str | None = None) -> None:
-        """Text-mode passive tap. Fired from `serial_session._drain` when a
-        `pio device monitor` subprocess is running.
-
-        Same parse + heap-synthesis path as `_on_log_line`, but receives
-        the raw text-formatted line (full level/clock/uptime/thread/`[heap N]`/
-        body). On DEBUG_HEAP builds in text mode this gives us per-log-line
-        heap data — far higher cadence than LocalStats, and works without
-        protobuf API mode (no SerialInterface required).
-        """
-        files = self._files_snapshot()
-        if files is None:
-            return
-        try:
-            parsed = parsers.parse_log_line(str(line))
-            ts = time.time()
-            record: dict[str, Any] = {
-                "ts": ts,
-                "port": port,
-                "role": "serial_session",
-                "level": parsed.get("level"),
-                "tag": parsed.get("tag"),
-                "uptime_s": parsed.get("uptime_s"),
-                "line": parsed["line"],
-            }
-            if "heap_free" in parsed:
-                record["heap_free"] = parsed["heap_free"]
-            if "heap_total" in parsed:
-                record["heap_total"] = parsed["heap_total"]
-            if "heap_delta" in parsed:
-                record["heap_delta"] = parsed["heap_delta"]
-            heap_event = parsed.get("heap_event")
-            if heap_event:
-                record["heap_event"] = heap_event
-            files["logs"].write(record)
-
-            # Synthesize a heap_free telemetry sample whenever the line
-            # carries one — same logic as _on_log_line, tagged source so
-            # consumers can distinguish text-mode tap from protobuf path.
-            heap_free = parsed.get("heap_free")
-            if isinstance(heap_free, int):
-                fields: dict[str, Any] = {"heap_free_bytes": heap_free}
-                heap_total = parsed.get("heap_total")
-                if isinstance(heap_total, int):
-                    fields["heap_total_bytes"] = heap_total
-                files["telemetry"].write(
-                    {
-                        "ts": ts,
-                        "port": port,
-                        "role": "serial_session",
-                        "from_node": None,
-                        "variant": "local",
-                        "fields": fields,
-                        "source": "debug_heap_serial",
-                    }
-                )
-        except Exception:
-            pass
-
-    def _on_telemetry(self, packet: dict[str, Any], interface: Any = None) -> None:
-        files = self._files_snapshot()
-        if files is None:
-            return
-        try:
-            tags = parsers.interface_label(interface)
-            extracted = parsers.extract_telemetry(packet)
-            if extracted is None:
-                # Couldn't extract a known variant — fall through to the
-                # generic `_on_receive` path, which will still fire for
-                # this packet via the parent topic.
-                return
-            record = {
-                "ts": time.time(),
-                "port": tags["port"],
-                "role": tags["role"],
-                "from_node": packet.get("fromId") or packet.get("from"),
-                "variant": extracted["variant"],
-                "fields": extracted["fields"],
-                "device_time": extracted.get("time"),
-            }
-            files["telemetry"].write(record)
-        except Exception:
-            pass
-
-    def _on_receive(self, packet: dict[str, Any], interface: Any = None) -> None:
-        # Generic-receive fires for EVERY packet. Telemetry packets get
-        # recorded twice (here and in _on_telemetry) — that's intentional:
-        # packets.jsonl is the universal record, telemetry.jsonl is the
-        # structured timeseries view.
-        files = self._files_snapshot()
-        if files is None:
-            return
-        try:
-            tags = parsers.interface_label(interface)
-            summary = parsers.summarize_packet(packet)
-            record = {
-                "ts": time.time(),
-                "port": tags["port"],
-                "role": tags["role"],
-                **summary,
-            }
-            files["packets"].write(record)
-        except Exception:
-            pass
-
-    def _on_connection_established(self, interface: Any = None) -> None:
-        self._write_event(
-            kind="connection_established",
-            interface=interface,
-        )
-
-    def _on_connection_lost(self, interface: Any = None) -> None:
-        self._write_event(
-            kind="connection_lost",
-            interface=interface,
-        )
-
-    def _on_node_updated(
-        self, node: dict[str, Any] | None = None, interface: Any = None
-    ) -> None:
-        # Lower-volume than packets but informative — node ID, hops away,
-        # last heard. Skip the user dict if absent.
-        try:
-            user = (node or {}).get("user") if isinstance(node, dict) else None
-            self._write_event(
-                kind="node_updated",
-                interface=interface,
-                data={
-                    "num": (node or {}).get("num"),
-                    "id": (user or {}).get("id"),
-                    "short": (user or {}).get("shortName"),
-                    "long": (user or {}).get("longName"),
-                    "hops_away": (node or {}).get("hopsAway"),
-                    "snr": (node or {}).get("snr"),
-                    "last_heard": (node or {}).get("lastHeard"),
-                },
-            )
-        except Exception:
-            pass
-
-    # -- public write helpers -----------------------------------------
-
-    def mark_event(
-        self,
-        label: str,
-        note: str | None = None,
-        data: dict[str, Any] | None = None,
-    ) -> dict[str, Any]:
-        """User-facing marker. Writes to events.jsonl AND emits a
-        synthetic logs.jsonl row tagged level=MARK so timelines align.
-        """
-        ts = self._write_event(kind="mark", label=label, note=note, data=data)
-        # Mirror into logs so a single logs_window grep finds it.
-        files = self._files_snapshot()
-        if files is not None:
-            try:
-                files["logs"].write(
-                    {
-                        "ts": ts,
-                        "port": None,
-                        "role": "marker",
-                        "level": "MARK",
-                        "tag": "mark_event",
-                        "line": f"[mark] {label}" + (f" — {note}" if note else ""),
-                    }
-                )
-            except Exception:
-                pass
-        return {"ts": ts, "label": label}
-
-    def _write_event(
-        self,
-        *,
-        kind: str,
-        label: str | None = None,
-        note: str | None = None,
-        interface: Any = None,
-        data: dict[str, Any] | None = None,
-    ) -> float:
-        ts = time.time()
-        # Lifecycle markers (recorder_start, recorder_pause, recorder_resume)
-        # arrive at choreographed moments — `pause()` writes BEFORE flipping
-        # the flag and `resume()` writes AFTER clearing it, so those calls
-        # see _paused=False here. Other event kinds short-circuit when
-        # paused via the snapshot guard below.
-        files = self._files_snapshot()
-        if files is None:
-            return ts
-        try:
-            tags = parsers.interface_label(interface)
-            files["events"].write(
-                {
-                    "ts": ts,
-                    "kind": kind,
-                    "label": label,
-                    "note": note,
-                    "port": tags["port"],
-                    "role": tags["role"],
-                    "data": data,
-                }
-            )
-        except Exception:
-            pass
-        return ts
-
-    # -- introspection ------------------------------------------------
-
-    def status(self) -> dict[str, Any]:
-        with self._lock:
-            return {
-                "running": self._started,
-                "paused": self._paused,
-                "pause_reason": self._pause_reason,
-                "started_at": self._started_at,
-                "base_dir": str(self.base_dir),
-                "files": {name: f.status() for name, f in self._files.items()},
-            }
-
-    def force_rotate_all(self) -> dict[str, Any]:
-        """Test/admin hook: rotate every stream right now."""
-        with self._lock:
-            files = list(self._files.values())
-        for f in files:
-            f.force_rotate()
-        # `status()` re-acquires `self._lock`; release before calling it.
-        return self.status()
-
-
-# -- module-level singleton accessor ------------------------------------
-
-_INSTANCE_LOCK = threading.Lock()
-_INSTANCE: Recorder | None = None
-
-
-def get_recorder() -> Recorder:
-    """Return the process-global Recorder. Created on first call.
-
-    Honors `MESHTASTIC_MCP_LOG_DIR` env var for the base directory
-    (used by tests to redirect to a tmpdir).
-    """
-    global _INSTANCE
-    with _INSTANCE_LOCK:
-        if _INSTANCE is None:
-            override = os.environ.get("MESHTASTIC_MCP_LOG_DIR")
-            base = Path(override) if override else None
-            _INSTANCE = Recorder(base_dir=base)
-        return _INSTANCE

mcp-server/src/meshtastic_mcp/recorder/rotating.py

@@ -1,163 +0,0 @@
-"""Append-only JSONL writer with size-capped rotation.
-
-A `_RotatingJsonl` owns one live `.jsonl` file. Writes are line-delimited
-JSON objects (one row per call). When the live file exceeds `max_bytes`,
-it is closed, gzipped to `<name>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`,
-and the live file resets to empty. Old archives past `keep_archives` are
-unlinked oldest-first.
-
-Size check is amortized — `os.fstat` runs every `check_every` writes,
-not per-write, so the hot path stays at one `fh.write` + one `fh.flush`.
-
-Threading: every public method acquires `self._lock`. The recorder runs
-several pubsub handlers on whatever thread the meshtastic library
-dispatches from (varies by interface), and queries from MCP tool calls
-arrive on the FastMCP request thread, so this lock is not optional.
-"""
-
-from __future__ import annotations
-
-import gzip
-import json
-import os
-import shutil
-import threading
-from datetime import datetime, timezone
-from pathlib import Path
-from typing import Any
-
-
-class _RotatingJsonl:
-    """Append-only JSONL with size rotation. Thread-safe."""
-
-    def __init__(
-        self,
-        path: Path,
-        *,
-        max_bytes: int = 100 * 1024 * 1024,
-        keep_archives: int = 5,
-        check_every: int = 1000,
-    ) -> None:
-        self.path = path
-        self.max_bytes = max_bytes
-        self.keep_archives = keep_archives
-        self.check_every = check_every
-        self._lock = threading.Lock()
-        self._fh: Any = None
-        self._writes_since_check = 0
-        self._rotations = 0
-        self._lines_written = 0
-        self._last_ts: float | None = None
-        self._open()
-
-    # -- lifecycle ----------------------------------------------------
-
-    def _open(self) -> None:
-        self.path.parent.mkdir(parents=True, exist_ok=True)
-        self._fh = self.path.open("a", encoding="utf-8")
-
-    def close(self) -> None:
-        with self._lock:
-            if self._fh is not None:
-                try:
-                    self._fh.close()
-                finally:
-                    self._fh = None
-
-    # -- write --------------------------------------------------------
-
-    def write(self, record: dict[str, Any]) -> None:
-        """Append one JSON object as a line. Triggers rotation if oversized."""
-        line = json.dumps(record, separators=(",", ":"), default=str) + "\n"
-        with self._lock:
-            if self._fh is None:
-                return
-            try:
-                self._fh.write(line)
-                self._fh.flush()
-            except Exception:
-                # Best-effort: a failed write must not crash the pubsub
-                # handler. Caller has no way to react anyway.
-                return
-            self._lines_written += 1
-            ts = record.get("ts")
-            if isinstance(ts, (int, float)):
-                self._last_ts = float(ts)
-            self._writes_since_check += 1
-            if self._writes_since_check >= self.check_every:
-                self._writes_since_check = 0
-                self._maybe_rotate()
-
-    # -- rotation -----------------------------------------------------
-
-    def _maybe_rotate(self) -> None:
-        # Caller holds self._lock.
-        try:
-            size = os.fstat(self._fh.fileno()).st_size
-        except OSError:
-            return
-        if size < self.max_bytes:
-            return
-        self._rotate_locked()
-
-    def _rotate_locked(self) -> None:
-        # Close, gzip-rename, reopen empty, prune oldest archives.
-        try:
-            self._fh.close()
-        except Exception:
-            pass
-        self._fh = None
-        # Microsecond-resolution timestamp + per-instance counter so back-
-        # to-back rotations (small max_bytes, repeated `force_rotate()`,
-        # or chatty test loops) get unique archive filenames. The lex
-        # sort order of `YYYYMMDD-HHMMSS-uuuuuu-NNNNN` is chronological,
-        # which `_prune_archives()` and `log_query._iter_jsonl()` both
-        # rely on.
-        stamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S-%f")
-        archive = self.path.with_suffix(f".{stamp}-{self._rotations:05d}.jsonl.gz")
-        try:
-            with self.path.open("rb") as src, gzip.open(archive, "wb") as dst:
-                shutil.copyfileobj(src, dst, length=1024 * 1024)
-            self.path.unlink()
-        except Exception:
-            # Rotation is best-effort. If gzip fails, leave the file
-            # in place and re-open it; we'll try again next check.
-            pass
-        self._open()
-        self._rotations += 1
-        self._prune_archives()
-
-    def _prune_archives(self) -> None:
-        # Match siblings of self.path.name with `.jsonl.gz` suffix.
-        prefix = self.path.stem  # "logs" for "logs.jsonl"
-        # Archive filenames are already lexicographically chronological.
-        # Prune by name, not mtime, so copied/restored files don't reorder.
-        archives = sorted(self.path.parent.glob(f"{prefix}.*.jsonl.gz"))
-        excess = len(archives) - self.keep_archives
-        for old in archives[: max(0, excess)]:
-            try:
-                old.unlink()
-            except OSError:
-                pass
-
-    def force_rotate(self) -> None:
-        """Test/admin hook: rotate immediately regardless of size."""
-        with self._lock:
-            if self._fh is not None:
-                self._rotate_locked()
-
-    # -- introspection ------------------------------------------------
-
-    def status(self) -> dict[str, Any]:
-        with self._lock:
-            try:
-                size = os.fstat(self._fh.fileno()).st_size if self._fh else 0
-            except OSError:
-                size = 0
-            return {
-                "path": str(self.path),
-                "size": size,
-                "lines": self._lines_written,
-                "last_ts": self._last_ts,
-                "rotations": self._rotations,
-            }

mcp-server/src/meshtastic_mcp/serial_session.py

@@ -46,23 +46,7 @@ class SerialSession:
 
 
 def _drain(session: SerialSession) -> None:
-    """Reader thread: line-by-line pull stdout into buffer.
-
-    Each line is also published to the `meshtastic.serial.line` pubsub
-    topic so the persistent recorder can capture it without holding its
-    own port. This is the text-mode tap path: when no SerialInterface is
-    open, the firmware emits full formatted lines (level + clock + uptime
-    + thread + `[heap N]` prefix on DEBUG_HEAP builds + body), and we
-    fan them out to whoever is listening. Pubsub is best-effort —
-    publish failures must never block the reader.
-    """
-    # Lazy import: pubsub isn't required just to import this module
-    # (e.g., during static analysis), and we want a clean test surface.
-    try:
-        from pubsub import pub  # type: ignore[import-untyped]
-    except Exception:  # pragma: no cover - defensive
-        pub = None
-
+    """Reader thread: line-by-line pull stdout into buffer."""
     assert session.proc.stdout is not None
     try:
         for line in session.proc.stdout:
@@ -70,16 +54,6 @@ def _drain(session: SerialSession) -> None:
             with session.lock:
                 session.buffer.append(line_stripped)
                 session.total_lines += 1
-            if pub is not None:
-                try:
-                    pub.sendMessage(
-                        "meshtastic.serial.line",
-                        line=line_stripped,
-                        port=session.port,
-                    )
-                except Exception:
-                    # A subscriber raising must not break the reader.
-                    pass
     except Exception:  # pragma: no cover - defensive
         pass
     finally:

mcp-server/src/meshtastic_mcp/server.py

@@ -6,7 +6,6 @@ etc.). Business logic does not live here.
 
 from __future__ import annotations
 
-import logging
 from typing import Any
 
 from mcp.server.fastmcp import FastMCP
@@ -15,38 +14,17 @@ from . import (
     admin,
     boards,
     devices,
-    fixtures,
     flash,
     hw_tools,
     info,
-    log_query,
     registry,
     serial_session,
 )
 from . import userprefs as userprefs_mod
-from .recorder import get_recorder
-
-log = logging.getLogger(__name__)
 
 app = FastMCP("meshtastic-mcp")
 
 
-def _start_recorder() -> None:
-    # Persistent device-log capture. Starts on first import — pubsub fan-out
-    # is process-global, so subscribing here captures every active interface
-    # (whether opened by an MCP tool, a pytest fixture, or a serial_session).
-    # Files land in mcp-server/.mtlog/ (gitignored). See recorder/recorder.py
-    # for the full design. Recorder startup is best-effort: an unwritable
-    # log dir or pubsub mismatch should not take the MCP server down.
-    try:
-        get_recorder().start()
-    except Exception as exc:
-        log.warning("Failed to start persistent recorder: %s", exc)
-
-
-_start_recorder()
-
-
 # ---------- Discovery & metadata ------------------------------------------
 
 
@@ -97,7 +75,6 @@ def build(
     env: str,
     with_manifest: bool = True,
     userprefs: dict[str, Any] | None = None,
-    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Build firmware for one env via `pio run -e <env>`.
 
@@ -109,21 +86,8 @@ def build(
     build via userPrefs.jsonc injection. The file is restored after the build
     completes. Use `userprefs_manifest` to discover available keys. Use
     `userprefs_set` for persistent changes.
-
-    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for this build
-    only, injected via `PLATFORMIO_BUILD_FLAGS`. Common pattern:
-    `build_flags={"DEBUG_HEAP": 1}` enables per-thread leak detection + a
-    `[heap N]` prefix on every log line. The recorder picks the prefix up
-    automatically and synthesizes a high-resolution heap timeline that
-    `telemetry_timeline(field="free_heap")` can read alongside the normal
-    ~60 s LocalStats packets. Pair with `/leakhunt` for classification.
     """
-    return flash.build(
-        env,
-        with_manifest=with_manifest,
-        userprefs_overrides=userprefs,
-        build_flags=build_flags,
-    )
+    return flash.build(env, with_manifest=with_manifest, userprefs_overrides=userprefs)
 
 
 @app.tool()
@@ -141,7 +105,6 @@ def pio_flash(
     port: str,
     confirm: bool = False,
     userprefs: dict[str, Any] | None = None,
-    build_flags: dict[str, Any] | None = None,
 ) -> dict[str, Any]:
     """Flash firmware via `pio run -e <env> -t upload --upload-port <port>`.
 
@@ -151,19 +114,8 @@ def pio_flash(
 
     `userprefs` (optional): dict of `USERPREFS_<KEY>: value` baked into this
     build via userPrefs.jsonc injection; restored after upload.
-
-    `build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros for the
-    rebuild-before-upload, e.g. `{"DEBUG_HEAP": 1}`. Required for the flags
-    to actually land in the uploaded firmware — without it, the implicit
-    rebuild relinks without the env var and silently drops them.
     """
-    return flash.flash(
-        env,
-        port,
-        confirm=confirm,
-        userprefs_overrides=userprefs,
-        build_flags=build_flags,
-    )
+    return flash.flash(env, port, confirm=confirm, userprefs_overrides=userprefs)
 
 
 @app.tool()
@@ -782,227 +734,3 @@ def picotool_load(uf2_path: str, confirm: bool = False) -> dict[str, Any]:
 def picotool_raw(args: list[str], confirm: bool = False) -> dict[str, Any]:
     """Pass-through to `picotool`. load/reboot/save/erase require confirm=True."""
     return hw_tools.picotool_raw(args, confirm=confirm)
-
-
-# ---------- Persistent device-log capture (recorder) ----------------------
-#
-# The recorder is autouse — it starts at server import and continuously
-# writes every meshtastic pubsub event to JSONL files under .mtlog/. These
-# tools are query-only over those files, plus a few lifecycle controls.
-
-
-@app.tool()
-def logs_window(
-    start: str = "-15m",
-    end: str = "now",
-    grep: str | None = None,
-    level: str | None = None,
-    tag: str | None = None,
-    port: str | None = None,
-    max_lines: int = 200,
-) -> dict[str, Any]:
-    """Recent firmware log lines from the persistent recorder.
-
-    Filters by time window, regex over the line, level (single or
-    pipe-separated set like "WARN|ERROR|CRIT"), thread-name tag, and
-    interface port. Returns up to max_lines most-recent matches.
-
-    Time strings: "-15m", "-2h", "-3d", "now", or ISO 8601.
-
-    Note: lines arriving via the LogRecord protobuf path (when
-    set_debug_log_api(True) is on) come without level prefix — the
-    meshtastic Python lib drops record.level before fan-out. For those,
-    `level` filter won't match; use `grep` instead.
-    """
-    return log_query.logs_window(
-        start=start,
-        end=end,
-        grep=grep,
-        level=level,
-        tag=tag,
-        port=port,
-        max_lines=max_lines,
-    )
-
-
-@app.tool()
-def telemetry_timeline(
-    window: str = "1h",
-    variant: str = "local",
-    field: str = "free_heap",
-    port: str | None = None,
-    max_points: int = 200,
-) -> dict[str, Any]:
-    """Time series of one telemetry field, downsampled to <= max_points.
-
-    `variant` ∈ device, local, environment, power, airQuality, health, host.
-    `field` accepts snake_case or camelCase; common aliases (free_heap ↔
-    heap_free_bytes) are normalized.
-
-    Returns slope_per_min (linear-regression slope, units/minute) so a
-    leak detector can read one number — negative slope on free_heap over
-    a long window indicates a real leak.
-
-    LocalStats variant ("local") cadence is ~60 s (whatever the device's
-    `device_update_interval` is set to), so a 1 h window gives ~60 raw
-    points. Bucket-mean downsampling preserves shape.
-    """
-    return log_query.telemetry_timeline(
-        window=window,
-        variant=variant,
-        field=field,
-        port=port,
-        max_points=max_points,
-    )
-
-
-@app.tool()
-def packets_window(
-    start: str = "-5m",
-    end: str = "now",
-    portnum: str | None = None,
-    from_node: str | None = None,
-    to_node: str | None = None,
-    max: int = 200,
-) -> dict[str, Any]:
-    """Recent mesh packets recorded by the recorder.
-
-    Each row is a summary (portnum, from/to, hop_limit, RSSI/SNR, payload
-    size + first 64 bytes hex) — full payload bytes are not stored.
-    `portnum` accepts a pipe-separated set like "TEXT_MESSAGE_APP|POSITION_APP".
-    """
-    return log_query.packets_window(
-        start=start,
-        end=end,
-        portnum=portnum,
-        from_node=from_node,
-        to_node=to_node,
-        max=max,
-    )
-
-
-@app.tool()
-def events_window(
-    start: str = "-1h",
-    end: str = "now",
-    kind: str | None = None,
-    max: int = 200,
-) -> dict[str, Any]:
-    """Return recorder events: connection lifecycle, node updates, and `mark_event` markers.
-
-    `kind` ∈ recorder_start, recorder_pause, recorder_resume,
-    connection_established, connection_lost, node_updated, mark.
-    Pipe-separated sets ("connection_lost|connection_established") work.
-    """
-    return log_query.events_window(start=start, end=end, kind=kind, max=max)
-
-
-@app.tool()
-def mark_event(
-    label: str,
-    note: str | None = None,
-    data: dict[str, Any] | None = None,
-) -> dict[str, Any]:
-    """Drop a named marker into events.jsonl AND logs.jsonl.
-
-    Useful for aligning a timeline around a known stimulus: call before
-    and after a stress workload, then query telemetry_timeline /
-    logs_window with the markers' timestamps as bounds.
-
-    The marker also lands in logs.jsonl with level=MARK so a single
-    grep over logs picks it up.
-    """
-    return get_recorder().mark_event(label=label, note=note, data=data)
-
-
-@app.tool()
-def recorder_status() -> dict[str, Any]:
-    """Return recorder runtime info: running, paused, file sizes, last_ts per stream.
-
-    Use this to sanity-check that capture is working before you trust a
-    `logs_window` / `telemetry_timeline` result.
-    """
-    return get_recorder().status()
-
-
-@app.tool()
-def recorder_pause(reason: str | None = None) -> dict[str, Any]:
-    """Pause writes to all four streams. Pubsub subscriptions stay active —
-    we just drop events on the floor while paused. Resume with `recorder_resume`.
-
-    Use when capturing a known-good baseline that you don't want to
-    pollute with pre-test noise. Default state is recording; this is
-    rarely needed.
-    """
-    get_recorder().pause(reason=reason)
-    return {"ok": True, "paused": True, "reason": reason}
-
-
-@app.tool()
-def recorder_resume() -> dict[str, Any]:
-    """Resume writes after `recorder_pause`. No-op if already running."""
-    get_recorder().resume()
-    return {"ok": True, "paused": False}
-
-
-@app.tool()
-def recorder_export(
-    start: str,
-    end: str,
-    dest_dir: str,
-    streams: list[str] | None = None,
-) -> dict[str, Any]:
-    """Bundle a slice of the recorder's streams into `dest_dir`.
-
-    Writes one uncompressed JSONL per requested stream (logs / telemetry /
-    packets / events). Useful for: attaching to a bug report, feeding a
-    notebook, or backfilling Datadog after the fact.
-    """
-    return log_query.export(
-        start=start,
-        end=end,
-        dest_dir=dest_dir,
-        streams=streams,
-    )
-
-
-# ---------- Fixture / test-data push --------------------------------------
-
-
-@app.tool()
-def push_fake_nodedb(
-    size: int,
-    target: str = "portduino",
-    port: str | None = None,
-    portduino_config: str = "default",
-    backup_existing: bool = True,
-    confirm: bool = False,
-    reboot_after: bool = True,
-    custom_seed_jsonl: str | None = None,
-) -> dict[str, Any]:
-    """Push a fake-NodeDB v25 fixture (250/500/1000/2000 nodes) onto a device.
-
-    Two transports:
-      target="portduino" — file copy to ~/.portduino/<portduino_config>/prefs/nodes.proto.
-                            Fast, no device connection needed.
-      target="hardware"  — XModem upload over serial/BLE to /prefs/nodes.proto.
-                            Requires `port` + `confirm=True`. Triggers a reboot
-                            so loadFromDisk picks up the new file at next boot.
-
-    Compiles a fresh-timestamp proto from the committed JSONL seed under
-    test/fixtures/nodedb/seed_v25_<N>.jsonl each invocation, so the loaded
-    NodeDB always looks "recent" to the connecting phone. Structural data
-    (names, IDs, positions, telemetries) is deterministic per the seed.
-
-    Override the JSONL via `custom_seed_jsonl` to push a hand-edited scenario.
-    """
-    return fixtures.push_fake_nodedb(
-        size=size,
-        target=target,  # type: ignore[arg-type]
-        port=port,
-        portduino_config=portduino_config,
-        backup_existing=backup_existing,
-        confirm=confirm,
-        reboot_after=reboot_after,
-        custom_seed_jsonl=custom_seed_jsonl,
-    )

mcp-server/tests/unit/test_build_flags.py

@@ -1,88 +0,0 @@
-"""Unit tests for the `build_flags` injection on `flash.build()`.
-
-We don't actually run pio here — too slow, requires hardware-aware envs.
-We test the translation layer (`_build_flags_env`) and that the env vars
-are threaded through pio.run correctly via mock.
-"""
-
-from __future__ import annotations
-
-from unittest.mock import patch
-
-from meshtastic_mcp import flash, pio
-
-
-class TestBuildFlagsEnv:
-    def test_simple_value(self) -> None:
-        out = flash._build_flags_env({"DEBUG_HEAP": 1})
-        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"}
-
-    def test_string_value(self) -> None:
-        out = flash._build_flags_env({"FOO": "bar"})
-        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DFOO=bar"}
-
-    def test_bool_true_is_bare_flag(self) -> None:
-        out = flash._build_flags_env({"DEBUG_HEAP": True})
-        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP"}
-
-    def test_bool_false_dropped(self) -> None:
-        out = flash._build_flags_env({"DEBUG_HEAP": False, "OTHER": 1})
-        assert out == {"PLATFORMIO_BUILD_FLAGS": "-DOTHER=1"}
-
-    def test_none_dropped(self) -> None:
-        out = flash._build_flags_env({"DEBUG_HEAP": None})
-        assert out == {}
-
-    def test_multiple_combined(self) -> None:
-        out = flash._build_flags_env({"DEBUG_HEAP": 1, "FOO": "x", "BAR": True})
-        # Order isn't guaranteed in dict iteration, so check membership.
-        flags = out["PLATFORMIO_BUILD_FLAGS"].split()
-        assert set(flags) == {"-DDEBUG_HEAP=1", "-DFOO=x", "-DBAR"}
-
-
-class TestBuildPropagatesFlags:
-    def test_extra_env_passed_to_pio_run(self) -> None:
-        # Mock pio.run so we don't actually invoke pio. Capture extra_env.
-        captured = {}
-
-        class _StubResult:
-            returncode = 0
-            stdout = ""
-            stderr = ""
-            duration_s = 0.1
-
-        def _stub(args, **kwargs):
-            captured["args"] = args
-            captured["kwargs"] = kwargs
-            return _StubResult()
-
-        with patch.object(pio, "run", side_effect=_stub):
-            with patch.object(flash, "_artifacts_for", return_value=[]):
-                out = flash.build(
-                    "fake-env",
-                    with_manifest=False,
-                    build_flags={"DEBUG_HEAP": 1},
-                )
-        assert captured["args"] == ["run", "-e", "fake-env"]
-        assert captured["kwargs"]["extra_env"] == {
-            "PLATFORMIO_BUILD_FLAGS": "-DDEBUG_HEAP=1"
-        }
-        assert out["build_flags"] == {"DEBUG_HEAP": 1}
-
-    def test_no_flags_means_no_extra_env(self) -> None:
-        captured = {}
-
-        class _StubResult:
-            returncode = 0
-            stdout = ""
-            stderr = ""
-            duration_s = 0.1
-
-        def _stub(args, **kwargs):
-            captured["kwargs"] = kwargs
-            return _StubResult()
-
-        with patch.object(pio, "run", side_effect=_stub):
-            with patch.object(flash, "_artifacts_for", return_value=[]):
-                flash.build("fake-env", with_manifest=False)
-        assert captured["kwargs"]["extra_env"] is None

mcp-server/tests/unit/test_fake_nodedb_generator.py

@@ -1,364 +0,0 @@
-"""Tests for the fake-NodeDB fixture pipeline (bin/gen-fake-nodedb-seed.py
-+ bin/seed-json-to-proto.py + mcp-server fixtures.push_fake_nodedb).
-
-Lives under tests/unit/ because none of these touch real hardware — they
-shell out to the bin/ scripts and decode the resulting protobufs in-process.
-"""
-
-from __future__ import annotations
-
-import json
-import pathlib
-import subprocess
-import sys
-import time
-
-import pytest
-
-REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
-SEED_GEN = REPO_ROOT / "bin" / "gen-fake-nodedb-seed.py"
-COMPILE = REPO_ROOT / "bin" / "seed-json-to-proto.py"
-FIXTURES_DIR = REPO_ROOT / "test" / "fixtures" / "nodedb"
-
-# Ensure the locally-generated Python protobuf bindings are importable.
-# These live under `meshtastic_v25` (not `meshtastic`) so they don't shadow
-# the PyPI `meshtastic` package that the rest of the mcp-server depends on.
-_BINDINGS_DIR = REPO_ROOT / "bin" / "_generated"
-if _BINDINGS_DIR.is_dir() and str(_BINDINGS_DIR) not in sys.path:
-    sys.path.insert(0, str(_BINDINGS_DIR))
-
-try:
-    from meshtastic_v25.deviceonly_pb2 import (
-        NodeDatabase,  # type: ignore[import-not-found]
-    )
-except ImportError:
-    NodeDatabase = None  # type: ignore[assignment]
-
-
-def _require_v25_bindings() -> None:
-    if NodeDatabase is None:
-        pytest.skip(
-            "v25 Python protobuf bindings missing; run `./bin/regen-py-protos.sh`."
-        )
-    if "positions" not in NodeDatabase.DESCRIPTOR.fields_by_name:
-        pytest.skip(
-            "Loaded NodeDatabase predates v25 — run `./bin/regen-py-protos.sh`."
-        )
-
-
-def _run(cmd: list[str]) -> None:
-    subprocess.check_call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
-
-
-# ---------------------------------------------------------------------------
-# Seed generator: deterministic for given --seed (no wall-clock dependence).
-# ---------------------------------------------------------------------------
-def test_seed_generator_is_deterministic(tmp_path: pathlib.Path) -> None:
-    a = tmp_path / "a.jsonl"
-    b = tmp_path / "b.jsonl"
-    _run(
-        [
-            sys.executable,
-            str(SEED_GEN),
-            "--count",
-            "100",
-            "--seed",
-            "42",
-            "--out",
-            str(a),
-        ]
-    )
-    # Sleep so any sneaky wall-clock leak in the generator would surface as
-    # a byte diff between the two runs.
-    time.sleep(0.8)
-    _run(
-        [
-            sys.executable,
-            str(SEED_GEN),
-            "--count",
-            "100",
-            "--seed",
-            "42",
-            "--out",
-            str(b),
-        ]
-    )
-    assert a.read_bytes() == b.read_bytes()
-
-
-def test_seed_generator_meta_line(tmp_path: pathlib.Path) -> None:
-    out = tmp_path / "seed.jsonl"
-    _run(
-        [
-            sys.executable,
-            str(SEED_GEN),
-            "--count",
-            "50",
-            "--seed",
-            "1",
-            "--out",
-            str(out),
-        ]
-    )
-    lines = out.read_text(encoding="utf-8").splitlines()
-    assert len(lines) == 51  # 1 meta + 50 nodes
-    meta = json.loads(lines[0])
-    assert "_meta" in meta
-    assert meta["_meta"]["version"] == 25
-    assert meta["_meta"]["count"] == 50
-    assert meta["_meta"]["seed"] == 1
-
-
-def test_seed_only_uses_active_hardware_and_roles(tmp_path: pathlib.Path) -> None:
-    """Confirm no deprecated roles + no off-list HW models leak through."""
-    out = tmp_path / "seed.jsonl"
-    _run(
-        [
-            sys.executable,
-            str(SEED_GEN),
-            "--count",
-            "500",
-            "--seed",
-            "7",
-            "--out",
-            str(out),
-        ]
-    )
-    forbidden_roles = {"ROUTER_CLIENT", "REPEATER"}
-    forbidden_hw = {
-        "TLORA_V1",
-        "TLORA_V2",
-        "TLORA_V1_1P3",
-        "TLORA_V2_1_1P6",
-        "TLORA_V2_1_1P8",
-        "HELTEC_V1",
-        "HELTEC_V2_0",
-        "HELTEC_V2_1",
-        "TBEAM",
-        "TBEAM_V0P7",
-        "NANO_G1",
-        "NANO_G1_EXPLORER",
-        "NANO_G2_ULTRA",
-        "STATION_G1",
-        "STATION_G2",
-        "PORTDUINO",
-        "ANDROID_SIM",
-        "DIY_V1",
-        "LORA_RELAY_V1",
-        "NRF52840_PCA10059",
-        "NRF52_UNKNOWN",
-        "DR_DEV",
-        "GENIEBLOCKS",
-        "M5STACK",
-        "RP2040_LORA",
-        "PPR",
-    }
-    for raw in out.read_text(encoding="utf-8").splitlines()[1:]:
-        node = json.loads(raw)
-        assert node["role"] not in forbidden_roles, f"deprecated role: {node['role']}"
-        assert (
-            node["hw_model"] not in forbidden_hw
-        ), f"non-tier-1 HW: {node['hw_model']}"
-
-
-# ---------------------------------------------------------------------------
-# Compile step + committed seeds.
-# ---------------------------------------------------------------------------
-@pytest.mark.parametrize("size", [250, 500, 1000, 2000])
-def test_committed_seed_compiles_and_decodes(size: int, tmp_path: pathlib.Path) -> None:
-    _require_v25_bindings()
-    proto = tmp_path / "out.proto"
-    jsonl = FIXTURES_DIR / f"seed_v25_{size:04d}.jsonl"
-    if not jsonl.is_file():
-        pytest.skip(f"{jsonl} not present — run ./bin/regen-fake-nodedbs.sh")
-    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(proto)])
-
-    db = NodeDatabase()
-    db.ParseFromString(proto.read_bytes())
-    assert db.version == 25
-    assert len(db.nodes) == size
-    nums = {n.num for n in db.nodes}
-    assert len(nums) == size, "node numbers must be unique"
-    assert all(n.long_name and n.short_name for n in db.nodes)
-    assert all(len(n.long_name) <= 24 for n in db.nodes)  # max_size:25 - NUL
-
-    # Coverage sanity (±10pp tolerance for binomial fluctuation).
-    def in_range(actual: int, expected_ratio: float, tol_pp: float = 0.10) -> bool:
-        lo = max(0, int((expected_ratio - tol_pp) * size))
-        hi = min(size, int((expected_ratio + tol_pp) * size))
-        return lo <= actual <= hi
-
-    assert in_range(len(db.positions), 0.85)
-    assert in_range(len(db.telemetry), 0.70)
-    assert in_range(len(db.environment), 0.25)
-    assert in_range(len(db.status), 0.40)
-
-
-def test_compile_freshens_timestamps(tmp_path: pathlib.Path) -> None:
-    """Same JSONL compiled twice → identical structure, different timestamps."""
-    _require_v25_bindings()
-    jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
-    if not jsonl.is_file():
-        pytest.skip("250-node seed not present — run ./bin/regen-fake-nodedbs.sh")
-    a = tmp_path / "a.proto"
-    b = tmp_path / "b.proto"
-    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(a)])
-    time.sleep(1.2)
-    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(b)])
-
-    da = NodeDatabase()
-    db_ = NodeDatabase()
-    da.ParseFromString(a.read_bytes())
-    db_.ParseFromString(b.read_bytes())
-
-    # Zero out timestamp fields and confirm everything else is byte-identical.
-    for d in (da, db_):
-        for n in d.nodes:
-            n.last_heard = 0
-        for p in d.positions:
-            p.position.time = 0
-    assert da.SerializeToString() == db_.SerializeToString()
-
-    # Re-load fresh copies to confirm timestamps actually moved.
-    aa = NodeDatabase()
-    bb = NodeDatabase()
-    aa.ParseFromString(a.read_bytes())
-    bb.ParseFromString(b.read_bytes())
-    aa_max = max(n.last_heard for n in aa.nodes if n.last_heard)
-    bb_max = max(n.last_heard for n in bb.nodes if n.last_heard)
-    assert bb_max >= aa_max
-    assert bb_max - aa_max < 5  # within a few seconds
-
-
-def test_compile_pinned_now_epoch_is_byte_identical(tmp_path: pathlib.Path) -> None:
-    """With --now-epoch pinned, two compiles produce identical bytes."""
-    _require_v25_bindings()
-    jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
-    if not jsonl.is_file():
-        pytest.skip("250-node seed not present")
-    a = tmp_path / "a.proto"
-    b = tmp_path / "b.proto"
-    for o in (a, b):
-        _run(
-            [
-                sys.executable,
-                str(COMPILE),
-                "--in",
-                str(jsonl),
-                "--now-epoch",
-                "1700000000",
-                "--out",
-                str(o),
-            ]
-        )
-    assert a.read_bytes() == b.read_bytes()
-
-
-def test_compile_timestamps_are_recent(tmp_path: pathlib.Path) -> None:
-    _require_v25_bindings()
-    jsonl = FIXTURES_DIR / "seed_v25_0250.jsonl"
-    if not jsonl.is_file():
-        pytest.skip("250-node seed not present")
-    out = tmp_path / "out.proto"
-    _run([sys.executable, str(COMPILE), "--in", str(jsonl), "--out", str(out)])
-    db = NodeDatabase()
-    db.ParseFromString(out.read_bytes())
-    now = int(time.time())
-    # No timestamp older than 7 days, none in the future.
-    for n in db.nodes:
-        if n.last_heard:
-            assert now - 7 * 86400 <= n.last_heard <= now
-    # At least half should be within the last hour
-    # (matches expovariate(mean=3600s)).
-    recent = sum(1 for n in db.nodes if n.last_heard and n.last_heard >= now - 3600)
-    assert recent >= 0.4 * len(db.nodes)
-
-
-def test_compile_hand_edit_round_trip(tmp_path: pathlib.Path) -> None:
-    """Edit one JSONL line, recompile, confirm edit appears in the proto."""
-    _require_v25_bindings()
-    src = FIXTURES_DIR / "seed_v25_0250.jsonl"
-    if not src.is_file():
-        pytest.skip("250-node seed not present")
-    dst = tmp_path / "edited.jsonl"
-    lines = src.read_text(encoding="utf-8").splitlines()
-
-    # Find a node that already has telemetry so the index relationship is
-    # easy to assert on the other side.
-    edit_idx = None
-    for i, raw in enumerate(lines[1:], start=1):
-        node = json.loads(raw)
-        if node.get("telemetry") is not None:
-            edit_idx = i
-            break
-    assert edit_idx is not None, "expected at least one node with telemetry"
-
-    node = json.loads(lines[edit_idx])
-    target_num = int(node["num"], 16)
-    node["long_name"] = "Hand Edited Node"
-    node["telemetry"] = {
-        "battery_level": 42,
-        "voltage": 3.71,
-        "channel_utilization": 0.0,
-        "air_util_tx": 0.0,
-        "uptime_seconds": 1,
-    }
-    lines[edit_idx] = json.dumps(node, ensure_ascii=False, sort_keys=True)
-    dst.write_text("\n".join(lines) + "\n", encoding="utf-8")
-
-    out = tmp_path / "out.proto"
-    _run([sys.executable, str(COMPILE), "--in", str(dst), "--out", str(out)])
-    db = NodeDatabase()
-    db.ParseFromString(out.read_bytes())
-    edited = next((n for n in db.nodes if n.num == target_num), None)
-    assert edited is not None
-    assert edited.long_name == "Hand Edited Node"
-    tel = next((t for t in db.telemetry if t.num == target_num), None)
-    assert tel is not None
-    assert tel.device_metrics.battery_level == 42
-
-
-# ---------------------------------------------------------------------------
-# Misc smoke checks on the module surface.
-# ---------------------------------------------------------------------------
-def test_crc16_ccitt_matches_known_vectors() -> None:
-    """Sanity-check the hand-rolled CRC16-CCITT matches well-known vectors.
-
-    Test vectors from the XModem-CRC spec (init=0, poly=0x1021):
-      crc16("123456789") = 0x31C3
-      crc16("")          = 0x0000
-    """
-    from meshtastic_mcp.fixtures import _crc16_ccitt
-
-    assert _crc16_ccitt(b"") == 0x0000
-    assert _crc16_ccitt(b"123456789") == 0x31C3
-
-
-def test_push_fake_nodedb_rejects_invalid_size() -> None:
-    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
-
-    with pytest.raises(FixtureError, match="size must be one of"):
-        push_fake_nodedb(size=999, target="portduino")  # type: ignore[arg-type]
-
-
-def test_push_fake_nodedb_hardware_requires_confirm() -> None:
-    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
-
-    with pytest.raises(FixtureError, match="confirm=True"):
-        push_fake_nodedb(size=250, target="hardware", port="/dev/cu.fake")
-
-
-def test_push_fake_nodedb_hardware_requires_port() -> None:
-    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
-
-    with pytest.raises(FixtureError, match="requires a port"):
-        push_fake_nodedb(size=250, target="hardware", confirm=True)
-
-
-def test_push_fake_nodedb_hardware_rejects_tcp_port() -> None:
-    from meshtastic_mcp.fixtures import FixtureError, push_fake_nodedb
-
-    with pytest.raises(FixtureError, match="not supported"):
-        push_fake_nodedb(
-            size=250, target="hardware", confirm=True, port="tcp://localhost:4403"
-        )

mcp-server/tests/unit/test_recorder.py

@@ -1,548 +0,0 @@
-"""Unit tests for the persistent device-log recorder.
-
-Hardware-free: drives the Recorder through its `_on_*` handlers with
-synthetic packet/line dicts, then queries via log_query. Validates
-prefix parsing, telemetry variant dispatch, marker round-trip, time
-window filtering, downsampling, slope estimation, and gzip rotation
-+ archive pruning.
-"""
-
-from __future__ import annotations
-
-import gzip
-import json
-import logging
-import os
-import time
-from pathlib import Path
-
-import pubsub
-import pytest
-from meshtastic_mcp import log_query
-from meshtastic_mcp.recorder.parsers import (
-    extract_telemetry,
-    interface_label,
-    parse_log_line,
-    summarize_packet,
-)
-from meshtastic_mcp.recorder.recorder import Recorder
-from meshtastic_mcp.recorder.rotating import _RotatingJsonl
-
-# -- isolation: every test gets a fresh Recorder + tmp dir -----------
-
-
-@pytest.fixture
-def recorder(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Recorder:
-    # Redirect both the Recorder and the module-level singleton lookup
-    # to the same tmp dir so log_query queries the same files we write.
-    monkeypatch.setenv("MESHTASTIC_MCP_LOG_DIR", str(tmp_path))
-    monkeypatch.setattr(
-        "meshtastic_mcp.recorder.recorder._INSTANCE", None, raising=False
-    )
-    r = Recorder(base_dir=tmp_path)
-    r.start()
-    monkeypatch.setattr("meshtastic_mcp.recorder.recorder._INSTANCE", r, raising=False)
-    yield r
-    r.stop()
-
-
-class _FakeIface:
-    devPath = "/dev/cu.fake"
-
-
-# -- parsers ---------------------------------------------------------
-
-
-class TestParseLogLine:
-    def test_full_prefix(self) -> None:
-        out = parse_log_line("INFO  | 12:34:56 12345 [Main] Booting")
-        assert out["level"] == "INFO"
-        assert out["tag"] == "Main"
-        assert out["uptime_s"] == 12345
-        assert out["msg"] == "Booting"
-        assert out["clock"] == "12:34:56"
-
-    def test_invalid_clock(self) -> None:
-        out = parse_log_line("WARN  | ??:??:?? 7 [SerialConsole] Boot")
-        assert out["level"] == "WARN"
-        assert out["clock"] == "??:??:??"
-        assert out["uptime_s"] == 7
-
-    def test_no_thread_bracket(self) -> None:
-        out = parse_log_line("DEBUG | 00:00:00 0 raw message body")
-        assert out["level"] == "DEBUG"
-        assert out.get("tag") is None
-        assert out["msg"] == "raw message body"
-
-    def test_bare_message(self) -> None:
-        # LogRecord.message path — no level prefix at all.
-        out = parse_log_line("just a bare message")
-        assert "level" not in out or out.get("level") is None
-        assert out["line"] == "just a bare message"
-
-    def test_empty(self) -> None:
-        assert parse_log_line("") == {"line": ""}
-
-    def test_debug_heap_prefix_extracted(self) -> None:
-        out = parse_log_line("INFO  | 12:34:56 12345 [Main] [heap 92344] Booting")
-        assert out["level"] == "INFO"
-        assert out["tag"] == "Main"
-        assert out["heap_free"] == 92344
-        assert out["msg"] == "Booting"
-
-    def test_debug_heap_prefix_on_bare_line(self) -> None:
-        # LogRecord.message path: no level prefix but still has [heap N].
-        out = parse_log_line("[heap 12345] some message")
-        assert out["heap_free"] == 12345
-        assert out["msg"] == "some message"
-
-    def test_thread_leak_event(self) -> None:
-        out = parse_log_line(
-            "HEAP  | 00:00:01 100 [Power] [heap 90000] "
-            "------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------"
-        )
-        assert out["level"] == "HEAP"
-        assert out["heap_free"] == 90000
-        ev = out["heap_event"]
-        assert ev["kind"] == "leaked"
-        assert ev["thread"] == "MeshPacket"
-        assert ev["before"] == 92344
-        assert ev["after"] == 90000
-        assert ev["delta"] == -2344
-
-    def test_thread_freed_event(self) -> None:
-        out = parse_log_line(
-            "++++++ Thread Router freed heap 1000 -> 1500 (500) ++++++"
-        )
-        ev = out["heap_event"]
-        assert ev["kind"] == "freed"
-        assert ev["thread"] == "Router"
-        assert ev["delta"] == 500
-
-    def test_heap_status_periodic(self) -> None:
-        out = parse_log_line(
-            "HEAP  | 00:00:30 30 [Power] "
-            "Heap status: 92344/200000 bytes free (-128), running 8/12 threads"
-        )
-        assert out["heap_free"] == 92344
-        assert out["heap_total"] == 200000
-        assert out["heap_delta"] == -128
-
-
-class TestRecorderDebugHeapSynthesis:
-    def test_log_with_heap_writes_telemetry(self, recorder: "Recorder") -> None:
-        # When a log line carries [heap N], the recorder should also
-        # emit a synthesized telemetry row tagged source=debug_heap.
-        recorder._on_log_line(
-            "INFO  | 00:00:00 1 [Main] [heap 88888] hello",
-            _FakeIface(),
-        )
-        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
-        synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
-        assert len(synth) == 1
-        assert synth[0]["fields"]["heap_free_bytes"] == 88888
-        assert synth[0]["variant"] == "local"
-
-    def test_heap_status_writes_total_too(self, recorder: "Recorder") -> None:
-        recorder._on_log_line(
-            "HEAP  | 00:00:30 30 [Power] "
-            "Heap status: 50000/200000 bytes free (-100), running 8/12 threads",
-            _FakeIface(),
-        )
-        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
-        synth = [json.loads(r) for r in telem if '"source":"debug_heap"' in r]
-        assert synth[-1]["fields"]["heap_free_bytes"] == 50000
-        assert synth[-1]["fields"]["heap_total_bytes"] == 200000
-
-    def test_no_heap_no_synthesis(self, recorder: "Recorder") -> None:
-        # Plain log line (no [heap N], no Heap status) — telemetry.jsonl
-        # should NOT gain a synth row.
-        before = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
-        recorder._on_log_line("INFO  | 00:00:00 1 [Main] just a message", _FakeIface())
-        after = (recorder.base_dir / "telemetry.jsonl").read_text().count("\n")
-        assert after == before
-
-    def test_thread_leak_event_persists_on_log_row(self, recorder: "Recorder") -> None:
-        recorder._on_log_line(
-            "HEAP  | 00:00:01 100 [Power] [heap 90000] "
-            "------ Thread MeshPacket leaked heap 92344 -> 90000 (-2344) ------",
-            _FakeIface(),
-        )
-        rows = [
-            json.loads(r)
-            for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
-            if r
-        ]
-        evt_rows = [r for r in rows if r.get("heap_event")]
-        assert len(evt_rows) == 1
-        assert evt_rows[0]["heap_event"]["thread"] == "MeshPacket"
-        assert evt_rows[0]["heap_event"]["delta"] == -2344
-
-
-class TestSerialTap:
-    def test_serial_line_records_log_and_synthesizes_heap(
-        self, recorder: "Recorder"
-    ) -> None:
-        recorder._on_serial_line(
-            "INFO  | 00:00:00 5 [Main] [heap 88888] tap-line",
-            port="/dev/cu.tap",
-        )
-        logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
-        telem = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
-        log_rows = [json.loads(r) for r in logs if r]
-        # Find the row from this call (port=/dev/cu.tap, role=serial_session)
-        tap_rows = [r for r in log_rows if r.get("port") == "/dev/cu.tap"]
-        assert len(tap_rows) == 1
-        assert tap_rows[0]["role"] == "serial_session"
-        assert tap_rows[0]["level"] == "INFO"
-        assert tap_rows[0]["tag"] == "Main"
-        assert tap_rows[0]["heap_free"] == 88888
-        synth = [json.loads(r) for r in telem if '"source":"debug_heap_serial"' in r]
-        assert len(synth) == 1
-        assert synth[0]["fields"]["heap_free_bytes"] == 88888
-        assert synth[0]["role"] == "serial_session"
-
-    def test_serial_line_thread_leak_event(self, recorder: "Recorder") -> None:
-        recorder._on_serial_line(
-            "HEAP  | 00:00:30 30 [Power] [heap 53484] "
-            "------ Thread Router leaked heap 53612 -> 53484 (-128) ------",
-            port="/dev/cu.tap",
-        )
-        rows = [
-            json.loads(r)
-            for r in (recorder.base_dir / "logs.jsonl").read_text().splitlines()
-            if r
-        ]
-        evt = [r for r in rows if r.get("heap_event")]
-        assert len(evt) == 1
-        assert evt[0]["heap_event"]["thread"] == "Router"
-        assert evt[0]["heap_event"]["delta"] == -128
-        # Heap also synthesized.
-        telem = (recorder.base_dir / "telemetry.jsonl").read_text()
-        assert '"source":"debug_heap_serial"' in telem
-
-    def test_serial_line_pause(self, recorder: "Recorder") -> None:
-        recorder.pause("baseline")
-        recorder._on_serial_line(
-            "INFO  | 00:00:00 1 [t] [heap 1000] dropped",
-            port="/dev/cu.tap",
-        )
-        # Only the pause event row should exist; no tap row.
-        logs = (recorder.base_dir / "logs.jsonl").read_text()
-        assert "dropped" not in logs
-
-    def test_serial_line_handler_swallows_exceptions(
-        self, recorder: "Recorder"
-    ) -> None:
-        # Hostile input — should not raise.
-        recorder._on_serial_line(None, port="/dev/cu.tap")  # type: ignore[arg-type]
-        recorder._on_serial_line(b"\x00\x01\x02\x03", port="/dev/cu.tap")  # type: ignore[arg-type]
-        # Survived.
-
-
-class TestExtractTelemetry:
-    def test_local_stats_camel(self) -> None:
-        pkt = {
-            "decoded": {
-                "telemetry": {
-                    "localStats": {"heap_total_bytes": 1000, "heap_free_bytes": 600}
-                }
-            }
-        }
-        out = extract_telemetry(pkt)
-        assert out is not None
-        assert out["variant"] == "local"
-        assert out["fields"]["heap_free_bytes"] == 600
-
-    def test_device_metrics_snake(self) -> None:
-        pkt = {
-            "decoded": {
-                "telemetry": {"device_metrics": {"battery_level": 88, "voltage": 4.1}}
-            }
-        }
-        out = extract_telemetry(pkt)
-        assert out is not None
-        assert out["variant"] == "device"
-        assert out["fields"]["battery_level"] == 88
-
-    def test_unknown_variant_returns_none(self) -> None:
-        assert extract_telemetry({"decoded": {"telemetry": {"weird": {}}}}) is None
-        assert extract_telemetry({}) is None
-        assert extract_telemetry({"decoded": "not-a-dict"}) is None
-
-
-class TestSummarizePacket:
-    def test_text_with_payload(self) -> None:
-        pkt = {
-            "fromId": "!abc",
-            "toId": "!def",
-            "decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hello"},
-            "hopLimit": 3,
-        }
-        out = summarize_packet(pkt)
-        assert out["from_node"] == "!abc"
-        assert out["portnum"] == "TEXT_MESSAGE_APP"
-        assert out["payload_size"] == 5
-        assert out["payload_hex_prefix"] == "68656c6c6f"
-
-    def test_no_decoded(self) -> None:
-        out = summarize_packet({"fromId": "!abc"})
-        assert out["from_node"] == "!abc"
-        assert out["portnum"] is None
-
-
-class TestInterfaceLabel:
-    def test_serial(self) -> None:
-        assert interface_label(_FakeIface()) == {
-            "port": "/dev/cu.fake",
-            "role": "serial",
-        }
-
-    def test_tcp(self) -> None:
-        class T:
-            hostname = "node.lan"
-            portNumber = 4403
-
-        assert interface_label(T()) == {"port": "tcp://node.lan:4403", "role": "tcp"}
-
-    def test_unknown(self) -> None:
-        assert interface_label(object()) == {"port": "object", "role": None}
-
-    def test_none(self) -> None:
-        assert interface_label(None) == {"port": None, "role": None}
-
-
-# -- recorder write side ---------------------------------------------
-
-
-class TestRecorderWrites:
-    def test_log_line_is_recorded(self, recorder: Recorder) -> None:
-        recorder._on_log_line("INFO  | 12:34:56 99 [T] hi", _FakeIface())
-        path = recorder.base_dir / "logs.jsonl"
-        rows = [json.loads(line) for line in path.read_text().splitlines() if line]
-        # First row is recorder_start_event mirror? No — that's events.jsonl only.
-        assert any(r.get("level") == "INFO" and r.get("tag") == "T" for r in rows)
-
-    def test_telemetry_recorded_and_packet_double(self, recorder: Recorder) -> None:
-        # _on_telemetry alone — only telemetry.jsonl
-        recorder._on_telemetry(
-            {
-                "fromId": "!abc",
-                "decoded": {"telemetry": {"localStats": {"heap_free_bytes": 600}}},
-            },
-            _FakeIface(),
-        )
-        telem_rows = (recorder.base_dir / "telemetry.jsonl").read_text().splitlines()
-        assert any('"variant":"local"' in r for r in telem_rows)
-
-    def test_packets_summary(self, recorder: Recorder) -> None:
-        recorder._on_receive(
-            {
-                "fromId": "!abc",
-                "toId": "!def",
-                "decoded": {"portnum": "TEXT_MESSAGE_APP", "payload": b"hi"},
-            },
-            _FakeIface(),
-        )
-        rows = (recorder.base_dir / "packets.jsonl").read_text().splitlines()
-        assert any('"portnum":"TEXT_MESSAGE_APP"' in r for r in rows)
-
-    def test_mark_event_round_trip(self, recorder: Recorder) -> None:
-        out = recorder.mark_event("checkpoint", note="midpoint")
-        assert "ts" in out
-        events = (recorder.base_dir / "events.jsonl").read_text().splitlines()
-        logs = (recorder.base_dir / "logs.jsonl").read_text().splitlines()
-        assert any('"label":"checkpoint"' in r and '"kind":"mark"' in r for r in events)
-        assert any('"level":"MARK"' in r and "checkpoint" in r for r in logs)
-
-    def test_pause_drops_writes(self, recorder: Recorder) -> None:
-        before = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
-        recorder.pause(reason="baseline")
-        recorder._on_log_line("INFO  | 00:00:00 1 [t] swallowed", _FakeIface())
-        after = len((recorder.base_dir / "logs.jsonl").read_text().splitlines())
-        assert after == before
-        recorder.resume()
-        recorder._on_log_line("INFO  | 00:00:00 2 [t] kept", _FakeIface())
-        post_resume = (recorder.base_dir / "logs.jsonl").read_text()
-        assert "kept" in post_resume
-
-    def test_pubsub_handler_swallows_exceptions(self, recorder: Recorder) -> None:
-        # If the writer dies, the pubsub callback must NOT raise — that
-        # would crash the meshtastic receive thread.
-        bad_packet = object()  # not a dict
-        recorder._on_receive(bad_packet, _FakeIface())  # type: ignore[arg-type]
-        recorder._on_telemetry(bad_packet, _FakeIface())  # type: ignore[arg-type]
-        recorder._on_log_line(None, _FakeIface())  # type: ignore[arg-type]
-        # No assertion needed — survival is the test.
-
-
-# -- log_query read side ---------------------------------------------
-
-
-class TestLogQuery:
-    def test_logs_window_grep_and_level(self, recorder: Recorder) -> None:
-        recorder._on_log_line("INFO  | 12:00:00 1 [A] alpha", _FakeIface())
-        recorder._on_log_line("WARN  | 12:00:01 2 [B] bravo failed", _FakeIface())
-        recorder._on_log_line("ERROR | 12:00:02 3 [C] charlie failed", _FakeIface())
-
-        out = log_query.logs_window(start="-1m", level="WARN|ERROR", max_lines=10)
-        assert out["total_matched"] == 2
-        levels = {r["level"] for r in out["lines"]}
-        assert levels == {"WARN", "ERROR"}
-
-        out2 = log_query.logs_window(start="-1m", grep=r"failed$", max_lines=10)
-        assert out2["total_matched"] == 2
-
-    def test_logs_window_invalid_regex(self, recorder: Recorder) -> None:
-        recorder._on_log_line("INFO  | 12:00:00 1 [A] alpha", _FakeIface())
-        with pytest.raises(ValueError, match="invalid grep regex"):
-            log_query.logs_window(start="-1m", grep="(")
-
-    def test_telemetry_timeline_slope_and_downsample(self, recorder: Recorder) -> None:
-        # Synthesize a downward leak: 100 points, free_heap drops 1 byte/sample.
-        base_ts = time.time() - 60
-        for i in range(100):
-            recorder._files["telemetry"].write(
-                {
-                    "ts": base_ts + i * 0.5,
-                    "port": "/dev/cu.fake",
-                    "role": "serial",
-                    "from_node": "!abc",
-                    "variant": "local",
-                    "fields": {"heap_free_bytes": 10000 - i},
-                }
-            )
-
-        out = log_query.telemetry_timeline(
-            window="2m", variant="local", field="free_heap", max_points=10
-        )
-        assert out["samples"] == 100
-        assert len(out["points"]) <= 10
-        # Negative slope (heap dropping). Magnitude: 1 byte every 0.5s = 120/min.
-        assert out["slope_per_min"] is not None
-        assert out["slope_per_min"] < -100
-
-    def test_export_bundles_slice(self, recorder: Recorder, tmp_path: Path) -> None:
-        recorder._on_log_line("INFO  | 00:00:00 1 [t] one", _FakeIface())
-        recorder._on_log_line("INFO  | 00:00:00 2 [t] two", _FakeIface())
-        dest = tmp_path / "bundle"
-        out = log_query.export(start="-1m", end="now", dest_dir=str(dest))
-        assert (dest / "logs.jsonl").exists()
-        assert "logs" in out["paths"]
-
-
-# -- time parser -----------------------------------------------------
-
-
-class TestParseTime:
-    def test_relative(self) -> None:
-        now = 1_000_000.0
-        assert log_query._parse_time("-15m", now=now) == now - 900
-        assert log_query._parse_time("-2h", now=now) == now - 7200
-        assert log_query._parse_time("-1d", now=now) == now - 86400
-
-    def test_now_and_epoch(self) -> None:
-        now = 1_000_000.0
-        assert log_query._parse_time("now", now=now) == now
-        assert log_query._parse_time(now) == now
-
-    def test_iso(self) -> None:
-        ts = log_query._parse_time("2026-01-01T00:00:00Z")
-        assert isinstance(ts, float) and ts > 1_700_000_000
-
-    def test_naive_iso_assumes_utc(self) -> None:
-        assert log_query._parse_time("2026-01-01T00:00:00") == log_query._parse_time(
-            "2026-01-01T00:00:00Z"
-        )
-
-    def test_invalid(self) -> None:
-        with pytest.raises(ValueError):
-            log_query._parse_time("not a time")
-
-
-# -- rotation --------------------------------------------------------
-
-
-class TestRotation:
-    def test_size_cap_rotates_and_gzips(self, tmp_path: Path) -> None:
-        path = tmp_path / "rot.jsonl"
-        r = _RotatingJsonl(path, max_bytes=512, keep_archives=5, check_every=1)
-        for i in range(100):
-            r.write({"ts": float(i), "i": i, "pad": "x" * 40})
-        r.close()
-        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
-        assert archives, "expected at least one rotation"
-        # Archive content is valid gzip + valid JSONL
-        with gzip.open(archives[0], "rt") as fh:
-            first = json.loads(fh.readline())
-            assert "ts" in first
-
-    def test_archive_pruning(self, tmp_path: Path) -> None:
-        path = tmp_path / "rot.jsonl"
-        r = _RotatingJsonl(path, max_bytes=200, keep_archives=2, check_every=1)
-        # Force several rotations.
-        for _ in range(8):
-            for i in range(20):
-                r.write({"ts": float(i), "pad": "x" * 30})
-            r.force_rotate()
-        r.close()
-        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
-        assert len(archives) <= 2, f"expected ≤2 kept archives, got {len(archives)}"
-
-    def test_archive_pruning_uses_filename_order(self, tmp_path: Path) -> None:
-        path = tmp_path / "rot.jsonl"
-        r = _RotatingJsonl(path, keep_archives=2)
-        old = tmp_path / "rot.20260101-000000-000000-00000.jsonl.gz"
-        mid = tmp_path / "rot.20260101-000001-000000-00000.jsonl.gz"
-        new = tmp_path / "rot.20260101-000002-000000-00000.jsonl.gz"
-        for archive in (old, mid, new):
-            with gzip.open(archive, "wt", encoding="utf-8") as fh:
-                fh.write('{"ts":1}\n')
-        # Deliberately scramble mtimes so lexicographic filename order is
-        # the only stable chronological signal.
-        os.utime(old, (300, 300))
-        os.utime(mid, (100, 100))
-        os.utime(new, (200, 200))
-
-        r._prune_archives()
-        r.close()
-
-        archives = sorted(p.name for p in tmp_path.glob("rot.*.jsonl.gz"))
-        assert archives == [mid.name, new.name]
-
-    def test_force_rotate_when_below_threshold(self, tmp_path: Path) -> None:
-        path = tmp_path / "rot.jsonl"
-        r = _RotatingJsonl(path, max_bytes=10_000_000, check_every=999_999)
-        r.write({"ts": 1.0, "msg": "tiny"})
-        r.force_rotate()
-        r.write({"ts": 2.0, "msg": "after-rotate"})
-        r.close()
-        archives = sorted(tmp_path.glob("rot.*.jsonl.gz"))
-        assert len(archives) == 1
-        assert path.exists()
-        assert "after-rotate" in path.read_text()
-
-
-class TestRecorderLocks:
-    def test_force_rotate_all_returns_status(self, recorder: Recorder) -> None:
-        out = recorder.force_rotate_all()
-        assert out["running"] is True
-        assert out["files"]
-
-    def test_wire_pubsub_logs_subscription_failure(
-        self,
-        tmp_path: Path,
-        monkeypatch: pytest.MonkeyPatch,
-        caplog: pytest.LogCaptureFixture,
-    ) -> None:
-        class FailingPubSubMock:
-            def subscribe(self, callback: object, topic: str) -> None:
-                raise RuntimeError("boom")
-
-        monkeypatch.setattr(pubsub, "pub", FailingPubSubMock())
-        recorder = Recorder(base_dir=tmp_path)
-        with caplog.at_level(logging.WARNING):
-            recorder._wire_pubsub()
-        assert (
-            "Recorder failed to subscribe to meshtastic.log.line: boom" in caplog.text
-        )

meshtasticd.spec.rpkg

@@ -34,7 +34,6 @@ BuildRequires: python3dist(grpcio-tools)
 BuildRequires: git-core
 BuildRequires: gcc-c++
 BuildRequires: pkgconfig(yaml-cpp)
-BuildRequires: pkgconfig(jsoncpp)
 BuildRequires: pkgconfig(libgpiod)
 BuildRequires: pkgconfig(bluez)
 BuildRequires: pkgconfig(libusb-1.0)

platformio.ini

@@ -2,42 +2,21 @@
 ; https://docs.platformio.org/page/projectconf.html
 
 [platformio]
-default_envs = heltec-v3
+default_envs = tbeam
 
 extra_configs =
 	variants/*/*.ini
 	variants/*/*/platformio.ini
 	variants/*/diy/*/platformio.ini
+	src/graphics/niche/InkHUD/PlatformioConfig.ini
 
 description = Meshtastic
 
-; E-Ink / NicheGraphics build helpers.
-[niche]
-build_src_filter =
-  +<graphics/eink/>
-build_flags =
-  -D MESHTASTIC_INCLUDE_NICHE_GRAPHICS
-
-[inkhud]
-build_src_filter =
-  ${niche.build_src_filter}
-  +<graphics/niche/>
-build_flags =
-  ${niche.build_flags}
-  -D MESHTASTIC_INCLUDE_INKHUD ; Use InkHUD as the UI
-  -D MESHTASTIC_EXCLUDE_SCREEN ; Suppress default Screen class
-  -D MESHTASTIC_EXCLUDE_INPUTBROKER ; Suppress default input handling
-  -D HAS_BUTTON=0 ; Suppress default ButtonThread
-lib_deps =
-  # renovate: datasource=github-tags depName=GFX_Root packageName=ZinggJM/GFX_Root
-  https://github.com/ZinggJM/GFX_Root/archive/3195764e352a0d2567c8d277ac408ca7293a99b0.zip ; Used by InkHUD as a "slimmer" version of AdafruitGFX
-
 [env]
 test_build_src = true
 extra_scripts =
 	pre:bin/platformio-pre.py
 	bin/platformio-custom.py
-	post:extra_scripts/nrf54l15_linker.py
 ; note: we add src to our include search path so that lmic_project_config can override
 ; note: TINYGPS_OPTION_NO_CUSTOM_FIELDS is VERY important.  We don't use custom fields and somewhere in that pile
 ; of code is a heap corruption bug!
@@ -70,7 +49,6 @@ build_flags = -Wno-missing-field-initializers
 	-DRADIOLIB_EXCLUDE_PAGER=1
 	-DRADIOLIB_EXCLUDE_FSK4=1
 	-DRADIOLIB_EXCLUDE_APRS=1
-	-DRADIOLIB_EXCLUDE_ADSB=1
 	-DRADIOLIB_EXCLUDE_LORAWAN=1
 	-DMESHTASTIC_EXCLUDE_DROPZONE=1
 	-DMESHTASTIC_EXCLUDE_REPLYBOT=1
@@ -123,7 +101,7 @@ build_unflags =
 	-std=gnu++11
 build_flags = ${env.build_flags} -Os
 	-std=gnu++17
-build_src_filter = ${env.build_src_filter} -<platform/portduino/> -<graphics/niche/> -<graphics/eink/>
+build_src_filter = ${env.build_src_filter} -<platform/portduino/> -<graphics/niche/>
 
 ; Common libs for communicating over TCP/IP networks such as MQTT
 [networking_base]
@@ -159,7 +137,7 @@ lib_deps =
 	# renovate: datasource=github-tags depName=Adafruit GFX packageName=adafruit/Adafruit-GFX-Library
 	https://github.com/adafruit/Adafruit-GFX-Library/archive/refs/tags/1.12.6.zip
 	# renovate: datasource=github-tags depName=NeoPixel packageName=adafruit/Adafruit_NeoPixel
-	https://github.com/adafruit/Adafruit_NeoPixel/archive/1.15.5.zip
+	https://github.com/adafruit/Adafruit_NeoPixel/archive/refs/tags/1.15.4.zip
 	# renovate: datasource=github-tags depName=Adafruit SSD1306 packageName=adafruit/Adafruit_SSD1306
 	https://github.com/adafruit/Adafruit_SSD1306/archive/refs/tags/2.5.16.zip
 	# renovate: datasource=github-tags depName=Adafruit BMP280 packageName=adafruit/Adafruit_BMP280_Library
@@ -206,16 +184,12 @@ lib_deps =
 	https://github.com/sparkfun/SparkFun_MAX3010x_Sensor_Library/archive/refs/tags/v1.1.2.zip
 	# renovate: datasource=github-tags depName=SparkFun 9DoF IMU Breakout ICM 20948 packageName=sparkfun/SparkFun_ICM-20948_ArduinoLibrary
 	https://github.com/sparkfun/SparkFun_ICM-20948_ArduinoLibrary/archive/refs/tags/v1.3.2.zip
-	# renovate: datasource=github-tags depName=TDK InvenSense ICM42670P packageName=tdk-invn-oss/motion.arduino.ICM42670P
-	https://github.com/tdk-invn-oss/motion.arduino.ICM42670P/archive/refs/tags/1.0.8.zip
 	# renovate: datasource=github-tags depName=Adafruit LTR390 Library packageName=adafruit/Adafruit_LTR390
 	https://github.com/adafruit/Adafruit_LTR390/archive/refs/tags/1.1.2.zip
 	# renovate: datasource=github-tags depName=Adafruit PCT2075 packageName=adafruit/Adafruit_PCT2075
 	https://github.com/adafruit/Adafruit_PCT2075/archive/refs/tags/1.0.6.zip
 	# renovate: datasource=github-tags depName=DFRobot_BMM150 packageName=dfrobot/DFRobot_BMM150
 	https://github.com/DFRobot/DFRobot_BMM150/archive/refs/tags/V1.0.0.zip
-	# renovate: datasource=github-tags depName=SparkFun MMC5983MA Magnetometer packageName=sparkfun/SparkFun_MMC5983MA_Magnetometer_Arduino_Library
-	https://github.com/sparkfun/SparkFun_MMC5983MA_Magnetometer_Arduino_Library/archive/refs/tags/v1.1.4.zip
 	# renovate: datasource=github-tags depName=Adafruit_TSL2561 packageName=adafruit/Adafruit_TSL2561
 	https://github.com/adafruit/Adafruit_TSL2561/archive/refs/tags/1.1.3.zip
 	# renovate: datasource=github-tags depName=BH1750_WE packageName=wollewald/BH1750_WE
@@ -228,10 +202,16 @@ lib_deps =
 	https://github.com/adafruit/Adafruit_BMP3XX/archive/refs/tags/2.1.6.zip
 	# renovate: datasource=github-tags depName=Adafruit MAX1704X packageName=adafruit/Adafruit_MAX1704X
 	https://github.com/adafruit/Adafruit_MAX1704X/archive/refs/tags/1.0.3.zip
+	# renovate: datasource=github-tags depName=Adafruit SHTC3 packageName=adafruit/Adafruit_SHTC3
+	https://github.com/adafruit/Adafruit_SHTC3/archive/refs/tags/1.0.2.zip
 	# renovate: datasource=github-tags depName=Adafruit LPS2X packageName=adafruit/Adafruit_LPS2X
 	https://github.com/adafruit/Adafruit_LPS2X/archive/refs/tags/2.0.6.zip
+	# renovate: datasource=github-tags depName=Adafruit SHT31 packageName=adafruit/Adafruit_SHT31
+	https://github.com/adafruit/Adafruit_SHT31/archive/refs/tags/2.2.2.zip
 	# renovate: datasource=github-tags depName=Adafruit VEML7700 packageName=adafruit/Adafruit_VEML7700
 	https://github.com/adafruit/Adafruit_VEML7700/archive/refs/tags/2.1.6.zip
+	# renovate: datasource=github-tags depName=Adafruit SHT4x packageName=adafruit/Adafruit_SHT4X
+	https://github.com/adafruit/Adafruit_SHT4X/archive/refs/tags/1.0.5.zip
 	# renovate: datasource=github-tags depName=SparkFun Qwiic Scale NAU7802 packageName=sparkfun/SparkFun_Qwiic_Scale_NAU7802_Arduino_Library
 	https://github.com/sparkfun/SparkFun_Qwiic_Scale_NAU7802_Arduino_Library/archive/refs/tags/v1.0.6.zip
 	# renovate: datasource=custom.pio depName=ClosedCube OPT3001 packageName=closedcube/library/ClosedCube OPT3001

src/DebugConfiguration.cpp

@@ -151,7 +151,7 @@ inline bool Syslog::_sendLog(uint16_t pri, const char *appName, const char *mess
     if (!this->_enabled)
         return false;
 
-    if ((this->_server == NULL && this->_ip == IPAddress(0, 0, 0, 0)) || this->_port == 0)
+    if ((this->_server == NULL && this->_ip == INADDR_NONE) || this->_port == 0)
         return false;
 
     // Check priority against priMask values.

src/DebugConfiguration.h

@@ -13,11 +13,6 @@ extern MemGet memGet;
 #define LED_STATE_ON 1
 #endif
 
-// WIFI LED
-#ifndef WIFI_STATE_ON
-#define WIFI_STATE_ON 1
-#endif
-
 // -----------------------------------------------------------------------------
 // DEBUG
 // -----------------------------------------------------------------------------
@@ -152,16 +147,13 @@ extern "C" void logLegacy(const char *level, const char *fmt, ...);
 // Default Bluetooth PIN
 #define defaultBLEPin 123456
 
-#if HAS_ETHERNET && defined(USE_ARDUINO_ETHERNET)
-#include <Ethernet.h> // arduino-libraries/Ethernet — supports W5500 auto-detect
-#elif HAS_ETHERNET && defined(USE_CH390D)
-#include <ESP32_CH390.h>
-#elif HAS_ETHERNET && !defined(USE_WS5500)
+#if HAS_ETHERNET && !defined(USE_WS5500)
 #include <RAK13800_W5100S.h>
 #endif // HAS_ETHERNET
 
-#if HAS_ETHERNET && defined(ARCH_ESP32)
-#include <ETH.h>
+#if HAS_ETHERNET && defined(USE_WS5500)
+#include <ETHClass2.h>
+#define ETH ETH2
 #endif // HAS_ETHERNET
 
 #if HAS_WIFI

src/DisplayFormatters.cpp

@@ -1,5 +1,4 @@
 #include "DisplayFormatters.h"
-#include "MeshRadio.h"
 
 const char *DisplayFormatters::getModemPresetDisplayName(meshtastic_Config_LoRaConfig_ModemPreset preset, bool useShortName,
                                                          bool usePreset)
@@ -12,51 +11,33 @@ const char *DisplayFormatters::getModemPresetDisplayName(meshtastic_Config_LoRaC
     }
 
     switch (preset) {
-    case PRESET(SHORT_TURBO):
+    case meshtastic_Config_LoRaConfig_ModemPreset_SHORT_TURBO:
         return useShortName ? "ShortT" : "ShortTurbo";
         break;
-    case PRESET(SHORT_SLOW):
+    case meshtastic_Config_LoRaConfig_ModemPreset_SHORT_SLOW:
         return useShortName ? "ShortS" : "ShortSlow";
         break;
-    case PRESET(SHORT_FAST):
+    case meshtastic_Config_LoRaConfig_ModemPreset_SHORT_FAST:
         return useShortName ? "ShortF" : "ShortFast";
         break;
-    case PRESET(MEDIUM_SLOW):
+    case meshtastic_Config_LoRaConfig_ModemPreset_MEDIUM_SLOW:
         return useShortName ? "MedS" : "MediumSlow";
         break;
-    case PRESET(MEDIUM_FAST):
+    case meshtastic_Config_LoRaConfig_ModemPreset_MEDIUM_FAST:
         return useShortName ? "MedF" : "MediumFast";
         break;
-    case PRESET(LONG_SLOW):
+    case meshtastic_Config_LoRaConfig_ModemPreset_LONG_SLOW:
         return useShortName ? "LongS" : "LongSlow";
         break;
-    case PRESET(LONG_FAST):
+    case meshtastic_Config_LoRaConfig_ModemPreset_LONG_FAST:
         return useShortName ? "LongF" : "LongFast";
         break;
-    case PRESET(LONG_TURBO):
+    case meshtastic_Config_LoRaConfig_ModemPreset_LONG_TURBO:
         return useShortName ? "LongT" : "LongTurbo";
         break;
-    case PRESET(LONG_MODERATE):
+    case meshtastic_Config_LoRaConfig_ModemPreset_LONG_MODERATE:
         return useShortName ? "LongM" : "LongMod";
         break;
-    case PRESET(LITE_FAST):
-        return useShortName ? "LiteF" : "LiteFast";
-        break;
-    case PRESET(LITE_SLOW):
-        return useShortName ? "LiteS" : "LiteSlow";
-        break;
-    case PRESET(NARROW_FAST):
-        return useShortName ? "NarF" : "NarrowFast";
-        break;
-    case PRESET(NARROW_SLOW):
-        return useShortName ? "NarS" : "NarrowSlow";
-        break;
-    case PRESET(TINY_FAST):
-        return useShortName ? "TinyF" : "TinyFast";
-        break;
-    case PRESET(TINY_SLOW):
-        return useShortName ? "TinyS" : "TinySlow";
-        break;
     default:
         return useShortName ? "Custom" : "Invalid";
         break;

src/FSCommon.cpp

@@ -277,7 +277,7 @@ void fsInit()
  */
 void setupSDCard()
 {
-#if defined(HAS_SDCARD) && !defined(SDCARD_USE_SOFT_SPI) && !defined(HAS_SD_MMC)
+#if defined(HAS_SDCARD) && !defined(SDCARD_USE_SOFT_SPI)
     concurrency::LockGuard g(spiLock);
     SDHandler.begin(SPI_SCK, SPI_MISO, SPI_MOSI);
     if (!SD.begin(SDCARD_CS, SDHandler, SD_SPI_FREQUENCY)) {

src/FSCommon.h

@@ -48,14 +48,6 @@ using namespace STM32_LittleFS_Namespace;
 using namespace Adafruit_LittleFS_Namespace;
 #endif
 
-#if defined(ARCH_NRF54L15)
-// nRF54L15 — Zephyr LittleFS on 36 KB storage_partition (internal RRAM)
-#include "InternalFileSystem.h"
-#define FSCom InternalFS
-#define FSBegin() FSCom.begin()
-using namespace Adafruit_LittleFS_Namespace;
-#endif
-
 void fsInit();
 void fsListFiles();
 bool copyFile(const char *from, const char *to);

src/MessageStore.cpp

@@ -1,5 +1,5 @@
 #include "configuration.h"
-#if HAS_SCREEN || defined(MESHTASTIC_INCLUDE_NICHE_GRAPHICS)
+#if HAS_SCREEN
 #include "FSCommon.h"
 #include "MessageStore.h"
 #include "NodeDB.h"

src/MessageStore.h

@@ -1,6 +1,6 @@
 #pragma once
 
-#if HAS_SCREEN || defined(MESHTASTIC_INCLUDE_NICHE_GRAPHICS)
+#if HAS_SCREEN
 
 // Disable debug logging entirely on release builds of HELTEC_MESH_SOLAR for space constraints
 #if defined(HELTEC_MESH_SOLAR)

src/Power.cpp

@@ -24,13 +24,6 @@
 #include "meshUtils.h"
 #include "power/PowerHAL.h"
 #include "sleep.h"
-#ifdef ARCH_ESP32
-// #include <driver/adc.h>
-#include <esp_adc/adc_cali.h>
-#include <esp_adc/adc_cali_scheme.h>
-#include <esp_adc/adc_oneshot.h>
-#include <esp_err.h>
-#endif
 
 #if defined(ARCH_PORTDUINO)
 #include "api/WiFiServerAPI.h"
@@ -70,8 +63,9 @@
 #include <WiFi.h>
 #endif
 
-#if HAS_ETHERNET && defined(ARCH_ESP32)
-#include <ETH.h>
+#if HAS_ETHERNET && defined(USE_WS5500)
+#include <ETHClass2.h>
+#define ETH ETH2
 #endif // HAS_ETHERNET
 
 #endif
@@ -83,86 +77,21 @@
 #if defined(BATTERY_PIN) && defined(ARCH_ESP32)
 
 #ifndef BAT_MEASURE_ADC_UNIT // ADC1 is default
-static const adc_channel_t adc_channel = ADC_CHANNEL;
+static const adc1_channel_t adc_channel = ADC_CHANNEL;
 static const adc_unit_t unit = ADC_UNIT_1;
-#else  // ADC2
-static const adc_channel_t adc_channel = ADC_CHANNEL;
+#else // ADC2
+static const adc2_channel_t adc_channel = ADC_CHANNEL;
 static const adc_unit_t unit = ADC_UNIT_2;
+RTC_NOINIT_ATTR uint64_t RTC_reg_b;
+
 #endif // BAT_MEASURE_ADC_UNIT
 
-static adc_oneshot_unit_handle_t adc_handle = nullptr;
-static adc_cali_handle_t adc_cali_handle = nullptr;
-static bool adc_calibrated = false;
+esp_adc_cal_characteristics_t *adc_characs = (esp_adc_cal_characteristics_t *)calloc(1, sizeof(esp_adc_cal_characteristics_t));
 #ifndef ADC_ATTENUATION
 static const adc_atten_t atten = ADC_ATTEN_DB_12;
 #else
 static const adc_atten_t atten = ADC_ATTENUATION;
 #endif
-#ifdef ADC_BITWIDTH
-static const adc_bitwidth_t adc_width = ADC_BITWIDTH;
-#else
-static const adc_bitwidth_t adc_width = ADC_BITWIDTH_DEFAULT;
-#endif
-
-static int adcBitWidthToBits(adc_bitwidth_t width)
-{
-    switch (width) {
-    case ADC_BITWIDTH_9:
-        return 9;
-    case ADC_BITWIDTH_10:
-        return 10;
-    case ADC_BITWIDTH_11:
-        return 11;
-    case ADC_BITWIDTH_12:
-        return 12;
-#ifdef ADC_BITWIDTH_13
-    case ADC_BITWIDTH_13:
-        return 13;
-#endif
-    default:
-        return 12;
-    }
-}
-
-static bool initAdcCalibration()
-{
-#if ADC_CALI_SCHEME_CURVE_FITTING_SUPPORTED
-    adc_cali_curve_fitting_config_t cali_config = {
-        .unit_id = unit,
-        .atten = atten,
-        .bitwidth = adc_width,
-    };
-    esp_err_t ret = adc_cali_create_scheme_curve_fitting(&cali_config, &adc_cali_handle);
-    if (ret == ESP_OK) {
-        LOG_INFO("ADC calibration: curve fitting enabled");
-        return true;
-    }
-    if (ret != ESP_ERR_NOT_SUPPORTED) {
-        LOG_WARN("ADC calibration: curve fitting failed: %s", esp_err_to_name(ret));
-    }
-#endif
-
-#if ADC_CALI_SCHEME_LINE_FITTING_SUPPORTED
-    adc_cali_line_fitting_config_t cali_config = {
-        .unit_id = unit,
-        .atten = atten,
-        .bitwidth = adc_width,
-        .default_vref = DEFAULT_VREF,
-    };
-    esp_err_t ret = adc_cali_create_scheme_line_fitting(&cali_config, &adc_cali_handle);
-    if (ret == ESP_OK) {
-        LOG_INFO("ADC calibration: line fitting enabled");
-        return true;
-    }
-    if (ret != ESP_ERR_NOT_SUPPORTED) {
-        LOG_WARN("ADC calibration: line fitting failed: %s", esp_err_to_name(ret));
-    }
-#endif
-
-    LOG_INFO("ADC calibration not supported; using approximate scaling");
-    return false;
-}
-
 #endif // BATTERY_PIN && ARCH_ESP32
 
 #ifdef EXT_PWR_DETECT
@@ -438,20 +367,8 @@ class AnalogBatteryLevel : public HasBatteryLevel
             scaled *= operativeAdcMultiplier;
 #elif defined(ARCH_ESP32) // ADC block for espressif platforms
             raw = espAdcRead();
-            int voltage_mv = 0;
-            if (adc_calibrated && adc_cali_handle) {
-                if (adc_cali_raw_to_voltage(adc_cali_handle, raw, &voltage_mv) != ESP_OK) {
-                    LOG_WARN("ADC calibration read failed; using raw value");
-                    voltage_mv = 0;
-                }
-            }
-            if (voltage_mv == 0) {
-                // Fallback approximate conversion without calibration
-                const int bits = adcBitWidthToBits(adc_width);
-                const float max_code = powf(2.0f, bits) - 1.0f;
-                voltage_mv = (int)((raw / max_code) * DEFAULT_VREF);
-            }
-            scaled = voltage_mv * operativeAdcMultiplier;
+            scaled = esp_adc_cal_raw_to_voltage(raw, adc_characs);
+            scaled *= operativeAdcMultiplier;
 #else                     // block for all other platforms
 #ifdef ARCH_NRF52
             concurrency::LockGuard saadcGuard(concurrency::nrf52SaadcLock);
@@ -493,22 +410,51 @@ class AnalogBatteryLevel : public HasBatteryLevel
         uint32_t raw = 0;
         uint8_t raw_c = 0; // raw reading counter
 
-        if (!adc_handle) {
-            LOG_ERROR("ADC oneshot handle not initialized");
-            return 0;
-        }
-
+#ifndef BAT_MEASURE_ADC_UNIT // ADC1
         for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
-            int val = 0;
-            esp_err_t err = adc_oneshot_read(adc_handle, adc_channel, &val);
-            if (err == ESP_OK) {
-                raw += val;
+            int val_ = adc1_get_raw(adc_channel);
+            if (val_ >= 0) { // save only valid readings
+                raw += val_;
                 raw_c++;
+            }
+            // delayMicroseconds(100);
+        }
+#else                            // ADC2
+#ifdef CONFIG_IDF_TARGET_ESP32S3 // ESP32S3
+        // ADC2 wifi bug workaround not required, breaks compile
+        // On ESP32S3, ADC2 can take turns with Wifi (?)
+
+        int32_t adc_buf;
+        esp_err_t read_result;
+
+        // Multiple samples
+        for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
+            adc_buf = 0;
+            read_result = -1;
+
+            read_result = adc2_get_raw(adc_channel, ADC_WIDTH_BIT_12, &adc_buf);
+            if (read_result == ESP_OK) {
+                raw += adc_buf;
+                raw_c++; // Count valid samples
             } else {
-                LOG_DEBUG("ADC read failed: %s", esp_err_to_name(err));
+                LOG_DEBUG("An attempt to sample ADC2 failed");
             }
         }
 
+#else  // Other ESP32
+        int32_t adc_buf = 0;
+        for (int i = 0; i < BATTERY_SENSE_SAMPLES; i++) {
+            // ADC2 wifi bug workaround, see
+            // https://github.com/espressif/arduino-esp32/issues/102
+            WRITE_PERI_REG(SENS_SAR_READ_CTRL2_REG, RTC_reg_b);
+            SET_PERI_REG_MASK(SENS_SAR_READ_CTRL2_REG, SENS_SAR2_DATA_INV);
+            adc2_get_raw(adc_channel, ADC_WIDTH_BIT_12, &adc_buf);
+            raw += adc_buf;
+            raw_c++;
+        }
+#endif // BAT_MEASURE_ADC_UNIT
+
+#endif // End BAT_MEASURE_ADC_UNIT
         return (raw / (raw_c < 1 ? 1 : raw_c));
     }
 #endif
@@ -720,31 +666,42 @@ bool Power::analogInit()
 #ifdef ARCH_STM32WL
     analogReadResolution(BATTERY_SENSE_RESOLUTION_BITS);
 #elif defined(ARCH_ESP32) // ESP32 needs special analog stuff
-    adc_oneshot_unit_init_cfg_t init_config = {
-        .unit_id = unit,
-    };
 
-    if (!adc_handle) {
-        esp_err_t err = adc_oneshot_new_unit(&init_config, &adc_handle);
-        if (err != ESP_OK) {
-            LOG_ERROR("ADC oneshot init failed: %s", esp_err_to_name(err));
-            return false;
-        }
+#ifndef ADC_WIDTH // max resolution by default
+    static const adc_bits_width_t width = ADC_WIDTH_BIT_12;
+#else
+    static const adc_bits_width_t width = ADC_WIDTH;
+#endif
+#ifndef BAT_MEASURE_ADC_UNIT // ADC1
+    adc1_config_width(width);
+    adc1_config_channel_atten(adc_channel, atten);
+#else                        // ADC2
+    adc2_config_channel_atten(adc_channel, atten);
+#ifndef CONFIG_IDF_TARGET_ESP32S3
+    // ADC2 wifi bug workaround
+    // Not required with ESP32S3, breaks compile
+    RTC_reg_b = READ_PERI_REG(SENS_SAR_READ_CTRL2_REG);
+#endif
+#endif
+    // calibrate ADC
+    esp_adc_cal_value_t val_type = esp_adc_cal_characterize(unit, atten, width, DEFAULT_VREF, adc_characs);
+    // show ADC characterization base
+    if (val_type == ESP_ADC_CAL_VAL_EFUSE_TP) {
+        LOG_INFO("ADC config based on Two Point values stored in eFuse");
+    } else if (val_type == ESP_ADC_CAL_VAL_EFUSE_VREF) {
+        LOG_INFO("ADC config based on reference voltage stored in eFuse");
     }
-
-    adc_oneshot_chan_cfg_t chan_cfg = {
-        .atten = atten,
-        .bitwidth = adc_width,
-    };
-
-    esp_err_t err = adc_oneshot_config_channel(adc_handle, adc_channel, &chan_cfg);
-    if (err != ESP_OK) {
-        LOG_ERROR("ADC channel config failed: %s", esp_err_to_name(err));
-        return false;
+#ifdef CONFIG_IDF_TARGET_ESP32S3
+    // ESP32S3
+    else if (val_type == ESP_ADC_CAL_VAL_EFUSE_TP_FIT) {
+        LOG_INFO("ADC config based on Two Point values and fitting curve "
+                 "coefficients stored in eFuse");
     }
-
-    adc_calibrated = initAdcCalibration();
-#endif                    // ARCH_ESP32
+#endif
+    else {
+        LOG_INFO("ADC config based on default reference voltage");
+    }
+#endif // ARCH_ESP32
 
     // NRF52 ADC init moved to powerHAL_init in nrf52 platform
 
@@ -947,16 +904,7 @@ void Power::readPowerStatus()
 
     // Notify any status instances that are observing us
     const PowerStatus powerStatus2 = PowerStatus(hasBattery, usbPowered, isChargingNow, batteryVoltageMv, batteryChargePercent);
-
-    // Log battery-presence transitions once; skip OptUnknown so we don't lie before the first probe.
-    static OptionalBool prevHasBattery = OptUnknown;
-    if (hasBattery != OptUnknown && hasBattery != prevHasBattery) {
-        LOG_INFO("Power: battery hardware %s", hasBattery == OptTrue ? "detected" : "absent (USB-only)");
-        prevHasBattery = hasBattery;
-    }
-
-    // Periodic telemetry only emits when a battery is actually present (otherwise values are constant -1/0).
-    if (hasBattery == OptTrue && !Throttle::isWithinTimespanMs(lastLogTime, 50 * 1000)) {
+    if (millis() > lastLogTime + 50 * 1000) {
         LOG_DEBUG("Battery: usbPower=%d, isCharging=%d, batMv=%d, batPct=%d", powerStatus2.getHasUSB(),
                   powerStatus2.getIsCharging(), powerStatus2.getBatteryVoltageMv(), powerStatus2.getBatteryChargePercent());
         lastLogTime = millis();

src/RF95Configuration.h

@@ -1,16 +1,6 @@
 // TODO refactor this out with better radio configuration system
 #ifdef USE_RF95
-
-#ifndef RF95_RESET
 #define RF95_RESET LORA_RESET
-#endif
-
-#ifndef RF95_IRQ
-#define RF95_IRQ LORA_DIO0 // on SX1262 version this is a no connect DIO0
-#endif
-
-#ifndef RF95_DIO1
+#define RF95_IRQ LORA_DIO0  // on SX1262 version this is a no connect DIO0
 #define RF95_DIO1 LORA_DIO1 // Note: not really used for RF95, but used for pure SX127x
 #endif
-
-#endif

src/RedirectablePrint.cpp

@@ -51,7 +51,7 @@ size_t RedirectablePrint::write(uint8_t c)
 size_t RedirectablePrint::vprintf(const char *logLevel, const char *format, va_list arg)
 {
     va_list copy;
-#if ARCH_PORTDUINO
+#if ENABLE_JSON_LOGGING || ARCH_PORTDUINO
     static char printBuf[512];
 #else
     static char printBuf[160];
@@ -225,16 +225,14 @@ void RedirectablePrint::log_to_ble(const char *logLevel, const char *format, va_
         isBleConnected = nimbleBluetooth && nimbleBluetooth->isActive() && nimbleBluetooth->isConnected();
 #elif defined(ARCH_NRF52)
         isBleConnected = nrf52Bluetooth != nullptr && nrf52Bluetooth->isConnected();
-#elif defined(ARCH_NRF54L15)
-        isBleConnected = nrf54l15Bluetooth != nullptr && nrf54l15Bluetooth->isConnected();
 #endif
         if (isBleConnected) {
             auto thread = concurrency::OSThread::currentThread;
             meshtastic_LogRecord logRecord = meshtastic_LogRecord_init_zero;
             logRecord.level = getLogLevel(logLevel);
-            vsnprintf(logRecord.message, sizeof(logRecord.message), format, arg);
+            vsprintf(logRecord.message, format, arg);
             if (thread)
-                strlcpy(logRecord.source, thread->ThreadName.c_str(), sizeof(logRecord.source));
+                strcpy(logRecord.source, thread->ThreadName.c_str());
             logRecord.time = getValidTime(RTCQuality::RTCQualityDevice, true);
 
             auto buffer = std::unique_ptr<uint8_t[]>(new uint8_t[meshtastic_LogRecord_size]);
@@ -243,8 +241,6 @@ void RedirectablePrint::log_to_ble(const char *logLevel, const char *format, va_
             nimbleBluetooth->sendLog(buffer.get(), size);
 #elif defined(ARCH_NRF52)
             nrf52Bluetooth->sendLog(buffer.get(), size);
-#elif defined(ARCH_NRF54L15)
-            nrf54l15Bluetooth->sendLog(buffer.get(), size);
 #endif
         }
     }

src/airtime.cpp

@@ -133,12 +133,11 @@ bool AirTime::isTxAllowedChannelUtil(bool polite)
 
 bool AirTime::isTxAllowedAirUtil()
 {
-    float effectiveDutyCycle = getEffectiveDutyCycle();
-    if (!config.lora.override_duty_cycle && effectiveDutyCycle < 100) {
-        if (utilizationTXPercent() < effectiveDutyCycle * polite_duty_cycle_percent / 100) {
+    if (!config.lora.override_duty_cycle && myRegion->dutyCycle < 100) {
+        if (utilizationTXPercent() < myRegion->dutyCycle * polite_duty_cycle_percent / 100) {
             return true;
         } else {
-            LOG_WARN("TX air util. >%f%%. Skip send", effectiveDutyCycle * polite_duty_cycle_percent / 100);
+            LOG_WARN("TX air util. >%f%%. Skip send", myRegion->dutyCycle * polite_duty_cycle_percent / 100);
             return false;
         }
     }

src/configuration.h

@@ -31,11 +31,6 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #endif
 #if __has_include("SensorRtcHelper.hpp")
 #include "SensorRtcHelper.hpp"
-// SensorLib defines isBitSet as a macro; undefine it here to avoid conflicts
-// with the SparkFun MMC5983MA library, which has a class method of the same name.
-#ifdef isBitSet
-#undef isBitSet
-#endif
 #endif
 
 /* Offer chance for variant-specific defines */
@@ -248,7 +243,6 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #define QMI8658_ADDR 0x6B
 #define QMC5883L_ADDR 0x0D
 #define HMC5883L_ADDR 0x1E
-#define MMC5983MA_ADDR 0x30
 #define SHTC3_ADDR 0x70
 #define LPS22HB_ADDR 0x5C
 #define LPS22HB_ADDR_ALT 0x5D
@@ -298,8 +292,6 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #define DA217_ADDR 0x26
 #define BMI270_ADDR 0x68
 #define BMI270_ADDR_ALT 0x69
-#define ICM42607P_ADDR 0x68
-#define ICM42607P_ADDR_ALT 0x69
 
 // -----------------------------------------------------------------------------
 // LED
@@ -569,9 +561,5 @@ along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #define HAS_SCREEN 0
 #endif
 
-#ifndef USE_ETHERNET_DEFAULT
-#define USE_ETHERNET_DEFAULT 0
-#endif
-
 #include "DebugConfiguration.h"
 #include "RF95Configuration.h"

src/detect/ScanI2C.cpp

@@ -37,15 +37,8 @@ ScanI2C::FoundDevice ScanI2C::firstKeyboard() const
 
 ScanI2C::FoundDevice ScanI2C::firstAccelerometer() const
 {
-    ScanI2C::DeviceType types[] = {MPU6050, LIS3DH, BMA423, LSM6DS3, BMX160, STK8BAXX,
-                                   ICM20948, QMA6100P, BMM150,  BMI270,  ICM42607P};
-    return firstOfOrNONE(11, types);
-}
-
-ScanI2C::FoundDevice ScanI2C::firstMagnetometer() const
-{
-    ScanI2C::DeviceType types[] = {MMC5983MA};
-    return firstOfOrNONE(1, types);
+    ScanI2C::DeviceType types[] = {MPU6050, LIS3DH, BMA423, LSM6DS3, BMX160, STK8BAXX, ICM20948, QMA6100P, BMM150, BMI270};
+    return firstOfOrNONE(10, types);
 }
 
 ScanI2C::FoundDevice ScanI2C::firstAQI() const

src/detect/ScanI2C.h

@@ -41,7 +41,6 @@ class ScanI2C
         QMI8658,
         QMC5883L,
         HMC5883L,
-        MMC5983MA,
         PMSA003I,
         QMA6100P,
         MPU6050,
@@ -66,7 +65,6 @@ class ScanI2C
         FT6336U,
         STK8BAXX,
         ICM20948,
-        ICM42607P,
         SCD4X,
         MAX30102,
         TPS65233,
@@ -151,8 +149,6 @@ class ScanI2C
 
     FoundDevice firstAccelerometer() const;
 
-    FoundDevice firstMagnetometer() const;
-
     FoundDevice firstAQI() const;
 
     FoundDevice firstRGBLED() const;

src/detect/ScanI2CTwoWire.cpp

@@ -179,22 +179,8 @@ String readSEN5xProductName(TwoWire *i2cBus, uint8_t address)
 #endif
 
 #if HAS_TELEMETRY && !MESHTASTIC_EXCLUDE_ENVIRONMENTAL_SENSOR
-static uint8_t crcSHT2X(const uint8_t *data, uint8_t len)
-{
-    uint8_t crc = 0;
-    for (uint8_t i = 0; i < len; i++) {
-        crc ^= data[i];
-        for (uint8_t bit = 0; bit < 8; bit++) {
-            crc = (crc & 0x80) ? (crc << 1) ^ 0x31 : crc << 1;
-        }
-    }
-    return crc;
-}
-
 bool detectSHT21SerialNumber(TwoWire *i2cBus, uint8_t address)
 {
-    uint8_t serialA[8] = {0};
-    uint8_t serialB[6] = {0};
 
     i2cBus->beginTransmission(address);
     i2cBus->write(0xFA);
@@ -203,13 +189,12 @@ bool detectSHT21SerialNumber(TwoWire *i2cBus, uint8_t address)
     if (i2cBus->endTransmission() != 0)
         return false;
 
-    if (i2cBus->requestFrom(address, (uint8_t)sizeof(serialA)) != sizeof(serialA))
+    if (i2cBus->requestFrom(address, (uint8_t)8) != 8)
         return false;
 
-    for (uint8_t i = 0; i < sizeof(serialA); i++) {
-        if (!i2cBus->available())
-            return false;
-        serialA[i] = i2cBus->read();
+    // Just flush the data
+    while (i2cBus->available() < 8) {
+        i2cBus->read();
     }
 
     i2cBus->beginTransmission(address);
@@ -219,18 +204,16 @@ bool detectSHT21SerialNumber(TwoWire *i2cBus, uint8_t address)
     if (i2cBus->endTransmission() != 0)
         return false;
 
-    if (i2cBus->requestFrom(address, (uint8_t)sizeof(serialB)) != sizeof(serialB))
+    if (i2cBus->requestFrom(address, (uint8_t)6) != 6)
         return false;
 
-    for (uint8_t i = 0; i < sizeof(serialB); i++) {
-        if (!i2cBus->available())
-            return false;
-        serialB[i] = i2cBus->read();
+    // Just flush the data
+    while (i2cBus->available() < 6) {
+        i2cBus->read();
     }
 
-    return crcSHT2X(&serialA[0], 1) == serialA[1] && crcSHT2X(&serialA[2], 1) == serialA[3] &&
-           crcSHT2X(&serialA[4], 1) == serialA[5] && crcSHT2X(&serialA[6], 1) == serialA[7] &&
-           crcSHT2X(&serialB[0], 2) == serialB[2] && crcSHT2X(&serialB[3], 2) == serialB[5];
+    // Assume we detect the SHT21 if something came back from the request
+    return true;
 }
 #endif
 
@@ -360,6 +343,9 @@ void ScanI2CTwoWire::scanPort(I2CPort port, uint8_t *address, uint8_t asize)
                 SCAN_SIMPLE_CASE(ST7567_ADDRESS, SCREEN_ST7567, "ST7567", (uint8_t)addr.address);
 #ifdef HAS_NCP5623
                 SCAN_SIMPLE_CASE(NCP5623_ADDR, NCP5623, "NCP5623", (uint8_t)addr.address);
+#endif
+#ifdef HAS_LP5562
+                SCAN_SIMPLE_CASE(LP5562_ADDR, LP5562, "LP5562", (uint8_t)addr.address);
 #endif
             case XPOWERS_AXP192_AXP2101_ADDRESS:
                 // Do we have the axp2101/192 or the TCA8418
@@ -597,18 +583,6 @@ void ScanI2CTwoWire::scanPort(I2CPort port, uint8_t *address, uint8_t asize)
 #else
                 SCAN_SIMPLE_CASE(PMSA003I_ADDR, PMSA003I, "PMSA003I", (uint8_t)addr.address)
 #endif
-            case MMC5983MA_ADDR:
-                registerValue = getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0x2F), 1);
-                if (registerValue == 0x30) {
-                    type = MMC5983MA;
-                    logFoundDevice("MMC5983MA", (uint8_t)addr.address);
-#ifdef HAS_LP5562
-                } else {
-                    type = LP5562;
-                    logFoundDevice("LP5562", (uint8_t)addr.address);
-#endif
-                }
-                break;
             case BMA423_ADDR: // this can also be LIS3DH_ADDR_ALT
                 registerValue = getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0x0F), 2);
                 if (registerValue == 0x3300 || registerValue == 0x3333) { // RAK4631 WisBlock has LIS3DH register at 0x3333
@@ -763,8 +737,8 @@ void ScanI2CTwoWire::scanPort(I2CPort port, uint8_t *address, uint8_t asize)
                 }
                 break;
 
-            case ICM20948_ADDR:     // same as BMX160_ADDR, BMI270_ADDR_ALT, ICM42607P_ADDR_ALT, and SEN5X_ADDR
-            case ICM20948_ADDR_ALT: // same as MPU6050_ADDR, BMI270_ADDR, and ICM42607P_ADDR
+            case ICM20948_ADDR:     // same as BMX160_ADDR, BMI270_ADDR_ALT, and SEN5X_ADDR
+            case ICM20948_ADDR_ALT: // same as MPU6050_ADDR, BMI270_ADDR
                 registerValue = getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0x00), 1);
 #ifdef HAS_ICM20948
                 type = ICM20948;
@@ -784,12 +758,6 @@ void ScanI2CTwoWire::scanPort(I2CPort port, uint8_t *address, uint8_t asize)
                     logFoundDevice("BMX160", (uint8_t)addr.address);
                     break;
                 } else {
-                    registerValue = getRegisterValue(ScanI2CTwoWire::RegisterLocation(addr, 0x75), 1); // WHO_AM_I
-                    if (registerValue == 0x60) {
-                        type = ICM42607P;
-                        logFoundDevice("ICM-42607-P", (uint8_t)addr.address);
-                        break;
-                    }
 #if HAS_TELEMETRY && !MESHTASTIC_EXCLUDE_AIR_QUALITY_SENSOR
                     String prod = "";
                     prod = readSEN5xProductName(i2cBus, addr.address);

src/graphics/BaseUIEInkDisplay.cpp

@@ -1,211 +0,0 @@
-#ifdef MESHTASTIC_INCLUDE_NICHE_GRAPHICS
-
-#include "./BaseUIEInkDisplay.h"
-
-#include "configuration.h"
-#include "main.h"
-
-using namespace NicheGraphics;
-
-BaseUIEInkDisplay::BaseUIEInkDisplay(Drivers::EInk *driver, uint8_t rotation) : driver(driver), rotation(rotation & 0x3)
-{
-    this->geometry = GEOMETRY_RAWMODE;
-
-    // BaseUI draws in UI orientation. Physical panel dimensions are swapped for 90°/270°.
-    const bool swap = (this->rotation == 1) || (this->rotation == 3);
-    this->displayWidth = swap ? driver->height : driver->width;
-    this->displayHeight = swap ? driver->width : driver->height;
-
-    uint16_t shortSide = min(displayWidth, displayHeight);
-    uint16_t longSide = max(displayWidth, displayHeight);
-    if (shortSide % 8 != 0)
-        shortSide = (shortSide | 7) + 1;
-    this->displayBufferSize = longSide * (shortSide / 8);
-
-    // Panel-native row-major buffer
-    panelRowBytes = ((driver->width - 1) / 8) + 1;
-    panelBufferSize = panelRowBytes * driver->height;
-    panelBuffer = new uint8_t[panelBufferSize];
-    memset(panelBuffer, 0xFF, panelBufferSize); // All white
-}
-
-BaseUIEInkDisplay::~BaseUIEInkDisplay()
-{
-    delete[] panelBuffer;
-}
-
-bool BaseUIEInkDisplay::connect()
-{
-    LOG_INFO("Init BaseUI E-Ink (%u x %u, rot %u)", driver->width, driver->height, rotation);
-    return true;
-}
-
-void BaseUIEInkDisplay::addFrameFlag(frameFlagTypes flag)
-{
-    frameFlags = (frameFlagTypes)(frameFlags | flag);
-}
-
-void BaseUIEInkDisplay::setDisplayResilience(uint8_t fastPerFull, float stressMultiplier)
-{
-    this->fastPerFull = (fastPerFull == 0) ? 1 : fastPerFull;
-    this->stressMultiplier = stressMultiplier;
-}
-
-void BaseUIEInkDisplay::joinAsyncRefresh()
-{
-    if (driver->busy())
-        driver->await();
-}
-
-// OLEDDisplayUi tick path. Honours rate-limit unless flags demand otherwise.
-void BaseUIEInkDisplay::display()
-{
-    const bool demandFast = frameFlags & DEMAND_FAST;
-    const bool cosmetic = frameFlags & COSMETIC;
-    const bool unlimitedFast = frameFlags & UNLIMITED_FAST;
-
-    if (!demandFast && !cosmetic && !unlimitedFast) {
-        if (!forceDisplay(lastDrawMsec == 0 ? 0 : 1000))
-            return;
-        return;
-    }
-
-    forceDisplay(0);
-}
-
-// Keyframe path. Returns true if a frame was pushed (sets lastDrawMsec).
-bool BaseUIEInkDisplay::forceDisplay(uint32_t msecLimit)
-{
-    const uint32_t now = millis();
-    if (lastDrawMsec != 0 && (now - lastDrawMsec) < msecLimit)
-        return false;
-
-    const bool blocking = frameFlags & BLOCKING;
-    Drivers::EInk::UpdateTypes type = decide();
-
-    // Don't pile frames on top of a running update - wait it out.
-    if (driver->busy())
-        driver->await();
-
-    const bool pushed = commit(type, blocking);
-    if (pushed)
-        lastDrawMsec = now;
-
-    // Reset flags for next frame
-    frameFlags = BACKGROUND;
-    return pushed;
-}
-
-bool BaseUIEInkDisplay::commit(Drivers::EInk::UpdateTypes type, bool blocking)
-{
-    uint32_t hash = repack();
-
-    // Skip if frame unchanged. Exception: caller explicitly wants a refresh (COSMETIC or FULL).
-    if (hash == lastHash && type != Drivers::EInk::UpdateTypes::FULL && lastDrawMsec != 0)
-        return false;
-    lastHash = hash;
-
-    // Fall back to FULL on panels that don't advertise FAST support.
-    if (type == Drivers::EInk::UpdateTypes::FAST && !driver->supports(Drivers::EInk::UpdateTypes::FAST))
-        type = Drivers::EInk::UpdateTypes::FULL;
-
-    driver->update(panelBuffer, type);
-
-    if (blocking)
-        driver->await();
-    return true;
-}
-
-Drivers::EInk::UpdateTypes BaseUIEInkDisplay::decide()
-{
-    typedef Drivers::EInk::UpdateTypes UT;
-
-    const bool unlimitedFast = frameFlags & UNLIMITED_FAST;
-
-    // Explicit flag wins outright
-    if (frameFlags & COSMETIC) {
-        fullRefreshDebt = max(fullRefreshDebt - 1.0f, 0.0f);
-        return UT::FULL;
-    }
-    if (frameFlags & DEMAND_FAST) {
-        if (!unlimitedFast) {
-            fullRefreshDebt += (fullRefreshDebt < 1.0f) ? (1.0f / fastPerFull) : (stressMultiplier * (1.0f / fastPerFull));
-        }
-        return UT::FAST;
-    }
-
-    const bool explicitFast = frameFlags & RESPONSIVE;
-
-    if (explicitFast || unlimitedFast) {
-        if (!unlimitedFast) {
-            fullRefreshDebt += (fullRefreshDebt < 1.0f) ? (1.0f / fastPerFull) : (stressMultiplier * (1.0f / fastPerFull));
-        }
-        return UT::FAST;
-    }
-
-    // BACKGROUND / unspecified: let debt decide
-    if (fullRefreshDebt >= 1.0f) {
-        fullRefreshDebt = max(fullRefreshDebt - 1.0f, 0.0f);
-        return UT::FULL;
-    }
-    fullRefreshDebt += 1.0f / fastPerFull;
-    return UT::FAST;
-}
-
-uint32_t BaseUIEInkDisplay::repack()
-{
-    memset(panelBuffer, 0xFF, panelBufferSize); // start all-white
-
-    const uint16_t pw = driver->width;
-    const uint16_t ph = driver->height;
-
-    // OLEDDisplay buffer: byte = buffer[x + (y/8) * displayWidth]; bit = 1 << (y & 7); 1 = black
-    // Niche buffer:       byte = (y * panelRowBytes) + (x/8);      bit = 1 << (7 - x%8); 1 = white
-    for (uint16_t oy = 0; oy < displayHeight; oy++) {
-        for (uint16_t ox = 0; ox < displayWidth; ox++) {
-            const uint8_t b = buffer[ox + (oy / 8) * displayWidth];
-            const bool isBlack = b & (1 << (oy & 7));
-
-            uint16_t px, py;
-            switch (rotation) {
-            case 1: // 90° CW: OLED (ox,oy) → panel (pw-1-oy, ox)
-                px = pw - 1 - oy;
-                py = ox;
-                break;
-            case 2: // 180°
-                px = pw - 1 - ox;
-                py = ph - 1 - oy;
-                break;
-            case 3: // 270° CW
-                px = oy;
-                py = ph - 1 - ox;
-                break;
-            case 0:
-            default:
-                px = ox;
-                py = oy;
-                break;
-            }
-
-            if (px >= pw || py >= ph)
-                continue;
-
-            const uint32_t byteNum = (py * panelRowBytes) + (px / 8);
-            const uint8_t bitNum = 7 - (px % 8);
-            if (isBlack)
-                panelBuffer[byteNum] &= ~(1 << bitNum);
-            else
-                panelBuffer[byteNum] |= (1 << bitNum);
-        }
-    }
-
-    // FNV-1a
-    uint32_t h = 2166136261u;
-    for (uint32_t i = 0; i < panelBufferSize; i++) {
-        h ^= panelBuffer[i];
-        h *= 16777619u;
-    }
-    return h;
-}
-
-#endif // MESHTASTIC_INCLUDE_NICHE_GRAPHICS

src/graphics/BaseUIEInkDisplay.h

@@ -1,98 +0,0 @@
-/*
-
-OLEDDisplay adapter that routes BaseUI pixel output to a NicheGraphics::Drivers::EInk driver.
-
-One adapter serves all E-Ink variants: the panel driver and orientation are injected at construction,
-and FULL/FAST selection is made by the shared DisplayHealth model (same as InkHUD).
-
-Replaces the per-board branching in EInkDisplay2 / EInkDynamicDisplay / EInkParallelDisplay.
-
-*/
-
-#pragma once
-
-#ifdef MESHTASTIC_INCLUDE_NICHE_GRAPHICS
-
-#include "configuration.h"
-
-#include "graphics/eink/Drivers/EInk.h"
-
-#include <OLEDDisplay.h>
-
-namespace NicheGraphics
-{
-
-class BaseUIEInkDisplay : public OLEDDisplay
-{
-  public:
-    // Flags Screen.cpp sets via EINK_ADD_FRAMEFLAG before triggering a draw.
-    // Bits are combined; decided at render time.
-    enum frameFlagTypes : uint8_t {
-        BACKGROUND = (1 << 0),     // Regular OLEDDisplayUi tick - no urgency, UNSPECIFIED
-        RESPONSIVE = (1 << 1),     // User-driven refresh - prefer FAST
-        COSMETIC = (1 << 2),       // Clean splash / wake-from-sleep - force FULL
-        DEMAND_FAST = (1 << 3),    // Menu interaction - force FAST
-        BLOCKING = (1 << 4),       // Wait for update to finish before returning
-        UNLIMITED_FAST = (1 << 5), // Suppress health-driven FULL promotion (typing modes)
-    };
-
-    BaseUIEInkDisplay(Drivers::EInk *driver, uint8_t rotation);
-    ~BaseUIEInkDisplay() override;
-
-    // OLEDDisplay overrides
-    bool connect() override;
-    void display() override;
-    void sendCommand(uint8_t com) override { (void)com; }
-    int getBufferOffset(void) override { return 0; }
-
-    // BaseUI public API (same shape as the old EInkDynamicDisplay)
-    bool forceDisplay(uint32_t msecLimit = 1000);
-    void addFrameFlag(frameFlagTypes flag);
-    void joinAsyncRefresh();
-    void enableUnlimitedFastMode() { addFrameFlag(UNLIMITED_FAST); }
-    void disableUnlimitedFastMode() { frameFlags = (frameFlagTypes)(frameFlags & ~UNLIMITED_FAST); }
-
-    // Tuning, called once per panel profile
-    void setDisplayResilience(uint8_t fastPerFull, float stressMultiplier = 2.0f);
-
-    // Exposed so Screen.cpp / variants can read the rotation passed in at construction
-    uint8_t getRotation() const { return rotation; }
-
-  private:
-    // Perform an update now, unconditionally. Returns true if a frame was pushed to the driver.
-    bool commit(Drivers::EInk::UpdateTypes type, bool blocking);
-
-    // Convert OLEDDisplay's column-major buffer into the panel's row-major MSB-left buffer.
-    // Applies rotation. Returns the hash of the panel buffer for frame-skip comparison.
-    uint32_t repack();
-
-    // Decide FULL vs FAST based on current frame flags + accumulated debt.
-    Drivers::EInk::UpdateTypes decide();
-
-    Drivers::EInk *driver = nullptr;
-    uint8_t rotation = 0; // 0=0°, 1=90°CW, 2=180°, 3=270°CW
-    uint8_t *panelBuffer = nullptr;
-    uint32_t panelBufferSize = 0;
-    uint16_t panelRowBytes = 0;
-
-    frameFlagTypes frameFlags = BACKGROUND;
-    uint32_t lastDrawMsec = 0;
-    uint32_t lastHash = 0;
-
-    // DisplayHealth-style debt tracking
-    float fullRefreshDebt = 0.0f;
-    uint8_t fastPerFull = 7;
-    float stressMultiplier = 2.0f;
-};
-
-} // namespace NicheGraphics
-
-// Compat macros used throughout Screen.cpp - route straight to the adapter.
-#define EINK_ADD_FRAMEFLAG(display, flag)                                                                                        \
-    static_cast<NicheGraphics::BaseUIEInkDisplay *>(display)->addFrameFlag(NicheGraphics::BaseUIEInkDisplay::flag)
-#define EINK_JOIN_ASYNCREFRESH(display) static_cast<NicheGraphics::BaseUIEInkDisplay *>(display)->joinAsyncRefresh()
-
-#else // !MESHTASTIC_INCLUDE_NICHE_GRAPHICS
-#define EINK_ADD_FRAMEFLAG(display, flag)
-#define EINK_JOIN_ASYNCREFRESH(display)
-#endif

src/graphics/EInkDisplay2.cpp

@@ -0,0 +1,298 @@
+#include "configuration.h"
+
+#if defined(USE_EINK) && !defined(USE_EINK_PARALLELDISPLAY)
+#include "EInkDisplay2.h"
+#include "SPILock.h"
+#include "main.h"
+#include <SPI.h>
+
+#ifdef GXEPD2_DRIVER_0
+#include "einkDetect.h"
+#endif
+
+/*
+    The macros EINK_DISPLAY_MODEL, EINK_WIDTH, and EINK_HEIGHT are defined as build_flags in a variant's platformio.ini
+    Previously, these macros were defined at the top of this file.
+
+    For archival reasons, note that the following configurations had also been tested during this period:
+    * ifdef RAK4631
+        - 4.2 inch
+          EINK_DISPLAY_MODEL: GxEPD2_420_M01
+          EINK_WIDTH: 300
+          EINK_WIDTH: 400
+
+        - 2.9 inch
+          EINK_DISPLAY_MODEL: GxEPD2_290_T5D
+          EINK_WIDTH: 296
+          EINK_HEIGHT: 128
+
+        - 1.54 inch
+          EINK_DISPLAY_MODEL: GxEPD2_154_M09
+          EINK_WIDTH: 200
+          EINK_HEIGHT: 200
+*/
+
+// Constructor
+EInkDisplay::EInkDisplay(uint8_t address, int sda, int scl, OLEDDISPLAY_GEOMETRY geometry, HW_I2C i2cBus)
+{
+    // Set dimensions in OLEDDisplay base class
+    this->geometry = GEOMETRY_RAWMODE;
+    this->displayWidth = EINK_WIDTH;
+    this->displayHeight = EINK_HEIGHT;
+
+    // Round shortest side up to nearest byte, to prevent truncation causing an undersized buffer
+    uint16_t shortSide = min(EINK_WIDTH, EINK_HEIGHT);
+    uint16_t longSide = max(EINK_WIDTH, EINK_HEIGHT);
+    if (shortSide % 8 != 0)
+        shortSide = (shortSide | 7) + 1;
+
+    this->displayBufferSize = longSide * (shortSide / 8);
+}
+
+/**
+ * Force a display update if we haven't drawn within the specified msecLimit
+ */
+bool EInkDisplay::forceDisplay(uint32_t msecLimit)
+{
+    // No need to grab this lock because we are on our own SPI bus
+    // concurrency::LockGuard g(spiLock);
+
+    uint32_t now = millis();
+    uint32_t sinceLast = now - lastDrawMsec;
+
+    if (adafruitDisplay && (sinceLast > msecLimit || lastDrawMsec == 0))
+        lastDrawMsec = now;
+    else
+        return false;
+
+    // FIXME - only draw bits have changed (use backbuf similar to the other displays)
+    const bool flipped = config.display.flip_screen;
+    // HACK for L1 EInk
+#if defined(SEEED_WIO_TRACKER_L1_EINK)
+    // For SEEED_WIO_TRACKER_L1_EINK, setRotation(3) is correct but mirrored; flip both axes
+    for (uint32_t y = 0; y < displayHeight; y++) {
+        for (uint32_t x = 0; x < displayWidth; x++) {
+            auto b = buffer[x + (y / 8) * displayWidth];
+            auto isset = b & (1 << (y & 7));
+            adafruitDisplay->drawPixel((displayWidth - 1) - x, (displayHeight - 1) - y, isset ? GxEPD_BLACK : GxEPD_WHITE);
+        }
+    }
+#else
+    for (uint32_t y = 0; y < displayHeight; y++) {
+        for (uint32_t x = 0; x < displayWidth; x++) {
+            auto b = buffer[x + (y / 8) * displayWidth];
+            auto isset = b & (1 << (y & 7));
+            if (flipped)
+                adafruitDisplay->drawPixel((displayWidth - 1) - x, (displayHeight - 1) - y, isset ? GxEPD_BLACK : GxEPD_WHITE);
+            else
+                adafruitDisplay->drawPixel(x, y, isset ? GxEPD_BLACK : GxEPD_WHITE);
+        }
+    }
+#endif
+
+    // Trigger the refresh in GxEPD2
+    LOG_DEBUG("Update E-Paper");
+    adafruitDisplay->nextPage();
+
+    // End the update process
+    endUpdate();
+
+    LOG_DEBUG("done");
+    return true;
+}
+
+// End the update process - virtual method, overridden in derived class
+void EInkDisplay::endUpdate()
+{
+#ifndef EINK_NOT_HIBERNATE
+    // By default, power off the E-Ink display hardware and enter hibernate().
+    // Boards/panels that define EINK_NOT_HIBERNATE intentionally skip this step.
+    // Skipping hibernate() can help avoid panel-specific wake/refresh or ghosting issues,
+    // but it typically trades lower power savings for that compatibility.
+    adafruitDisplay->hibernate();
+#endif
+}
+
+// Write the buffer to the display memory
+void EInkDisplay::display(void)
+{
+    // We don't allow regular 'dumb' display() calls to draw on eink until we've shown
+    // at least one forceDisplay() keyframe.  This prevents flashing when we should the critical
+    // bootscreen (that we want to look nice)
+
+    if (lastDrawMsec) {
+        forceDisplay(slowUpdateMsec); // Show the first screen a few seconds after boot, then slower
+    }
+}
+
+// Send a command to the display (low level function)
+void EInkDisplay::sendCommand(uint8_t com)
+{
+    (void)com;
+    // Drop all commands to device (we just update the buffer)
+}
+
+void EInkDisplay::setDetected(uint8_t detected)
+{
+    (void)detected;
+}
+
+// Connect to the display - variant specific
+bool EInkDisplay::connect()
+{
+    LOG_INFO("Do EInk init");
+
+#ifdef PIN_EINK_EN
+    // backlight power, HIGH is backlight on, LOW is off
+    pinMode(PIN_EINK_EN, OUTPUT);
+#ifdef ELECROW_ThinkNode_M1
+    // ThinkNode M1 has a hardware dimmable backlight. Start enabled
+    digitalWrite(PIN_EINK_EN, HIGH);
+#elif defined(MINI_EPAPER_S3)
+    // T-Mini Epaper S3 requires panel power rail enabled before SPI transfer.
+    digitalWrite(PIN_EINK_EN, HIGH);
+    delay(10);
+#else
+    digitalWrite(PIN_EINK_EN, LOW);
+#endif
+#endif
+
+#if defined(TTGO_T_ECHO) || defined(ELECROW_ThinkNode_M1) || defined(T_ECHO_LITE) || defined(TTGO_T_ECHO_PLUS)
+    {
+        auto lowLevel = new EINK_DISPLAY_MODEL(PIN_EINK_CS, PIN_EINK_DC, PIN_EINK_RES, PIN_EINK_BUSY, SPI1);
+
+        adafruitDisplay = new GxEPD2_BW<EINK_DISPLAY_MODEL, EINK_DISPLAY_MODEL::HEIGHT>(*lowLevel);
+        adafruitDisplay->init();
+#if defined(ELECROW_ThinkNode_M1) || defined(T_ECHO_LITE)
+        adafruitDisplay->setRotation(4);
+#else
+        adafruitDisplay->setRotation(3);
+#endif
+        adafruitDisplay->setPartialWindow(0, 0, displayWidth, displayHeight);
+    }
+#elif defined(ELECROW_ThinkNode_M5)
+    {
+        // Start HSPI
+        hspi = new SPIClass(HSPI);
+        hspi->begin(PIN_EINK_SCLK, -1, PIN_EINK_MOSI, PIN_EINK_CS); // SCLK, MISO, MOSI, SS
+
+        auto lowLevel = new EINK_DISPLAY_MODEL(PIN_EINK_CS, PIN_EINK_DC, PIN_EINK_RES, PIN_EINK_BUSY, *hspi);
+
+        adafruitDisplay = new GxEPD2_BW<EINK_DISPLAY_MODEL, EINK_DISPLAY_MODEL::HEIGHT>(*lowLevel);
+        adafruitDisplay->init();
+
+        adafruitDisplay->setRotation(4);
+
+        adafruitDisplay->setPartialWindow(0, 0, displayWidth, displayHeight);
+    }
+#elif defined(MESHLINK)
+    {
+        auto lowLevel = new EINK_DISPLAY_MODEL(PIN_EINK_CS, PIN_EINK_DC, PIN_EINK_RES, PIN_EINK_BUSY, SPI1);
+
+        adafruitDisplay = new GxEPD2_BW<EINK_DISPLAY_MODEL, EINK_DISPLAY_MODEL::HEIGHT>(*lowLevel);
+        adafruitDisplay->init();
+        adafruitDisplay->setRotation(3);
+        adafruitDisplay->setPartialWindow(0, 0, displayWidth, displayHeight);
+    }
+#elif defined(RAK4630) || defined(MAKERPYTHON)
+    {
+        if (eink_found) {
+            auto lowLevel = new EINK_DISPLAY_MODEL(PIN_EINK_CS, PIN_EINK_DC, PIN_EINK_RES, PIN_EINK_BUSY);
+            adafruitDisplay = new GxEPD2_BW<EINK_DISPLAY_MODEL, EINK_DISPLAY_MODEL::HEIGHT>(*lowLevel);
+            adafruitDisplay->init(115200, true, 10, false, SPI1, SPISettings(4000000, MSBFIRST, SPI_MODE0));
+            // RAK14000 2.13 inch b/w 250x122 does actually now support fast refresh
+            adafruitDisplay->setRotation(3);
+            // Fast refresh support for  1.54, 2.13 RAK14000 b/w , 2.9 and 4.2
+            // adafruitDisplay->setRotation(1);
+            adafruitDisplay->setPartialWindow(0, 0, displayWidth, displayHeight);
+        } else {
+            (void)adafruitDisplay;
+        }
+    }
+
+#elif defined(HELTEC_WIRELESS_PAPER_V1_0) || defined(HELTEC_VISION_MASTER_E290) || defined(TLORA_T3S3_EPAPER) ||                 \
+    defined(CROWPANEL_ESP32S3_5_EPAPER) || defined(CROWPANEL_ESP32S3_4_EPAPER) || defined(CROWPANEL_ESP32S3_2_EPAPER) ||         \
+    defined(MINI_EPAPER_S3)
+    {
+        // Start HSPI
+        hspi = new SPIClass(HSPI);
+        hspi->begin(PIN_EINK_SCLK, -1, PIN_EINK_MOSI, PIN_EINK_CS); // SCLK, MISO, MOSI, SS
+        // VExt already enabled in setup()
+        // RTC GPIO hold disabled in setup()
+
+        // Create GxEPD2 objects
+        auto lowLevel = new EINK_DISPLAY_MODEL(PIN_EINK_CS, PIN_EINK_DC, PIN_EINK_RES, PIN_EINK_BUSY, *hspi);
+        adafruitDisplay = new GxEPD2_BW<EINK_DISPLAY_MODEL, EINK_DISPLAY_MODEL::HEIGHT>(*lowLevel);
+
+        // Init GxEPD2
+        adafruitDisplay->init();
+#if defined(MINI_EPAPER_S3)
+        adafruitDisplay->setRotation(3);
+#else
+        adafruitDisplay->setRotation(3);
+#if defined(CROWPANEL_ESP32S3_5_EPAPER) || defined(CROWPANEL_ESP32S3_4_EPAPER)
+        adafruitDisplay->setRotation(0);
+#endif
+#endif
+    }
+#elif defined(PCA10059) || defined(ME25LS01)
+    {
+        auto lowLevel = new EINK_DISPLAY_MODEL(PIN_EINK_CS, PIN_EINK_DC, PIN_EINK_RES, PIN_EINK_BUSY);
+        adafruitDisplay = new GxEPD2_BW<EINK_DISPLAY_MODEL, EINK_DISPLAY_MODEL::HEIGHT>(*lowLevel);
+        adafruitDisplay->init(115200, true, 40, false, SPI1, SPISettings(4000000, MSBFIRST, SPI_MODE0));
+        adafruitDisplay->setRotation(0);
+        adafruitDisplay->setPartialWindow(0, 0, EINK_WIDTH, EINK_HEIGHT);
+    }
+#elif defined(M5_COREINK) || defined(T_DECK_PRO)
+    auto lowLevel = new EINK_DISPLAY_MODEL(PIN_EINK_CS, PIN_EINK_DC, PIN_EINK_RES, PIN_EINK_BUSY);
+    adafruitDisplay = new GxEPD2_BW<EINK_DISPLAY_MODEL, EINK_DISPLAY_MODEL::HEIGHT>(*lowLevel);
+    adafruitDisplay->init(115200, true, 40, false, SPI, SPISettings(4000000, MSBFIRST, SPI_MODE0));
+    adafruitDisplay->setRotation(0);
+    adafruitDisplay->setPartialWindow(0, 0, EINK_WIDTH, EINK_HEIGHT);
+#elif defined(my) || defined(ESP32_S3_PICO)
+    {
+        auto lowLevel = new EINK_DISPLAY_MODEL(PIN_EINK_CS, PIN_EINK_DC, PIN_EINK_RES, PIN_EINK_BUSY);
+        adafruitDisplay = new GxEPD2_BW<EINK_DISPLAY_MODEL, EINK_DISPLAY_MODEL::HEIGHT>(*lowLevel);
+        adafruitDisplay->init(115200, true, 40, false, SPI, SPISettings(4000000, MSBFIRST, SPI_MODE0));
+        adafruitDisplay->setRotation(1);
+        adafruitDisplay->setPartialWindow(0, 0, EINK_WIDTH, EINK_HEIGHT);
+    }
+#elif defined(HELTEC_MESH_POCKET) || defined(SEEED_WIO_TRACKER_L1_EINK) || defined(HELTEC_MESH_SOLAR_EINK)
+    {
+        spi1 = &SPI1;
+        spi1->begin();
+        // VExt already enabled in setup()
+        // RTC GPIO hold disabled in setup()
+
+        // Create GxEPD2 objects
+        auto lowLevel = new EINK_DISPLAY_MODEL(PIN_EINK_CS, PIN_EINK_DC, PIN_EINK_RES, PIN_EINK_BUSY, *spi1);
+        adafruitDisplay = new GxEPD2_BW<EINK_DISPLAY_MODEL, EINK_DISPLAY_MODEL::HEIGHT>(*lowLevel);
+
+        // Init GxEPD2
+        adafruitDisplay->init();
+        adafruitDisplay->setRotation(3);
+        adafruitDisplay->setPartialWindow(0, 0, EINK_WIDTH, EINK_HEIGHT);
+    }
+#elif defined(HELTEC_WIRELESS_PAPER) || defined(HELTEC_VISION_MASTER_E213)
+
+    // Detect display model, before starting SPI
+    EInkDetectionResult displayModel = detectEInk();
+
+    // Start HSPI
+    hspi = new SPIClass(HSPI);
+    hspi->begin(PIN_EINK_SCLK, -1, PIN_EINK_MOSI, PIN_EINK_CS); // SCLK, MISO, MOSI, SS
+
+    // Create GxEPD2 object
+    adafruitDisplay = new GxEPD2_Multi<GXEPD2_DRIVER_0, GXEPD2_DRIVER_1>((uint8_t)displayModel, PIN_EINK_CS, PIN_EINK_DC,
+                                                                         PIN_EINK_RES, PIN_EINK_BUSY, *hspi);
+
+    // Init GxEPD2
+    adafruitDisplay->init();
+    adafruitDisplay->setRotation(3);
+
+#endif
+
+    return true;
+}
+
+#endif

src/graphics/EInkDisplay2.h

@@ -0,0 +1,106 @@
+#pragma once
+
+#if defined(USE_EINK) && !defined(USE_EINK_PARALLELDISPLAY)
+
+#include "GxEPD2_BW.h"
+#include <OLEDDisplay.h>
+
+#ifdef GXEPD2_DRIVER_0 // If variant has multiple possible display models
+#include "GxEPD2Multi.h"
+#endif
+
+// Limit how often we push a full E-Ink refresh. T-Deck Pro needs faster updates for typing.
+#ifndef EINK_FORCE_DISPLAY_THROTTLE_MS
+#if defined(T_DECK_PRO)
+#define EINK_FORCE_DISPLAY_THROTTLE_MS 200
+#else
+#define EINK_FORCE_DISPLAY_THROTTLE_MS 1000
+#endif
+#endif
+
+/**
+ * An adapter class that allows using the GxEPD2 library as if it was an OLEDDisplay implementation.
+ *
+ * Note: EInkDynamicDisplay derives from this class.
+ *
+ * Remaining TODO:
+ * optimize display() to only draw changed pixels (see other OLED subclasses for examples)
+ * implement displayOn/displayOff to turn off the TFT device (and backlight)
+ * Use the fast NRF52 SPI API rather than the slow standard arduino version
+ *
+ * turn radio back on - currently with both on spi bus is fucked? or are we leaving chip select asserted?
+ * Suggestion: perhaps similar to HELTEC_WIRELESS_PAPER issue, which resolved with rtc_gpio_hold_dis()
+ */
+class EInkDisplay : public OLEDDisplay
+{
+    /// How often should we update the display
+    /// thereafter we do once per 5 minutes
+    uint32_t slowUpdateMsec = 5 * 60 * 1000;
+
+  public:
+    /* constructor
+    FIXME - the parameters are not used, just a temporary hack to keep working like the old displays
+    */
+    EInkDisplay(uint8_t, int, int, OLEDDISPLAY_GEOMETRY, HW_I2C);
+
+    // Write the buffer to the display memory (for eink we only do this occasionally)
+    virtual void display(void) override;
+
+    /**
+     * Force a display update if we haven't drawn within the specified msecLimit
+     *
+     * @return true if we did draw the screen
+     */
+    virtual bool forceDisplay(uint32_t msecLimit = EINK_FORCE_DISPLAY_THROTTLE_MS);
+
+    /**
+     * Run any code needed to complete an update, after the physical refresh has completed.
+     * Split from forceDisplay(), to enable async refresh in derived EInkDynamicDisplay class.
+     *
+     */
+    virtual void endUpdate();
+
+    /**
+     * shim to make the abstraction happy
+     *
+     */
+    void setDetected(uint8_t detected);
+
+  protected:
+    // the header size of the buffer used, e.g. for the SPI command header
+    virtual int getBufferOffset(void) override { return 0; }
+
+    // Send a command to the display (low level function)
+    virtual void sendCommand(uint8_t com) override;
+
+    // Connect to the display
+    virtual bool connect() override;
+
+#ifdef GXEPD2_DRIVER_0
+    // AdafruitGFX display object - wrapper for multiple drivers
+    // Allows runtime detection of multiple displays
+    // Avoid this situation if possible!
+    GxEPD2_Multi<GXEPD2_DRIVER_0, GXEPD2_DRIVER_1> *adafruitDisplay = NULL;
+#else
+    // AdafruitGFX display object (for single display model) - instantiated in connect(), variant specific
+    GxEPD2_BW<EINK_DISPLAY_MODEL, EINK_DISPLAY_MODEL::HEIGHT> *adafruitDisplay = NULL;
+#endif
+
+    // If display uses HSPI
+#if defined(HELTEC_WIRELESS_PAPER) || defined(HELTEC_WIRELESS_PAPER_V1_0) || defined(HELTEC_VISION_MASTER_E213) ||               \
+    defined(HELTEC_VISION_MASTER_E290) || defined(TLORA_T3S3_EPAPER) || defined(CROWPANEL_ESP32S3_5_EPAPER) ||                   \
+    defined(CROWPANEL_ESP32S3_4_EPAPER) || defined(CROWPANEL_ESP32S3_2_EPAPER) || defined(ELECROW_ThinkNode_M5) ||               \
+    defined(MINI_EPAPER_S3)
+    SPIClass *hspi = NULL;
+#endif
+
+#if defined(HELTEC_MESH_POCKET) || defined(SEEED_WIO_TRACKER_L1_EINK) || defined(HELTEC_MESH_SOLAR_EINK)
+    SPIClass *spi1 = NULL;
+#endif
+
+  private:
+    // FIXME quick hack to limit drawing to a very slow rate
+    uint32_t lastDrawMsec = 0;
+};
+
+#endif

src/graphics/EInkDynamicDisplay.cpp

@@ -0,0 +1,564 @@
+#include "Throttle.h"
+#include "configuration.h"
+
+#if defined(USE_EINK) && defined(USE_EINK_DYNAMICDISPLAY)
+#include "EInkDynamicDisplay.h"
+
+// Constructor
+EInkDynamicDisplay::EInkDynamicDisplay(uint8_t address, int sda, int scl, OLEDDISPLAY_GEOMETRY geometry, HW_I2C i2cBus)
+    : EInkDisplay(address, sda, scl, geometry, i2cBus), NotifiedWorkerThread("EInkDynamicDisplay")
+{
+    // If tracking ghost pixels, grab memory
+#ifdef EINK_LIMIT_GHOSTING_PX
+    dirtyPixels = std::unique_ptr<uint8_t[]>(new uint8_t[EInkDisplay::displayBufferSize]()); // Init with zeros
+#endif
+}
+
+// Destructor
+EInkDynamicDisplay::~EInkDynamicDisplay()
+{
+    // If we were tracking ghost pixels, free the memory
+#ifdef EINK_LIMIT_GHOSTING_PX
+    dirtyPixels = nullptr;
+#endif
+}
+
+// Screen requests a BACKGROUND frame
+void EInkDynamicDisplay::display()
+{
+    addFrameFlag(BACKGROUND);
+    update();
+}
+
+// Screen requests a RESPONSIVE frame
+bool EInkDynamicDisplay::forceDisplay(uint32_t msecLimit)
+{
+    addFrameFlag(RESPONSIVE);
+    return update(); // (Unutilized) Base class promises to return true if update ran
+}
+
+// Add flag for the next frame
+void EInkDynamicDisplay::addFrameFlag(frameFlagTypes flag)
+{
+    // OR the new flag into the existing flags
+    this->frameFlags = (frameFlagTypes)(this->frameFlags | flag);
+}
+
+// GxEPD2 code to set fast refresh
+void EInkDynamicDisplay::configForFastRefresh()
+{
+    // Variant-specific code can go here
+#if defined(PRIVATE_HW)
+#else
+    // Otherwise:
+    adafruitDisplay->setPartialWindow(0, 0, adafruitDisplay->width(), adafruitDisplay->height());
+#endif
+}
+
+// GxEPD2 code to set full refresh
+void EInkDynamicDisplay::configForFullRefresh()
+{
+    // Variant-specific code can go here
+#if defined(PRIVATE_HW)
+#else
+    // Otherwise:
+    adafruitDisplay->setFullWindow();
+#endif
+}
+
+// Run any relevant GxEPD2 code, so next update will use correct refresh type
+void EInkDynamicDisplay::applyRefreshMode()
+{
+    // Change from FULL to FAST
+    if (currentConfig == FULL && refresh == FAST) {
+        configForFastRefresh();
+        currentConfig = FAST;
+    }
+
+    // Change from FAST back to FULL
+    else if (currentConfig == FAST && refresh == FULL) {
+        configForFullRefresh();
+        currentConfig = FULL;
+    }
+}
+
+// Update fastRefreshCount
+void EInkDynamicDisplay::adjustRefreshCounters()
+{
+    if (refresh == FAST)
+        fastRefreshCount++;
+
+    else if (refresh == FULL)
+        fastRefreshCount = 0;
+}
+
+// Trigger the display update by calling base class
+bool EInkDynamicDisplay::update()
+{
+    // Determine the refresh mode to use, and start the update
+    bool refreshApproved = determineMode();
+    if (refreshApproved) {
+        EInkDisplay::forceDisplay(0); // Bypass base class' own rate-limiting system
+        storeAndReset();              // Store the result of this loop for next time. Note: call *before* endOrDetach()
+        endOrDetach();                // endUpdate() right now, or set the async refresh flag (if FULL and HAS_EINK_ASYNCFULL)
+    } else
+        storeAndReset(); // No update, no post-update code, just store the results
+
+    return refreshApproved; // (Unutilized) Base class promises to return true if update ran
+}
+
+// Figure out who runs the post-update code
+void EInkDynamicDisplay::endOrDetach()
+{
+    // If the GxEPD2 version reports that it has the async modifications
+#ifdef HAS_EINK_ASYNCFULL
+    if (previousRefresh == FULL) {
+        asyncRefreshRunning = true; // Set the flag - checked in determineMode(); cleared by onNotify()
+
+        if (previousFrameFlags & BLOCKING)
+            awaitRefresh();
+        else {
+            // Async begins
+            LOG_DEBUG("Async full-refresh begins (drop frames)");
+            notifyLater(intervalPollAsyncRefresh, DUE_POLL_ASYNCREFRESH, true); // Hand-off to NotifiedWorkerThread
+        }
+    }
+
+    // Fast Refresh
+    else if (previousRefresh == FAST)
+        EInkDisplay::endUpdate(); // Still block while updating, but EInkDisplay needs us to call endUpdate() ourselves.
+
+        // Fallback - If using an unmodified version of GxEPD2 for some reason
+#else
+    if (previousRefresh == FULL || previousRefresh == FAST) { // If refresh wasn't skipped (on unspecified..)
+        LOG_WARN(
+            "GxEPD2 version has not been modified to support async refresh; using fallback behavior. Please update lib_deps in "
+            "variant's platformio.ini file");
+        EInkDisplay::endUpdate();
+    }
+#endif
+}
+
+// Assess situation, pick a refresh type
+bool EInkDynamicDisplay::determineMode()
+{
+    checkInitialized();
+    checkForPromotion();
+#if defined(HAS_EINK_ASYNCFULL)
+    checkBusyAsyncRefresh();
+#endif
+    checkRateLimiting();
+
+    // If too soon for a new frame, or display busy, abort early
+    if (refresh == SKIPPED)
+        return false; // No refresh
+
+    // -- New frame is due --
+
+    resetRateLimiting(); // Once determineMode() ends, will have to wait again
+    hashImage();         // Generate here, so we can still copy it to previousImageHash, even if we skip the comparison check
+    LOG_DEBUG("determineMode(): "); // Begin log entry
+
+    // Once mode determined, any remaining checks will bypass
+    checkCosmetic();
+    checkDemandingFast();
+    checkFrameMatchesPrevious();
+    checkConsecutiveFastRefreshes();
+#ifdef EINK_LIMIT_GHOSTING_PX
+    checkExcessiveGhosting();
+#endif
+    checkFastRequested();
+
+    if (refresh == UNSPECIFIED)
+        LOG_WARN("There was a flaw in the determineMode() logic");
+
+    // -- Decision has been reached --
+    applyRefreshMode();
+    adjustRefreshCounters();
+
+#ifdef EINK_LIMIT_GHOSTING_PX
+    // Full refresh clears any ghosting
+    if (refresh == FULL)
+        resetGhostPixelTracking();
+#endif
+
+    // Return - call a refresh or not?
+    if (refresh == SKIPPED)
+        return false; // Don't trigger a refresh
+    else
+        return true; // Do trigger a refresh
+}
+
+// Is this the very first frame?
+void EInkDynamicDisplay::checkInitialized()
+{
+    if (!initialized) {
+        // Undo GxEPD2_BW::partialWindow(), if set by developer in EInkDisplay::connect()
+        configForFullRefresh();
+
+        // Clear any existing image, so we can draw logo with fast-refresh, but also to set GxEPD2_EPD::_initial_write
+        adafruitDisplay->clearScreen();
+
+        LOG_DEBUG("initialized, ");
+        initialized = true;
+
+        // Use a fast-refresh for the next frame; no skipping or else blank screen when waking from deep sleep
+        addFrameFlag(DEMAND_FAST);
+    }
+}
+
+// Was a frame skipped (rate, display busy) that should have been a FAST refresh?
+void EInkDynamicDisplay::checkForPromotion()
+{
+    // If a frame was skipped (rate, display busy), then promote a BACKGROUND frame
+    // Because we DID want a RESPONSIVE/COSMETIC/DEMAND_FULL frame last time, we just didn't get it
+
+    switch (previousReason) {
+    case ASYNC_REFRESH_BLOCKED_DEMANDFAST:
+        addFrameFlag(DEMAND_FAST);
+        break;
+    case ASYNC_REFRESH_BLOCKED_COSMETIC:
+        addFrameFlag(COSMETIC);
+        break;
+    case ASYNC_REFRESH_BLOCKED_RESPONSIVE:
+    case EXCEEDED_RATELIMIT_FAST:
+        addFrameFlag(RESPONSIVE);
+        break;
+    default:
+        break;
+    }
+}
+
+// Is it too soon for another frame of this type?
+void EInkDynamicDisplay::checkRateLimiting()
+{
+    // Sanity check: millis() overflow - just let the update run..
+    if (previousRunMs > millis())
+        return;
+
+    // Skip update: too soon for BACKGROUND
+    if (frameFlags == BACKGROUND) {
+        if (Throttle::isWithinTimespanMs(previousRunMs, 30000)) {
+            refresh = SKIPPED;
+            reason = EXCEEDED_RATELIMIT_FULL;
+            return;
+        }
+    }
+
+    // No rate-limit for these special cases
+    if (frameFlags & COSMETIC || frameFlags & DEMAND_FAST)
+        return;
+
+    // Skip update: too soon for RESPONSIVE
+    if (frameFlags & RESPONSIVE) {
+        if (Throttle::isWithinTimespanMs(previousRunMs, 1000)) {
+            refresh = SKIPPED;
+            reason = EXCEEDED_RATELIMIT_FAST;
+            LOG_DEBUG("refresh=SKIPPED, reason=EXCEEDED_RATELIMIT_FAST, frameFlags=0x%x", frameFlags);
+            return;
+        }
+    }
+}
+
+// Is this frame COSMETIC (splash screens?)
+void EInkDynamicDisplay::checkCosmetic()
+{
+    // If a decision was already reached, don't run the check
+    if (refresh != UNSPECIFIED)
+        return;
+
+    // A full refresh is requested for cosmetic purposes: we have a decision
+    if (frameFlags & COSMETIC) {
+        refresh = FULL;
+        reason = FLAGGED_COSMETIC;
+        LOG_DEBUG("refresh=FULL, reason=FLAGGED_COSMETIC, frameFlags=0x%x", frameFlags);
+    }
+}
+
+// Is this a one-off special circumstance, where we REALLY want a fast refresh?
+void EInkDynamicDisplay::checkDemandingFast()
+{
+    // If a decision was already reached, don't run the check
+    if (refresh != UNSPECIFIED)
+        return;
+
+    // A fast refresh is demanded: we have a decision
+    if (frameFlags & DEMAND_FAST) {
+        refresh = FAST;
+        reason = FLAGGED_DEMAND_FAST;
+        LOG_DEBUG("refresh=FAST, reason=FLAGGED_DEMAND_FAST, frameFlags=0x%x", frameFlags);
+    }
+}
+
+// Does the new frame match the currently displayed image?
+void EInkDynamicDisplay::checkFrameMatchesPrevious()
+{
+    // If a decision was already reached, don't run the check
+    if (refresh != UNSPECIFIED)
+        return;
+
+    // If frame is *not* a duplicate, abort the check
+    if (imageHash != previousImageHash)
+        return;
+
+#if !defined(EINK_BACKGROUND_USES_FAST)
+    // If BACKGROUND, and last update was FAST: redraw the same image in FULL (for display health + image quality)
+    if (frameFlags == BACKGROUND && fastRefreshCount > 0) {
+        refresh = FULL;
+        reason = REDRAW_WITH_FULL;
+        LOG_DEBUG("refresh=FULL, reason=REDRAW_WITH_FULL, frameFlags=0x%x", frameFlags);
+        return;
+    }
+#endif
+
+    // Not redrawn, not COSMETIC, not DEMAND_FAST
+    refresh = SKIPPED;
+    reason = FRAME_MATCHED_PREVIOUS;
+    LOG_DEBUG("refresh=SKIPPED, reason=FRAME_MATCHED_PREVIOUS, frameFlags=0x%x", frameFlags);
+}
+
+// Have too many fast-refreshes occurred consecutively, since last full refresh?
+void EInkDynamicDisplay::checkConsecutiveFastRefreshes()
+{
+    // If a decision was already reached, don't run the check
+    if (refresh != UNSPECIFIED)
+        return;
+
+    // Bypass limit if UNLIMITED_FAST mode is active
+    if (frameFlags & UNLIMITED_FAST) {
+        refresh = FAST;
+        reason = NO_OBJECTIONS;
+        LOG_DEBUG("refresh=FAST, reason=UNLIMITED_FAST_MODE_ACTIVE, frameFlags=0x%x", frameFlags);
+        return;
+    }
+
+    // If too many FAST refreshes consecutively - force a FULL refresh
+    if (fastRefreshCount >= EINK_LIMIT_FASTREFRESH) {
+        refresh = FULL;
+        reason = EXCEEDED_LIMIT_FASTREFRESH;
+        LOG_DEBUG("refresh=FULL, reason=EXCEEDED_LIMIT_FASTREFRESH, frameFlags=0x%x", frameFlags);
+    }
+}
+
+// No objections, we can perform fast-refresh, if desired
+void EInkDynamicDisplay::checkFastRequested()
+{
+    if (refresh != UNSPECIFIED)
+        return;
+
+    if (frameFlags == BACKGROUND) {
+#ifdef EINK_BACKGROUND_USES_FAST
+        // If we want BACKGROUND to use fast. (FULL only when a limit is hit)
+        refresh = FAST;
+        reason = BACKGROUND_USES_FAST;
+        LOG_DEBUG("refresh=FAST, reason=BACKGROUND_USES_FAST, fastRefreshCount=%lu, frameFlags=0x%x", fastRefreshCount,
+                  frameFlags);
+#else
+        // If we do want to use FULL for BACKGROUND updates
+        refresh = FULL;
+        reason = FLAGGED_BACKGROUND;
+        LOG_DEBUG("refresh=FULL, reason=FLAGGED_BACKGROUND");
+#endif
+    }
+
+    // Sanity: confirm that we did ask for a RESPONSIVE frame.
+    if (frameFlags & RESPONSIVE) {
+        refresh = FAST;
+        reason = NO_OBJECTIONS;
+        LOG_DEBUG("refresh=FAST, reason=NO_OBJECTIONS, fastRefreshCount=%lu, frameFlags=0x%x", fastRefreshCount, frameFlags);
+    }
+}
+
+// Reset the timer used for rate-limiting
+void EInkDynamicDisplay::resetRateLimiting()
+{
+    previousRunMs = millis();
+}
+
+// Generate a hash of this frame, to compare against previous update
+void EInkDynamicDisplay::hashImage()
+{
+    imageHash = 0;
+
+    // Sum all bytes of the image buffer together
+    for (uint16_t b = 0; b < (displayWidth / 8) * displayHeight; b++) {
+        imageHash ^= buffer[b] << b;
+    }
+}
+
+// Store the results of determineMode() for future use, and reset for next call
+void EInkDynamicDisplay::storeAndReset()
+{
+    previousFrameFlags = frameFlags;
+    previousRefresh = refresh;
+    previousReason = reason;
+
+    // Only store image hash if the display will update
+    if (refresh != SKIPPED) {
+        previousImageHash = imageHash;
+    }
+
+    frameFlags = BACKGROUND;
+    refresh = UNSPECIFIED;
+}
+
+#ifdef EINK_LIMIT_GHOSTING_PX
+// Count how many ghost pixels the new image will display
+void EInkDynamicDisplay::countGhostPixels()
+{
+    // If a decision was already reached, don't run the check
+    if (refresh != UNSPECIFIED)
+        return;
+
+    // Start a new count
+    ghostPixelCount = 0;
+
+    // Check new image, bit by bit, for any white pixels at locations marked "dirty"
+    for (uint16_t i = 0; i < displayBufferSize; i++) {
+        for (uint8_t bit = 0; bit < 7; bit++) {
+
+            const bool dirty = (dirtyPixels[i] >> bit) & 1;       // Has pixel location been drawn to since full-refresh?
+            const bool shouldBeBlank = !((buffer[i] >> bit) & 1); // Is pixel location white in the new image?
+
+            // If pixel is (or has been) black since last full-refresh, and now is white: ghosting
+            if (dirty && shouldBeBlank)
+                ghostPixelCount++;
+
+            // Update the dirty status for this pixel - will this location become a ghost if set white in future?
+            if (!dirty && !shouldBeBlank)
+                dirtyPixels[i] |= (1 << bit);
+        }
+    }
+
+    LOG_DEBUG("ghostPixels=%hu, ", ghostPixelCount);
+}
+
+// Check if ghost pixel count exceeds the defined limit
+void EInkDynamicDisplay::checkExcessiveGhosting()
+{
+    // If a decision was already reached, don't run the check
+    if (refresh != UNSPECIFIED)
+        return;
+
+    countGhostPixels();
+
+    // If too many ghost pixels, select full refresh
+    if (ghostPixelCount > EINK_LIMIT_GHOSTING_PX) {
+        refresh = FULL;
+        reason = EXCEEDED_GHOSTINGLIMIT;
+        LOG_DEBUG("refresh=FULL, reason=EXCEEDED_GHOSTINGLIMIT, frameFlags=0x%x", frameFlags);
+    }
+}
+
+// Clear the dirty pixels array. Call when full-refresh cleans the display.
+void EInkDynamicDisplay::resetGhostPixelTracking()
+{
+    // Copy the current frame into dirtyPixels[] from the display buffer
+    memcpy(dirtyPixels.get(), EInkDisplay::buffer, EInkDisplay::displayBufferSize);
+}
+#endif // EINK_LIMIT_GHOSTING_PX
+
+// Handle any asyc tasks
+void EInkDynamicDisplay::onNotify(uint32_t notification)
+{
+    // Which task
+    switch (notification) {
+    case DUE_POLL_ASYNCREFRESH:
+        pollAsyncRefresh();
+        break;
+    }
+}
+
+#ifdef HAS_EINK_ASYNCFULL
+// Public: wait for an refresh already in progress, then run the post-update code. See Screen::setScreensaverFrames()
+void EInkDynamicDisplay::joinAsyncRefresh()
+{
+    // If no async refresh running, nothing to do
+    if (!asyncRefreshRunning)
+        return;
+
+    LOG_DEBUG("Join an async refresh in progress");
+
+    // Continually poll the BUSY pin
+    while (adafruitDisplay->epd2.isBusy())
+        yield();
+
+    // If asyncRefreshRunning flag is still set, but display's BUSY pin reports the refresh is done
+    adafruitDisplay->endAsyncFull(); // Run the end of nextPage() code
+    EInkDisplay::endUpdate();        // Run base-class code to finish off update (NOT our derived class override)
+    asyncRefreshRunning = false;     // Unset the flag
+    LOG_DEBUG("Refresh complete");
+
+    // Note: this code only works because of a modification to meshtastic/GxEPD2.
+    // It is only equipped to intercept calls to nextPage()
+}
+
+// Called from NotifiedWorkerThread. Run the post-update code if the hardware is ready
+void EInkDynamicDisplay::pollAsyncRefresh()
+{
+    // In theory, this condition should never be met
+    if (!asyncRefreshRunning)
+        return;
+
+    // Still running, check back later
+    if (adafruitDisplay->epd2.isBusy()) {
+        // Schedule next call of pollAsyncRefresh()
+        NotifiedWorkerThread::notifyLater(intervalPollAsyncRefresh, DUE_POLL_ASYNCREFRESH, true);
+        return;
+    }
+
+    // If asyncRefreshRunning flag is still set, but display's BUSY pin reports the refresh is done
+    adafruitDisplay->endAsyncFull(); // Run the end of nextPage() code
+    EInkDisplay::endUpdate();        // Run base-class code to finish off update (NOT our derived class override)
+    asyncRefreshRunning = false;     // Unset the flag
+    LOG_DEBUG("Async full-refresh complete");
+
+    // Note: this code only works because of a modification to meshtastic/GxEPD2.
+    // It is only equipped to intercept calls to nextPage()
+}
+
+// Check the status of "async full-refresh"; skip if running
+void EInkDynamicDisplay::checkBusyAsyncRefresh()
+{
+    // No refresh taking place, continue with determineMode()
+    if (!asyncRefreshRunning)
+        return;
+
+    // Full refresh still running
+    if (adafruitDisplay->epd2.isBusy()) {
+        // No refresh
+        refresh = SKIPPED;
+
+        // Set the reason, marking what type of frame we're skipping
+        if (frameFlags & DEMAND_FAST)
+            reason = ASYNC_REFRESH_BLOCKED_DEMANDFAST;
+        else if (frameFlags & COSMETIC)
+            reason = ASYNC_REFRESH_BLOCKED_COSMETIC;
+        else if (frameFlags & RESPONSIVE)
+            reason = ASYNC_REFRESH_BLOCKED_RESPONSIVE;
+        else
+            reason = ASYNC_REFRESH_BLOCKED_BACKGROUND;
+
+        return;
+    }
+
+    // Async refresh appears to have stopped, but wasn't caught by onNotify()
+    else
+        pollAsyncRefresh(); // Check (and terminate) the async refresh manually
+}
+
+// Hold control while an async refresh runs
+void EInkDynamicDisplay::awaitRefresh()
+{
+    // Continually poll the BUSY pin
+    while (adafruitDisplay->epd2.isBusy())
+        yield();
+
+    // End the full-refresh process
+    adafruitDisplay->endAsyncFull(); // Run the end of nextPage() code
+    EInkDisplay::endUpdate();        // Run base-class code to finish off update (NOT our derived class override)
+    asyncRefreshRunning = false;     // Unset the flag
+}
+#endif // HAS_EINK_ASYNCFULL
+
+#endif // USE_EINK_DYNAMICDISPLAY

src/graphics/EInkDynamicDisplay.h

@@ -0,0 +1,155 @@
+#pragma once
+
+#include "configuration.h"
+#include <memory>
+
+#if defined(USE_EINK) && defined(USE_EINK_DYNAMICDISPLAY)
+
+#include "EInkDisplay2.h"
+#include "GxEPD2_BW.h"
+#include "concurrency/NotifiedWorkerThread.h"
+
+/*
+    Derives from the EInkDisplay adapter class.
+    Accepts suggestions from Screen class about frame type.
+    Determines which refresh type is most suitable.
+    (Full, Fast, Skip)
+*/
+
+class EInkDynamicDisplay : public EInkDisplay, protected concurrency::NotifiedWorkerThread
+{
+  public:
+    // Constructor
+    // ( Parameters unused, passed to EInkDisplay. Maintains compatibility OLEDDisplay class )
+    EInkDynamicDisplay(uint8_t address, int sda, int scl, OLEDDISPLAY_GEOMETRY geometry, HW_I2C i2cBus);
+    ~EInkDynamicDisplay();
+
+    // Methods to enable or disable unlimited fast refresh mode
+    void enableUnlimitedFastMode() { addFrameFlag(UNLIMITED_FAST); }
+    void disableUnlimitedFastMode() { frameFlags = (frameFlagTypes)(frameFlags & ~UNLIMITED_FAST); }
+
+    // What kind of frame is this
+    enum frameFlagTypes : uint8_t {
+        BACKGROUND = (1 << 0),  // For frames via display()
+        RESPONSIVE = (1 << 1),  // For frames via forceDisplay()
+        COSMETIC = (1 << 2),    // For splashes
+        DEMAND_FAST = (1 << 3), // Special case only
+        BLOCKING = (1 << 4),    // Modifier - block while refresh runs
+        UNLIMITED_FAST = (1 << 5)
+    };
+    void addFrameFlag(frameFlagTypes flag);
+
+    // Set the correct frame flag, then call universal "update()" method
+    void display() override;
+    bool forceDisplay(uint32_t msecLimit) override; // Shadows base class. Parameter and return val unused.
+
+  protected:
+    enum refreshTypes : uint8_t { // Which refresh operation will be used
+        UNSPECIFIED,
+        FULL,
+        FAST,
+        SKIPPED,
+    };
+    enum reasonTypes : uint8_t { // How was the decision reached
+        NO_OBJECTIONS,
+        ASYNC_REFRESH_BLOCKED_DEMANDFAST,
+        ASYNC_REFRESH_BLOCKED_COSMETIC,
+        ASYNC_REFRESH_BLOCKED_RESPONSIVE,
+        ASYNC_REFRESH_BLOCKED_BACKGROUND,
+        EXCEEDED_RATELIMIT_FAST,
+        EXCEEDED_RATELIMIT_FULL,
+        FLAGGED_COSMETIC,
+        FLAGGED_DEMAND_FAST,
+        EXCEEDED_LIMIT_FASTREFRESH,
+        EXCEEDED_GHOSTINGLIMIT,
+        FRAME_MATCHED_PREVIOUS,
+        BACKGROUND_USES_FAST,
+        FLAGGED_BACKGROUND,
+        REDRAW_WITH_FULL,
+    };
+
+    enum notificationTypes : uint8_t { // What was onNotify() called for
+        NONE = 0,                      // This behavior (NONE=0) is fixed by NotifiedWorkerThread class
+        DUE_POLL_ASYNCREFRESH = 1,
+    };
+    const uint32_t intervalPollAsyncRefresh = 100;
+
+    void onNotify(uint32_t notification) override; // Handle any async tasks - overrides NotifiedWorkerThread
+    void configForFastRefresh();                   // GxEPD2 code to set fast-refresh
+    void configForFullRefresh();                   // GxEPD2 code to set full-refresh
+    bool determineMode();                          // Assess situation, pick a refresh type
+    void applyRefreshMode();                       // Run any relevant GxEPD2 code, so next update will use correct refresh type
+    void adjustRefreshCounters();                  // Update fastRefreshCount
+    bool update();                                 // Trigger the display update - determine mode, then call base class
+    void endOrDetach();                            // Run the post-update code, or delegate it off to checkBusyAsyncRefresh()
+
+    // Checks as part of determineMode()
+    void checkInitialized();              // Is this the very first frame?
+    void checkForPromotion();             // Was a frame skipped (rate, display busy) that should have been a FAST refresh?
+    void checkRateLimiting();             // Is this frame too soon?
+    void checkCosmetic();                 // Was the COSMETIC flag set?
+    void checkDemandingFast();            // Was the DEMAND_FAST flag set?
+    void checkFrameMatchesPrevious();     // Does the new frame match the existing display image?
+    void checkConsecutiveFastRefreshes(); // Too many fast-refreshes consecutively?
+    void checkFastRequested();            // Was the flag set for RESPONSIVE, or only BACKGROUND?
+
+    void resetRateLimiting(); // Set previousRunMs - this now counts as an update, for rate-limiting
+    void hashImage();         // Generate a hashed version of this frame, to compare against previous update
+    void storeAndReset();     // Keep results of determineMode() for later, tidy-up for next call
+
+    // What we are determining for this frame
+    frameFlagTypes frameFlags = BACKGROUND; // Frame characteristics - determineMode() input
+    refreshTypes refresh = UNSPECIFIED;     // Refresh type - determineMode() output
+    reasonTypes reason = NO_OBJECTIONS;     // Reason - why was refresh type used
+
+    // What happened last time determineMode() ran
+    frameFlagTypes previousFrameFlags = BACKGROUND; // (Previous) Frame flags
+    refreshTypes previousRefresh = UNSPECIFIED;     // (Previous) Outcome
+    reasonTypes previousReason = NO_OBJECTIONS;     // (Previous) Reason
+
+    bool initialized = false;          // Have we drawn at least one frame yet?
+    uint32_t previousRunMs = -1;       // When did determineMode() last run (rather than rejecting for rate-limiting)
+    uint32_t imageHash = 0;            // Hash of the current frame. Don't bother updating if nothing has changed!
+    uint32_t previousImageHash = 0;    // Hash of the previous update's frame
+    uint32_t fastRefreshCount = 0;     // How many fast-refreshes consecutively since last full refresh?
+    refreshTypes currentConfig = FULL; // Which refresh type is GxEPD2 currently configured for
+
+    // Optional - track ghosting, pixel by pixel
+    // May 2024: no longer used by any display. Kept for possible future use.
+#ifdef EINK_LIMIT_GHOSTING_PX
+    void countGhostPixels();                // Count any pixels which have moved from black to white since last full-refresh
+    void checkExcessiveGhosting();          // Check if ghosting exceeds defined limit
+    void resetGhostPixelTracking();         // Clear the dirty pixels array. Call when full-refresh cleans the display.
+    std::unique_ptr<uint8_t[]> dirtyPixels; // Any pixels that have been black since last full-refresh (dynamically allocated mem)
+    uint32_t ghostPixelCount = 0;           // Number of pixels with problematic ghosting. Retained here for LOG_DEBUG use
+#endif
+
+    // Conditional - async full refresh - only with modified meshtastic/GxEPD2
+#if defined(HAS_EINK_ASYNCFULL)
+  public:
+    void joinAsyncRefresh(); // Main thread joins an async refresh already in progress. Blocks, then runs post-update code
+
+  protected:
+    void pollAsyncRefresh();          // Run the post-update code if the hardware is ready
+    void checkBusyAsyncRefresh();     // Check if display is busy running an async full-refresh (rejecting new frames)
+    void awaitRefresh();              // Hold control while an async refresh runs
+    void endUpdate() override {}      // Disable base-class behavior of running post-update immediately after forceDisplay()
+    bool asyncRefreshRunning = false; // Flag, checked by checkBusyAsyncRefresh()
+#else
+  public:
+    void joinAsyncRefresh() {} // Dummy method
+
+  protected:
+    void pollAsyncRefresh() {} // Dummy method. In theory, not reachable
+#endif
+};
+
+// Hide the ugly casts used in Screen.cpp
+#define EINK_ADD_FRAMEFLAG(display, flag) static_cast<EInkDynamicDisplay *>(display)->addFrameFlag(EInkDynamicDisplay::flag)
+#define EINK_JOIN_ASYNCREFRESH(display) static_cast<EInkDynamicDisplay *>(display)->joinAsyncRefresh()
+
+#else // !USE_EINK_DYNAMICDISPLAY
+// Dummy-macro, removes the need for include guards
+#define EINK_ADD_FRAMEFLAG(display, flag)
+#define EINK_JOIN_ASYNCREFRESH(display)
+#endif

src/graphics/EInkParallelDisplay.cpp

@@ -0,0 +1,427 @@
+#include "EInkParallelDisplay.h"
+
+#ifdef USE_EINK_PARALLELDISPLAY
+
+#include "Wire.h"
+#include "variant.h"
+#include <Arduino.h>
+#include <atomic>
+#include <stdlib.h>
+#include <string.h>
+
+#include "FastEPD.h"
+
+// Thresholds for choosing partial vs full update
+#ifndef EPD_PARTIAL_THRESHOLD_ROWS
+#define EPD_PARTIAL_THRESHOLD_ROWS 128 // if changed region <= this many rows, prefer partial
+#endif
+#ifndef EPD_FULLSLOW_PERIOD
+#define EPD_FULLSLOW_PERIOD 100 // every N full updates do a slow (CLEAR_SLOW) full refresh
+#endif
+#ifndef EPD_RESPONSIVE_MIN_MS
+#define EPD_RESPONSIVE_MIN_MS 1000 // simple rate-limit (ms) for responsive updates
+#endif
+
+EInkParallelDisplay::EInkParallelDisplay(uint16_t width, uint16_t height, EpdRotation rot) : epaper(nullptr), rotation(rot)
+{
+    LOG_INFO("init EInkParallelDisplay");
+    // Set dimensions in OLEDDisplay base class
+    this->geometry = GEOMETRY_RAWMODE;
+    this->displayWidth = width;
+    this->displayHeight = height;
+
+    // Round shortest side up to nearest byte, to prevent truncation causing an undersized buffer
+    uint16_t shortSide = min(width, height);
+    uint16_t longSide = max(width, height);
+    if (shortSide % 8 != 0)
+        shortSide = (shortSide | 7) + 1;
+
+    this->displayBufferSize = longSide * (shortSide / 8);
+
+#ifdef EINK_LIMIT_GHOSTING_PX
+    // allocate dirty pixel buffer same size as epaper buffers (rowBytes * height)
+    size_t rowBytes = (this->displayWidth + 7) / 8;
+    dirtyPixelsSize = rowBytes * this->displayHeight;
+    dirtyPixels = (uint8_t *)calloc(dirtyPixelsSize, 1);
+    ghostPixelCount = 0;
+#endif
+}
+
+EInkParallelDisplay::~EInkParallelDisplay()
+{
+#ifdef EINK_LIMIT_GHOSTING_PX
+    if (dirtyPixels) {
+        free(dirtyPixels);
+        dirtyPixels = nullptr;
+    }
+#endif
+    // If an async full update is running, wait for it to finish
+    if (asyncFullRunning.load()) {
+        // wait a short while for task to finish
+        for (int i = 0; i < 50 && asyncFullRunning.load(); ++i) {
+            delay(50);
+        }
+        if (asyncTaskHandle) {
+            // Let it finish or delete it
+            vTaskDelete(asyncTaskHandle);
+            asyncTaskHandle = nullptr;
+        }
+    }
+
+    delete epaper;
+}
+
+/*
+ * Called by the OLEDDisplay::init() path.
+ */
+bool EInkParallelDisplay::connect()
+{
+    LOG_INFO("Do EPD init");
+    if (!epaper) {
+        epaper = new FASTEPD;
+#if defined(T5_S3_EPAPER_PRO_V1)
+        epaper->initPanel(BB_PANEL_LILYGO_T5PRO, 28000000);
+#elif defined(T5_S3_EPAPER_PRO_V2)
+        epaper->initPanel(BB_PANEL_LILYGO_T5PRO_V2, 28000000);
+        // initialize all port 0 pins (0-7) as outputs / HIGH
+        for (int i = 0; i < 8; i++) {
+            epaper->ioPinMode(i, OUTPUT);
+            epaper->ioWrite(i, HIGH);
+        }
+#else
+#error "unsupported EPD device!"
+#endif
+    }
+
+    // epaper->setRotation(rotation); // does not work, messes up width/height
+    epaper->setMode(BB_MODE_1BPP);
+    epaper->clearWhite();
+    epaper->fullUpdate(true);
+
+#ifdef EINK_LIMIT_GHOSTING_PX
+    // After a full/clear the dirty tracking should be reset
+    resetGhostPixelTracking();
+#endif
+
+    return true;
+}
+
+/*
+ * sendCommand - simple passthrough (not required for epd_driver-based path)
+ */
+void EInkParallelDisplay::sendCommand(uint8_t com)
+{
+    LOG_DEBUG("EInkParallelDisplay::sendCommand %d", (int)com);
+}
+
+/*
+ * Start a background task that will perform a blocking fullUpdate(). This lets
+ * display() return quickly while the heavy refresh runs in the background.
+ */
+void EInkParallelDisplay::startAsyncFullUpdate(int clearMode)
+{
+    if (asyncFullRunning.load())
+        return; // already running
+
+    asyncFullRunning.store(true);
+    // pass 'this' as parameter
+    BaseType_t rc = xTaskCreatePinnedToCore(EInkParallelDisplay::asyncFullUpdateTask, "epd_full", 4096 / sizeof(StackType_t),
+                                            this, 2, &asyncTaskHandle,
+#if CONFIG_FREERTOS_UNICORE
+                                            0
+#else
+                                            1
+#endif
+    );
+    if (rc != pdPASS) {
+        LOG_WARN("Failed to create async full-update task, falling back to blocking update");
+        epaper->fullUpdate(clearMode, false);
+        epaper->backupPlane();
+        asyncFullRunning.store(false);
+        asyncTaskHandle = nullptr;
+    }
+}
+
+/*
+ * FreeRTOS task entry: runs the full update and then backs up plane.
+ */
+void EInkParallelDisplay::asyncFullUpdateTask(void *pvParameters)
+{
+    EInkParallelDisplay *self = static_cast<EInkParallelDisplay *>(pvParameters);
+    if (!self) {
+        vTaskDelete(nullptr);
+        return;
+    }
+
+    // choose CLEAR_SLOW occasionally
+    int clearMode = CLEAR_FAST;
+    if (self->fastRefreshCount >= EPD_FULLSLOW_PERIOD) {
+        clearMode = CLEAR_SLOW;
+        self->fastRefreshCount = 0;
+    } else {
+        // when running async full, treat it as a full so reset fast count
+        self->fastRefreshCount = 0;
+    }
+
+    self->epaper->fullUpdate(clearMode, false);
+    self->epaper->backupPlane();
+
+#ifdef EINK_LIMIT_GHOSTING_PX
+    // A full refresh clears ghosting state
+    self->resetGhostPixelTracking();
+#endif
+
+    self->asyncFullRunning.store(false);
+    self->asyncTaskHandle = nullptr;
+
+    // delete this task
+    vTaskDelete(nullptr);
+}
+
+/*
+ * Convert the OLEDDisplay buffer (vertical byte layout) into the 1bpp horizontal-bytes
+ * buffer used by the FASTEPD library. For performance we write directly into FASTEPD's
+ * currentBuffer() while comparing against previousBuffer() to detect changed rows.
+ * After conversion we call FASTEPD::partialUpdate() or FASTEPD::fullUpdate() according
+ * to a heuristic so only the minimal region is refreshed.
+ */
+void EInkParallelDisplay::display(void)
+{
+    const uint16_t w = this->displayWidth;
+    const uint16_t h = this->displayHeight;
+
+    // Simple rate limiting: avoid very-frequent responsive updates
+    uint32_t nowMs = millis();
+    if (lastUpdateMs != 0 && (nowMs - lastUpdateMs) < EPD_RESPONSIVE_MIN_MS) {
+        LOG_DEBUG("rate-limited, skipping update");
+        return;
+    }
+
+    // bytes per row in epd format (one byte = 8 horizontal pixels)
+    const uint32_t rowBytes = (w + 7) / 8;
+
+    // Get pointers to internal buffers
+    uint8_t *cur = epaper->currentBuffer();
+    const uint8_t *prev = epaper->previousBuffer(); // may be NULL on first init
+
+    // Track changed row range while converting
+    int newTop = h;     // min changed row (initialized to out-of-range)
+    int newBottom = -1; // max changed row
+
+#ifdef FAST_EPD_PARTIAL_UPDATE_BUG
+    // Track changed byte column range (for clipped fullUpdate fallback)
+    int newLeftByte = (int)rowBytes;
+    int newRightByte = -1;
+#endif
+
+    // Compute a quick hash of the incoming OLED buffer (so we can skip identical frames)
+    uint32_t imageHash = 0;
+    uint32_t bufBytes = (w / 8) * h; // vertical-byte layout size
+    for (uint32_t bi = 0; bi < bufBytes; ++bi) {
+        imageHash ^= ((uint32_t)buffer[bi]) << (bi & 31);
+    }
+    if (imageHash == previousImageHash) {
+        // LOG_DEBUG("image identical to previous, skipping update");
+        return;
+    }
+
+#ifdef EINK_LIMIT_GHOSTING_PX
+    // reset ghost count for this conversion pass; we'll mark bits that change
+    ghostPixelCount = 0;
+#endif
+
+    // Convert: OLED buffer layout -> FASTEPD 1bpp horizontal-bytes layout into cur,
+    // comparing against prev when available to detect changes.
+    for (uint32_t y = 0; y < h; ++y) {
+        const uint32_t base = (y >> 3) * w;               // (y/8) * width
+        const uint8_t bitMask = (uint8_t)(1u << (y & 7)); // mask for this row in vertical-byte layout
+        const uint32_t rowBase = y * rowBytes;
+
+        // process full 8-pixel bytes
+        for (uint32_t xb = 0; xb < rowBytes; ++xb) {
+            uint32_t x0 = xb * 8;
+            // read up to 8 source bytes (vertical-byte per column)
+            uint8_t b0 = (x0 + 0 < w) ? buffer[base + x0 + 0] : 0;
+            uint8_t b1 = (x0 + 1 < w) ? buffer[base + x0 + 1] : 0;
+            uint8_t b2 = (x0 + 2 < w) ? buffer[base + x0 + 2] : 0;
+            uint8_t b3 = (x0 + 3 < w) ? buffer[base + x0 + 3] : 0;
+            uint8_t b4 = (x0 + 4 < w) ? buffer[base + x0 + 4] : 0;
+            uint8_t b5 = (x0 + 5 < w) ? buffer[base + x0 + 5] : 0;
+            uint8_t b6 = (x0 + 6 < w) ? buffer[base + x0 + 6] : 0;
+            uint8_t b7 = (x0 + 7 < w) ? buffer[base + x0 + 7] : 0;
+
+            // build output byte: MSB = leftmost pixel
+            uint8_t out = 0;
+            out |= (uint8_t)((b0 & bitMask) ? 0x80 : 0x00);
+            out |= (uint8_t)((b1 & bitMask) ? 0x40 : 0x00);
+            out |= (uint8_t)((b2 & bitMask) ? 0x20 : 0x00);
+            out |= (uint8_t)((b3 & bitMask) ? 0x10 : 0x00);
+            out |= (uint8_t)((b4 & bitMask) ? 0x08 : 0x00);
+            out |= (uint8_t)((b5 & bitMask) ? 0x04 : 0x00);
+            out |= (uint8_t)((b6 & bitMask) ? 0x02 : 0x00);
+            out |= (uint8_t)((b7 & bitMask) ? 0x01 : 0x00);
+
+            // handle partial byte at end of row by masking off invalid bits
+            uint8_t mask = 0xFF;
+            uint32_t bitsRemain = (w > x0) ? (w - x0) : 0;
+            if (bitsRemain > 0 && bitsRemain < 8) {
+                mask = (uint8_t)(0xFF << (8 - bitsRemain));
+                out &= mask;
+            }
+
+            // invert to FASTEPD polarity
+            out = (~out) & mask;
+
+            uint32_t pos = rowBase + xb;
+            uint8_t prevVal = prev ? (prev[pos] & mask) : 0x00;
+            // Consider this byte changed if previous buffer differs (or prev is null)
+            bool changed = (prev == nullptr) || (prevVal != out);
+
+#ifdef EINK_LIMIT_GHOSTING_PX
+            if (changed && prev)
+                markDirtyBits(prev, pos, mask, out);
+#endif
+
+            // mark row changed only if the previous buffer differs
+            if (changed) {
+                if (y < (uint32_t)newTop)
+                    newTop = y;
+                if ((int)y > newBottom)
+                    newBottom = y;
+#ifdef FAST_EPD_PARTIAL_UPDATE_BUG
+                // record changed column bytes
+                if ((int)xb < newLeftByte)
+                    newLeftByte = (int)xb;
+                if ((int)xb > newRightByte)
+                    newRightByte = (int)xb;
+#endif
+            }
+
+            // Always write the computed value into the current buffer (avoid leaving stale bytes)
+            cur[pos] = (cur[pos] & ~mask) | out;
+        }
+    }
+
+    // If nothing changed, avoid any panel update
+    if (newBottom < 0) {
+        LOG_DEBUG("no pixel changes detected, skipping update (conv)");
+        previousImageHash = imageHash; // still remember that frame
+        return;
+    }
+
+    // Choose partial vs full update using heuristic
+    // Decide if we should force a full update after many fast updates
+    bool forceFull = (fastRefreshCount >= EPD_FULLSLOW_PERIOD);
+
+#ifdef EINK_LIMIT_GHOSTING_PX
+    // If ghost pixels exceed limit, force a full update to clear ghosting
+    if (ghostPixelCount > ghostPixelLimit) {
+        LOG_WARN("ghost pixels %u > limit %u, forcing full refresh", ghostPixelCount, ghostPixelLimit);
+        forceFull = true;
+    }
+#endif
+
+    // Compute pixel bounds from newTop/newBottom
+    int startRow = (newTop / 8) * 8;
+    int endRow = (newBottom / 8) * 8 + 7;
+
+    LOG_DEBUG("EPD update rows=%d..%d alignedRows=%d..%d rowBytes=%u", newTop, newBottom, startRow, endRow, rowBytes);
+
+    if (epaper->getMode() == BB_MODE_1BPP && !forceFull && (newBottom - newTop) <= EPD_PARTIAL_THRESHOLD_ROWS) {
+        // Prefer partial update path if driver is reliable; otherwise use clipped fullUpdate fallback.
+#ifdef FAST_EPD_PARTIAL_UPDATE_BUG
+        // Workaround for FastEPD partial update bug: use clipped fullUpdate instead
+        // Build a pixel rectangle for a clipped fullUpdate using the changed columns
+        int startCol = (newLeftByte <= newRightByte) ? (newLeftByte * 8) : 0;
+        int endCol = (newLeftByte <= newRightByte) ? ((newRightByte + 1) * 8 - 1) : (w - 1);
+
+        BB_RECT rect{startCol, startRow, endCol - startCol + 1, endRow - startRow + 1};
+        // LOG_DEBUG("Using clipped fullUpdate rect x=%d y=%d w=%d h=%d", rect.x, rect.y, rect.w, rect.h);
+        epaper->fullUpdate(CLEAR_FAST, false, &rect);
+#else
+        // Use rows for partial update
+        LOG_DEBUG("calling partialUpdate startRow=%d endRow=%d", startRow, endRow);
+        epaper->partialUpdate(true, startRow, endRow);
+#endif
+        epaper->backupPlane();
+        fastRefreshCount++;
+    } else {
+        // Full update: run async if possible (startAsyncFullUpdate will fall back to blocking)
+        startAsyncFullUpdate(forceFull ? CLEAR_SLOW : CLEAR_FAST);
+    }
+
+    lastUpdateMs = millis();
+    previousImageHash = imageHash;
+
+    // Keep same behavior as before
+    lastDrawMsec = millis();
+}
+
+#ifdef EINK_LIMIT_GHOSTING_PX
+// markDirtyBits: mark per-bit dirty flags and update ghostPixelCount
+void EInkParallelDisplay::markDirtyBits(const uint8_t *prevBuf, uint32_t pos, uint8_t mask, uint8_t out)
+{
+    // defensive: need dirtyPixels allocated and prevBuf valid
+    if (!dirtyPixels || !prevBuf)
+        return;
+
+    // 'out' is in FASTEPD polarity (1 = black, 0 = white)
+    uint8_t newBlack = out & mask;    // bits that will be black now
+    uint8_t newWhite = (~out) & mask; // bits that will be white now
+
+    // previously recorded dirty bits for this byte
+    uint8_t before = dirtyPixels[pos];
+
+    // Ghost bits: bits that were previously marked dirty and are now being driven white
+    uint8_t ghostBits = before & newWhite;
+    if (ghostBits) {
+        ghostPixelCount += __builtin_popcount((unsigned)ghostBits);
+    }
+
+    // Only mark bits dirty when they turn black now (accumulate until a full refresh)
+    uint8_t newlyDirty = newBlack & (~before);
+    if (newlyDirty) {
+        dirtyPixels[pos] |= newlyDirty;
+    }
+}
+
+// reset ghost tracking (call after a full refresh)
+void EInkParallelDisplay::resetGhostPixelTracking()
+{
+    if (!dirtyPixels)
+        return;
+    memset(dirtyPixels, 0, dirtyPixelsSize);
+    ghostPixelCount = 0;
+}
+#endif
+
+/*
+ * forceDisplay: use lastDrawMsec
+ */
+bool EInkParallelDisplay::forceDisplay(uint32_t msecLimit)
+{
+    uint32_t now = millis();
+    if (lastDrawMsec == 0 || (now - lastDrawMsec) > msecLimit) {
+        display();
+        return true;
+    }
+    return false;
+}
+
+void EInkParallelDisplay::endUpdate()
+{
+    {
+        // ensure any async full update is started/completed
+        if (asyncFullRunning.load()) {
+            // nothing to do; background task will run and call backupPlane when done
+        } else {
+            epaper->fullUpdate(CLEAR_FAST, false);
+            epaper->backupPlane();
+#ifdef EINK_LIMIT_GHOSTING_PX
+            resetGhostPixelTracking();
+#endif
+        }
+    }
+}
+
+#endif