Compare commits

..
321 Commits
Author SHA1 Message Date
Jonathan Bennett 11a14bdda5 latest test changes 2026-05-31 17:02:08 -04:00
Jonathan Bennett f25e3e893d talking stick 2026-05-29 22:18:32 +01:00
Jonathan Bennett 134ec6dc54 Add real Lock handling on Portduino 2026-05-29 22:17:10 +01:00
Jonathan BennettandGitHub a04b7c4a87 Merge branch 'develop' into vibe-coded-dmshell 2026-05-29 13:49:20 -05:00
60303968bb Add Heltec mesh node t1 (#10416)
* add heltec-mesh-node-t1

* fixed low power

* Update the sensor enumeration values.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix memory leak in ICM42607PSensor

* fix  ST7735_MISO  error

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-29 12:47:09 -05:00
Catalin PatuleaandGitHub d3c7f05baa Simplify tracking of BLE connection handle & improve thread safety. (#10390)
- Redefine isConnected in terms of nimbleBluetoothConnHandle. isConnected was not safe because BLEServer::getConnectedCount is not thread-safe (https://github.com/espressif/arduino-esp32/issues/12538) while isConnected is called from various threads. Now we can avoid checking bleServer every time before calling isConnected.
- getRssi: don't try to "populate nimbleBluetoothConnHandle", that requires calling BLEServer::getPeerDevices which is not thread-safe (same issue as above).
2026-05-29 06:48:43 -05:00
Ben Meadors 1971e5ab13 Remove now unused payload variant settings in allocAtakPli function 2026-05-29 06:15:11 -05:00
Ben Meadors e2fda6598c Update protobufs 2026-05-28 21:01:30 -05:00
982440d21d Noise floor (#9347)
* add noise floor

* Sliding window noise floor

* Add getCurrentRSSI() to SimRadio for noise floor support

* Remove sendLocalStatsToPhone call from runOnce

* Change noise floor to int32_t type

* Use int32_t for RSSI sample storage in noise floor

* Remove float cast from noise floor assignment

* Fix Copilot review issues: fix noise floor logic, types, and null pointer

- Use robust busyTx/busyRx checks instead of simple isReceiving check
- Initialize noiseFloorSamples to NOISE_FLOOR_MIN instead of 0
- Move noise_floor assignment inside null check to prevent potential crash
- Change getNoiseFloor() and getAverageNoiseFloor() to return int32_t
- Fix RSSI validation to check for positive values (rssi > 0)
- Fix format specifier from %.1f to %d for int32_t
- Update comments to accurately reflect the sampling logic

* Fix RSSI condition to include zero value

* Change noise floor initialization to zero

* Disable noise floor for LR11x0 chips: getRSSI(bool) unsupported

* Remove updateNoiseFloor call from onNotify to avoid radio queue overflow

Per PR review feedback, calling updateNoiseFloor() in onNotify() for every
ISR event (ISR_TX, ISR_RX, TRANSMIT_DELAY_COMPLETED) can cause the LoRa
radio queue to get full. The noise floor sampling still happens in
startReceive() and after transmitting.

* fix lr11x0 current rssi

* Address noise floor review comments

* Address Copilot SimRadio noise floor comments

* Fix RadioLibInterface formatting

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
2026-05-28 19:39:41 -05:00
Ben Meadors f9fea562aa Add TAKTALK message and room data structures to support voice/text chat 2026-05-27 09:05:38 -05:00
Ben Meadors c366296ab4 Update LoRaConfig region codes and add new amateur radio bands 2026-05-26 18:56:16 -05:00
Ben MeadorsGitHubCopilotcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
a076d97eb6 Implement ATAK Plugin V2 and drop unishox2 compression support (#10105)
* Implement ATAK Plugin V2 and drop unishox2 compression support

* Fix course calculation and improve error handling in allocAtakPli

* Update src/modules/AtakPluginModule.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix buffer overflow in allocAtakPli by ensuring null termination for callsigns

* Add missing include for concurrency::OSThread in AtakPluginModule.h

* Update src/modules/AtakPluginModule.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Refactor allocAtakPli to use configurable team and role; improve position source mapping

* Update src/modules/PositionModule.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remove OSThread inheritance from AtakPluginModule - pure passthrough needs no periodic scheduling

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/bdc82eb6-77c4-4711-839c-04bcbb1aa9cd

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Apply clang-format (trunk fmt)

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-05-26 15:16:47 -05:00
Clive BlackledgeandGitHub 32dcd90abf Preserve forwarded position payload precision (#10554) 2026-05-26 06:33:11 -05:00
5b7a5b2c22 feat: add Raspberry Pi Pico 2 + W5500 + E22-900M30S variant (#10135)
* feat: add Raspberry Pi Pico 2 + W5500 + E22-900M30S variant

Adds community variant for Raspberry Pi Pico 2 (RP2350, 4 MB flash)
with external WIZnet W5500 Ethernet module and EBYTE E22-900M30S LoRa
module (SX1262, 30 dBm PA, 868/915 MHz).

Key details:
- LoRa on SPI1: GP10/11/12/13 (SCK/MOSI/MISO/CS), RST=GP15,
  DIO1=GP14, BUSY=GP2, RXEN=GP3 (held HIGH via SX126X_ANT_SW)
- W5500 on SPI0: GP16/17/18/19/20 (MISO/CS/SCK/MOSI/RST)
- SX126X_DIO2_AS_RF_SWITCH: DIO2→TXEN bridge on module handles PA
- SX126X_DIO3_TCXO_VOLTAGE 1.8: TCXO support via EBYTE_E22 flags
- DHCP timeout reduced to 10 s to avoid blocking LoRa startup
- GPS on UART1/Serial2: GP8 TX, GP9 RX
- Reuses WIZNET_5500_EVB_PICO2 code paths for Ethernet init

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* pico2_w5500_e22: rename define and address review feedback

Rename WIZNET_5500_EVB_PICO2 to PICO2_W5500_E22 so the variant-specific
define matches the variant directory name and isn't confused with an
on-board EVB SKU.

Review fixes from PR #10135:
- Gate the 10 s Ethernet DHCP timeout behind PICO2_W5500_E22 so other
  Ethernet builds keep the default 60 s behavior; apply the same timeout
  to reconnectETH() for consistency.
- Drop the unused -D EBYTE_E22 flag; EBYTE_E22_900M30S already selects
  TX_GAIN_LORA / SX126X_MAX_POWER in src/configuration.h.
- Rewrite "on-board W5500" comments to describe the external module.
- Correct README TX_GAIN_LORA value (7, not 10) and drop the EBYTE_E22
  row.

* fix(pico2_w5500_e22): drop DEBUG_RP2040_PORT=Serial

The arduino-pico framework hooks _write() when DEBUG_RP2040_PORT=Serial
is set and dumps raw debug bytes onto USB CDC, corrupting any binary
protobuf stream sent through StreamAPI (e.g. `meshtastic --port COMx`).

The variant excludes BT and WiFi, so the primary client transport is
Ethernet TCP via ethServerAPI — unaffected — but users who configure
the node over USB serial would see protobuf decode failures from
debug-byte interleaving. Removing the flag restores clean USB CDC.

Debug output can still be enabled per-build by adding -D DEBUG_RP2040_PORT=Serial1
to redirect to UART0 instead of USB CDC.

* style(pico2_w5500_e22): apply trunk fmt — fixes Trunk Check

- variant.h: clang-format 16.0.3 (drop manual #define alignment)
- README.md: prettier + add `text` language to fenced code blocks (markdownlint MD040)
- wiring.svg: svgo optimization

Resolves the Trunk Check Runner failure on this PR (3 unformatted
files + 8 markdownlint issues). No functional changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(pico2_w5500_e22): address review — move to rp2350/diy, generic guards

Maintainer feedback from NomDeTom on PR #10135:

- Move the variant from variants/rp2350/ to variants/rp2350/diy/ to
  distinguish DIY from prebuilt boards (matches the variants/*/diy/
  pattern; still discovered via the existing variants/*/diy/*/platformio.ini
  glob in the root platformio.ini).
- Replace the board-name macro PICO2_W5500_E22 in shared code with a
  generic capability macro USE_ARDUINO_ETHERNET, defined from variant.h.
  DebugConfiguration.h / ethServerAPI.h / ethClient.cpp no longer
  reference a board name.
- Drop the architecture.h hook entirely: variant.h now defines PRIVATE_HW,
  which the existing `#elif defined(PRIVATE_HW)` branch already handles.

No functional change. Build verified: pico2_w5500_e22 SUCCESS
(RAM 19.2%, Flash 28.1%).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(pico2_w5500_e22): drop unoptimized wiring.svg to fix trunk fmt check

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-25 14:05:46 -05:00
Alexander BalyaandGitHub 5bce26d9b7 Fix SHT2x detection for INA219 addresses (#10482) 2026-05-24 21:03:24 -05:00
ManuelandGitHub ef734b73c7 fix: mbed TLS crash in Arduino 3.x (pioarduino) (#10535)
* fix mbed TLS crash

* adapt SSL_MAX_CONTENT_LEN size to framework-libs

* found two more
2026-05-22 20:14:42 -05:00
Ben Meadors c7748a1602 Fix update neighbor_info before checking update_interval in handleSetModuleConfig 2026-05-22 14:53:52 -05:00
Jaime RoldanandGitHub 91f930d5c0 fix(telemetry): stop emitting -0.001V sentinel when battery unavailable (#7958) (#10217)
* fix(telemetry): stop emitting -0.001V sentinel when battery unavailable (#7958)

* address review: use int32_t for batteryMv to avoid uint16_t signed wrap
2026-05-22 06:58:21 -05:00
κρμγandGitHub f2c5cb0a05 fix: first set pinMode, then write to pin (#10520) 2026-05-21 13:30:27 -05:00
e10e13226d nrf54l15: fix SHT4x libdep -- arduino-sht, not Adafruit_SHT4X (#10515)
SHTXXSensor (the SHT4x driver) includes <SHTSensor.h>, gated by
__has_include(<SHTSensor.h>). That header ships in Sensirion/arduino-sht.
Adafruit_SHT4X ships Adafruit_SHT4X.h and has no consumer anywhere in
src/, so the SHT40 driver was silently excluded from the build -- the
nRF54L15 variant could not read an SHT4x sensor as committed in #10193.

Replace the dead Adafruit_SHT4X libdep with arduino-sht v1.2.6.
Validated on nRF54L15-DK: SHT40-AD1B @0x44, 24.3h soak, 0 reboots,
temperature/humidity telemetry stable end-to-end.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-21 11:00:10 -05:00
5e69bc6c3f Enable Narrow and Lite regions for EU (#10120)
* Enable Lite and Narrow regions and introduce getEffectiveDutyCycle for Lite profiles

* Add TrafficType enum and extend getConfiguredOrDefaultMsScaled to manage based on regionProfile settings

* Refactor telemetry modules to include TrafficType in getConfiguredOrDefaultMsScaled calls

* Update submodule protobufs to latest commit

* Add support for new region presets and modem presets in menu options

* Add new LoRa region codes and modem presets for EU bands

* boof

* Add modem presets for LITE and NARROW configurations

* Update subproject commit reference in protobufs

* Update protobufs

* Refactor modem preset definitions to use macro for consistency and clarity

* Refactor modem preset cases to use PRESET macro for consistency

* fix: update LoRa region code for EU 868 narrowband configuration

Co-authored-by: Copilot <copilot@github.com>

* Fix test suite failure

Co-authored-by: Copilot <copilot@github.com>

* Add override slot override - for when one override isn't enough.

Co-authored-by: Copilot <copilot@github.com>

* address copilot comments

---------

Co-authored-by: Copilot <copilot@github.com>
2026-05-21 10:20:09 -05:00
f3cb2bff78 Refactor keyboard cell height logic for consistency (#10501)
Adjust keyboard cell height calculation for better layout consistency across different screen sizes.

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-21 06:12:34 -05:00
AustinandGitHub 894c5556cf Actions: Fix tagging upon release. (#10521)
Current release tags are actually based upon the latest state of `develop` currently...
Specify target_commitish to always use the commit that triggered the build
2026-05-21 06:09:57 -05:00
Ben MeadorsandGitHub 2f92eb8499 Refactor position precision handling to honor explicit channel settings and prevent location leaks (#10513) 2026-05-20 10:18:46 -05:00
Ben Meadors 8d08077412 Add Ethernet configuration to platformio.ini for ThinkNode variants 2026-05-20 07:12:43 -05:00
Ben Meadors 00ec69201d Develop protos should be on develop 2026-05-19 06:57:05 -05:00
Ben Meadors 4304480ca3 Merge remote-tracking branch 'origin/master' into develop 2026-05-19 05:37:01 -05:00
82aefd1af1 Upgrade trunk (#10503)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-05-19 05:23:01 -05:00
0f9eb86830 Enabled SX_LNA_EN by default (#10469)
* Enabled SX_LNA_EN by default
* Update I2C configuration for IO direction and pull settings

---------

Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com>
2026-05-19 04:56:01 -05:00
Thomas Göttgens e2aa44ec54 T-Echo-Card support (#10267)
# Conflicts:
#	src/graphics/draw/UIRenderer.cpp
2026-05-19 09:53:45 +02:00
Thomas GöttgensandGitHub 1747e2d8e5 T-Echo-Card support (#10267) 2026-05-19 09:31:04 +02:00
622aa046f1 Enabled SX_LNA_EN by default (#10469)
* Enabled SX_LNA_EN by default
* Update I2C configuration for IO direction and pull settings

---------

Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com>
2026-05-19 08:31:42 +02:00
Thomas GöttgensandGitHub 0832330327 Fix antenna switch initialization logic 2026-05-19 08:00:02 +02:00
Thomas GöttgensandGitHub 98e0604edf Fix antenna switch initialization logic once more 2026-05-19 07:58:24 +02:00
23ead5f2f1 Update protobufs (#10500)
Co-authored-by: caveman99 <25002+caveman99@users.noreply.github.com>
2026-05-19 07:47:06 +02:00
af3739fd63 Update protobufs (#10499)
Co-authored-by: caveman99 <25002+caveman99@users.noreply.github.com>
2026-05-19 07:22:15 +02:00
0148a89ddb Upgrade trunk (#10493)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-05-18 20:49:28 -05:00
Thomas Göttgens 6199faacf1 cherry pick backport fix for cardputer 2026-05-18 23:24:32 +02:00
Thomas GöttgensandGitHub 3261c04afb Fix Antenna Switch on Cardputer (#10491) 2026-05-18 11:18:40 +02:00
AustinGitHubmverch67ManuelCatalin Patuleacopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>vidplace7CopilotthebenternBen Meadors
a541957480 ESP32: Migrate to Arduino 3.x (pioarduino) (#9122)
* Migrate esp32 families to pioarduino platform

* ESP32c6 align text.handler_execute same as C3

* Use pioarduino `develop`

The latest fixes and the latest bugs!

* preliminary esp32p4.ini

* pioarduino: Update LovyanGFX

Includes Manuel's recent commit

* pioarduino 3.3.6

* pioarduino 3.3.6 *release*

chasing the release

* pioarduino: Fix OG ESP32 duplicate libs

* pioarduino: T-Beam 1W CDC mode

* pioarduino: disable network provisioning (wifiprov)

* pioarduino: use legacy esptoolpy naming (forward-compatible)

* Update lovyangfx from `develop` commit to 1.2.19

* fix esp32p4.ini

* check for esp32 w/ wifi

* esp32-p4 specific adaptations

* Switch to meshtastic/esp32_https_server fork (idf5 branch)

* don't ignore esp_lcd

* config for MUI

* fix/workaround SDMMC

* revert a6f6175, update to 3.3.8

* enable esp_hosted for esp32-p4 (experimental)

* Pioarduino 55.03.38-1

* NimBLE-Arduino -> Arduino "BLE" (3.3.x) migration (#10164)

* NimBLE-Arduino -> Arduino "BLE" (3.3.x) migration

* More NimBLE

* Fix Device Name in ATT Read Request (0x2A00).

Device Name is exposed in two places:

- Advertisement data: this is set properly in startAdvertising.
- GATT attribute Device Name (0x2A00). This one is handled internally in NimBLE
  and comes from ble_svc_gap_device_name_set. This is set initially, but then
  BLEDevice::createServer calls ble_svc_gap_init which resets the device name.
  This causes the device to apparently "change name after pairing":

< ACL Data TX:... flags 0x00 dlen 7  #113 [hci0] 14.241149
      ATT: Read Request (0x0a) len 2
        Handle: 0x0003 Type: Device Name (0x2a00)
> ACL Data RX: Handle 2048 flags 0x02 dlen 11             #115 [hci0] 14.269050
      ATT: Read Response (0x0b) len 6
        Value[6]: 6e696d626c65   # "nimble"

Workaround this by setting the device name once again after
BLEDevice::createServer.

* Temporarily lower CORE_DEBUG_LEVEL to INFO to avoid triggering an apparent ESP-IDF Bluetooth bug when re-connecting to Pixel 8 Android devices.

Initial pairing works, but after ESP32 is rebooted, phone fails to reconnect. Meshtastic app shows it as disconnecting immediately. LightBlue shows a more detailed error "Peripheral Connection - Warning: onConnectionStatusChange: status 61" (0x3D - MIC Failure).

Bug report to Espresssif: https://github.com/espressif/esp-idf/issues/18126#issuecomment-4286197744

* Temporarily disable ble_gap_set_data_len, causes crash with Pixel 8 Android reconnect.

Crash looks like this:
  [ 11966][E][BLEAdvertising.cpp:341] setScanResponseData(): ble_gap_adv_rsp_set_data: 22
  [ 11975][E][BLEAdvertising.cpp:1554] start(): Host reset, wait for sync.
  ERROR | ??:??:?? 11 BLE failed to start advertising
  Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.

  Core  0 register dump:
  PC      : 0x420e6190  PS      : 0x00060730  A0      : 0x820e158b  A1      : 0x3fce50c0
  A2      : 0x00000000  A3      : 0x3fcb8600  A4      : 0x3fcb85cc  A5      : 0x00000000
  A6      : 0x00000000  A7      : 0x00000c03  A8      : 0x00000000  A9      : 0x3fce50b0
  A10     : 0x0000000e  A11     : 0x00000000  A12     : 0x00000010  A13     : 0x3fce50e0
  A14     : 0x00000c03  A15     : 0x00000001  SAR     : 0x0000001e  EXCCAUSE: 0x0000001c
  EXCVADDR: 0x00000000  LBEG    : 0x400570e8  LEND    : 0x400570f3  LCOUNT  : 0x00000000

  Backtrace: 0x420e618d:0x3fce50c0 0x420e1588:0x3fce5110 0x420dfe87:0x3fce5200 0x420dfefb:0x3fce5220 0x420dff3f:0x3fce5240 0x4219602b:0x3fce5260 0x4037b0e5:0x3fce5280 0x4201edf3:0x3fce52a0

Connection seems fast enough even without this. We'll investigate the
reason for the crash and re-enable once it's safe.

---------

Co-authored-by: Catalin Patulea <cronos586@gmail.com>

* Add extension from pioarduino nag
"Jason2866.esp-decoder"

* Cleanup after merge

* ESP32: Disable classic bluetooth

* Cleanup: Fix ADC channels on new variants

* InkHUD: Fix type casting for message size in saveToFlash method

inkhud compiles again!

* update p4 esp_hosted for BT

* I thought I fixed this

* fix linker error using response file (p4 only)

* fix infinite loop

* Fix Power.cpp check warning

Local variable 'config' shadows outer variable [shadowVariable]

* Build ESP32 original with NimBLE ('custom_sdkconfig' approach). (#10235)

* Re-enable littlefs json manifest

This works locally again :)
Not sure what changed

* Re-add tool-mklittlefs

* sensecap indicator fixes after upgrade arduino-esp & lovyanGFX libs

* hackaday fix

* robot tbeam cache error fix

Co-authored-by: Copilot <copilot@github.com>

* trunk fmt

* ignore trunk

* BLEDevice::deinit() added

Co-authored-by: Copilot <copilot@github.com>

* platformio-custom: Modify mtjson target dependency to prevent fake-success. (#10291)

Co-authored-by: Copilot <copilot@github.com>

* Fix ESP32-C6 linker errors.

Align .text.handler_execute section to 4 bytes and update watchdog timer core mask configuration

Co-authored-by: Copilot <copilot@github.com>

* tlora-c6: Disable Screen

MESHTASTIC_EXCLUDE_SCREEN=1 on tlora-c6.

It doesn't have a screen, and this gets it compiling again (saving flash).

* Use mverch's iram_memset hack for all OG-ESP32

* Refactor watchdog timer initialization and handling

* use adc_channel_t in variant.h

* Fix variant headers

* More idiomatic default ethernet that doesn't break the build

* Elecrows: Delete problematic variant.cpp

Not needed after USE_ETHERNET_DEFAULT

---------

Co-authored-by: mverch67 <manuel.verch@gmx.de>
Co-authored-by: Manuel <71137295+mverch67@users.noreply.github.com>
Co-authored-by: Catalin Patulea <cronos586@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-17 18:53:32 -05:00
Jonathan BennettandGitHub 767a748188 add optional LED_LORA to indicate LoRa TX (#10465) 2026-05-16 22:11:49 -05:00
fc5556b8e6 Clamp direct position packets to channel precision (fixes #8640) (#10383)
* Fix position precision for direct sends

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Clarify zero position precision logging

* Use const channel reference for position precision

* Use C linkage for position precision test entrypoints

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-16 14:08:46 -05:00
4a1ff18f57 feat: add Nordic nRF54L15-DK variant (Zephyr + BLE + LoRa) (#10193)
* feat: add Nordic nRF54L15-DK variant (Zephyr + BLE + LoRa)

Adds a community hardware variant for the Nordic nRF54L15-DK (PCA10156)
with an external EBYTE E22-900M30S (SX1262) LoRa module. First Meshtastic
port running on the Zephyr RTOS; all other Nordic targets use the nRF5
SoftDevice stack.

Scope
-----
- New Zephyr-based platform layer under src/platform/nrf54l15/ providing
  Arduino-compatible shims (Arduino.h, SPI, Wire, Print, Stream) over the
  Zephyr APIs plus a LittleFS-backed InternalFileSystem on SPIM20.
- Bluetooth LE peripheral (NRF54L15Bluetooth.*) built on the Zephyr BT
  host stack, exposing the Meshtastic GATT service with legacy
  connectable advertising, just-works pairing, dynamic MTU exchange
  (up to 247 bytes), and iOS connection-parameter tweaks.
- Variant directory variants/nrf54l15/nrf54l15dk/ with pin map for the
  E22 module on connector J1, PlatformIO env (nrf54l15dk), Zephyr
  DT overlay and a wiring README.
- Zephyr project config (zephyr/prj.conf + board overlay) tuned for
  BT + LoRa: 16 KB main stack, 4 KB BT RX thread, RTT logging in
  immediate mode, newlib-nano heap sized to leave room for the GATT
  pools while still allowing ATT MTU=247.
- extra_scripts/nrf54l15_linker.py works around a PlatformIO + old Ninja
  issue where Zephyr's two-pass linker script generation does not run
  automatically; the post-script parses build.ninja and invokes the
  gcc -E step directly before the final link.
- boards/nrf54l15dk.json board definition (PlatformIO needs it for the
  DK; the Seeed platform only ships the XIAO variants).
- variants/rp2350/rp2350.ini excludes platform/nrf54l15/ from RP2350
  build_src_filter so the shared platform tree does not leak between
  targets.
- .gitignore: add nRF J-Link / RTT debug artifacts (flash.jlink,
  rtt_*.txt).

Shared source changes
---------------------
- src/main.{cpp,h}, src/RedirectablePrint.cpp, src/FSCommon.{cpp,h},
  src/mesh/{Channels,NodeDB,RadioLibInterface,MeshService,PhoneAPI}.cpp,
  src/mesh/RadioLibInterface.h, src/modules/AdminModule.cpp: add small
  guards / helpers so the Zephyr build compiles alongside the Arduino
  targets. Behavior on existing boards is unchanged.

Hardware model
--------------
HW_VENDOR maps to meshtastic_HardwareModel_PRIVATE_HW until a dedicated
protobuf enum value is assigned upstream. The variant declares
custom_meshtastic_hw_model = 132 so the maintainers can wire the new
enum value through the protobufs repo after merge.

Hardware note
-------------
The E22-900M30S does not connect its DIO2 pin to TXEN internally — a
wire/solder bridge between DIO2 and TXEN on the module is required for
TX to work. Details and full pin map are in the variant README.

Validation
----------
Built clean against develop. On real hardware (April 2026) the port
passes end-to-end: iOS companion app pairs and connects, configuration
round-trip works, LoRa TX/RX reaches a canonical tbeam on the same mesh
channel, NodeDB updates propagate both ways, and traceroute completes.

* fix(nrf54l15): use atomic fs_rename instead of copy fallback

Zephyr LittleFS on nrf54l15 supports fs_rename natively, so route it
through the same atomic path as ESP32. The previous copyFile+remove
fallback truncated the destination before copying, leaving 0-byte files
if interrupted mid-write.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(nrf54l15): expand storage_partition from 36KB to 700KB

LittleFS on the default 9-block (36KB) storage_partition ran out of
space during copy-on-write of config.proto, causing fs_write to return
ENOSPC and pb_encode to surface "io error" when saving configuration
via the mobile app.

Reclaim slot1_partition (the MCUboot secondary slot — unused since we
flash directly via J-Link) and grow storage_partition to span
0xb6000..0x165000 (~175 blocks).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(nrf54l15): drop USERPREFS_LORACONFIG_* so LoRa config stays mutable

NodeDB rewrites LoRa config from USERPREFS_LORACONFIG_* on every boot,
which prevented reconfiguration via the BLE/serial app. Drop the
variant-level defaults; users configure region and modem preset through
the app like every other variant.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(nrf54l15): enforce MITM passkey pairing on GATT service

- Add MESH_PERM_READ/MESH_PERM_WRITE macros (READ_AUTHEN/WRITE_AUTHEN)
  on all mesh service characteristics so clients must complete passkey
  exchange before accessing fromNum/fromRadio/toRadio/logRadio.
- Wire FIXED_PIN mode to bt_passkey_set() so the device advertises a
  known PIN (config.bluetooth.fixed_pin); RANDOM_PIN keeps default
  per-pairing random passkey.
- Reduce BleDeferredThread HARD_WATCHDOG_MS from 3min to 1min.
- prj.conf: CONFIG_BT_SMP_ENFORCE_MITM=y, CONFIG_BT_FIXED_PASSKEY=y,
  CONFIG_BT_SMP_SC_PAIR_ONLY=n (legacy fallback for clients that abort
  SC pairing with reason 0x01 within 150ms).

* fix(nrf54l15): resolve develop-merge conflict + cppcheck warnings

The `Merge branch 'develop'` left two ~RadioLibInterface() declarations
in src/mesh/RadioLibInterface.h: the inline version added upstream by
PR #10254 (which independently applied the same UAF guard this PR was
carrying) and the out-of-line version this PR introduced. GCC rejects
the duplicate, breaking every platform build. Drop the out-of-line
declaration + definition; keep upstream's inline form.

Also silence the 13 cppcheck low warnings introduced by the new
nrf54l15 Arduino shim — Arduino's `String`/`SPISettings` API contract
relies on implicit single-arg constructors used pervasively by
existing Meshtastic code, so suppress `noExplicitConstructor` inline
with a comment instead of breaking the API. The few mechanical wins
(`const tmp[2]`, `const uint32_t *sp`) are applied directly.

* fmt: fix Trunk Check lint issues on nrf54l15-port

- extra_scripts/nrf54l15_linker.py: move regular imports above
  Import("env") to silence E402, add trunk-ignore-all(F821) for the
  PIO/SCons SConstruct injection (matches esp32_pre.py / nrf52_extra.py
  convention)
- src/platform/nrf54l15/NRF54L15Bluetooth.cpp: clang-format 16.0.3
- boards/nrf54l15dk.json + variants/nrf54l15/nrf54l15dk/README.md:
  prettier 3.8.3 (also resolves markdownlint MD060 on README tables)

No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(nrf54l15): address Copilot review comments + correct clang-format style

Six review threads from the 2026-04-30 Copilot review:

- src/platform/nrf54l15/nrf54l15_main.cpp: validate PSP against the nRF54L15
  SRAM range (0x20000000..0x20040000) and 4-byte alignment before walking the
  faulting thread's stack, and clamp the walk so it never reads past the end
  of RAM. Prevents a second fault inside the fatal handler when PSP is
  corrupted (common in real faults).

- src/platform/nrf54l15/nrf54l15_arduino.cpp: gate the bring-up printk traces
  in digitalWrite/digitalRead (CS/NRESET toggle log, BUSY-before-NRESET
  snapshot, BUSY periodic timeline) behind a new -DNRF54L15_GPIO_DEBUG flag
  that is off by default. The "dev NOT READY" message stays unconditional —
  it indicates a genuine hardware/DTS misconfig.

- src/modules/AdminModule.cpp: don't mutate config.device.output_gpio_enabled
  from handleGetConfig(). Reflect the live pin state in the response payload
  only — a getter must not write back to disk-persisted state.

- src/platform/nrf54l15/InternalFileSystem.h: derive totalBytes() from
  FIXED_PARTITION_SIZE(storage_partition) at compile time so it tracks the
  DK overlay's ~700 KB partition instead of the stale 36 KB hard-coded value.
  Updated the file header comment accordingly.

- extra_scripts/nrf54l15_linker.py: make _extract_gcc_command() handle the
  POSIX Ninja COMMAND format (no `cmd.exe /C "..."` wrapper) in addition to
  the Windows form, so the script doesn't hard-fail on Linux/macOS hosts.

- src/platform/nrf54l15/NRF54L15Bluetooth.cpp: clamp NO_PIN to RANDOM_PIN
  with a one-shot LOG_WARN. The mesh GATT permissions are declared with
  BT_GATT_PERM_*_AUTHEN and prj.conf sets CONFIG_BT_SMP_ENFORCE_MITM=y, so
  NO_PIN with no auth callbacks would leave every characteristic returning
  BT_ATT_ERR_AUTHENTICATION. Falling back to RANDOM_PIN keeps the link
  usable instead of silently broken. Also re-formatted this file with the
  project's .trunk/configs/.clang-format (Linux braces, 4-space indent,
  130-col) — the previous lint-fix commit a2aca3234 accidentally used the
  default LLVM style here, which CI's clang-format would have rejected.

Build verified: pio run -e nrf54l15dk passes, RAM 47.4%, Flash 28.6%.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(nrf54l15): address remaining Copilot review threads

Round 2/3 review fixes — bugs first, then docs/portability:

BLE concurrency (NRF54L15Bluetooth.cpp):
- onNowHasData / sendLog / BleDeferredThread / shutdown: acquire
  active_conn under ble_mutex via new acquire_active_conn() helper so
  disconnected_cb can't free the conn between the null check and
  bt_conn_ref/bt_gatt_notify (use-after-unref).
- write_toradio: reject writes that exceed MAX_TO_FROM_RADIO_SIZE with
  ATT_ERR_INVALID_ATTRIBUTE_LEN instead of returning success and silently
  dropping the payload (would hide failed config writes from the phone).
- start_advertising: truncate the device name to fit the 31-byte legacy
  scan-response limit and switch to BT_DATA_NAME_SHORTENED so
  bt_le_adv_start() doesn't reject the payload when the name approaches
  CONFIG_BT_DEVICE_NAME_MAX=32.

Linker / portability:
- main.h: drop the rp2040Loop() forward declaration that had no
  definition and no callers — would surface as a link error if any RP2040
  build added a call to the symbol.
- nrf54l15_arduino.cpp: transfer16() now uses static __aligned(4) DMA
  buffers (matching transfer()), removing the EasyDMA-reachability hazard
  of caller-stack buffers on this part.

Filesystem (InternalFileSystem.h):
- usedBytes(): return real usage from fs_statvfs() instead of 0 so OTA
  / range-test free-space guards work.
- rewindDirectory(): close the dir before reopening — Zephyr fs_dir_t has
  no rewind, and re-fs_opendir on an open handle leaks LittleFS state.

Crash handler (nrf54l15_main.cpp):
- After the stack walk, busy-wait 50 ms to flush RTT/printk and call
  sys_reboot(SYS_REBOOT_COLD) directly so the saved_crash record is
  actually reported on the next boot. Default Zephyr config has
  RESET_ON_FATAL_ERROR=n, so the previous k_fatal_halt() spun forever.

Generalization / config:
- PhoneAPI.cpp: replace the NRF54L15_DK ifdef with a
  MESHTASTIC_EXCLUDE_FILES_MANIFEST capability flag (defined in the
  nrf54l15dk env) so future variants can opt in/out without touching
  shared code.
- variants/nrf54l15/nrf54l15.ini: parameterize libdeps include paths via
  ${PIOENV} so additional nRF54L15 envs sharing nrf54l15_base don't break.
- prj.conf: drop the stale "36 KB storage_partition" comments — the DK
  overlay reclaims slot1 to ~700 KB and runtime size comes from
  FIXED_PARTITION_SIZE.
- nrf54l15dk overlay: remove the zephyr,console / zephyr,shell-uart
  chosen entries that conflicted with CONFIG_UART_CONSOLE=n + RTT
  console; keep uart30 enabled so swapping the console is one Kconfig
  flip away.

Build: nrf54l15dk SUCCESS (flash 28.6%, RAM 47.4%); wiznet_5500_evb_pico2
SUCCESS (verifies the shared main.h change).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(nrf54l15dk): use PRIVATE_HW (255) for custom_meshtastic_hw_model

Per @thebentern's review: the nRF54L15-DK is a development kit, not a
canonical Meshtastic SKU, so it falls under HardwareModel::PRIVATE_HW
(255) — the same enum value already used at runtime via HW_VENDOR. The
placeholder 132 is removed; no dedicated enum number will be assigned
for DK boards. Slug stays NRF54L15_DK as a human-readable identifier.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(nrf54l15): unstarve bt_long_wq so SC pairing completes

bt_pub_key_gen() runs the ECC P256 key generation on bt_long_wq.  At default
prio=10 (preemptible) and stack=1400 it gets starved by Meshtastic app
threads at boot — sc_public_key stays NULL for minutes, smp_public_key()
defers with SMP_FLAG_PKEY_SEND, and every SC pairing attempt stalls right
after the public-key exchange.  iOS shows "Connecting…" forever with no
PIN prompt; bleak/CLI fails the first CCC notify write with
"Protocol Error 0x05: Insufficient Authentication".

Set CONFIG_BT_LONG_WQ_PRIO=0 (highest preemptible, ties with main) and
CONFIG_BT_LONG_WQ_STACK_SIZE=4096 (margin for the P256M driver frames).

Validated E2E with iOS Meshtastic app: bt_smp_pkey_ready fires within ~40 s
of boot, 20 SC Passkey Entry rounds complete with matching pcnf/cfm,
encrypt 0x01 / sec_level 0x04 (Authenticated MITM), bonded=1.

* feat(nrf54l15): hardware I2C bus via TWIM30 + sensor telemetry

Adds the Arduino TwoWire layer for the nRF54L15-DK so Meshtastic's
sensor drivers can talk to external I2C devices over the hardware
TWIM30 peripheral.

Bus binding:
- &uart30 disabled in the board overlay (peripheral instance 30 is
  shared between UARTE30 / TWIM30 / SPIM30 — pick one). Console stays
  on RTT via CONFIG_RTT_CONSOLE.
- New i2c30_default / i2c30_sleep pinctrl with SDA=P0.03 / SCL=P0.04.
  External 4.7 kOhm pull-ups required on both lines.
- &i2c30 enabled at I2C_BITRATE_FAST (400 kHz).
- button_3 (SW3, P0.04) deleted from DTS so the pad can be claimed by
  i2c30 pinctrl; SW3 is still wired to the pad on the DK, do not press
  it during I2C use or it will short SCL to GND.

Arduino layer:
- src/platform/nrf54l15/Wire.cpp resolves the DT node at compile time
  via DEVICE_DT_GET(DT_NODELABEL(i2c30)) and dispatches Arduino's
  beginTransmission / write / endTransmission / requestFrom to
  i2c_write / i2c_write_read / i2c_read. Buffer is sized to 256 bytes
  for forward compatibility with the SE050 secure element on the
  custom PCB.
- Wire.h drops the prior compile-only stubs and exposes the real
  TwoWire surface.
- Arduino.h: BitOrder becomes an enum (not #define) so Adafruit_BusIO's
  `typedef BitOrder BusIOBitOrder;` compiles.

Variant + build flags:
- nrf54l15.ini flips HAS_WIRE / HAS_SENSOR / HAS_TELEMETRY from 0 to 1
  and cherry-picks the sensor libs Meshtastic needs (BusIO, Sensor,
  BMP280, BME280, INA219/226/260/3221, SHT4X). The full
  environmental_base group is avoided because it pulls
  Adafruit_SSD1306 / Adafruit_GFX which rely on Arduino pin macros the
  Zephyr shim does not implement.
- nrf54l15dk variant.h defines PIN_WIRE_SDA / PIN_WIRE_SCL for parity
  with the Arduino convention used by other variants. The actual bus
  wiring is fixed by the overlay pinctrl above.

Validated 2026-05-14/15 on the DK with BMP280 @ 0x76 (temperature +
barometric pressure) and INA3221 @ 0x42 (rail voltage / current);
EnvironmentTelemetry / PowerTelemetry packets transmit successfully
over LoRa.

Footprint cost on nrf54l15dk: +45 KB flash, +1.7 KB RAM.

* feat(nodedb): honor USERPREFS for environment telemetry on first boot

installDefaultConfig() now respects two new compile-time prefs:

  USERPREFS_CONFIG_ENV_TELEM_UPDATE_INTERVAL
  USERPREFS_CONFIG_ENVIRONMENT_MEASUREMENT_ENABLED

The mobile apps enforce a 30 min floor on environment_update_interval
in the settings UI, which makes short-interval bring-up testing of new
sensor hardware painful — you have to wait half an hour for the first
LoRa packet to confirm wiring + driver. With these prefs baked into
the variant, the firmware can ship a freshly-flashed device that
broadcasts on a shorter cadence (e.g. 900 s) the moment storage_partition
is empty.

Both prefs are gated on #ifdef so the behavior is unchanged for any
variant that does not opt in. Documented in userPrefs.jsonc with the
existing telemetry-interval pref block.

* fix(nrf54l15): allow multiple bonded BLE peers

CONFIG_BT_MAX_PAIRED defaults to 1, so once the first peer (e.g. an
iOS phone) has paired and bonded, every subsequent pairing attempt
from a different MAC fails inside bt_keys_get_addr() with no free
key slot — the host returns BT_SECURITY_ERR_KEY_DOES_NOT_EXIST and
the second peer never gets past SMP.

Raise the slot count to 4 so the device can simultaneously hold an
iOS phone, a Windows host, a Linux host, and one spare bond. Add
BT_KEYS_OVERWRITE_OLDEST so that once the table fills, the LRU peer
is evicted on the next pairing rather than rejecting the new peer.
This matches the behavior other Meshtastic ports already provide
(nRF52 uses CONFIG_BT_PERIPHERAL_PRIO_CONN with similar semantics).

Discovered while bringing up the Python CLI on Windows alongside
the existing iOS bond.

* fix(nrf54l15): zero-initialize TwoWire buffers + clang-format Wire

cppcheck on every CI target (esp32s3, rp2040, rp2350, nrf52840, ...) was
failing the build with two `uninitMemberVar` warnings on TwoWire's
constructor: `txBuf` and `rxBuf` (256-byte arrays) were not initialized.
Even though the buffers are only read after txLen/rxLen is set, leaving
them uninitialized is a footgun if any future caller bypasses the
len-set step. Use C++11 value-initialization in the member initializer
list — costs ~512 B of memset at boot, gains a clean cppcheck pass and
defensive-against-future-changes semantics.

Also reformat Wire.{cpp,h} with the project's `.trunk/configs/.clang-format`
config so the Trunk Check Runner passes — clang-format moved the
`<errno.h>` include before the Zephyr-namespaced ones in Wire.cpp and
collapsed two inline overloads to single lines in Wire.h.

* fix(AdminModule): remove dead OUTPUT_GPIO_PIN/GpioOutputModule references

OUTPUT_GPIO_PIN is never defined and modules/GpioOutputModule.h doesn't
exist in the codebase; all #ifdef OUTPUT_GPIO_PIN branches were dead code
introduced by the nRF54L15-DK variant commit. Strips the include, the
output_gpio_enabled OFF→ON/ON→OFF transition logic in handleSetConfig(),
and the digitalRead() reflection in handleGetConfig().

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-16 06:16:11 -05:00
502c5af524 Upgrade trunk (#10481)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-05-15 20:03:50 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
05707079bd Update libch341-spi-userspace digest to 2e5ff75 (#10485)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-05-15 19:30:04 -05:00
Ben Meadors 2a91d186eb Add max_session_seconds to LockdownAuth for session management 2026-05-15 15:14:29 -05:00
Ben MeadorsandGitHub 1c05633fcd Add more support for small fonts in screen resolution determination (#10480) 2026-05-15 05:59:15 -05:00
4827498188 Clamp direct position packets to channel precision (fixes #8640) (#10383)
* Fix position precision for direct sends

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Clarify zero position precision logging

* Use const channel reference for position precision

* Use C linkage for position precision test entrypoints

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-14 20:51:44 -05:00
fce419b335 Upgrade trunk (#10476)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-05-14 06:43:06 -05:00
Ben MeadorsandGitHub 3a0c08b695 Merge pull request #10474 from meshtastic/2.8
Develop is 2.8 WIP now
2026-05-13 19:36:15 -05:00
Ben Meadors 7bdff8ff70 Bump protos 2026-05-13 14:33:15 -05:00
Ben Meadors 35b0590408 Develop is 2.8 WIP now 2026-05-13 13:50:17 -05:00
Ben MeadorsandGitHub cbddf07bc8 Merge pull request #10472 from meshtastic/remove-arial 2026-05-13 12:01:02 -05:00
Jonathan BennettGitHubBen MeadorsHarukiToredaJason Pcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>thebentern
871194517d Ble banner (#8902)
* Drop unneeded Sizeof() instances

* Use SimpleBanner for BLE pin

* Support for different font sizes on notification banner

* Fix NRF52 BLE cppcheck shadow warning

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/de12b52c-49d5-452a-b3fb-344724649270

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: HarukiToreda <116696711+HarukiToreda@users.noreply.github.com>
Co-authored-by: Jason P <applewiz@mac.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
2026-05-13 11:02:19 -05:00
Ben Meadors 5a1d2b9ef4 Refine nRF52 flash optimization comment for FONT_LARGE_LOCAL definition 2026-05-13 10:52:05 -05:00
Ben Meadors 9cd3a86938 Cleanup 2026-05-13 10:43:16 -05:00
Ben Meadors 748668b8e9 Remove ARIAL24 on NRF52 2026-05-13 10:13:53 -05:00
Ben Meadors 778d1ad90f Merge remote-tracking branch 'origin/master' into develop 2026-05-13 09:40:16 -05:00
Ben Meadors 1ae4a538f5 Trunk 2026-05-13 09:27:05 -05:00
Andros FenollosaandBen Meadors c756bbe2c1 Fix WiFi TCP/HTTP services not starting without USB serial connected (#10460)
Move WiFi.onEvent(WiFiEvent) registration before createSSLCert() to
prevent a race where the ESP32 auto-reconnects during cert generation
and fires GOT_IP before the handler is attached, causing
onNetworkConnected() to never run and the TCP/HTTP API services to
never initialize when booting without USB serial.

Also call onNetworkConnected() from reconnectWiFi() on all platforms
(not just RP2040) as a safety net; it is already guarded by
APStartupComplete so it only runs once.
2026-05-13 09:26:44 -05:00
AustinandGitHub 4c3ba612bb VSCode: Prepare for pioarduino transition (#10471)
Start reccomending the pioarduino VS Code extension instead of the PlatformIO extension.

pioarduino-based builds cannot complete correctly using the platformio extension. Normal platformio builds (nrf52, stm32) are unaffected//still work correctly.

Devs may need to delete their ~.platformio and .pio directories once after install in order to build properly.
2026-05-13 09:25:11 -05:00
Ben Meadors 75b7a7df4f Missed one 2026-05-13 08:50:15 -05:00
59025e4820 Add initial support for Station G3 variant (#10457)
* Add initial support for Station G3 variant

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-13 08:07:24 -05:00
Ben Meadors 8110887be2 Turns out it's already excluded 2026-05-13 07:57:54 -05:00
Ben Meadors 5f734dabf9 Trunk 2026-05-13 06:44:51 -05:00
Andros FenollosaandGitHub 039ad42758 Fix WiFi TCP/HTTP services not starting without USB serial connected (#10460)
Move WiFi.onEvent(WiFiEvent) registration before createSSLCert() to
prevent a race where the ESP32 auto-reconnects during cert generation
and fires GOT_IP before the handler is attached, causing
onNetworkConnected() to never run and the TCP/HTTP API services to
never initialize when booting without USB serial.

Also call onNetworkConnected() from reconnectWiFi() on all platforms
(not just RP2040) as a safety net; it is already guarded by
APStartupComplete so it only runs once.
2026-05-13 06:43:36 -05:00
Ben Meadors 593909c26b Radiolib excludes 2026-05-13 06:42:49 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
0a7b3c723e Update NeoPixel to v1.15.5 (#10466)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-05-13 10:57:48 +02:00
Ben Meadors 29a61dc75c Fix type declaration for ambientLightingThread and correct uint32_t usage in PacketHistory 2026-05-12 21:45:16 -05:00
Ben MeadorsandGitHub eead467ce6 Added NodeDB fixtures and refactored to use std maps for better memory efficiency (#10464)
* Added NodeDB fixtures and refactored to use std maps for better efficiency

* Defer NodeDB save during xmodem transfer to prevent mid-transfer fsFormat
2026-05-12 17:23:29 -05:00
Ben Meadors f3ae02c425 Cleanup comments 2026-05-12 16:32:00 -05:00
Ben Meadors d9cb74e4dd XModemAdapter: ensure file truncation before receiving and add isBusy() method to prevent concurrent writes 2026-05-12 15:38:56 -05:00
7ff6641f97 Fix missing potential null termination in xmodem filename handling (#10308)
* Fix missing potential null termination in xmodem filename handling

The packet size max is 128 bytes, and the filename is 128 bytes, so
potentially there is no NUL at the end. use strlcpy() as that takes
care of null termination even if buffer size is exceeded.

* Protect against theoretical buffer overflows in BLE logging

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-12 06:26:13 -05:00
cd5d608e8d Upgrade trunk (#10461)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-05-12 06:07:09 -05:00
Ben MeadorsGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
f7548e7c25 Remove gradient sync nonce and simplify replay handling (#10459)
* Remove gradient sync nonce and simplify replay handling

* Fix ONLY_CONFIG replay gating and stale gradient-sync comments

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/cfa93978-e2e0-4dc2-ba5f-b82b5b43cef8

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Add transport mechanism to replay packets for client filtering

* Comments

* Update protobuf definitions to include precision_bits in PositionLite

* Propagate position precision_bits and remove verbose NodeInfo sync log

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/41572cbc-408e-499d-b59e-00f330b5789f

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-05-11 21:42:07 -05:00
b960121464 BaseUI: remove legacy single-message runtime path and keep multimessage flow (#10450)
* cleanup

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-11 20:53:41 -05:00
Ben Meadors b074645586 Change node pointer to const in JsonSerialize function 2026-05-11 17:16:32 -05:00
Ben Meadors 811dd427dd Update protobufs 2026-05-11 16:36:35 -05:00
Ben MeadorsandGitHub 7f5184281d Make power status logging less chatty and track battery presence transitions (#10453) 2026-05-11 16:09:33 -05:00
Jonathan Bennett 1c06b702dc Merge remote-tracking branch 'origin/master' into develop 2026-05-11 15:07:22 -05:00
59b4993861 Update protobufs (#10456)
Co-authored-by: jp-bennett <5630967+jp-bennett@users.noreply.github.com>
2026-05-11 15:06:37 -05:00
Jonathan BennettandGitHub 4446b0f1a2 Add variantDefaultConfig and set eth_enabled to default true (#10454) 2026-05-11 14:51:21 -05:00
Thomas GöttgensJonathan BennettBen Meadorscopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>caveman99
64fd61706d ThinkNode M7 (#8077)
* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* rename variant and add guard macros

* older G3 operational. M7 next.

* Split out G3 and M7 to different variants. Completely new PCB design. The G3 stays on 'PRIVATE_HW'

* Define button behaviour and use all of the device flash

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: caveman99 <25002+caveman99@users.noreply.github.com>
Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
2026-05-11 11:46:13 -05:00
Jonathan BennettandGitHub da61a0db7d Refactor mutex handling in PhoneAPI.cpp
Replace LockGuard with explicit lock and unlock calls to avoid deadlock
2026-05-11 11:45:14 -05:00
Thomas GöttgensGitHubBen Meadorscopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>caveman99Jonathan Bennett
8e99ffbe7e ThinkNode M7 (#8077)
* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* ThinkNode G3, ETH support WIP

* rename variant and add guard macros

* older G3 operational. M7 next.

* Split out G3 and M7 to different variants. Completely new PCB design. The G3 stays on 'PRIVATE_HW'

* Define button behaviour and use all of the device flash

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: caveman99 <25002+caveman99@users.noreply.github.com>
Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
2026-05-11 11:33:13 -05:00
Ben Meadors 8877608858 Protos 2026-05-11 09:32:55 -05:00
Ben Meadors a23f923e64 Update subproject commit reference in protobufs 2026-05-11 08:09:39 -05:00
Ben Meadors 0522039830 Merge branch 'master' into develop 2026-05-11 08:09:10 -05:00
Ben Meadors dfcb685963 Update protos 2026-05-11 08:08:15 -05:00
Ben Meadors 9bc25b34fd Add guidance to use Throttle for time-based rate limiting in agent instructions 2026-05-11 07:42:04 -05:00
33319aa4e2 Upgrade trunk (#10451)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-05-11 07:30:03 -05:00
Ben Meadors d79e62fd2a Chatty LLMs should pipe down 2026-05-10 10:20:10 -05:00
Ben MeadorsGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>Claude Opus 4.7
f6a954b97e Implement rotating JSONL recorder for persistent logging (#10428)
* Implement rotating JSONL recorder for persistent logging

* Fixes

* Update documentation and clean up imports in command files

* Address remaining recorder review feedback

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/2541773c-869a-463f-9fae-8505272c06ff

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* recorder: fix lock re-entry deadlock on start() and force_rotate_all()

The previous "Fixes" commit added `_files_snapshot()` which acquires
`self._lock` so handlers don't race with `stop()` clearing `_files`.
But two callers were already holding `self._lock` when they invoked
methods that go through the snapshot:

  - `start()` writes the `recorder_start` event from inside its `with
    self._lock:` block. `_write_event` -> `_files_snapshot` re-acquires
    the same non-reentrant `threading.Lock`, freezing process startup.

  - `force_rotate_all()` calls `self.status()` (which also acquires
    `self._lock`) while still holding the lock from rotating each file.

Both fixes release the lock before the call. The recorder_start marker
still lands in events.jsonl because the started/started_at flags are
already set when we write it.

Verified end-to-end against the standalone /tmp/verify_pr_fixes.py
harness — all 9 PR review-comment fixes pass, including pause/resume
event ordering and concurrent start/stop without KeyError.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Fix markdown linting issues in leakhunt.md and repro.md

* Handle recorder startup and query review fixes

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/78540a9f-fe62-4350-b252-0ae5621f0b8a

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Tighten recorder follow-up tests

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/78540a9f-fe62-4350-b252-0ae5621f0b8a

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Stabilize recorder startup tests

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/78540a9f-fe62-4350-b252-0ae5621f0b8a

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Remove brittle recorder startup test

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/78540a9f-fe62-4350-b252-0ae5621f0b8a

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Polish recorder follow-up errors

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/78540a9f-fe62-4350-b252-0ae5621f0b8a

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Refine recorder startup and regex errors

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/78540a9f-fe62-4350-b252-0ae5621f0b8a

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Clean up recorder follow-up nits

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/78540a9f-fe62-4350-b252-0ae5621f0b8a

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Trunk

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 09:22:40 -05:00
Ben MeadorsGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>Copilot Autofix powered by AI
94bb21ecc7 2.8: NodeDB shrink, decoupling, and restructuring (#10413)
* 2.8: NodeDB refactor to decouple satellite entries and decrease size

* Regen

* Refactor node mute handling to use dedicated functions for clarity and consistency

* Develop ref

* Fix NodeDB review follow-ups

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/6b1d6cf6-ed6b-43b6-95cb-8e141757664e

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Address review validation nits

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/6b1d6cf6-ed6b-43b6-95cb-8e141757664e

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Trunk

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Extract legacy NodeDatabase migration

* Fix remaining NodeDB review issues

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/c76b9a5a-7244-4fbc-9ef0-98091d8caaea

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Fixes

* Trunk

* Fix latest review compile follow-ups

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/5198da01-ec4c-4c16-8a09-68b8e6d5d410

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Fix cppcheck style warnings

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/e60287ba-4ece-46e0-83d8-a6d89664c0bb

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Change pointer type for mesh node in set_favorite function

* Change pointer types for mesh node references to const in multiple applets

* Add NodeDB layout v25 documentation and migration guidelines

* Remove tests for uninitialized PacketHistory state due to undefined behavior

* Fix code block formatting in copilot instructions

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-09 15:12:10 -05:00
1bcabb893b mesh: bound the user-facing notification sprintf calls (#10437)
Two sites built ClientNotification messages with sprintf into a
fixed-size proto buffer with no length cap. The current format strings
fit comfortably, but a future caller editing either format string
without rechecking the buffer size would get a silent stack/heap
overrun. Switch to snprintf with sizeof so the bound is enforced at
the call site.

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-09 13:32:42 -05:00
BJKandBen Meadors 10a7f1042b Fix screen geometry update for SH1107 display (#10444)
Added conditional block to update screen geometry for SH1107 128x128.
2026-05-09 13:20:50 -05:00
BJKandGitHub fefd424901 Fix screen geometry update for SH1107 display (#10444)
Added conditional block to update screen geometry for SH1107 128x128.
2026-05-09 12:53:07 -05:00
b4234b7f11 Automated version bumps (#10419)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
2026-05-08 19:05:55 -05:00
f63716c322 Automated version bumps (#10419)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
2026-05-08 19:05:11 -05:00
Jonathan BennettandGitHub 5512185cfe Make heartbeat LED play nice with other LEDs (#10423) 2026-05-08 16:03:39 -05:00
a8a785bbb7 Upgrade trunk (#10418)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-05-08 05:43:02 -05:00
5e2ca8aed4 LR2021 radio on NRF_Promicro (#10401)
* LR2021 radio on NRF_Promicro

Co-authored-by: Copilot <copilot@github.com>

* Refactor LR2021 interface includes and conditional compilation for improved clarity

Co-authored-by: Copilot <copilot@github.com>

* Refactor LR20x0 interface: remove unused includes and update comments for clarity

* Fix LR2021 max power definitions and add radio type detection tests

* remove potato radio type detection tests

* Include placeholder for DCDC - currently requires godmode

* Added godmode features - not enabled by default

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-07 20:24:39 -05:00
Jonathan BennettandGitHub 0f854862e7 Give ThinkNode-m4 a heartbeat (#10408) 2026-05-07 13:17:29 -05:00
4553d1e0b1 Skip MQTT allocation when disabled (#10411)
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-07 11:51:55 -05:00
Ben Meadors 784e3748e2 Merge remote-tracking branch 'origin/master' into develop 2026-05-07 11:45:04 -05:00
TomandGitHub b11d29ff31 fix(mesh): update reconfigure methods to return true instead of RADIOLIB_ERR_NONE (#10407) 2026-05-07 05:57:34 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
b246bcd72e Update libpax digest to df42474 (#10406)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-05-06 20:52:30 -05:00
33e2bb70e6 Enhance GPS search failure handling backoff logic (#10404)
* Enhance GPS search failure handling backoff logic

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Remove stray submodule gitlink for .claude worktree

A 160000 (gitlink) entry for .claude/worktrees/naughty-payne-60fdb7
pointing at f2923590bc was accidentally committed in 9db15780f. The
path isn't a real submodule — it's a Claude Code agent worktree that
shouldn't be tracked.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-06 20:44:04 -05:00
Ben MeadorsandGitHub 220bb4d186 Smart pointers and memory management cleanup (#10400)
* Refactor memory management in Syslog and StoreForwardModule

* Implement destructor for Lock

* Refactor RotaryEncoder and PacketHistory to use smart pointers for better memory management

* CH341 should use unique_ptr for improved memory management

* Fix checks in PH

* Improve Syslog::vlogf to handle variable argument lists more safely

* Fix initOk method to use nullptr for null pointer check
2026-05-06 15:33:59 -05:00
6e810741f3 Fix GPS initialization logic for Portduino configuration (#10395)
Co-authored-by: jessm33 <root@example.com>
2026-05-05 17:28:22 -05:00
cdc47a2aea Fix GPS initialization logic for Portduino configuration (#10395)
Co-authored-by: jessm33 <root@example.com>
2026-05-05 14:17:04 -05:00
Ben MeadorsandGitHub 603cce2988 Add informSearchFailed method to update GPS power state handling (#10394) 2026-05-05 10:12:50 -05:00
fcef46f4b0 dependency swap - INA3221Sensor (#10379)
* dependency swap - INA3221Sensor
update INA3221 initialization and measurement methods for compatibility with Rob Tillaart's library

Co-authored-by: Copilot <copilot@github.com>

* Addresses copilot review

Co-authored-by: Copilot <copilot@github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Refine comments on USB detection and INA3221

Updated comments regarding USB detection and INA3221 usage.

* Fix static_assert conditions for INA3221 channel definitions

Co-authored-by: Copilot <copilot@github.com>

* moved macro defines earlier to allow better use.

Co-authored-by: Copilot <copilot@github.com>

* Add raw register read methods for bus voltage and shunt current in INA3221Sensor

Co-authored-by: Copilot <copilot@github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-05 06:52:07 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
d559af8477 Update LovyanGFX to v1.2.21 (#10373)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-05-05 12:29:04 +02:00
Jonathan Bennett bb86cf4e81 Merge remote-tracking branch 'origin/master' into develop 2026-05-04 18:10:59 -05:00
Jonathan BennettandGitHub 6ea0d5ebba Add TFT_BACKLIGHT_ON for cardputer to fix builds (#10387) 2026-05-04 16:19:48 -05:00
JordandGitHub b2bda3b07a Enable MESHTASTIC_PREHOP_DROP by default (#9924) 2026-05-02 16:38:06 -05:00
Ben MeadorsandGitHub 41f53177a1 Use OBS instead of launchpad (#10375) 2026-05-02 09:25:24 -04:00
2d761f6453 Upgrade trunk (#10364)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-05-02 07:23:49 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
7cb071c780 Update platform-native digest to cab4b21 (#10372)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-05-01 18:25:08 -04:00
Jonathan BennettandGitHub b7a9555905 Update RadioLib dependency to a specific commit (#10370)
Exploratory PR to test new radiolib change
2026-05-01 12:50:07 -05:00
Austin Lane 0240a00d09 MacOS: Re-Add Orcania/Yder 2026-05-01 10:55:32 -04:00
AustinandGitHub c0fcf807c0 MacOS: Correct pkg-config name openssl for ulfius. (#10369) 2026-05-01 10:42:17 -04:00
Ben MeadorsGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>Copilot Autofix powered by AI
55f40ecdfd Add ulfius webserver support to macos native target (#10366)
* Add ulfius webserver support to macos native target

* fix: update PiWebServer docs for macOS and add explicit cstring include

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/3ce82582-23e0-4afe-b22f-b24f81721488

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix: add --cflags to openssl@3 pkg-config and fix apt package name

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/1a6c59aa-4393-4134-8cee-61eeee0e9127

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-01 08:56:49 -05:00
Jason PandGitHub 90744ee0b7 Update PhoneAPI.cpp to reduce chattiness (#10367) 2026-05-01 08:46:53 -05:00
1eb860a3fc fix(stm32wl,nrf52,fs): flash hardening, FS platform unification, write-behind LFS cache (FORMAT BREAK) (#10171)
* stm32wl: check HAL_FLASH_Unlock() return in _internal_flash_erase

_internal_flash_prog already checks HAL_FLASH_Unlock() and returns
LFS_ERR_IO on failure. _internal_flash_erase discarded the return
value, proceeding to erase even if the flash was not unlocked.

Apply the same check for consistency and safety.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl: fix _internal_flash_prog to abort on first write error

Previously the programming loop continued to the next doubleword after
HAL_FLASH_Program() failed, potentially writing to invalid addresses
and returning a misleading error code only at the end (last iteration).
HAL_FLASH_Lock() was also skipped on the mid-loop early return path.

- Move bounds check before the loop (validate full range at once)
- Break on first HAL error so subsequent doublewords are not written
- Move HAL_FLASH_Lock() after the loop so it always runs

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl: clear stale flash SR error flags before erase and program

Stale error flags in FLASH->SR from a previous failed operation can
cause HAL_FLASH_Program() or HAL_FLASHEx_Erase() to return HAL_ERROR
immediately without attempting the operation.

Add __HAL_FLASH_CLEAR_FLAG(FLASH_FLAG_ALL_ERRORS) after each
HAL_FLASH_Unlock() in both _internal_flash_prog and
_internal_flash_erase to ensure a clean state before each operation.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl: reject flash prog writes not aligned to 8-byte doubleword

The STM32WL HAL minimum write unit is one 64-bit doubleword (8 bytes).
_internal_flash_prog silently truncated any trailing bytes when size % 8
!= 0 because dw_count = size / 8 drops the remainder. Return LFS_ERR_INVAL
early so LittleFS sees the error rather than a silent short write.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(nrf52,fs): use atomic SafeFile rename instead of direct write

NRF52 was bypassing the .tmp/readback/rename path entirely — openFile()
deleted the target file and wrote directly to it, and close() returned
true without verifying the write or renaming anything.

Adafruit_LittleFS::rename() calls lfs_rename() directly (confirmed at
Adafruit_LittleFS.cpp:205). Remove both ARCH_NRF52 guards so NRF52
follows the same write-to-.tmp → readback-hash → rename path used by
all other platforms.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(admin): skip uiconfig.proto save on devices without a screen

handleStoreDeviceUIConfig() was writing /prefs/uiconfig.proto
unconditionally. MenuHandler.cpp is already gated behind #if HAS_SCREEN,
so there is no path that populates UI config on screen-less platforms.
Guard the save with #if HAS_SCREEN to avoid wasting a flash block on
devices that will never use it.

The read path (handleGetDeviceUIConfig) does not touch the filesystem
and needs no change.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* fs: enable format-on-retry for all platforms in saveToDisk

The FSCom.format() call on save failure was guarded to ARCH_NRF52 with
a comment that other platforms were not ready (bug #4184). STM32WL was
added to the guard in a prior commit. All platforms now expose format
semantics and the retry logic is identical — remove the guard.

To keep NodeDB.cpp platform-agnostic and fix a CI failure on native-tft
(portduino's fs::FS has no format() method), introduce fsFormat() in
FSCommon as the single call-site for all callers:

  - Embedded (ESP32, NRF52, STM32WL, RP2040): delegates to FSCom.format()
  - Portduino: rmDir("/prefs") + FSBegin() (a no-op on portduino).
    rmDir("/prefs") is already called unconditionally by factoryReset()
    (NodeDB.cpp:504), so both primitives are proven on portduino.

Replace both direct FSCom.format() calls in NodeDB.cpp with fsFormat().

Note: we do not run portduino locally — portduino/native build testers
please verify the format-on-retry path.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* DO NOT MERGE: nrf52(fs): add File() default constructor bound to InternalFS

Adds File() to the Adafruit LittleFS File class (in the Meshtastic
Adafruit_nRF52_Arduino fork), delegating to File(InternalFS). This
matches the default-constructible File API on all other platforms.

The constructor is implemented in Adafruit_LittleFS_File.cpp rather
than inline in the header to avoid a circular include between
Adafruit_LittleFS_File.h and InternalFileSystem.h.

FOLLOW-UP REQUIRED: nrf52.ini points to a commit SHA on the
mesh-malaysia/Adafruit_nRF52_Arduino fork instead of the upstream
meshtastic framework. Once meshtastic/Adafruit_nRF52_Arduino#5 is
merged, revert nrf52.ini to point back to the upstream meshtastic
framework URL.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl(fs): add File() default constructor and document LFS tunables

Adds File() to STM32_LittleFS_Namespace::File, delegating to
File(InternalFS). Implemented in the .cpp to avoid a circular include
between STM32_LittleFS_File.h (which cannot include LittleFS.h) and
the InternalFS extern declaration.

This matches the File API on ESP32/RP2040/Portduino and is a
prerequisite for removing the ARCH_STM32WL guard in xmodem.h.

No behavior change — the constructor leaves the file in the same
closed/unattached state as File(InternalFS) would.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* fs: remove arch-specific ifdefs from FSCommon, SafeFile, xmodem

Now that NRF52 and STM32WL have File() default constructors and NRF52
has working atomic SafeFile rename, the capability gaps are closed.
Remove all per-arch guards across the shared FS layer:

FSCommon.cpp — renameFile():
  Use FSCom.rename() on all platforms. Adafruit_LittleFS::rename()
  calls lfs_rename() directly (Adafruit_LittleFS.cpp:205). The
  copy+delete fallback on NRF52/RP2040 was never necessary.

FSCommon.cpp — getFiles():
  Replace four ARCH_ESP32 guards with a single filepath pointer at
  the top of the loop (file.path() on ESP32, file.name() elsewhere).
  Fix strcpy(fileInfo.file_name, filepath): bounded to
  sizeof(fileInfo.file_name)-1 with explicit NUL termination to prevent
  overflow of the 228-byte meshtastic_FileInfo::file_name array.

FSCommon.cpp — listDir():
  Same filepath pointer approach. NRF52/STM32WL were in an else-branch
  that only logged but never deleted — now all platforms follow the
  unified del path. 12 guards → 2.
  Fix three strncpy(buffer, ..., sizeof(buffer)) calls that did not
  NUL-terminate when source length >= sizeof(buffer) (255 bytes).
  Add explicit buffer[sizeof(buffer)-1] = '\0' after each.

FSCommon.cpp — rmDir():
  Use listDir(del=true) everywhere. The ARCH_NRF52 rmdir_r() path and
  the ARCH_ESP32|RP2040|PORTDUINO listDir() path collapse to one line.

SafeFile.cpp:
  ARCH_NRF52 bypass removed (handled in preceding commit).

xmodem.h:
  File file; now works on all platforms via default constructors
  added in the two preceding commits.

Remaining #ifdef ARCH_ESP32 in FSCommon.cpp: exactly 4, all for the
file.path() vs file.name() API difference (ESP32 Arduino LittleFS
returns the full path; all others return only the name). That
difference lives in the framework and cannot be closed here.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl(fs): add write-behind page cache, reduce virtual block size and FS reservation (FORMAT BREAK)

Adds a write-behind (RMW) page cache to the STM32WL LittleFS driver,
modelled after the NRF52 Adafruit approach (flash_cache.c). This allows
LFS to use 256-byte virtual blocks backed by 2048-byte physical pages:
the erase/prog callbacks accumulate changes in a 2 KB RAM buffer; the
sync callback (and page eviction on page-change) flushes with a single
HAL physical-erase + doubleword-program pass.

LFS tunables changed (FORMAT BREAK — superblock parameters):
  block_size:  2048 B → 256 B  (8 virtual blocks per physical page)
  read_size:   2048 B → 256 B  (= block_size)
  prog_size:   2048 B → 256 B  (= block_size; hardware min is 8 B)
  block_count: 112   → 80     (14 phys pages → 10 phys pages = 20 KiB)

Benefits:
  - Internal fragmentation: max 2047 B/file → max 255 B/file
  - Heap per open LFS file: ~4 KB → 512 B (prog + read buffers)
  - Code flash headroom: 6.7 KB → ~14.1 KB (+7.4 KB)
  - Block budget: 80 virtual blocks, worst-case peak ~20, ~60 free

Updates board_upload.maximum_size in wio-e5/platformio.ini from 233472
(256 KB − 28 KB) to 241664 (256 KB − 20 KB) to match the reduced FS
reservation.

Justification for the format break: the prior STM32WL firmware had
several flash write bugs fixed earlier in this series (missing error
flag clearing, no abort on first write failure, unaligned write
acceptance). These bugs very likely caused silent config corruption on
deployed devices. The format break should be treated as an enhancement:
it provides a clean, reliably-written starting point. Users will need
to reconfigure their device once after this update.

Correctness fixes applied to the cache implementation:
  - alignas(8) on _page_cache: the buffer was uint8_t[] (alignment 1)
    but _flash_cache_flush casts it to const uint64_t* — undefined
    behaviour per C++ standard, potential Cortex-M hardfault. alignas(8)
    guarantees the required alignment for the doubleword cast.
  - HAL_FLASH_Lock() return value: was discarded. Now assigned to
    lock_rc and propagated into rc if prior writes succeeded, so LFS
    sees the error rather than a false success.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

* stm32wl(fs): reduce FS reservation from 10 pages to 7 pages (FORMAT BREAK)

Reduces LFS_FLASH_TOTAL_SIZE from 10 × 2 KiB pages (20 KiB) to
7 × 2 KiB pages (14 KiB), freeing 6 KiB for firmware.

board_upload.maximum_size updated accordingly across all STM32WL variants:
  241664 (256 KiB - 20 KiB) → 247808 (256 KiB - 14 KiB)

This is a FORMAT BREAK: existing filesystems must be erased before use.

Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Andrew Yong <me@ndoo.sg>

* fix(fs): return false in renameFile() when FSCom is not defined

Avoids undefined behavior and -Wreturn-type warnings in configurations
that compile FSCommon.cpp without a filesystem backend.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Signed-off-by: Andrew Yong <me@ndoo.sg>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-05-01 08:25:19 -05:00
Jonathan Bennett 8ccb2c918f dmshell reliability tweaks 2026-04-30 21:28:28 -05:00
4ee9598107 Docker: Install grpcio-tools from distro (#10358)
Use distro provided Python at build time (instead of the `python` images from dockerhub) and install `grpcio-tools` using the distro provided packages.

This should speed up build times, ESPECIALLY on riscv64 (where prebuilt `grpcio-tools` wheels are not provided on pip).

Co-authored-by: Copilot <copilot@github.com>
2026-04-30 15:22:11 -05:00
Ben MeadorsandGitHub 7066abbb86 Fix MAC_from_string to use input parameter instead of global config for MAC address parsing (#10356)
* Fix MAC_from_string to use input parameter instead of global config for MAC address parsing

* Enhance MAC_from_string validation and error handling

* Add missing include for <cctype> in PortduinoGlue.cpp
2026-04-30 13:52:42 -05:00
Ben MeadorsandGitHub 21cef8c2e5 Add TCP support for Meshtastic MCP interface / tests and update docs (#10355)
* Add TCP support for Meshtastic MCP interface / tests and update docs

* Address TCP endpoint validation and error handling in connection

* TCP connection handling and device listing logic

* Fix docstring formatting in normalize_tcp_endpoint function
2026-04-30 13:51:29 -05:00
Jonathan BennettandGitHub 6b49b7228a Merge branch 'develop' into vibe-coded-dmshell 2026-04-30 10:56:15 -05:00
Jonathan Bennett 989b8620ba Merge remote-tracking branch 'origin/master' into develop 2026-04-30 10:49:26 -05:00
173ac58ed7 Update protobufs (#10357)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
2026-04-30 10:45:20 -05:00
83adfd417a Upgrade trunk (#10354)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-04-30 06:39:52 -05:00
a0951f23c3 fix: MQTT connection on Portduino/Linux native nodes (#10330)
isConnectedToNetwork() always returned false on ARCH_PORTDUINO
because none of HAS_WIFI, HAS_ETHERNET, or USE_WS5500 are defined
for Linux native builds. This caused wantsLink() to always return
false, preventing the MQTT thread from ever connecting at boot.

Fix: return true for ARCH_PORTDUINO since Linux always has network
access available.

Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
2026-04-30 06:00:50 -05:00
HarukiToredaandGitHub e19f531059 Update Screen.cpp (#10344) 2026-04-29 21:05:16 -05:00
24d64a0013 Docker: Build for riscv64 (#10345)
Upstream support has been added in Debian and Alpine.
Only build as part of `docker_manifest` (Beta/Alpha/Daily) releases, because these will take a **while** thanks to qemu.

Co-authored-by: Copilot <copilot@github.com>
2026-04-29 21:04:49 -05:00
Ben Meadors 3a87fc82c0 Add documentation for macOS support in Copilot and Agent instructions 2026-04-29 19:54:05 -05:00
AustinandGitHub 478444eb02 Docker-Alpine: Align version between build/main stages (#10347)
FROM python:3.14-alpine3.23 AS builder
FROM alpine:3.23

the alpine version needs to match in both stages 😅
2026-04-29 20:31:59 -04:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
ad23c42fcc Update meshtastic/device-ui digest to 4bf593a (#10346)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-29 19:09:21 -05:00
Ben Meadors 195f42af82 Doesn't FSCom 2026-04-29 16:57:21 -05:00
Ben Meadors 089af764ec Replace FSCom.format() with FSCom.rmDir() for directory cleanup in NodeDB::loadFromDisk() 2026-04-29 16:41:21 -05:00
AustinandGitHub 7be5426f34 Do not FACTORY_INSTALL on ARCH_PORTDUINO (#10343) 2026-04-29 13:00:01 -05:00
9ec63b5eb2 Upgrade trunk (#10336)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-04-29 12:55:48 -05:00
AustinandGitHub 22a9346fe0 Debian: Correctly fail upon failure (#10341)
Fake success is BS! We should fail when we fail.
Fixes issues with Debian sourcedebs silently failing to build ocassionally (due to github 502s, etc).
2026-04-29 11:16:25 -05:00
Jonathan BennettandGitHub c194666885 Merge branch 'develop' into vibe-coded-dmshell 2026-04-28 22:03:03 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
c0e52e6e1c Update meshtastic/device-ui digest to 1ddcc9d (#10328)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-28 19:10:17 -05:00
11df30a85f Upgrade trunk (#10324)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-04-28 19:04:16 -05:00
Ben MeadorsandGitHub 8d8ff21e7c Add clamping logic for milliseconds conversion and unit tests (#10326)
* Add clamping logic for milliseconds conversion and unit tests

* Simplify comments in secondsToMsClamped function

Removed detailed comments about seconds to milliseconds conversion.
2026-04-28 19:03:50 -05:00
9c72767c01 macOS: enable CH341 LoRa-hardware path (fix serial truncation, document setup) (#10320)
* macOS: enable CH341 LoRa-hardware path — fix serial truncation, document setup

Verified on Apple Silicon with a CH341A USB-SPI bridge (VID 0x1A86,
PID 0x5512) wired to an SX1262 (Meshstick variant) that the existing
`pine64/libch341-spi-userspace` lib_dep works on macOS as-is — Apple's
bundled CH34x driver only matches the CH340 *UART* variant
(PID 0x7523), so the CH341A's interface 0 is left unclaimed and
libusb opens / configures / claims it directly via IOUSBHostInterface.
End-to-end test: meshtasticd boots, libusb claim succeeds, SX1262 init
returns 0, TCP API serves the meshtastic CLI's --info / --sendtext flow.

Two changes:

1. **`PortduinoGlue.cpp:497`**: pass `sizeof(serial)` (= 9) instead of
   the literal `8` to `Ch341Hal::getSerialString()`. The function in
   `USBHal.h:61-68` treats `len` as buffer size and reserves one slot
   for the null terminator (`bytesCopied = (len - 1) < 8 ? (len - 1) : 8`),
   so passing 8 produced a 7-char serial — which then broke the
   `strlen(serial) == 8` check at line 502, skipping the auto-MAC
   derivation from serial + product string. On Linux this was masked
   by the BlueZ HCI MAC fallback in `getMacAddr()` at lines 139-157,
   but on macOS that fallback is `__linux__`-guarded so the serial path
   is mandatory and the truncation left `mac_address` empty, causing
   the daemon to exit with `*** Blank MAC Address not allowed!`.

2. **`variants/native/portduino/platformio.ini`**: expand the
   `[env:native-macos]` comment block with a "Real LoRa hardware on
   macOS" section. Documents:
   - Why no upstream library change is needed (Apple kext targets
     CH340/UART, not CH341A/SPI; libusb's `#ifdef __linux__` skip is
     correct for macOS in this case).
   - How to point `meshtasticd` at an existing platform-agnostic
     `bin/config.d/lora-*.yaml` for CH341 hardware.
   - The auto-MAC-derivation contract (now working with this fix).
   - `ioreg` and `LIBUSB_DEBUG=4` diagnostic recipes for the failure
     mode where a third-party WCH `CH34xVCPDriver` *would* claim
     interface 0 (`kmutil unload -b <bundleID>` workaround).

No upstream library forks, no PR chain, no additional lib_deps —
the existing `pine64/libch341-spi-userspace` + libusb-1.0 stack does
the right thing on macOS already.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-28 10:54:01 -05:00
6c7ffa1054 macOS: enable CH341 LoRa-hardware path (fix serial truncation, document setup) (#10320)
* macOS: enable CH341 LoRa-hardware path — fix serial truncation, document setup

Verified on Apple Silicon with a CH341A USB-SPI bridge (VID 0x1A86,
PID 0x5512) wired to an SX1262 (Meshstick variant) that the existing
`pine64/libch341-spi-userspace` lib_dep works on macOS as-is — Apple's
bundled CH34x driver only matches the CH340 *UART* variant
(PID 0x7523), so the CH341A's interface 0 is left unclaimed and
libusb opens / configures / claims it directly via IOUSBHostInterface.
End-to-end test: meshtasticd boots, libusb claim succeeds, SX1262 init
returns 0, TCP API serves the meshtastic CLI's --info / --sendtext flow.

Two changes:

1. **`PortduinoGlue.cpp:497`**: pass `sizeof(serial)` (= 9) instead of
   the literal `8` to `Ch341Hal::getSerialString()`. The function in
   `USBHal.h:61-68` treats `len` as buffer size and reserves one slot
   for the null terminator (`bytesCopied = (len - 1) < 8 ? (len - 1) : 8`),
   so passing 8 produced a 7-char serial — which then broke the
   `strlen(serial) == 8` check at line 502, skipping the auto-MAC
   derivation from serial + product string. On Linux this was masked
   by the BlueZ HCI MAC fallback in `getMacAddr()` at lines 139-157,
   but on macOS that fallback is `__linux__`-guarded so the serial path
   is mandatory and the truncation left `mac_address` empty, causing
   the daemon to exit with `*** Blank MAC Address not allowed!`.

2. **`variants/native/portduino/platformio.ini`**: expand the
   `[env:native-macos]` comment block with a "Real LoRa hardware on
   macOS" section. Documents:
   - Why no upstream library change is needed (Apple kext targets
     CH340/UART, not CH341A/SPI; libusb's `#ifdef __linux__` skip is
     correct for macOS in this case).
   - How to point `meshtasticd` at an existing platform-agnostic
     `bin/config.d/lora-*.yaml` for CH341 hardware.
   - The auto-MAC-derivation contract (now working with this fix).
   - `ioreg` and `LIBUSB_DEBUG=4` diagnostic recipes for the failure
     mode where a third-party WCH `CH34xVCPDriver` *would* claim
     interface 0 (`kmutil unload -b <bundleID>` workaround).

No upstream library forks, no PR chain, no additional lib_deps —
the existing `pine64/libch341-spi-userspace` + libusb-1.0 stack does
the right thing on macOS already.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-28 08:31:08 -05:00
c0425d7444 Actions: Build MacOS binary (#10319)
Preliminary CI for the MacOS builds

Co-authored-by: Copilot <copilot@github.com>
2026-04-27 13:33:19 -05:00
d7db0f5829 add heltec-v4-r8 board (#10268)
* add heltec-v4-r8 board

* Fixed default SPI pin and macro definition errors.

* update platformio.ini according device-ui LGFX display definitions

Co-authored-by: Copilot <copilot@github.com>

* fix commit reference

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: mverch67 <manuel.verch@gmx.de>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Manuel <71137295+mverch67@users.noreply.github.com>
2026-04-27 09:47:41 -05:00
f037ce2165 add heltec-v4-r8 board (#10268)
* add heltec-v4-r8 board

* Fixed default SPI pin and macro definition errors.

* update platformio.ini according device-ui LGFX display definitions

Co-authored-by: Copilot <copilot@github.com>

* fix commit reference

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: mverch67 <manuel.verch@gmx.de>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Manuel <71137295+mverch67@users.noreply.github.com>
2026-04-27 09:25:19 -05:00
Ben Meadors bc17d004ab Merge branch 'master' into develop 2026-04-27 08:05:22 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
048e5187ba Update platform-native digest to 4ea5e09 (#10314)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-27 07:50:13 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>Ben Meadors
4234fe6f86 Update meshtastic/device-ui digest to 7289329 (#10313)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-27 07:50:01 -05:00
Ben MeadorsJonathan BennettCopilotcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
126861fd16 Native MacOS hello world (#10309)
* Native MacOS hello world

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update variants/native/portduino/platformio.ini

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: ensure null-termination in getSerialString() and handle len==0

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/e5647919-2255-48ad-bcaa-7a2c2fdbf212

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

---------

Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-04-27 07:07:49 -05:00
47a6c4c6a0 Upgrade trunk (#10317)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-04-27 06:22:50 -05:00
Ben MeadorsGitHubJonathan BennettCopilotcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
06a6c3ee20 Native MacOS hello world (#10309)
* Native MacOS hello world

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update variants/native/portduino/platformio.ini

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: ensure null-termination in getSerialString() and handle len==0

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/e5647919-2255-48ad-bcaa-7a2c2fdbf212

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

---------

Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-04-26 22:07:07 -05:00
87f1f9d349 fix(Router): localize p_encrypted to prevent recursive-overwrite leak (#10311)
Router::handleReceived stores its allocCopy of the encrypted packet in the
class member p_encrypted. callModules() invokes module replies that re-enter
the router via MeshService::sendToMesh -> Router::sendLocal, which on a
broadcast reply recursively calls handleReceived. The inner call overwrites
the outer's p_encrypted without releasing it; on the way out it nulls the
member, the outer release(p_encrypted) now releases nullptr, and the original
allocation is permanently leaked. ~381 B per recursion.

Promote p_encrypted to a function-local so each invocation owns its own copy
for its full lifetime. The MQTT-publish null check at the call site (added by
PR #9136 as a workaround for this bug) stays in place because allocCopy can
still legitimately return nullptr on packetPool exhaustion.

Copilot's review of PR #8999 (the original introduction) flagged this exact
pattern at merge time:
  "Storing p_encrypted as a class member can cause issues with recursive or
  concurrent calls to handleReceived() since each call would overwrite the
  previous packet pointer."

The historical reason for the member (S&F needing to retain the encrypted
copy across calls) was satisfied differently by PR #9799 (ServerAPI converted
to std::unique_ptr + cleanup on connection close), so the member is no longer
load-bearing.

Reproduces issues #9632 / #10101 / #8729 (heap leak when MeshMonitor
connected; TCP drops on Station G2 / LILYGO ServerAPI dump abort).

Hardware A/B on Station G2 under sustained TCP-API retry storm (open :4403,
request config, disconnect mid-stream, repeat at ~0.6/s) - 9 min run:

  | Build         | heapFree drift | rebootCount delta |
  | this patch    | -1.5 KB (noise)| 0                 |
  | stock 2.7.13  | -73 KB (8.1KB/min) | +1 (OOM crash) |

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-26 21:03:34 -05:00
bfadf0c36a fix(Router): localize p_encrypted to prevent recursive-overwrite leak (#10311)
Router::handleReceived stores its allocCopy of the encrypted packet in the
class member p_encrypted. callModules() invokes module replies that re-enter
the router via MeshService::sendToMesh -> Router::sendLocal, which on a
broadcast reply recursively calls handleReceived. The inner call overwrites
the outer's p_encrypted without releasing it; on the way out it nulls the
member, the outer release(p_encrypted) now releases nullptr, and the original
allocation is permanently leaked. ~381 B per recursion.

Promote p_encrypted to a function-local so each invocation owns its own copy
for its full lifetime. The MQTT-publish null check at the call site (added by
PR #9136 as a workaround for this bug) stays in place because allocCopy can
still legitimately return nullptr on packetPool exhaustion.

Copilot's review of PR #8999 (the original introduction) flagged this exact
pattern at merge time:
  "Storing p_encrypted as a class member can cause issues with recursive or
  concurrent calls to handleReceived() since each call would overwrite the
  previous packet pointer."

The historical reason for the member (S&F needing to retain the encrypted
copy across calls) was satisfied differently by PR #9799 (ServerAPI converted
to std::unique_ptr + cleanup on connection close), so the member is no longer
load-bearing.

Reproduces issues #9632 / #10101 / #8729 (heap leak when MeshMonitor
connected; TCP drops on Station G2 / LILYGO ServerAPI dump abort).

Hardware A/B on Station G2 under sustained TCP-API retry storm (open :4403,
request config, disconnect mid-stream, repeat at ~0.6/s) - 9 min run:

  | Build         | heapFree drift | rebootCount delta |
  | this patch    | -1.5 KB (noise)| 0                 |
  | stock 2.7.13  | -73 KB (8.1KB/min) | +1 (OOM crash) |

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-26 21:02:42 -05:00
Jonathan BennettandGitHub 24c4162a75 Standardize PMU IRQ handling and enable power button cancel on tbeam-s3 (#10285)
* Standardize PMU IRQ handling and enable power button as cancel on tbeam s3

* Original T-beam, too
2026-04-26 19:58:23 -05:00
Ben Meadors b148fac340 Update framework version reference for Adafruit nRF52 to latest master branch 2026-04-26 09:49:39 -05:00
8dde4eeee1 BaseUI: Color Support for TFT Nodes (#10233)
* True Colors on TFT (Heltec Mesh Node T114, Heltec Vision Master T190, CardPuter Adv, T-Deck, T-Lora Pager)

* Theme support - New and some Classic Themes!

* Colored Compass

---------

Co-authored-by: Jason P <applewiz@mac.com>
Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-26 07:44:56 -05:00
Jonathan BennettGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>thebenternBen MeadorsCopilot
4ccdd80090 Add search duration check for exceeding 15 minutes (#10293)
* Add search duration check for exceeding 15 minutes

Added a condition to check if the search duration exceeds 15 minutes, indicating too long of a search.

* trunk

* Fix searchedTooLong: move 15-min cap before UINT32_MAX check, cache elapsed, add constexpr

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/b7f74430-9e7e-4a6f-8095-6176c1eee972

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Update src/gps/GPSUpdateScheduling.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Remove dead UINT32_MAX branch from searchedTooLong

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/6dad5b56-902e-4d0e-90c1-038a9c2df364

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-25 20:42:14 -05:00
Ben Meadors 4d4e14600c Merge remote-tracking branch 'origin/master' into develop 2026-04-25 15:55:16 -05:00
Ben Meadors aec0805a27 Trying another guard approach 2026-04-25 15:32:19 -05:00
Ben Meadors 0bd8dee346 Merge remote-tracking branch 'origin/master' into develop 2026-04-25 15:06:28 -05:00
Ben Meadors 7800dc3c8d Enhance UTF-8 sanitization logic and add delays in test setup for reliable timing 2026-04-25 15:04:58 -05:00
Ben Meadors 554188e90e Fix main function to setup and loop for Unity test framework 2026-04-25 14:49:37 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
fb678b9337 Update platform-native digest to 135b91e (#10300)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-25 14:44:06 -05:00
2828dbe4ca t5s3-epaper: Move variant.cpp -> extra_variants/variant.cpp ...again (#10297)
Fixes issues with #includes inherited from `configuration.h` when building for pioarduino.

Aligns t5s3_epaper with other variants like t_deck_pro.

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-25 13:36:31 -04:00
Ben Meadors a8c8fd7002 Remove redundant power interrupt methods for ESP32 2026-04-25 07:11:03 -05:00
Ben MeadorsandCopilot e7c02da24b Merge remote-tracking branch 'origin/master' into develop
Co-authored-by: Copilot <copilot@github.com>
2026-04-25 06:41:38 -05:00
Ben Meadors 7347091055 Add script to show unmerged commits from develop to master 2026-04-25 06:10:45 -05:00
55f15076ca Update protobufs (#10295)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
2026-04-25 06:08:07 -05:00
HarukiToredaandGitHub 7421953e8f InkHUD: Add full touch support to T5s3 (#10286)
* InkHUD touch rework

* Applet Switcher

* Update ED047TC1.cpp

* trunk fix

* Custom tip screen for T5s3

* Update TouchScreenImpl1.cpp

* Update ED047TC1.cpp

* Delete variant.cpp
2026-04-25 05:22:24 -05:00
9306e66067 fix(inkhud): scale MapApplet markers with fontSmall line height (#10288)
Marker boxes, the own-node bullseye, and the labeled-marker cross were
all hardcoded in pixels (11px box, r=8 circle, 12px cross). On the
T5S3 with a 12pt fontSmall (~17px line height) the hop-count digit
overflowed its box entirely. Sizes now derive from fontSmall.lineHeight()
so the applet renders correctly on both small (6pt) and large (12pt+)
display variants.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 21:20:39 -04:00
Jonathan BennettandBen Meadors 439b87b860 Detach power interrupts for sleep (#10230)
* Detach power interrupts for sleep

* Gate PMU IRQ behind a found PMU
2026-04-24 06:48:21 -05:00
Jonathan BennettandBen Meadors d47301defc Add PortduinoSetOptions to overwrite the realhardware bool (#10157)
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-24 06:48:09 -05:00
Jonathan BennettandBen Meadors 04b819a7b5 Remove incorrect LED_STATE_ON definition for t-beam-s3 (#10280)
Fixes #9912  and #10170
2026-04-24 06:47:38 -05:00
8e653122c7 Upgrade trunk (#10284)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-04-24 05:40:44 -05:00
Jonathan BennettandGitHub 7adfc3f992 Remove incorrect LED_STATE_ON definition for t-beam-s3 (#10280)
Fixes #9912  and #10170
2026-04-24 15:17:33 +10:00
ba9cadc14d Fix INA226 detection for non-TI compatible chip (Silergy) (#10247)
* Fix INA226 detection for non-TI compatible chip (Silergy)

* Removed extra I2C transaction + 20ms delay on every scan of address 0x40 (including real SHT2x sensors).

Changes suggested by Copilot

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Apply formatting (trunk fmt)

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-04-24 13:09:37 +10:00
924411de59 T-Watch S3 Power button managment (#9855)
* PMU interrupt pin defined in t-watch s3

* Implement button control on T-Watch S3

Added interrupt handling for the Power/Corona button on T-Watch S3, I use it to control screen state.

* Reducing labels

* Reducing labels

* Updated the comment

* ISR is now IRAM-safe

Updated interrupt management not to cause random crashes.

* Trunk

* Simplify and use INPUT_BROKER_CANCEL

---------

Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
2026-04-23 19:53:59 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
5ea3d143da Update meshtastic-esp8266-oled-ssd1306 digest to 6bfd1f1 (#10277)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-23 19:30:47 -05:00
d9195944df PositionModule::sendLostAndFoundText: use stack buffer, eliminate heap alloc (#10251)
* PositionModule::sendLostAndFoundText: use stack buffer, eliminate heap alloc

The lost-and-found message was built with an unnecessary heap allocation:

    char *message = new char[60];
    sprintf(message, "..."...);
    ...
    delete[] message;

Two problems:

1. **Buffer too small.** The format string expands with two %f (IEEE 754
   doubles), which `sprintf` prints with full precision — easily 15+
   digits each plus separators — so the actual rendered string can run
   40-50 characters before even considering the emoji (4 UTF-8 bytes)
   and the embedded BEL. A pathological lat/lon can overflow 60 bytes
   and corrupt heap metadata. Unbounded `sprintf` with no size check.

2. **Heap churn in a GPS callback.** This function is called from the
   position-update path which is already heap-sensitive. An infrequent
   60-byte transient alloc isn't catastrophic, but stack is trivially
   available here and removes the failure mode entirely.

Fix: replace with a 128-byte stack buffer and `snprintf` bounded by
`sizeof(message)`. Drop the matching `delete[]` since there's nothing
to delete.

Behavior is identical on the happy path; the overflow case now
truncates safely instead of scribbling over heap.

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* PositionModule.cpp: add trailing newline for clang-format

* Address Copilot review: cleaner snprintf size handling

Review feedback from @Copilot on PR #10251: the ternary-plus-static-cast
form mixed signed/unsigned types (int written vs. pb_size_t payload.size
vs. size_t sizeof(message)) and was harder to read than necessary.

Cleaner form:
  const size_t msg_len = std::min(static_cast<size_t>(written), sizeof(message) - 1);
  p->decoded.payload.size = msg_len;

Same behaviour (clamp to buffer-minus-NUL) with one explicit cast and
a size_t variable that names the meaning. Handles the encoding-error
path (written < 0) separately so no bad values leak into payload.size.

* Trunk

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-04-23 18:20:07 -05:00
83a98c81f6 Hash table index for O(1) packet history lookups (#9499)
* Use hash table for O(1) lookup of recently seen packets

* Eliminate a packet lookup during deduplication

* Infinite loop checks for find and remove

* Consolidate conditional compilation

* Exclude hash table from minimal build

* Additional comment on hash table capacity

* Unit tests for packet history changes

* Update incorrect comment about size clamp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Const

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-23 18:04:34 -05:00
Jonathan BennettandGitHub 837637b70c Only enable wakeup via EXT_CHRG_DETECT if we shut down due to low power (#10263) 2026-04-23 15:37:35 -05:00
Valentin V. BartenevandGitHub 7b3f58875a Fix example comment in airtime.h (#10275)
Looks like a copy'n'paste typo from the previous line.
It definitely meant to be RX_ALL_LOG according to comment.
2026-04-23 14:44:39 -05:00
Ben MeadorsandCopilot 56c897e826 Can't LOG when we don't have logging set up yet in the native test suite
Co-authored-by: Copilot <copilot@github.com>
2026-04-23 14:42:29 -05:00
Andrew YongandGitHub b2d980fc25 feat(Power): support EXT_PWR_DETECT_MODE & EXT_PWR_DETECT_VALUE, simplify EXT_PWR_DETECT (#10140)
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Signed-off-by: Andrew Yong <me@ndoo.sg>
2026-04-23 14:32:17 -05:00
Ben Meadors 2cc13a1132 Sane sanitization 2026-04-23 14:19:33 -05:00
2ed7bba5e7 fix(Power): refactor EXT_CHRG_DETECT to compile-time macros (#10191)
Mirror the EXT_PWR_DETECT pattern: replace runtime static variables
(ext_chrg_detect_mode, ext_chrg_detect_value) with compile-time macros.
Auto-infer EXT_CHRG_DETECT_VALUE from EXT_CHRG_DETECT_MODE when the mode
is INPUT_PULLUP (→ LOW) or INPUT_PULLDOWN (→ HIGH); default to HIGH.

This fixes inverted polarity on variants that define EXT_CHRG_DETECT_MODE
INPUT_PULLUP without an explicit EXT_CHRG_DETECT_VALUE (e.g. russell):
previously the runtime default of HIGH caused isCharging() to return the
opposite of the correct value. With auto-inference the correct LOW active
level is now derived at compile time.

Remove the now-redundant EXT_CHRG_DETECT_VALUE HIGH from ELECROW-ThinkNode-M4
variant.h since HIGH is the inferred default.


Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Signed-off-by: Andrew Yong <noreply@example.com>
Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
2026-04-23 13:24:05 -05:00
Jonathan BennettandGitHub 031f332ec1 We have HardwareRNG, let's use it! (#10274) 2026-04-23 13:16:49 -05:00
Catalin PatuleaandGitHub 22d50fe437 NimbleBluetooth misc cleanups (#10264)
* Delete unused clearNVS() (last used in commit 761804b1).

* virtual methods: add 'override' to ensure we get the signature right.

This is a safety net for pioarduino/NimBLE work where there's multiple
similar variants of the same method (eg. onConnect) and it's easy to get
the wrong one and accidentally miss a callback.
2026-04-23 09:18:41 -05:00
Ben Meadors cde5a08bc5 Merge remote-tracking branch 'origin/master' into develop 2026-04-23 06:48:43 -05:00
55bf8c25fc PhoneAPI: add missing tak_tag case + skip reserved gap in module-config iteration (#10256)
* PhoneAPI: add missing tak_tag case + skip reserved gap in module-config iteration

The STATE_SEND_MODULECONFIG state machine iterates config_state through
ModuleConfigType enum values (1..MAX+1 = 16) and switches on
meshtastic_ModuleConfig_*_tag values. Two of the resulting tag values
land in the default / LOG_ERROR path:

1. `tak_tag` (16) — the meshtastic_ModuleConfig_TAKConfig struct exists
   in the protobuf and has a `.tak` member in payload_variant, but no
   PhoneAPI case ever sends it to the phone. As a result, Android
   clients can't read the persisted TAK (Team Awareness Kit) module
   config at all. Added case that sends moduleConfig.tak, matching the
   pattern used for all other module-config tags (paxcounter,
   traffic_management, etc.). NodeDB already persists the struct via
   the moduleConfig protobuf save; this just wires the read path to
   the phone.

2. Tag 14 — reserved gap in the oneof numbering. No payload_variant
   member exists at tag 14. Without this patch, every phone reconnect
   walks through config_state=14 and hits
   `LOG_ERROR("Unknown module config type %d", config_state)`. On an
   active deployment that's ~1,400 LOG_ERROR lines per day per node —
   burying real errors. Added explicit `case 14: break;` so the gap
   is silently skipped.

Also: lowered the `default:` log level from LOG_ERROR to LOG_DEBUG. A
truly-new unknown type number would indicate firmware lagging the
protobuf — annoying but not an error event worth LOG_ERROR, especially
since this path runs on every phone handshake. If a new ModuleConfig
tag appears, devs will notice via the phone UI missing it, not via log.

Observed on a Station G2 fleet: 1403 "Unknown module config type 16"
and 1427 "Unknown module config type 14" LOG_ERROR lines in 24 hours
from routine phone reconnects.

* Get rid of the placeholder

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-23 06:42:05 -05:00
nightjoker7andGitHub 399dde0f4b Router: demote cross-channel decrypt failures from ERROR to DEBUG (#10259)
The "Invalid protobufs (bad psk?)" and "Invalid portnum (bad psk?)"
messages fire every time a neighbor transmits on a channel whose
8-bit hash matches one of ours but the PSK differs. In RF environments
with multiple mesh groups nearby this is routine — a single device
can see dozens of these per minute from SAR/MeshCA/private networks
sharing a hash collision.

LOG_ERROR for a benign "not our PSK" event:
- spams the log when you have any neighboring mesh group
- makes a genuine PSK misconfiguration on YOUR own channel
  indistinguishable from the constant cross-channel noise
- hides actual errors in the stream

LOG_DEBUG matches how similar decryption-failure paths are handled
elsewhere (eg. the PKC "decrypt attempted but failed" uses LOG_WARN).
Dropping the trailing "!" as well — these are expected events, not
exceptional ones.
2026-04-23 06:19:05 -05:00
48747ee43d Upgrade trunk (#10266)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-04-23 06:16:23 -05:00
66971a0a26 RadioLibInterface: clear static instance on destruction to prevent UAF (#10254)
The constructor sets `RadioLibInterface::instance = this` immediately,
before `init()` runs. `initLoRa()` in RadioInterface.cpp creates each
radio variant with `new SX1262Interface(...)` or similar, then calls
`init()`, and if init fails the `unique_ptr<RadioInterface>` is reset
to nullptr — destroying the object — while the static `instance`
pointer continues to point at the freed memory.

Main loop then checks `RadioLibInterface::instance != nullptr` and
calls `pollMissedIrqs()` or `resetAGC()` on the dangling pointer →
Guru Meditation (IllegalInstruction / LoadProhibited).

Reported in #9880 on an ESP32-S3 dev board without radio hardware
attached, where init always fails and the leftover pointer crashes
the device on the next `loop()` iteration.

Fix: add a virtual destructor to `RadioLibInterface` that clears the
static pointer iff it still references this object. A later successful
init() may have replaced `instance` with a different interface — the
`instance == this` guard preserves that case.

Fixes #9880

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-23 06:16:08 -05:00
Thomas GöttgensandGitHub 4c24218afb Revert "Update LovyanGFX to v1.2.20 (#10232)" (#10269) (#10270)
This reverts commit fb1de111d7.
2026-04-23 12:40:27 +02:00
Thomas GöttgensandGitHub 7c27f4e2df Revert "Update LovyanGFX to v1.2.20 (#10232)" (#10269)
This reverts commit fb1de111d7.
2026-04-23 12:37:05 +02:00
Ben Meadors 4b4914736f Merge remote-tracking branch 'origin/master' into develop 2026-04-22 21:52:25 -05:00
Jonathan BennettandGitHub 92c0133ef9 Finish evil merge cleanup (#10253) 2026-04-22 21:47:20 -05:00
nightjoker7andGitHub a6b1a69630 StoreForwardModule::historyAdd: memcpy source size, not buffer capacity (#10250)
`memcpy(... p.payload.bytes, meshtastic_Constants_DATA_PAYLOAD_LEN)`
reads past the actual payload when the incoming packet's payload is
shorter than `DATA_PAYLOAD_LEN` (237 bytes). The code just above
already records the correct size:

    this->packetHistory[...].payload_size = p.payload.size;

but then the memcpy ignores that and copies the full buffer capacity,
pulling uninitialized / adjacent memory bytes into the history entry.
Those extra bytes are later rebroadcast whenever the Store & Forward
module replays the packet.

Fix: memcpy using `p.payload.size` (the actual payload length) instead
of the constant buffer capacity.

Classification: bounded out-of-bounds READ into the protobuf scratch
buffer. Not directly exploitable for RCE (the destination buffer is
also DATA_PAYLOAD_LEN), but leaks adjacent memory into replayed
packets and is a latent correctness bug.
2026-04-22 21:04:37 -05:00
Ben Meadors 6171ad8c14 Remove duplicate GPIO init block in setup() from bad merge in PR #9378
The T-Deck-Pro V1.1 PR (#9378) had a backwards merge (e7b66281) that pulled
236 commits of master INTO a 1-commit fork branch, then a second merge
(8fd0a7f2) duplicated the entire T_DECK/T_DECK_PRO/T_LORA_PAGER/HACKADAY
peripheral init block in setup(). Also removes a stray
digitalWrite(BLE_LED, LED_STATE_OFF) in the HACKADAY section that was an
artifact of the evil merge at e7b66281.
2026-04-22 19:35:13 -05:00
Jonathan BennettandGitHub 28e705de5c Detach power interrupts for sleep (#10230)
* Detach power interrupts for sleep

* Gate PMU IRQ behind a found PMU
2026-04-22 14:27:48 -05:00
Jason PandGitHub d8b11f0b14 Improve options to align to names of UI options (#10240) 2026-04-22 11:42:25 -05:00
AustinandGitHub fcb9ec0c2d t5s3-epaper: Move variant.cpp -> extra_variants/variant.cpp (#10241)
Fixes issues with #includes inherited from `configuration.h` when building for pioarduino.

Aligns t5s3_epaper with other variants like t_deck_pro.
2026-04-22 11:42:02 -05:00
AustinandGitHub a4b55bc6f2 cardputer-adv: Move variant.cpp -> extra_variants/variant.cpp (#10242)
Fixes issues with #includes inherited from `configuration.h` when building for pioarduino.

Aligns cardputer-adv with other variants like t_deck_pro.
2026-04-22 11:41:49 -05:00
a710841723 Upgrade trunk (#10236)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-04-22 11:35:07 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
fb1de111d7 Update LovyanGFX to v1.2.20 (#10232)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-22 11:34:48 -05:00
Jonathan Bennett b53fe7a1e7 T watch pinfix (#10231)
* Minor button debugging bits

* pin0 is a pin, pin -1 means disabled
2026-04-21 20:03:40 -05:00
Jonathan BennettandGitHub 2b5daf2438 T watch pinfix (#10231)
* Minor button debugging bits

* pin0 is a pin, pin -1 means disabled
2026-04-21 20:02:56 -05:00
TomandGitHub db9fdd6794 Fix: filter out SKIPPED tests in PlatformIO output to improve log clarity (#10214) 2026-04-21 16:43:35 -05:00
9361b85f47 feat(t5s3-epaper): add InkHUD port for LilyGo T5 E-Paper S3 Pro (#10211)
* niche: add InkHUD port for LilyGo T5-E-Paper-S3-Pro (ED047TC1)

Add a NicheGraphics EInk driver adapter for the 4.7" ED047TC1 parallel
e-paper display used on the T5-E-Paper-S3-Pro (H752-01). The driver
wraps FastEPD and handles the polarity difference between InkHUD's
buffer format (0xFF = white) and FastEPD's (0x00 = white).

Rewrite variants/esp32s3/t5s3_epaper/nicheGraphics.h which was an
incomplete copy of the Heltec VM-E290 setup referencing undefined SPI
pin macros and a non-existent BUTTON_PIN_SECONDARY. The board uses a
parallel display, not the small SPI DEPG0290BNS800 that was referenced.

* fix: guard inputBroker null dereference in TouchScreenImpl1::init()

When MESHTASTIC_EXCLUDE_INPUTBROKER is defined (e.g. InkHUD builds),
inputBroker is nullptr. Calling inputBroker->registerSource() in that
state caused a LoadProhibited panic on any board that has both
HAS_TOUCHSCREEN=1 and the InputBroker excluded.

Add a null check before registerSource() to prevent the crash.

* niche: fix display rotation for T5-E-Paper-S3-Pro InkHUD port

Set rotation=3 (270° CW) in nicheGraphics.h to correct for FastEPD
scanning the ED047TC1 panel in portrait orientation, resulting in
correct landscape display output.

* fix: update buffer format descriptions and remove polarity inversion for InkHUD and FastEPD

* fix: update ED047TC1 driver to handle inactive pixel borders and adjust safe-area dimensions

* fix: comment out ruler diagnostic for E-Ink driver

* feat: implement TouchInkHUDBridge for direct touch event handling in InkHUD

* niche: add FreeSans 18pt/24pt Win1253 (Greek) fonts for larger InkHUD displays

Add Win1253-encoded FreeSans 18pt and 24pt font headers to support Greek
script on larger InkHUD screens (e.g., the 4.7" ED047TC1 at ~234 DPI).
Register FREESANS_24PT_WIN1253 and FREESANS_18PT_WIN1253 macros in AppletFont.h.
Set fontLarge=24pt, fontMedium=18pt, fontSmall=12pt in nicheGraphics.h for the
T5-E-Paper-S3-Pro.

* feat(ed047tc1): use true partial update for FAST refresh

Replace fullUpdate(CLEAR_FAST) with partialUpdate() for FAST display
updates. FastEPD's partialUpdate() diffs pCurrent against pPrevious
and only applies the update waveform to rows that have changed, leaving
unchanged rows with a neutral signal.

This reduces visible flicker on routine updates (new messages, position
changes) — only the affected region of the screen refreshes. Full-screen
CLEAR_SLOW updates are preserved for periodic ghosting cleanup, driven
by InkHUD's setDisplayResilience() ratio.

* feat(t5s3-epaper): enable frontlight via LatchingBacklight

Wire up BOARD_BL_EN (GPIO11) to InkHUD's LatchingBacklight driver.
Enable the backlight menu item so users can toggle "Keep Backlight On"
via Settings. The backlight turns on automatically when the menu opens
and off when it closes.

* Fix RTC chip (PCF8563 not PCF85063) and GT911 I2C address collision

- variant.h used PCF85063_RTC but the board has a PCF8563. The difference
  is the RAM register: PCF85063 has 1 byte of RAM; PCF8563 does not. The
  PCF85063 driver was trying to write this register on init, failing every
  time, and setDateTime writes were silently discarded — RTC time was
  never persisted across reboots. Switch to PCF8563_RTC/PCF8563_INT.

  Before:
    [E][SensorPCF85063.hpp:375] initImpl(): Failed to write to RAM memory
      register. Maybe this chip is pcf8563.
    Read RTC time from PCF85063 getDateTime as 2026-04-05 00:00:23
    PCF85063 setDateTime 2026-04-05 18:40:59
    Read RTC time from PCF85063 getDateTime as 2026-04-05 00:00:19  ← lost

  After:
    PCF8563 found at address 0x51
    Read RTC time from PCF8563 getDateTime as 2026-04-05 18:58:37  ← persisted
    PCF8563 setDateTime 2026-04-05 18:58:44
    Read RTC time from PCF8563 getDateTime as 2026-04-05 18:58:44  ← round-trips

- GT911 touch was initialized with GT911_SLAVE_ADDRESS_L (0x5D), which
  collides with the SFA30 air quality sensor also at 0x5D on the same
  I2C bus. Switch to GT911_SLAVE_ADDRESS_H (0x14): the library drives
  INT high during reset to program the GT911 to address 0x14,
  eliminating the address conflict.

  Before:
    SFA30 found at address 0x5d
    [I][TouchDrvGT911.hpp:568] initImpl(): Try using 0x5D as the device address

  After:
    SFA30 found at address 0x5d
    [I][TouchDrvGT911.hpp:544] initImpl(): Try using 0x14 as the device address

* t5s3_epaper: fix GT911 ghost-SFA30 via early I2C address latch

Investigation findings
----------------------
Boot logs showed "SFA30 found at address 0x5d" on every cold power-on,
and AirQualityTelemetry was registering an SFA30 sensor. However, every
readMeasuredValues() call returned error 268 (0x010C = Sensirion
WriteError | I2cAddressNack), meaning the I2C write to 0x5D was being
NACK'd — inconsistent with a real SFA30.

Root cause: the GT911 touch controller latches its I2C address from the
INT pin level at reset time (GT911 datasheet §4.3). GPIO3 (INT) defaults
LOW on ESP32-S3 cold boot → GT911 always powers up at 0x5D
(SLAVE_ADDRESS_L). The I2C scanner runs before lateInitVariant() had a
chance to reprogram the chip.

The scanner's SFA30 detection (ScanI2CTwoWire.cpp) sends the 2-byte
command 0xD060 to 0x5D and requests 48 bytes back. GT911 ACKs the
write (treating it as a register address) and returns 48 bytes of
register data, passing the length check — a false-positive SFA30
detection.

Confirmed via second cold-boot log: after the previous commit moved GT911
to 0x14 in lateInitVariant(), address 0x5D *still* appeared in the scan
because the scanner runs first. The board has no physical SFA30 fitted.

Fix
---
Add the GT911 address-latch reset sequence to earlyInitVariant(), before
Wire is initialised and before the I2C scan runs. Per the datasheet:
drive RST LOW, drive INT HIGH (selects address 0x14 / SLAVE_ADDRESS_H),
hold >100 µs, release RST, wait >5 ms startup. GPIO-only, no Wire
dependency. lateInitVariant() then repeats this sequence internally via
touch.begin(); the double-reset is harmless.

Verified in boot log:
  Before: "SFA30 found at address 0x5d", 5 I2C devices, NACK errors
  After:  no SFA30 entry, 4 I2C devices (TCA9535/PCF8563/BQ27220/BQ25896),
          GT911 found at 0x14 and touch initialised successfully,
          AirQualityTelemetry registers no sensors (correct — no SFA30 present)

* t5s3_epaper: add variant_shutdown() for touch sleep and backlight off

Put GT911 into low-power standby (command 0x05) and drive BOARD_BL_EN
LOW before deep sleep to avoid unnecessary current draw.

* t5s3_epaper: fix touch gesture routing and coordinate mapping

readTouch() now transforms raw GT911 axes to visual-frame coordinates
based on the current display rotation (rotation=3 is the hardware
identity). This ensures TouchScreenBase detects swipe direction
correctly regardless of which rotation the user has selected.

TouchInkHUDBridge dynamically sets joystick.alignment = (4-rotation)%4
on each touch event so that (rotation+alignment)%4==0 always, keeping
nav calls pass-through without remapping.

nicheGraphics.h now calls loadSettings() first so that rotation is
persisted across reboots. rotation=3 and other first-boot defaults are
only applied when tips.firstBoot is set. alignment is recomputed from
the loaded rotation on every boot.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* t5s3_epaper: fix GT911 sleep timing via notifyDeepSleep observer

touch.sleep() was called from variant_shutdown(), which runs inside
cpuDeepSleep() — after Wire.end() had already torn down the I2C bus in
doDeepSleep(). This caused Wire NULL TX buffer errors and left the GT911
awake during deep sleep.

Register a CallbackObserver on notifyDeepSleep, which fires before
Wire.end(), so the I2C command reaches the chip while the bus is live.
Pattern matches LatchingBacklight and other NicheGraphics components.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* t5s3_epaper: fix touch nav and applet defaults in nicheGraphics

Enable joystick mode post-begin so menu scroll and swipe-up/down
gestures are not silently dropped by the joystick.enabled gate in
Events.cpp. Activate DMs and Channel 0/1 applets with correct
autoshow defaults matching the mini-epaper-s3 reference pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update nicheGraphics.h

* t5s3_epaper: fix ED047TC1 driver docs and remove spurious beginPolling

Addressing PR review comments:

Remove beginPolling(1, 0) after the blocking FastEPD update — it
incorrectly set updateRunning=true for one loop cycle after the
hardware was already done, causing busy() to briefly return true.
Since isUpdateDone() always returns true, no polling is needed.

Also fix stale comments: safe-area buffer size was 944×532, now
944×523; V_OFFSET_ROWS didn't exist, replaced with the actual
V_OFFSET_TOP=9 / V_OFFSET_BOTTOM=8 constant names.

* t5s3_epaper: clean up applet addition formatting in setupNicheGraphics

* t5s3_epaper: guard ED047TC1.cpp against non-T5S3 InkHUD builds

The InkHUD base config pulls in all of src/graphics/niche/ so every
InkHUD device compiled ED047TC1.cpp, triggering the #error on line 48
for boards that define neither T5_S3_EPAPER_PRO_V1 nor V2.

Wrap the file body with #ifdef T5_S3_EPAPER_PRO so it is only compiled
for T5S3 targets. The #error is preserved inside the guard to catch
future hardware revisions that forget to update the driver.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: HarukiToreda <116696711+HarukiToreda@users.noreply.github.com>
2026-04-21 15:35:28 -05:00
3b4c66439d feat(t5s3-epaper): add InkHUD port for LilyGo T5 E-Paper S3 Pro (#10211)
* niche: add InkHUD port for LilyGo T5-E-Paper-S3-Pro (ED047TC1)

Add a NicheGraphics EInk driver adapter for the 4.7" ED047TC1 parallel
e-paper display used on the T5-E-Paper-S3-Pro (H752-01). The driver
wraps FastEPD and handles the polarity difference between InkHUD's
buffer format (0xFF = white) and FastEPD's (0x00 = white).

Rewrite variants/esp32s3/t5s3_epaper/nicheGraphics.h which was an
incomplete copy of the Heltec VM-E290 setup referencing undefined SPI
pin macros and a non-existent BUTTON_PIN_SECONDARY. The board uses a
parallel display, not the small SPI DEPG0290BNS800 that was referenced.

* fix: guard inputBroker null dereference in TouchScreenImpl1::init()

When MESHTASTIC_EXCLUDE_INPUTBROKER is defined (e.g. InkHUD builds),
inputBroker is nullptr. Calling inputBroker->registerSource() in that
state caused a LoadProhibited panic on any board that has both
HAS_TOUCHSCREEN=1 and the InputBroker excluded.

Add a null check before registerSource() to prevent the crash.

* niche: fix display rotation for T5-E-Paper-S3-Pro InkHUD port

Set rotation=3 (270° CW) in nicheGraphics.h to correct for FastEPD
scanning the ED047TC1 panel in portrait orientation, resulting in
correct landscape display output.

* fix: update buffer format descriptions and remove polarity inversion for InkHUD and FastEPD

* fix: update ED047TC1 driver to handle inactive pixel borders and adjust safe-area dimensions

* fix: comment out ruler diagnostic for E-Ink driver

* feat: implement TouchInkHUDBridge for direct touch event handling in InkHUD

* niche: add FreeSans 18pt/24pt Win1253 (Greek) fonts for larger InkHUD displays

Add Win1253-encoded FreeSans 18pt and 24pt font headers to support Greek
script on larger InkHUD screens (e.g., the 4.7" ED047TC1 at ~234 DPI).
Register FREESANS_24PT_WIN1253 and FREESANS_18PT_WIN1253 macros in AppletFont.h.
Set fontLarge=24pt, fontMedium=18pt, fontSmall=12pt in nicheGraphics.h for the
T5-E-Paper-S3-Pro.

* feat(ed047tc1): use true partial update for FAST refresh

Replace fullUpdate(CLEAR_FAST) with partialUpdate() for FAST display
updates. FastEPD's partialUpdate() diffs pCurrent against pPrevious
and only applies the update waveform to rows that have changed, leaving
unchanged rows with a neutral signal.

This reduces visible flicker on routine updates (new messages, position
changes) — only the affected region of the screen refreshes. Full-screen
CLEAR_SLOW updates are preserved for periodic ghosting cleanup, driven
by InkHUD's setDisplayResilience() ratio.

* feat(t5s3-epaper): enable frontlight via LatchingBacklight

Wire up BOARD_BL_EN (GPIO11) to InkHUD's LatchingBacklight driver.
Enable the backlight menu item so users can toggle "Keep Backlight On"
via Settings. The backlight turns on automatically when the menu opens
and off when it closes.

* Fix RTC chip (PCF8563 not PCF85063) and GT911 I2C address collision

- variant.h used PCF85063_RTC but the board has a PCF8563. The difference
  is the RAM register: PCF85063 has 1 byte of RAM; PCF8563 does not. The
  PCF85063 driver was trying to write this register on init, failing every
  time, and setDateTime writes were silently discarded — RTC time was
  never persisted across reboots. Switch to PCF8563_RTC/PCF8563_INT.

  Before:
    [E][SensorPCF85063.hpp:375] initImpl(): Failed to write to RAM memory
      register. Maybe this chip is pcf8563.
    Read RTC time from PCF85063 getDateTime as 2026-04-05 00:00:23
    PCF85063 setDateTime 2026-04-05 18:40:59
    Read RTC time from PCF85063 getDateTime as 2026-04-05 00:00:19  ← lost

  After:
    PCF8563 found at address 0x51
    Read RTC time from PCF8563 getDateTime as 2026-04-05 18:58:37  ← persisted
    PCF8563 setDateTime 2026-04-05 18:58:44
    Read RTC time from PCF8563 getDateTime as 2026-04-05 18:58:44  ← round-trips

- GT911 touch was initialized with GT911_SLAVE_ADDRESS_L (0x5D), which
  collides with the SFA30 air quality sensor also at 0x5D on the same
  I2C bus. Switch to GT911_SLAVE_ADDRESS_H (0x14): the library drives
  INT high during reset to program the GT911 to address 0x14,
  eliminating the address conflict.

  Before:
    SFA30 found at address 0x5d
    [I][TouchDrvGT911.hpp:568] initImpl(): Try using 0x5D as the device address

  After:
    SFA30 found at address 0x5d
    [I][TouchDrvGT911.hpp:544] initImpl(): Try using 0x14 as the device address

* t5s3_epaper: fix GT911 ghost-SFA30 via early I2C address latch

Investigation findings
----------------------
Boot logs showed "SFA30 found at address 0x5d" on every cold power-on,
and AirQualityTelemetry was registering an SFA30 sensor. However, every
readMeasuredValues() call returned error 268 (0x010C = Sensirion
WriteError | I2cAddressNack), meaning the I2C write to 0x5D was being
NACK'd — inconsistent with a real SFA30.

Root cause: the GT911 touch controller latches its I2C address from the
INT pin level at reset time (GT911 datasheet §4.3). GPIO3 (INT) defaults
LOW on ESP32-S3 cold boot → GT911 always powers up at 0x5D
(SLAVE_ADDRESS_L). The I2C scanner runs before lateInitVariant() had a
chance to reprogram the chip.

The scanner's SFA30 detection (ScanI2CTwoWire.cpp) sends the 2-byte
command 0xD060 to 0x5D and requests 48 bytes back. GT911 ACKs the
write (treating it as a register address) and returns 48 bytes of
register data, passing the length check — a false-positive SFA30
detection.

Confirmed via second cold-boot log: after the previous commit moved GT911
to 0x14 in lateInitVariant(), address 0x5D *still* appeared in the scan
because the scanner runs first. The board has no physical SFA30 fitted.

Fix
---
Add the GT911 address-latch reset sequence to earlyInitVariant(), before
Wire is initialised and before the I2C scan runs. Per the datasheet:
drive RST LOW, drive INT HIGH (selects address 0x14 / SLAVE_ADDRESS_H),
hold >100 µs, release RST, wait >5 ms startup. GPIO-only, no Wire
dependency. lateInitVariant() then repeats this sequence internally via
touch.begin(); the double-reset is harmless.

Verified in boot log:
  Before: "SFA30 found at address 0x5d", 5 I2C devices, NACK errors
  After:  no SFA30 entry, 4 I2C devices (TCA9535/PCF8563/BQ27220/BQ25896),
          GT911 found at 0x14 and touch initialised successfully,
          AirQualityTelemetry registers no sensors (correct — no SFA30 present)

* t5s3_epaper: add variant_shutdown() for touch sleep and backlight off

Put GT911 into low-power standby (command 0x05) and drive BOARD_BL_EN
LOW before deep sleep to avoid unnecessary current draw.

* t5s3_epaper: fix touch gesture routing and coordinate mapping

readTouch() now transforms raw GT911 axes to visual-frame coordinates
based on the current display rotation (rotation=3 is the hardware
identity). This ensures TouchScreenBase detects swipe direction
correctly regardless of which rotation the user has selected.

TouchInkHUDBridge dynamically sets joystick.alignment = (4-rotation)%4
on each touch event so that (rotation+alignment)%4==0 always, keeping
nav calls pass-through without remapping.

nicheGraphics.h now calls loadSettings() first so that rotation is
persisted across reboots. rotation=3 and other first-boot defaults are
only applied when tips.firstBoot is set. alignment is recomputed from
the loaded rotation on every boot.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* t5s3_epaper: fix GT911 sleep timing via notifyDeepSleep observer

touch.sleep() was called from variant_shutdown(), which runs inside
cpuDeepSleep() — after Wire.end() had already torn down the I2C bus in
doDeepSleep(). This caused Wire NULL TX buffer errors and left the GT911
awake during deep sleep.

Register a CallbackObserver on notifyDeepSleep, which fires before
Wire.end(), so the I2C command reaches the chip while the bus is live.
Pattern matches LatchingBacklight and other NicheGraphics components.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* t5s3_epaper: fix touch nav and applet defaults in nicheGraphics

Enable joystick mode post-begin so menu scroll and swipe-up/down
gestures are not silently dropped by the joystick.enabled gate in
Events.cpp. Activate DMs and Channel 0/1 applets with correct
autoshow defaults matching the mini-epaper-s3 reference pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update nicheGraphics.h

* t5s3_epaper: fix ED047TC1 driver docs and remove spurious beginPolling

Addressing PR review comments:

Remove beginPolling(1, 0) after the blocking FastEPD update — it
incorrectly set updateRunning=true for one loop cycle after the
hardware was already done, causing busy() to briefly return true.
Since isUpdateDone() always returns true, no polling is needed.

Also fix stale comments: safe-area buffer size was 944×532, now
944×523; V_OFFSET_ROWS didn't exist, replaced with the actual
V_OFFSET_TOP=9 / V_OFFSET_BOTTOM=8 constant names.

* t5s3_epaper: clean up applet addition formatting in setupNicheGraphics

* t5s3_epaper: guard ED047TC1.cpp against non-T5S3 InkHUD builds

The InkHUD base config pulls in all of src/graphics/niche/ so every
InkHUD device compiled ED047TC1.cpp, triggering the #error on line 48
for boards that define neither T5_S3_EPAPER_PRO_V1 nor V2.

Wrap the file body with #ifdef T5_S3_EPAPER_PRO so it is only compiled
for T5S3 targets. The #error is preserved inside the guard to catch
future hardware revisions that forget to update the driver.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: HarukiToreda <116696711+HarukiToreda@users.noreply.github.com>
2026-04-21 15:35:02 -05:00
Ben MeadorsandCopilot 68383c8bd5 Add encryption overview to agent instructions in AGENTS.md (#10207)
* Add encryption overview to agent instructions in AGENTS.md

* Update AGENTS.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update copilot-instructions.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update copilot-instructions.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Clarify nonce and wire overhead details in encryption section of copilot instructions

* Enhance encryption documentation in copilot instructions and agents guide for clarity on key management and reset behaviors

* Update .github/copilot-instructions.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix botched merge conflict resolution

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-21 10:49:13 -05:00
TomandBen Meadors 76dea77929 Add authoring guide for native unit tests in README.md (#10201)
* Add authoring guide for native unit tests in README.md

* Enhance documentation for agent tooling and native unit tests in README and related files

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-21 10:49:08 -05:00
Clive BlackledgeandBen Meadors d7ba178bf1 Fix: prompt markdownlint md040 fix for new prompts. (#10199)
* Add ESP32 Power Management lessons learned document

Documents our experimentation with ESP-IDF DFS and why it doesn't
work well for Meshtastic (RTOS locks, BLE locks, USB issues).

Proposes simpler alternative: manual setCpuFrequencyMhz() control
with explicit triggers for when to go fast vs slow.

* docs(prompts): fix markdown fence language tags

* docs: remove ESP32 power management notes
2026-04-21 10:49:01 -05:00
Jennifer SanchezandBen Meadors 5f836cdf3b Added support for Spreading Factors 5 and 6 on compatible radios (#10160) 2026-04-21 10:48:33 -05:00
HarukiToredaandGitHub 945f4780ea BaseUI: Nodelist screen/favorite screen cleanup (#10197)
* nodelist screen cleanup

* Update UIRenderer.cpp

* Update src/graphics/draw/UIRenderer.cpp

* removed brackets from hop and made signal mutually exclusive
2026-04-21 10:31:29 -05:00
a8a5315460 Upgrade trunk (#10221)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-04-21 17:14:12 +02:00
0e38a15d46 Update protobufs (#10223)
Co-authored-by: caveman99 <25002+caveman99@users.noreply.github.com>
2026-04-21 17:13:55 +02:00
Jonathan BennettandBen Meadors d8e4389da2 No longer need undefines, thanks to #10179 (#10180) 2026-04-21 09:57:48 -05:00
Jonathan BennettandBen Meadors 25febfdeee More cleanly remove LED_BUILTIN (#10179)
* Test PR to remove LED_BUILTIN 

Comment out the LED_BUILTIN definition in platformio.ini

* Add LED_BUILTIN definition to nrf52840.ini
2026-04-21 09:57:41 -05:00
RuledoandBen Meadors f5be09c123 Add Luckfox Pico Max Waveshare Pico LoRa config (#10175)
Add a meshtasticd config for the Luckfox Pico Max with the Waveshare Pico LoRa SX1262 TCXO HAT.

Tested on hardware with successful SX1262 init, broadcast, and direct messaging.
2026-04-21 09:57:33 -05:00
23321c4588 Reduce key duplication by enabling hardware RNG (#8803)
* Reduce key duplication by enabling hardware RNG

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Use micros() for worst case random seed for nrf52

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Minor cleanup, remove dead code and clarify comment

* trunk

* Add useRadioEntropy bool, default false.

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
2026-04-21 09:57:06 -05:00
Catalin PatuleaandBen Meadors e1f5043489 Delete PointerQueue::dequeuePtrFromISR, unused since commit db766f1 (#99). (#10090) 2026-04-21 09:57:00 -05:00
Jaime RoldanandBen Meadors 5d9a2564e4 fix(nodedb): force null-terminate name fields in UserLite/User conversions (#8174) (#10218) 2026-04-21 09:52:47 -05:00
Jaime RoldanandGitHub 63bce1f01a fix(nodedb): force null-terminate name fields in UserLite/User conversions (#8174) (#10218) 2026-04-21 09:52:19 -05:00
nightjoker7andBen Meadors 84ce1ea147 SX126x: re-apply 0x8B5 register in resetAGC() to preserve RX sensitivity (#10219)
The CALIBRATE_ALL (0x7F) command inside resetAGC() clears bit 0 of the
undocumented 0x8B5 register. That bit is set once in init() by #9571 and
#9777 to improve SX1262 RX sensitivity, and the AGC-reset path was not
re-applying it. Result: every SX1262 node silently loses the RX
sensitivity patch ~60s after boot and never recovers until reboot.

Empirically confirmed on Heltec Mesh Node T114 (nRF52840 + SX1262):
  - Post-calibration read of 0x8B5 = 0x04 (bit 0 cleared)
  - After re-apply: 0x05 (bit 0 set)
Reproducible every AGC_RESET_INTERVAL_MS tick.

Fix re-applies the register bit alongside the existing post-calibration
re-applies (setDio2AsRfSwitch, setRxBoostedGainMode).
2026-04-21 09:50:35 -05:00
nightjoker7andGitHub 4090d9f2b3 SX126x: re-apply 0x8B5 register in resetAGC() to preserve RX sensitivity (#10219)
The CALIBRATE_ALL (0x7F) command inside resetAGC() clears bit 0 of the
undocumented 0x8B5 register. That bit is set once in init() by #9571 and
#9777 to improve SX1262 RX sensitivity, and the AGC-reset path was not
re-applying it. Result: every SX1262 node silently loses the RX
sensitivity patch ~60s after boot and never recovers until reboot.

Empirically confirmed on Heltec Mesh Node T114 (nRF52840 + SX1262):
  - Post-calibration read of 0x8B5 = 0x04 (bit 0 cleared)
  - After re-apply: 0x05 (bit 0 set)
Reproducible every AGC_RESET_INTERVAL_MS tick.

Fix re-applies the register bit alongside the existing post-calibration
re-applies (setDio2AsRfSwitch, setRxBoostedGainMode).
2026-04-21 09:50:01 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
eba74fa6e2 Update GxEPD2 to v1.6.9 (#10212)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-21 06:08:28 -05:00
8627bce1a1 Upgrade trunk (#10125)
Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com>
2026-04-20 04:55:10 -05:00
d50caf231b Add encryption overview to agent instructions in AGENTS.md (#10207)
* Add encryption overview to agent instructions in AGENTS.md

* Update AGENTS.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update copilot-instructions.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update copilot-instructions.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Clarify nonce and wire overhead details in encryption section of copilot instructions

* Enhance encryption documentation in copilot instructions and agents guide for clarity on key management and reset behaviors

* Update .github/copilot-instructions.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix botched merge conflict resolution

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-19 16:05:28 -05:00
f396200d38 Add authoring guide for native unit tests in README.md (#10201)
* Add authoring guide for native unit tests in README.md

* Enhance documentation for agent tooling and native unit tests in README and related files

---------

Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-19 13:30:50 -05:00
Ben Meadors 6c04c37294 Merge remote-tracking branch 'origin/master' into develop 2026-04-19 12:50:12 -05:00
de23e5199d Add USB camera and uhubctl support for new test suite. Also included some bug fixes (#10204)
* Add USB camera and uhubctl support for new test suite. Also added some bug fixes

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Refactor test messages for clarity and consistency in regex tests

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-04-19 06:51:41 -05:00
Ben MeadorsCopilotcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
6b15571e14 Add MCP server for interacting with meshtastic devices and testing framework / TUI (#10194)
* Start of MCP server and test suite

* Add MCP server for interacting with meshtastic devices and testing framework / TUI

* Update mcp-server/README.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix mcp-server review feedback from thread

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/91dc128a-ed50-4d07-8bb2-3dc6623a05f7

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Enhance StreamAPI and PhoneAPI for improved log record handling and concurrency control

* Semgrep fixes

* Trunk and semgrep fixes

* optimize pio streaming tee file writes

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/04e26c6b-6a2b-45be-bbeb-79ae4d0be633

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* chore: remove redundant log handle assignment

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/04e26c6b-6a2b-45be-bbeb-79ae4d0be633

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Consolidate type imports and remove placeholder test files

* Add tests for config persistence and more exchange messages

* Refactor position test to validate on-demand request/reply behavior

* Remove  position request/reply test and update README for telemetry behavior

* Fix transmit history file to get removed on factory reset

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-04-18 11:29:02 -05:00
Ben MeadorsandGitHub 8fd0a7f283 Merge branch 'master' into master 2026-04-18 11:17:03 -05:00
Ben Meadors e589de2d6e Tronk 2026-04-18 11:12:05 -05:00
Ben MeadorsandCopilot a277108c84 Update src/graphics/EInkDisplay2.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-18 11:11:41 -05:00
Ben MeadorsandCopilot 9e26cc3795 Update src/main.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-18 11:11:41 -05:00
Ben MeadorsandCopilot 84bb909437 Update variants/esp32s3/t-deck-pro-v1_1/platformio.ini
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-18 11:11:41 -05:00
Ben MeadorsandCopilot 2beebea453 Update src/platform/extra_variants/t_deck_pro/variant.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-18 11:11:41 -05:00
Ben MeadorsandCopilot d0cd8ec366 Update variants/esp32s3/t-deck-pro-v1_1/variant.h
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-18 11:11:41 -05:00
lewisxheandBen Meadors d31d0f85fe Added compatibility with LilyGo T-Deck-Pro V1.1 2026-04-18 11:11:41 -05:00
Clive BlackledgeandGitHub 34aa5e995b Fix: prompt markdownlint md040 fix for new prompts. (#10199)
* Add ESP32 Power Management lessons learned document

Documents our experimentation with ESP-IDF DFS and why it doesn't
work well for Meshtastic (RTOS locks, BLE locks, USB issues).

Proposes simpler alternative: manual setCpuFrequencyMhz() control
with explicit triggers for when to go fast vs slow.

* docs(prompts): fix markdown fence language tags

* docs: remove ESP32 power management notes
2026-04-18 08:29:30 -05:00
Ben MeadorsGitHubCopilotcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
c8dac10348 Add MCP server for interacting with meshtastic devices and testing framework / TUI (#10194)
* Start of MCP server and test suite

* Add MCP server for interacting with meshtastic devices and testing framework / TUI

* Update mcp-server/README.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix mcp-server review feedback from thread

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/91dc128a-ed50-4d07-8bb2-3dc6623a05f7

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Enhance StreamAPI and PhoneAPI for improved log record handling and concurrency control

* Semgrep fixes

* Trunk and semgrep fixes

* optimize pio streaming tee file writes

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/04e26c6b-6a2b-45be-bbeb-79ae4d0be633

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* chore: remove redundant log handle assignment

Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/04e26c6b-6a2b-45be-bbeb-79ae4d0be633

Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>

* Consolidate type imports and remove placeholder test files

* Add tests for config persistence and more exchange messages

* Refactor position test to validate on-demand request/reply behavior

* Remove  position request/reply test and update README for telemetry behavior

* Fix transmit history file to get removed on factory reset

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-04-18 08:17:44 -05:00
aab4cd086f Compass improvements/refactoring (#10166)
* Infinite calibration loop fix

* Save calibration

* Screen refresh

* reduce repeated code

* reduce repeated code to reduce flash

* fix Waypoint compass size and no fix no heading labels

* Don't show compass unless we have a heading and location

* If no calculated heading from moving, we should have no heading

* Slow walking calculated heading and auto stale heading when not moving

* Triming flash space

* cleanup

* show "?" when no location or heading for distance and heading screen

* cleanup

* Stale heading logic

* final trim

* Compass Calibration screen redesign

* Trunk Fix

* Compile fix

* patch

* Update src/motion/MotionSensor.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update WaypointModule.cpp

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-18 06:53:22 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2c8dec2fbd Update meshtastic/device-ui digest to 56e1da4 (#10195)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-17 19:48:07 -05:00
Jonathan BennettandGitHub d7cb5d7885 Merge branch 'develop' into vibe-coded-dmshell 2026-04-17 12:17:55 -05:00
Jason PandGitHub 6208c243f9 BaseUI: Implementation of Status Message for Favorite and NodeList views (#9504)
* Implementation of Status Message

* Change drawNodeInfo to drawFavoriteNode

* Truncate overflow on Favorite frame

* Set MAX_RECENT_STATUSMESSAGES to 5 to meet memory usage targets
2026-04-17 08:42:56 -05:00
Ben MeadorsandGitHub c48b5ab556 Merge branch 'master' into master 2026-04-17 05:22:30 -05:00
Jonathan BennettandGitHub ffd144da83 Merge branch 'develop' into vibe-coded-dmshell 2026-04-16 22:48:26 -05:00
Jonathan BennettandGitHub 3a87e746a8 No longer need undefines, thanks to #10179 (#10180) 2026-04-16 21:34:28 -05:00
Jonathan BennettandGitHub 1e8c9e7071 Merge branch 'develop' into vibe-coded-dmshell 2026-04-16 21:29:43 -05:00
3c83e01d0e Update protobufs (#10188)
Co-authored-by: jp-bennett <5630967+jp-bennett@users.noreply.github.com>
2026-04-16 21:28:53 -05:00
Ben Meadors cd04206334 Merge branch 'master' into develop 2026-04-16 21:22:17 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
8e55a6e4c1 Update meshtastic/device-ui digest to 5305670 (#10183)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-16 20:19:52 -05:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
f14ef121ef Update meshtastic-st7789 digest to 92bae2e (#10182)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-16 20:19:30 -05:00
92263859a7 Update protobufs (#10184)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
2026-04-16 20:19:17 -05:00
Jonathan BennettandGitHub 2768080edf More cleanly remove LED_BUILTIN (#10179)
* Test PR to remove LED_BUILTIN 

Comment out the LED_BUILTIN definition in platformio.ini

* Add LED_BUILTIN definition to nrf52840.ini
2026-04-16 13:12:31 -05:00
RuledoandGitHub 466cc4cecd Add Luckfox Pico Max Waveshare Pico LoRa config (#10175)
Add a meshtasticd config for the Luckfox Pico Max with the Waveshare Pico LoRa SX1262 TCXO HAT.

Tested on hardware with successful SX1262 init, broadcast, and direct messaging.
2026-04-16 10:41:06 -05:00
fe90c49795 fix/feat(stm32/russell): Serial2 build fix and BME680 support (#10097)
* fix(stm32/russell): define ENABLE_HWSERIAL2 and Serial2 pins

The Russell board variant was missed during Initial serialModule cleanup
(PR #9465), which began requiring Serial2 to be explicitly defined via
ENABLE_HWSERIAL2 and PIN_SERIAL2_TX/RX rather than relying on implicit
defaults, causing a build error.

Signed-off-by: Andrew Yong <me@ndoo.sg>

* feat(stm32/russell): add BME680 support, exclude modules to fit flash

The BME680 is hardware footprint compatible with the BME280 already
present on the Russell board, so add it as an additional lib dep to
enable environment sensing (temperature, humidity, pressure,
gas resistance).

The STM32 target has very limited flash. Even traceroute alone causes
overflow, so the following modules are excluded to stay within budget:

  - RANGETEST
  - DETECTIONSENSOR
  - EXTERNALNOTIFICATION
  - POWERSTRESS
  - NEIGHBORINFO
  - TRACEROUTE
  - WAYPOINT

AIR_QUALITY_SENSOR is also excluded as it requires the BSEC2 library
for real IAQ output, which alone overflows flash by ~44KB on this
target. The Adafruit BME680 library is used instead for raw sensor
readings.

Signed-off-by: Andrew Yong <me@ndoo.sg>

---------

Signed-off-by: Andrew Yong <me@ndoo.sg>
Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-16 10:39:37 -05:00
Ben Meadors 23f92c1cbd Add comprehensive guides for new module, sensor, and hardware variant development 2026-04-16 08:01:03 -05:00
Chloe BethelandChloe Bethel 31418ca821 stm32wl: reserve 2KB of stack via linker script to match NRF52, change sbrkHeadroom to use the start of the reserved stack region instead of the current stack pointer
The linker script was created by merging variants/STM32WLxx/WL54JCI_WL55JCI_WLE4J(8-B-C)I_WLE5J(8-B-C)I/ldscript.ld and system/ldscript.ld from stm32duino/Arduino_Core_STM32.
2026-04-16 13:57:21 +01:00
Andrew YongandChloe Bethel 0ee5777c15 stm32wl(mem): fix getFreeHeap() underreporting on dynamic sbrk heap
mallinfo().fordblks counts only free bytes within the committed arena.
On STM32WL (newlib sbrk heap) the arena grows lazily from _end toward SP,
so fordblks reads near-zero at early boot even when ~48 KB of addressable
space remains. This caused NodeDB::isFull() to fire prematurely and evict
nodes on a freshly booted device.

Fix getFreeHeap() to include uncommitted sbrk headroom (SP - sbrk(0)) so
the returned value reflects true available memory throughout the boot
lifecycle.

Introduce MESHTASTIC_DYNAMIC_SBRK_HEAP as an opt-in build flag (set in
stm32.ini) so the fix is gated to platforms with a dynamic sbrk heap
rather than a static heap. Future platforms with the same heap model can
opt in by adding this flag.

Signed-off-by: Andrew Yong <me@ndoo.sg>
Assisted-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 13:57:21 +01:00
79e7ed30f1 Update src/graphics/EInkDisplay2.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-16 07:49:34 -05:00
d5af07e458 Update src/main.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-16 07:49:08 -05:00
7d957f8c7b Update variants/esp32s3/t-deck-pro-v1_1/platformio.ini
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-16 07:48:50 -05:00
5cae9e0183 Update src/platform/extra_variants/t_deck_pro/variant.cpp
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-16 07:48:35 -05:00
edf660ccb3 Update variants/esp32s3/t-deck-pro-v1_1/variant.h
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-16 07:48:19 -05:00
Ben MeadorsandGitHub e7b66281fa Merge branch 'master' into master 2026-04-16 07:07:03 -05:00
Andrew YongandGitHub 026213aab7 feat(stm32): Add STM32 ADC support to AnalogBatteryLevel (#9369)
Integrate STM32 battery monitoring into AnalogBatteryLevel, supporting
external GPIO ADC pins as well as internal VBAT channel.

Features:
- ADC reading using STM32 LL (Lower Layer) macros supporting external
  ADC channels and internal VBAT channel (AVBAT)
- ADC compensation using STM32 LL macros with factory-calibrated VREFINT
  (AVREF) for accurate voltage measurement
- LFP battery OCV curve for STM32WL using AVBAT (STM32 VDD absolute
  maximum supply voltage 3.9V, direct connection of Li-Po batteries is
  not supported)

Internal VBAT channel implemented in:
- Russell
- RAK3172

In these variants, ADC_MULTIPLIER = (1.01f * 3) = 3.30 as there is a
3:1 internal divider (DS13105 Rev 12 §5.3.21), and a bit of tolerance
as the actual 10% spec leads to readings much too high.


Assisted-by: Claude:sonnet-4-5

Signed-off-by: Andrew Yong <me@ndoo.sg>
2026-04-16 10:58:54 +01:00
renovate[bot]GitHubrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
c1bee82baf Update platform-native digest to 71ed55b (#10165)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-14 21:07:15 -05:00
Jonathan Bennett dc3947117e Make new protobuf value consistent 2026-04-14 18:47:53 -05:00
Jonathan Bennett 87d0850f95 Refactor and Simplify 2026-04-14 17:16:36 -05:00
0cab43fb43 Add PortduinoSetOptions to overwrite the realhardware bool (#10157)
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
2026-04-14 21:32:48 +02:00
47e129f4bd Automated version bumps (#10159)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
2026-04-14 13:13:45 -05:00
1341cd4078 Automated version bumps (#10159)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
2026-04-14 13:11:47 -05:00
Jennifer SanchezandGitHub 4059202a5c Added support for Spreading Factors 5 and 6 on compatible radios (#10160) 2026-04-14 13:11:36 -05:00
Ben MeadorsandCopilot d24d8806e1 Fix heap blowout on TBeams (#10155)
* Fix heap blowout on TBeams

* Update src/graphics/draw/MessageRenderer.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Set MESSAGE_HISTORY_LIMIT to 10 for original ESP32 to optimize RAM usage

* Optimize message frame allocation to prevent excessive memory usage

* Refine message history limits for resource-constrained builds and cap cached lines to prevent heap overflow

* Update src/graphics/draw/MessageRenderer.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-14 13:05:58 -05:00
Thomas GöttgensandBen Meadors a67eb15ad3 fix last cppcheck issue (#10154) 2026-04-14 13:05:58 -05:00
Ben Meadors 9e182a595c Enhance release notes generation with commit range comparison 2026-04-14 13:05:58 -05:00
Jonathan Bennett 5831952636 simplify pt 1 2026-04-14 12:57:29 -05:00
Tom FifieldandGitHub 4587dc2d64 Add RADIOLIB_EXCLUDE_LR2021 in places that excluded LR11x0 (#10112)
To save resources, some devices where LR11x0 was never an option
excluded it from compilation, using RADIOLIB_EXCLUDE_LR11X0 .

As we will soon have LR2021 support, apply the same treatment
to that chip to these devices, by adding RADIOLIB_EXCLUDE_LR2021
2026-04-14 16:55:11 +02:00
00fccd87f9 Update protobufs (#10161)
Co-authored-by: caveman99 <25002+caveman99@users.noreply.github.com>
2026-04-14 15:27:05 +02:00
Andrew YongandGitHub 01bd4cfb73 feat(stm32wl): add reboot-to-bootloader support via enter_dfu_mode_request (#10158) 2026-04-14 13:46:33 +02:00
Jonathan BennettandGitHub 125c1b7f13 Make consoleInit() Reentrant, and initialize it earlier on native (#10156) 2026-04-14 06:41:04 -05:00
Thomas GöttgensandGitHub d6cf67b6bc Clarify behavior when no radio instance is present
Add comment explaining silent behavior when no radio instance is available.
2026-04-14 12:19:58 +02:00
Jonathan Bennett a6d61413c3 Add PortduinoSetOptions to overwrite the realhardware bool 2026-04-13 22:20:38 -05:00
Jonathan Bennett e393a5c410 Make consoleInit() Reentrant, and initialize it earlier on native 2026-04-13 20:56:44 -05:00
Jonathan Bennett 8f2ecbdb4d No Child Left Behind 2026-04-13 20:32:38 -05:00
Jonathan Bennett 6c28d11cee Minor cleanups 2026-04-13 20:23:31 -05:00
Jonathan Bennett 69f1b502cc Harden against possible memory overflows 2026-04-13 20:05:50 -05:00
Jonathan Bennett 3a498fbbe4 Make the DMShell tests compile 2026-04-13 19:47:33 -05:00
Jonathan Bennett 8f7dea0580 Use RemoteShell in protobufs 2026-04-13 19:25:09 -05:00
96dd647882 Fix heap blowout on TBeams (#10155)
* Fix heap blowout on TBeams

* Update src/graphics/draw/MessageRenderer.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Set MESSAGE_HISTORY_LIMIT to 10 for original ESP32 to optimize RAM usage

* Optimize message frame allocation to prevent excessive memory usage

* Refine message history limits for resource-constrained builds and cap cached lines to prevent heap overflow

* Update src/graphics/draw/MessageRenderer.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-13 14:50:51 -05:00
Jonathan Bennett 322f0262a8 Trunk 2026-04-13 12:11:31 -05:00
Jonathan BennettandGitHub 00762393cf Merge branch 'develop' into vibe-coded-dmshell 2026-04-13 12:07:56 -05:00
Thomas GöttgensandGitHub e42ff3590c fix last cppcheck issue (#10154) 2026-04-13 15:48:30 +02:00
Ben Meadors 7527233130 Enhance release notes generation with commit range comparison 2026-04-13 06:43:11 -05:00
197226365b fix(native): implement BinarySemaphorePosix with proper pthread synchronization (#9895)
* fix(native): implement BinarySemaphorePosix with proper pthread synchronization

The BinarySemaphorePosix class (used on all Linux/portduino/native builds)
had stub implementations: give() was a no-op and take() just called
delay(msec) and returned false. This broke the cooperative thread scheduler
on native platforms — threads could not wake the main loop, radio RX
interrupts were missed, and telemetry never transmitted over the mesh.

Replace the stubs with a proper binary semaphore using pthread_mutex_t +
pthread_cond_t + bool signaled:

- take(msec): pthread_cond_timedwait with CLOCK_REALTIME timeout, consumes
  signal atomically (binary semaphore semantics)
- give(): sets signaled=true, signals condition variable
- giveFromISR(): delegates to give(), sets pxHigherPriorityTaskWoken

Tested on Raspberry Pi 3 Model B (ARM64, Debian Bookworm) with Adafruit
LoRa Radio Bonnet (SX1276). Before fix: no radio TX/RX, no telemetry on
mesh. After fix: bidirectional LoRa, MQTT gateway, telemetry all working.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ARCH_PORTDUINO

* Refactor BinarySemaphorePosix header for ARCH_PORTDUINO

* Change preprocessor directive from ifndef to ifdef

* Gate new Semaphore code to Portduino and fix STM compilation

* Binary Semaphore Posix better error handling

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz>
2026-04-13 06:31:38 -05:00
Jonathan BennettandGitHub 866c89f801 Merge branch 'develop' into vibe-coded-dmshell 2026-04-12 22:41:58 -05:00
Jonathan Bennett 8c248927c8 Remove some dead code and LLM overcomplication 2026-04-10 16:01:35 -05:00
Ben MeadorsandGitHub 188d895eb4 Merge branch 'develop' into vibe-coded-dmshell 2026-04-10 07:21:42 -05:00
Jonathan BennettandGitHub 6f476f3475 Merge branch 'develop' into vibe-coded-dmshell 2026-04-09 21:46:10 -05:00
Jonathan Bennett 50e1fe88e8 dmshell client serial support and tweaks 2026-04-09 18:31:29 -05:00
Jonathan Bennett f9bedd8adc DMShell heartbeat 2026-04-08 23:19:02 -05:00
Jonathan Bennett 5a619c9031 Attempt at better responsiveness 2026-04-08 21:15:23 -05:00
Jonathan Bennett 8d3f9222ff Don't firehose missing packets 2026-04-08 17:52:47 -05:00
Jonathan Bennett f5335f22ea troubleshoot dropped packets 2026-04-08 16:42:46 -05:00
Jonathan Bennett 4a3f449555 Try to re-request missed sequences 2026-04-08 15:16:33 -05:00
Jonathan Bennett 608713470b Interactive mode 2026-04-08 14:55:37 -05:00
Jonathan Bennett f475be19c6 Dumb fixes 2026-04-08 14:27:15 -05:00
Jonathan Bennett 27cc76d5ed Very WIP dmshell 2026-04-08 13:18:21 -05:00
lewisxhe 15b474172a Added compatibility with LilyGo T-Deck-Pro V1.1 2026-01-21 17:33:50 +08:00
683 changed files with 56282 additions and 8826 deletions
+49
View File
@@ -0,0 +1,49 @@
# Claude Code slash commands for the mcp-server test suite
Three AI-assisted workflows wrapping `mcp-server/run-tests.sh` and the meshtastic MCP tools. Each one has a twin in `.github/prompts/` for Copilot users.
| Slash command | What it does | Copilot equivalent |
| --------------------- | ------------------------------------------------------------------------- | ---------------------------------------- |
| `/test [args]` | Runs the test suite (auto-detects hardware) and interprets failures | `.github/prompts/mcp-test.prompt.md` |
| `/diagnose [role]` | Read-only device health report via the meshtastic MCP tools | `.github/prompts/mcp-diagnose.prompt.md` |
| `/repro <test> [n=5]` | Re-runs one test N times, diffs firmware logs between passes and failures | `.github/prompts/mcp-repro.prompt.md` |
## Why two surfaces
The Claude Code commands and Copilot prompts cover the same three workflows but each speaks its host's idiom:
- **Claude Code** (`/test`) uses `$ARGUMENTS` for pass-through, has direct access to Bash + all MCP tools registered in the user's settings, and runs in the terminal context.
- **Copilot** (`/mcp-test`) runs in VS Code's agent mode; it has terminal + MCP access too but typically asks the operator to confirm inputs interactively.
A contributor using either IDE gets equivalent assistance. Keep the two in sync when behavior changes — the diff of intent should be minimal.
## House rules
- **No destructive writes without explicit operator approval.** Skills that could reflash, factory-reset, or reboot a device must describe the action and stop — the operator authorizes.
- **Interpret failures, don't just echo them.** The skill body should pull firmware log lines from `mcp-server/tests/report.html` (the `Meshtastic debug` section, attached by `tests/conftest.py::pytest_runtest_makereport`) and classify the failure.
- **Keep MCP tool calls sequential per port.** SerialInterface holds an exclusive port lock; two parallel tool calls on the same port deadlock.
- **Never speculate about root cause.** If the evidence doesn't support a classification, say "unknown" and list what you'd need to disambiguate.
## Adding a new command
1. Write the Claude Code version at `.claude/commands/<name>.md` with YAML frontmatter:
```yaml
---
description: one-line purpose (used for auto-invocation by the model)
argument-hint: [optional-hint]
---
```
2. Write the Copilot equivalent at `.github/prompts/mcp-<name>.prompt.md` with:
```yaml
---
mode: agent
description: ...
---
```
3. Add the row to the table above. Cross-link in both bodies.
4. Smoke-test on Claude Code first (`/<name>` should appear in autocomplete), then in VS Code Copilot (`/mcp-<name>` in Chat).
+68
View File
@@ -0,0 +1,68 @@
---
description: Produce a device health report using the meshtastic MCP tools (device_info, list_nodes, get_config, short serial log capture)
argument-hint: [role=all|nrf52|esp32s3|<port>]
---
# `/diagnose` — device health report
Call the meshtastic MCP tool bundle and format a structured health report for one or all detected devices. Zero guesswork for the operator.
## What to do
1. **Enumerate hardware.** Call `mcp__meshtastic__list_devices(include_unknown=True)`. For each entry where `likely_meshtastic=True`, capture `port`, `vid`, `pid`, `description`.
2. **Filter by `$ARGUMENTS`**:
- No args, `all` → every likely-meshtastic device.
- `nrf52` → only devices with `vid == 0x239a`.
- `esp32s3` → only devices with `vid == 0x303a` or `vid == 0x10c4`.
- A `/dev/cu.*` path → only that one port.
- Anything else → treat as a substring match against the `port` string.
3. **For each selected device, in sequence (NOT parallel — SerialInterface holds an exclusive port lock):**
- `mcp__meshtastic__device_info(port=<p>)` — captures `my_node_num`, `long_name`, `short_name`, `firmware_version`, `hw_model`, `region`, `num_nodes`, `primary_channel`.
- `mcp__meshtastic__list_nodes(port=<p>)` — count of peers, which ones have `publicKey` set, SNR/RSSI distribution.
- `mcp__meshtastic__get_config(section="lora", port=<p>)` — region, preset, channel_num, tx_power, hop_limit.
- Optionally, if the device seems unhappy (fails to connect, `num_nodes==1` when ≥2 are plugged in, missing firmware*version), open a short firmware log window: `mcp__meshtastic__serial_open(port=<p>, env=<inferred-env>)`, wait 3s, `serial_read(session_id=<s>, max_lines=100)`, `serial_close(session_id=<s>)`. The env should be inferred from the VID map in `mcp-server/run-tests.sh` (nrf52 → rak4631, esp32s3 → heltec-v3) unless `MESHTASTIC_MCP_ENV*<ROLE>` is set.
4. **Hub health** (call once, not per-device): `mcp__meshtastic__uhubctl_list()` — enumerates every USB hub the host can see. Note which hubs advertise `ppps=true` and which hub hosts each Meshtastic device (cross-reference by VID). Flag it in the report if:
- No hub advertises PPPS → `tests/recovery/` can't run on this setup; hard-recovery via `uhubctl_cycle` isn't available.
- A Meshtastic device is on a non-PPPS hub → note it; operator may want to move the device to a PPPS hub to unlock auto-recovery.
- `uhubctl_list` raises `ConfigError: uhubctl not found` → just say `uhubctl not installed` in the report; don't treat as a fault.
5. **Render per-device report** as:
```text
[nrf52 @ /dev/cu.usbmodem1101] fw=2.7.23.bce2825, hw=RAK4631
owner : Meshtastic 40eb / 40eb
region/band : US, channel 88, LONG_FAST
tx_power : 30 dBm, hop_limit=3
peers : 1 (esp32s3 0x433c2428, pubkey ✓, SNR 6.0 / RSSI -24 dBm)
primary ch : McpTest
hub : 1-1.3 port 2 (PPPS, uhubctl-controllable)
firmware : no panics in last 3s; NodeInfoModule emitted 2 broadcasts
```
Keep it scannable. If a field is missing or abnormal (no pubkey for a known peer, region=UNSET, num_nodes inconsistent with the hub, device on non-PPPS hub), flag it inline with a short `⚠︎ <one-line reason>`.
6. **Cross-device correlation** (only when >1 device is inspected):
- Do both sides see each other in `nodesByNum`? If one does and the other doesn't, that's asymmetric NodeInfo — flag it.
- Do the LoRa configs match? (region, channel_num, modem_preset should all agree; mismatch = no mesh)
- Do the primary channel NAMES match? Mismatch = different PSK = no decode.
7. **Recorder slice (cheap, always available).** The mcp-server runs an autouse log recorder that's been collecting from every connected device. Pull two short slices to surface anything weird that's already happened:
- `mcp__meshtastic__logs_window(start="-2m", level="WARN|ERROR|CRIT", max_lines=20)` — recent firmware errors. If empty, say "no recent errors"; don't manufacture concern.
- `mcp__meshtastic__telemetry_timeline(window="1h", field="free_heap", max_points=60)` — heap trend. If `slope_per_min < -50`, flag it and recommend `/leakhunt window=6h` for a deeper read; otherwise just note the current free heap.
- If `recorder_status` shows `running:false` or `files.telemetry.last_ts` is null, note "recorder has no telemetry yet — enable `set_debug_log_api(True)` to populate" and skip this step gracefully.
8. **Suggest next actions only for specific, recognisable failure modes**:
- Stale PKI pubkey one-way → "run `/test tests/mesh/test_direct_with_ack.py` — the retry + nodeinfo-ping heals this in the test path."
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
- Device unreachable, reachable via DFU → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds AND the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
- CP2102-wedged-driver on macOS → see the note in `run-tests.sh`.
- Heap slope strongly negative → "run `/leakhunt window=6h` for a full timeline + classification."
## What NOT to do
- No writes. No `set_config`, no `reboot`, no `factory_reset`. This is a read-only diagnostic skill — if the operator wants to change state, they'll ask explicitly.
- No `flash` / `erase_and_flash`. Those are separate escalations.
- No holding SerialInterface across tool calls — open, query, close; next device. The port lock is exclusive.
+103
View File
@@ -0,0 +1,103 @@
---
description: Hunt for memory leaks (and other slow degradations) by reading the persistent recorder's heap timeline + log slice over a window
argument-hint: [window=1h] [field=free_heap] [variant=local]
---
<!-- markdownlint-disable MD029 -->
# `/leakhunt` — read the recorder, classify a memory leak
Use the always-on recorder (`mcp-server/.mtlog/`) to read a heap timeline plus the matching log slice and produce a one-page verdict: **steady / slow leak / fragmentation / OOM-imminent**. No firmware changes, no special build flags — the LocalStats telemetry packet that the firmware already broadcasts every ~60 s carries `heap_free_bytes` and `heap_total_bytes`.
## Two signal paths — pick the right one
| Path | Build flag | Cadence | Per-thread attribution | Cost |
| --------------------- | ---------------- | -------------- | ---------------------- | ------------------------- |
| LocalStats packet | (default) | ~60 s | No | Free — always on |
| `[heap N]` log prefix | `-DDEBUG_HEAP=1` | every log line | Yes (Thread X leaked) | Bigger flash + log volume |
Both feed the same `telemetry_timeline(field="free_heap")` query — when DEBUG_HEAP is on, the recorder synthesizes telemetry rows from log prefixes (tagged `source: debug_heap`), so a single timeline call gets whichever signal is available. **For a slow leak diagnosis, the default path is plenty** (60 s cadence over 6 h = 360 points; linear regression over that nails sub-100-byte/min slopes). **DEBUG_HEAP is for attribution** — when the slope is real and you need to know which thread is leaking.
## What to do
1. **Parse `$ARGUMENTS`**: optional `window` (default `1h`, accepts `30m`/`6h`/`-3d`/etc.), optional `field` (default `free_heap`; alternates: `total_heap`, `battery_level`, anything in the LocalStats variant), optional `variant` (default `local`; alternates: `device`, `environment`, `power`, `airQuality`, `health`).
2. **Verify the recorder is alive** — call `mcp__meshtastic__recorder_status`. Check:
- `running == True`
- `files.telemetry.lines > 0` (at least one telemetry packet recorded — if zero, the device hasn't broadcast LocalStats yet OR `set_debug_log_api` has never been on; tell the operator to run `mcp__meshtastic__set_debug_log_api(enabled=True)` and wait one device-update interval)
- `files.telemetry.last_ts` within the last 5 minutes (if older, the device is silent — log that, not "leak detected")
3. **Detect whether DEBUG_HEAP is active**`mcp__meshtastic__logs_window(start="-2m", grep=r"\\[heap \\d+\\]", max_lines=3)`. If any line matches, the firmware has the prefix → DEBUG_HEAP is on, expect higher-cadence data and `heap_event` rows. If zero matches over the last 2 minutes, you're on the LocalStats-only path.
4. **Pull the timeline**`mcp__meshtastic__telemetry_timeline(window=$window, variant=$variant, field=$field, max_points=200)`. Read:
- `samples` — how many raw points contributed
- `min`, `max` — total swing
- `slope_per_min` — units per minute (linear regression over the whole window)
5. **Pull the log context for the same window**`mcp__meshtastic__logs_window(start="-${window}", grep="Heap status|leaked heap|freed heap|out of memory|Alloc an err|panic|abort", max_lines=200)`. These are the strings the firmware emits when something memory-related happens (`DEBUG_HEAP` builds emit `"Heap status:"` and `"leaked heap"` lines; production builds emit `"Alloc an err"` on failure and `"out of memory"` on OOM).
6. **Pull marker events** so we know if the operator labeled phases — `mcp__meshtastic__events_window(start="-${window}", kind="mark|connection_lost|connection_established")`. If a `connection_lost` overlaps a sharp drop, that's not a leak; that's a reboot.
6a. **(DEBUG_HEAP only) Per-thread attribution** — `mcp__meshtastic__logs_window(start="-${window}", grep="leaked heap", max_lines=200)`. Each row has a structured `heap_event` field with `{kind, thread, before, after, delta}`. Aggregate by thread: sum the `delta` over the window per thread name. The thread with the largest cumulative negative delta is your suspect. Note the count too — a thread with 50× small leaks is different from 1× big leak.
7. **Classify** based on what the data says, NOT on what you wish it said. Use these rules in order:
- **Insufficient data** (< 5 samples): say so. Suggest a longer window or longer wait. Stop.
- **Reboot mid-window**: if any `connection_lost` event is present AND `free_heap` jumped UP at that timestamp, the device rebooted. Note it; pre-reboot trend may be a leak but you only have part of the curve.
- **OOM-imminent**: any `Alloc an err=` or `out of memory` line in the log slice. This trumps everything; flag urgently.
- **Slow leak**: `slope_per_min < -50` AND `max - min > 1000` AND no reboot. The heap is monotonically (or near-monotonically) declining. Estimate time-to-zero: `min / -slope_per_min` minutes. Surface it.
- **Fragmentation suspect**: `slope_per_min` close to zero (|x| < 50) BUT min trends down across the window AND the log slice shows `Alloc an err` warnings WITHOUT total OOM. Means free total is OK but largest contiguous block is shrinking. Recommend a `DEBUG_HEAP` build to confirm.
- **Steady**: |slope_per_min| < 50, no error lines. Heap is fine.
- **Recovery curve**: slope is POSITIVE — heap recovered. Either a workload completed or GC fired. Note it; not a leak.
8. **Report**:
```text
/leakhunt window=6h field=free_heap variant=local
────────────────────────────────────────────────────
recorder : running, telem last_ts 8s ago
build : DEBUG_HEAP=ON (per-line prefix detected)
samples : 14,200 over 6h (cadence ~1.5s, log-line synth)
free_heap : min 92,344 / max 124,008 / range 31,664
slope : -82 bytes/min (negative — heap declining)
reboots : none in window
OOM events : none
error lines : 3× "Alloc an err=ESP_ERR_NO_MEM" at +4h12m, +5h08m, +5h44m
thread leaks : (DEBUG_HEAP) MeshPacket -3,124 B over 18 events
Router -1,408 B over 4 events
others -240 B
verdict : SLOW LEAK — primary suspect MeshPacket thread
est. time-to-OOM: ~1,127 min (~18.8 h) at current slope
evidence : (3 log line citations with uptimes)
```
Then: **what to do next.**
- SLOW LEAK, **DEBUG_HEAP off** → recommend rebuilding with the flag and re-running this skill. Concrete one-liner the operator can copy:
```text
mcp__meshtastic__build(env="<env>", build_flags={"DEBUG_HEAP": 1})
mcp__meshtastic__pio_flash(env="<env>", port="<port>", confirm=True)
```
After flash, set debug_log_api back on and wait one window; re-run `/leakhunt`.
- SLOW LEAK, **DEBUG_HEAP on** → cite the top-leaking thread name from step 6a. Point at the corresponding source file (`grep -rn "ThreadName(\"<name>\")" src/`); the operator decides what to fix.
- FRAGMENTATION SUSPECT → propose pre-allocating any per-packet buffers; or rebuilding with `CONFIG_HEAP_TASK_TRACKING=y` on ESP32 to see who's holding the largest blocks.
- OOM-IMMINENT → flag for immediate attention; don't wait for the next telemetry interval.
- STEADY → say so; stop. Don't invent problems.
## What NOT to do
- Don't assume a leak from a single dip. LocalStats fires every ~60 s and the firmware naturally allocates+frees on each broadcast cycle; one packet sees the trough. Look at the slope, not the deltas.
- Don't recommend code changes. This skill diagnoses; the operator decides what to fix.
- Don't enable `set_debug_log_api` automatically — if it's off, telemetry isn't reaching pubsub anyway, and the recorder will be empty. Tell the operator to flip it on and wait, then re-run.
- Don't run heavy workloads to "trigger the leak." The recorder is passive; we read what's there.
## Companion: `mark_event` for stress runs
If the operator wants to test under stimulus (e.g. blast 50 broadcasts and see what the heap does), they can frame the experiment with markers:
```text
mark_event("burst-start")
… run the workload …
mark_event("burst-end")
/leakhunt window=15m
```
The markers land in both `events.jsonl` and `logs.jsonl`, so the report can show "free_heap dipped 8 KB during the burst window, recovered to baseline within 2 LocalStats cycles" → not a leak.
+70
View File
@@ -0,0 +1,70 @@
---
description: Re-run a specific test N times in isolation to triage flakes, diff firmware logs between passes and failures
argument-hint: <test-node-id> [count=5]
---
<!-- markdownlint-disable MD029 -->
# `/repro` — flakiness triage for one test
Re-run a single pytest node ID N times in isolation, track pass rate, and surface what's _different_ in the firmware logs between the passing attempts and the failing ones. Turns "it's flaky, I guess" into "it fails when X, passes when Y."
## What to do
1. **Parse `$ARGUMENTS`**: first token is the pytest node id (e.g. `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip[nrf52->esp32s3]`); second token is an integer count (default `5`, cap at `20`). If the first token doesn't look like a test path (no `::` and no `tests/` prefix), treat the whole `$ARGUMENTS` as a `-k` filter instead.
2. **Sanity-check the hub first** (so we're not measuring "nothing plugged in" N times): call `mcp__meshtastic__list_devices`. If the test name contains `nrf52` or `esp32s3` and the matching VID isn't present, stop and report — re-running won't help.
3. **Loop N times**. For each iteration:
```bash
./mcp-server/run-tests.sh <test-id> --tb=short -p no:cacheprovider
```
Capture: exit code, duration, and (on failure) the `Meshtastic debug` firmware log section from `mcp-server/tests/report.html`. `-p no:cacheprovider` suppresses pytest's `.pytest_cache` writes so iterations don't influence each other.
4. **Track a small structured tally**:
```text
attempt 1: PASS (42s)
attempt 2: FAIL (128s) ← firmware log 200-line tail captured
attempt 3: PASS (39s)
attempt 4: FAIL (121s)
attempt 5: PASS (41s)
--------------------------------------
pass rate: 3/5 (60%) | mean duration: 74s
```
5. **On mixed outcomes**: diff the firmware log tails between a representative passing attempt and a representative failing attempt. Focus on:
- Error-level lines only present in failures (`PKI_UNKNOWN_PUBKEY`, `Alloc an err=`, `Skip send`, `No suitable channel`)
- Timing around the assertion event — did a broadcast go out, was there an ACK, did NAK fire?
- Device state fields that changed (nodesByNum entries, region/preset, channel_num)
Surface the top 3 differences as a "passes when / fails when" table. Don't dump full logs — pull specific lines with uptime timestamps.
5a. **Archive recorder slices per attempt** (no extra device interaction; the recorder runs autouse). Right after each attempt finishes, capture its `(start_ts, end_ts)` and call `mcp__meshtastic__recorder_export(start=<start>, end=<end>, dest_dir="mcp-server/tests/repro_artifacts/<safe-test-id>/attempt_<n>/")`. This drops a `logs.jsonl`, `telemetry.jsonl`, `packets.jsonl`, and `events.jsonl` snapshot scoped to the attempt window. Use these for cross-attempt diffs in step 5: `jq '.line' logs.jsonl` is faster than re-running the test, and the telemetry slice lets you compare heap behavior across attempts.
6. **Classify the flake** into one of:
- **LoRa airtime collision** → pass rate improves with fewer concurrent transmitters; propose a `time.sleep` gap or retry bump in the test body.
- **PKI key staleness** → fails on first attempt, passes after self-heal; existing retry loop in `test_direct_with_ack.py` handles this.
- **NodeInfo cooldown** → `Skip send NodeInfo since we sent it <600s ago` in fail-only logs; needs `broadcast_nodeinfo_ping()` warmup.
- **Hardware-specific** (one direction fails, other passes; one device's firmware is older; driver wedged) → specific recovery pointer. For a device that's wedged past `touch_1200bps`, the next escalation is `uhubctl_cycle(role=..., confirm=True)` to hard-power-cycle its hub port (requires `uhubctl` installed).
- **Device went dark mid-run** → fails from some attempt onward, never recovers, firmware log stops arriving. Almost always hardware: a Guru crash + frozen CDC. Hard-power-cycle via `uhubctl_cycle(role=..., confirm=True)` before the next iteration; if that also fails, escalate to replug.
- **Genuinely unknown** → say so; don't invent a root cause.
7. **Report back** with:
- Pass rate and mean duration.
- Classification + evidence (the specific log lines that support it).
- A suggested next step (re-run with specific args, open `/diagnose`, edit a specific test file, nothing).
## Examples
- `/repro tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip[esp32s3->nrf52] 10` — runs 10 times, diffs firmware logs.
- `/repro broadcast_delivers` — no `::`, no `tests/`, so interpreted as `-k broadcast_delivers`; runs every matching test the default 5 times.
- `/repro tests/telemetry/test_device_telemetry_broadcast.py 3` — shorter run for a slow test.
## Constraints
- Don't exceed `count=20` per invocation — airtime and USB wear add up. If the user asks for 50, negotiate down.
- Don't rebuild firmware as part of triage; flakes that only reproduce under different firmware belong in a separate session.
- If the FIRST attempt fails AND the rest all pass, that's a classic "state leak from a prior test" → say so and suggest running with `--force-bake` or starting from a clean state rather than chasing the first failure.
+47
View File
@@ -0,0 +1,47 @@
---
description: Run the mcp-server test suite (auto-detects devices) and interpret the results
argument-hint: [pytest-args]
---
# `/test` — mcp-server test runner with interpretation
Run `mcp-server/run-tests.sh` and make sense of the output so the operator doesn't have to.
## What to do
1. **Invoke the wrapper.** From the firmware repo root, run:
```bash
./mcp-server/run-tests.sh $ARGUMENTS
```
The wrapper auto-detects connected Meshtastic devices, maps each to its PlatformIO env, exports the required `MESHTASTIC_MCP_ENV_*` env vars, and invokes pytest. If the user passed no arguments, the wrapper supplies a sensible default set (`tests/ --html=tests/report.html --self-contained-html --junitxml=tests/junit.xml -v --tb=short`). A `--report-log=tests/reportlog.jsonl` arg is always appended (unless the operator passed their own). `--assume-baked` is deliberately NOT in the defaults — `test_00_bake.py` has its own skip-if-already-baked check and runs the ~8 s verification by default. Operators can opt into the fast path with `--assume-baked`, or force a reflash with `--force-bake`.
2. **Read the pre-flight header.** First ~6 lines print the detected hub (role → port → env). If that line reads `detected hub : (none)`, the wrapper will narrow to `tests/unit` only — say so explicitly in your summary so the operator knows hardware tiers were skipped.
3. **On pass**: one-line summary of the form `N passed, M skipped in <duration>`. Don't enumerate the test names — the user can read those. Do mention any SKIPPED tests and name the cause:
- `"role not present on hub"` → device unplugged; operator knows to reconnect.
- `"firmware not baked with USERPREFS_UI_TEST_LOG"` → tests/ui skipped because the macro isn't in firmware yet; suggest `--force-bake`.
- `"uhubctl not installed"` → tests/recovery + peer-offline skipped; suggest `brew install uhubctl` / `apt install uhubctl`.
- `"no PPPS-capable hubs detected"` → tests/recovery skipped because the hub doesn't support per-port power; the tier will never run on that setup.
- `"opencv-python-headless is not installed"` → tests/ui auto-deselected by run-tests.sh; suggest `pip install -e 'mcp-server/.[ui]'`.
4. **On failure**: for every FAILED test, open `mcp-server/tests/report.html` and extract the `Meshtastic debug` section for that test. pytest-html embeds the firmware log stream + device state dump there; the 200-line firmware log tail is usually enough to explain the failure. Summarise: which test, one-line assertion message, the firmware log lines that matter (things like `PKI_UNKNOWN_PUBKEY`, `Skip send NodeInfo`, `Error=`, `Guru Meditation`, `assertion failed`). For UI-tier failures also glance at `mcp-server/tests/ui_captures/<session>/<test>/transcript.md` — it records each step's frame + OCR.
5. **Classify the failure** as one of:
- **Transient/flake**: LoRa collision, timing-sensitive assertion, first-attempt NAK + successful retry pattern. Propose `/repro <test_node_id>` to confirm.
- **Environmental**: device unreachable, port busy, CP2102 driver wedged. Suggest the specific recovery in escalation order: (a) replug USB, (b) `touch_1200bps(port=...)` + `pio_flash` for nRF52 DFU, (c) `uhubctl_cycle(role="nrf52", confirm=True)` when a device is fully wedged past DFU (needs `uhubctl` installed — `baked_single`'s auto-recovery hook does this once automatically). Also check `git status userPrefs.jsonc`.
- **Regression**: same assertion fails repeatedly, firmware log shows a new/unusual error. Surface the diff between expected and observed, identify the module likely responsible.
6. **Never run destructive recovery automatically.** If a failure looks like it needs a reflash, factory*reset, `uhubctl_cycle`, or USB replug, \_describe what to do* — don't execute. The operator decides.
## Arguments handling
- No args → wrapper's defaults (full suite).
- `$ARGUMENTS` passed verbatim to the wrapper, which passes them to pytest.
- Common operator invocations: `/test tests/mesh`, `/test tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip`, `/test --force-bake`, `/test -k telemetry`.
## Side-effects to mention in summary
- The session fixture snapshots `userPrefs.jsonc` at session start and restores at teardown (plus on `atexit`). After a clean run, `git status userPrefs.jsonc` should be empty. If the wrapper's pre-flight printed a warning about a stale sidecar, call that out — means a prior session crashed.
- `mcp-server/tests/report.html` and `junit.xml` are regenerated on every run; the HTML is self-contained (shareable).
+7 -3
View File
@@ -8,17 +8,21 @@
"features": {
"ghcr.io/devcontainers/features/python:1": {
"installTools": true,
"version": "3.14"
"version": "3.13"
}
},
"customizations": {
"vscode": {
"extensions": [
"ms-vscode.cpptools",
"platformio.platformio-ide",
"Jason2866.esp-decoder",
"pioarduino.pioarduino-ide",
"Trunk.io"
],
"unwantedRecommendations": ["ms-azuretools.vscode-docker"],
"unwantedRecommendations": [
"ms-azuretools.vscode-docker",
"platformio.platformio-ide"
],
"settings": {
"extensions.ignoreRecommendations": true
}
+4 -4
View File
@@ -75,11 +75,11 @@ body:
- type: checkboxes
id: mui
attributes:
label: Is this bug report about any UI component firmware like InkHUD or Meshtatic UI (MUI)?
label: Is this bug report about any UI (https://meshtastic.org/docs/configuration/device-uis/) component firmware?
options:
- label: Meshtastic UI aka MUI colorTFT
- label: InkHUD ePaper
- label: OLED slide UI on any display
- label: Meshtastic UI aka MUI
- label: InkHUD
- label: BaseUI
- type: input
id: version
+514 -27
View File
@@ -4,15 +4,16 @@ This document provides context and guidelines for AI assistants working with the
## Project Overview
Meshtastic is an open-source LoRa mesh networking project for long-range, low-power communication without relying on internet or cellular infrastructure. The firmware enables text messaging, location sharing, and telemetry over a decentralized mesh network.
Meshtastic is an open-source LoRa mesh networking project for long-range, low-power communication without relying on internet or cellular infrastructure. The firmware enables text messaging, location sharing, and telemetry over a decentralized mesh network. The project uses **C++17** as its language standard across all platforms.
### Supported Hardware Platforms
- **ESP32** (ESP32, ESP32-S3, ESP32-C3) - Most common platform
- **ESP32** (ESP32, ESP32-S3, ESP32-C3, ESP32-C6) - Most common platform
- **nRF52** (nRF52840, nRF52833) - Low power Nordic chips
- **RP2040/RP2350** - Raspberry Pi Pico variants
- **STM32WL** - STM32 with integrated LoRa
- **Linux/Portduino** - Native Linux builds (Raspberry Pi, etc.)
- **macOS native** - Headless `meshtasticd` on Apple Silicon / x86_64; see `variants/native/portduino/platformio.ini` for Homebrew prereqs + CH341 LoRa setup
### Supported Radio Chips
@@ -70,6 +71,163 @@ PKI (Public Key Infrastructure) messages have special handling:
- Accepted on a special "PKI" channel
- Allow encrypted DMs between nodes that discovered each other on downlink-enabled channels
## Encryption & Key Management
Meshtastic packets on the air are typically encrypted one of two ways: the **per-channel symmetric** layer (AES-CTR with a shared PSK) for broadcasts and channel traffic, and the **per-peer PKI** layer (X25519 ECDH → AES-256-CCM) for direct messages and remote admin. A channel with a 0-byte PSK (or Ham mode, which wipes PSKs) transmits cleartext — see the size table below. Both are implemented in `src/mesh/CryptoEngine.cpp`; the send/receive dispatch lives in `src/mesh/Router.cpp`; admin authorization lives in `src/modules/AdminModule.cpp`.
### High-level model
- **Channels** are symmetric rooms: anyone with the PSK can read any message on the channel. Channel 0 is the "primary" channel and ships with the short-form default PSK on factory devices, forming the public mesh most users join. (The LoRa modem preset `LONG_FAST` lives on `config.lora.modem_preset` and is an independent field — don't conflate "channel 0 default PSK" with the modem preset name.)
- **DMs** addressed to a single node require PKI so that other holders of the channel PSK can't read them. Outside Ham mode, Meshtastic does not fall back to channel-symmetric encryption when the destination public key is unknown.
- **Remote admin** is a DM carrying an `AdminMessage`. The receiver only acts on it if the sender's public key is on its allowlist (`config.security.admin_key[0..2]`).
- **Ham mode** (`owner.is_licensed=true`, where `owner` is the local `meshtastic_User` record) disables PKI entirely and sends cleartext — FCC Part 97 prohibits encryption on amateur bands.
- **No ratchet, no session.** Every packet is encrypted from scratch — a stateless design that matches the high-loss, store-and-forward nature of LoRa.
### Symmetric channel encryption (AES-CTR)
`CryptoEngine::encryptPacket` / `decrypt` / `encryptAESCtr` in `src/mesh/CryptoEngine.cpp`.
- **Cipher**: AES-CTR, AES-128 or AES-256 depending on key length. Same routine in both directions (CTR is a stream cipher, so encrypt == decrypt).
- **Key**: `ChannelSettings.psk` bytes. Size semantics:
- **0 bytes** → no encryption, cleartext on the air
- **1 byte** → short-form index into the well-known `defaultpsk[]` in `src/mesh/Channels.h`. Index 0 = cleartext; 1 = defaultpsk unchanged; 2..255 = defaultpsk with its last byte incremented by (index 1). This is what the CLI's `--ch-set psk default` produces.
- **16 bytes** → raw AES-128 key
- **32 bytes** → raw AES-256 key
- **2..15 bytes** → zero-padded to 16 and used as AES-128 (with a warn log); **17..31 bytes** → zero-padded to 32 and used as AES-256 (with a warn log). Defensive fallback for malformed PSK input, not something to rely on.
- **Nonce (128 bit)**: `packet_id` (u64 LE) ‖ `from_node` (u32 LE) ‖ `block_counter` (u32, starts at 0). Built in `CryptoEngine::initNonce`.
- **No AEAD**: channel packets carry no MAC, so the channel-hash byte is not an integrity or authenticity check. `Channels::getHash` is a 1-byte XOR-derived hint over the channel name bytes and PSK bytes that helps receivers pick a candidate channel/PSK for decryption. Because it is only a small hint and collisions are easy to find, it should be described purely as a PSK-selection aid, not as a security filter an attacker cannot bypass.
- **Channel 0 is special in one way only**: it's the channel the Router attempts PKI decryption on before falling through to AES-CTR. Non-zero channels always go straight to AES-CTR.
### PKI encryption for DMs (X25519 ECDH + AES-256-CCM)
`CryptoEngine::encryptCurve25519` / `decryptCurve25519` in `src/mesh/CryptoEngine.cpp`.
- **Keypair**: Curve25519 (aka X25519), 32-byte public + 32-byte private. Stored in `config.security.public_key` / `private_key`; the public half is mirrored into `owner.public_key` so it rides along in NodeInfo broadcasts and propagates through the mesh like any other identity field.
- **Key generation** (`generateKeyPair`): stirs `HardwareRNG::fill()` (64 B from platform TRNG when available), the 16-byte `myNodeInfo.device_id`, and a call to `random()` into the rweather/Crypto library's software RNG, then `Curve25519::dh1`. `regeneratePublicKey` recomputes the public half from a known private (used when restoring from backup).
- **Keygen entry points**: at boot, `NodeDB` calls `generateKeyPair` (or `regeneratePublicKey` when a stored private key is present and passes a low-entropy check) **directly** when `!owner.is_licensed` and `config.lora.region != UNSET`. `ensurePkiKeys` wraps the same logic for runtime/admin flows — it's the path `AdminModule::handleSetConfig` runs when first assigning a valid region or when security config is written; **do not assume it's the universal boot-time gate**, because the NodeDB path bypasses it.
- **Handshake**: `Curve25519::dh2(local_private, remote_public) → 32-byte shared secret → SHA-256 → 32-byte AES-256 key`. Recomputed per packet. The SHA-256 step is effectively a KDF over the raw ECDH output.
- **Cipher**: AES-256-CCM via `aes_ccm_ae` / `aes_ccm_ad` (`src/mesh/aes-ccm.cpp`). MAC length (the `M` parameter) is **8 bytes**. No AAD — the MAC covers ciphertext only.
- **Nonce (13 bytes / 104 bit)**: `aes_ccm_ae`/`aes_ccm_ad` use a 13-byte CCM nonce (`L = 2` is hardcoded in `src/mesh/aes-ccm.cpp`), not a 16-byte nonce. For PKI packets, `CryptoEngine::initNonce(fromNode, packetNum, extraNonce)` starts from the usual packet-derived nonce material, then overwrites nonce bytes `4..7` with a fresh 32-bit `extraNonce = random()`. The effective nonce bytes are therefore: bytes `0..3` = `packet_id`, bytes `4..7` = transmitted `extraNonce`, bytes `8..11` = `from_node`, byte `12` = `0x00`. The receiver reconstructs the same 13-byte nonce from the packet metadata plus the appended `extraNonce`.
- **Wire overhead**: 12 bytes appended to the ciphertext = 8-byte MAC ‖ 4-byte extraNonce. Defined as `MESHTASTIC_PKC_OVERHEAD = 12` in `src/mesh/RadioInterface.h`. Only the 4-byte `extraNonce` is sent; the rest of the 13-byte CCM nonce is reconstructed from packet fields as described above. The Router's send path checks this overhead against `MAX_LORA_PAYLOAD_LEN` before committing to PKI.
- **Send selection** (`Router::send`): the sender enters the PKI path when **all** hold — we're the originator AND not Ham mode AND not Portduino simradio AND not on the `serial`/`gpio` channels (unless the packet is already marked `pki_encrypted`) AND `config.security.private_key.size == 32` AND destination is a single node (not broadcast) AND the portnum isn't infrastructure. `TRACEROUTE_APP`, `NODEINFO_APP`, `ROUTING_APP`, and `POSITION_APP` are routed through channel encryption even when DMed (these need to be readable by relaying peers). Once on the PKI path, if the destination's public key isn't in our NodeDB the send **fails** with `PKI_SEND_FAIL_PUBLIC_KEY` — it does not silently fall back to channel encryption. If the client explicitly set `pki_encrypted=true` and any condition blocks PKI, the send fails with `PKI_FAILED`.
- **Receive selection** (`Router::perhapsDecode`): try PKI decrypt first when `channel == 0` AND `isToUs(p)` AND not broadcast AND both peers have public keys in NodeDB AND `rawSize > MESHTASTIC_PKC_OVERHEAD`. On success the packet gets `pki_encrypted=true` stamped and the sender's public key copied into `p->public_key` for downstream authorization.
### Remote admin authorization
Implemented in `src/modules/AdminModule.cpp``handleReceivedProtobuf`. The authorization check runs in this order:
1. **Response messages** — if `messageIsResponse(r)` is true (the payload is a response to one of our earlier admin requests), it's accepted without any further check. The in-file comment flags this as a known-untightened gap: a stricter implementation would remember which `public_key` we last queried and reject responses that don't match.
2. **Local admin**`mp.from == 0` (phone app over BLE, serial CLI, internal module); never travels over the air. **Rejected** if `config.security.is_managed` is true, because managed devices expect admin to arrive over the air through an authorized remote path.
3. **Legacy admin channel (deprecated)** — the packet arrived on a channel named literally `"admin"`. Gated by `config.security.admin_channel_enabled`; returns `NOT_AUTHORIZED` if the flag is false. Kept for backward compatibility; new deployments should use PKI admin.
4. **PKI admin (preferred for remote)**`mp.pki_encrypted == true` AND `mp.public_key` matches one of `config.security.admin_key[0..2]` (up to three authorized 32-byte Curve25519 public keys, typically copied from the admin node's own `user.public_key`).
5. **Fallthrough**`NOT_AUTHORIZED`.
On top of authorization, any remote admin message that **mutates** state (not a request, not a response) also has to pass a session-key check (`checkPassKey`): the client must first pull a fresh 8-byte `session_passkey` via `get_admin_session_key_request`, then echo that passkey back in the mutating message. The device rotates the passkey after 150 s and rejects values older than 300 s — a narrow anti-replay window on top of the PKI layer.
`config.security.is_managed = true` disables **local** admin writes (`mp.from == 0` is rejected). It does not by itself force every admin action through PKI — the legacy `"admin"` channel still authorizes remote admin when `config.security.admin_channel_enabled == true`. The AdminModule refuses to persist `is_managed=true` unless at least one `admin_key` is populated — a deliberate guard against operators locking themselves out.
### Key-rotation hazards (actions that invalidate peers)
- **`factory_reset_device`** (the "full" variant, calls `NodeDB::factoryReset(eraseBleBonds=true)`) → **wipes** the X25519 private key; a fresh keypair is generated on the next region-set. Every existing peer holds the old public key, so DMs to this node silently fail PKI decrypt until every peer re-exchanges NodeInfo.
- **`factory_reset_config`** (the "partial" variant, calls `NodeDB::factoryReset()` with `eraseBleBonds=false`) → **preserves** the X25519 private key in `installDefaultConfig(preserveKey=true)`; the public key is zeroed and gets rebuilt from the preserved private key on the next boot via the NodeDB path's `regeneratePublicKey` call. Identity is preserved and the mesh does not need to re-exchange keys.
- **`region=UNSET → valid region`** → `ensurePkiKeys` runs inside the same `handleSetConfig` path; missing keys get generated at that moment.
- **Ham mode transitions** — entering Ham mode (`user.is_licensed=true`) runs `Channels::ensureLicensedOperation`, which **wipes every channel PSK** (all traffic becomes cleartext) and disables the legacy admin channel. The X25519 private key is preserved on the device but not used because `Router::send` skips PKI when `owner.is_licensed` is true. Leaving Ham mode re-enables PKI with the preserved keypair but does not restore the wiped channel PSKs — the operator has to re-set them.
- **Channel 0 PSK change** → every peer must re-learn the channel hash; cached NodeInfo becomes temporarily unreachable until the next broadcast.
- **`security.private_key` blanked via admin** → regenerates both halves (unless in Ham mode) and propagates the new public key via NodeInfo.
## NodeDB Layout (v25)
`DEVICESTATE_CUR_VER = 25`, `DEVICESTATE_MIN_VER = 24`. The on-device NodeDB was split in v25 into a slim header table plus four optional satellite stores. Older v24 saves auto-migrate at boot. Old training-data instincts (`node->user.long_name`, `node->position.latitude_i`, `node->is_favorite`, `node->device_metrics.battery_level`) are wrong now — the fields aren't there. Read this section before touching anything that walks `nodeDB->meshNodes`.
### Slim `NodeInfoLite`
`UserLite` is flattened onto `NodeInfoLite` (no nested sub-message); `position` and `device_metrics` are removed entirely (tags reserved). MAC address is dropped. Long names are capped at 25 chars (`max_size:25` in `deviceonly.options`); `hw_model` and `role` are `int_size:8`. Encoded size dropped from ~166 B → ~105 B per node.
Booleans are bit-packed into `NodeInfoLite.bitfield`. **Do not read or write the bits directly** — use the inline helpers in `src/mesh/NodeDB.h`:
```cpp
nodeInfoLiteHasUser(n) // bit 5 — user fields populated
nodeInfoLiteIsFavorite(n) // bit 3
nodeInfoLiteIsIgnored(n) // bit 4
nodeInfoLiteIsMuted(n) // bit 1
nodeInfoLiteIsLicensed(n) // bit 6 — Ham mode peer
nodeInfoLiteIsKeyManuallyVerified(n) // bit 0
nodeInfoLiteHasIsUnmessagable(n) // bit 8 — "is_unmessagable was sent"
nodeInfoLiteIsUnmessagable(n) // bit 7
// via_mqtt is bit 2 (mask exposed; predicate uses the mask directly)
nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true); // setter
```
### Satellite stores
Four `std::unordered_map<NodeNum, …>` members on `NodeDB`, each gated by its own build flag:
| Map | Value type | Build flag |
| ----------------- | ------------------------------- | ---------------------------------- |
| `nodePositions` | `meshtastic_PositionLite` | `MESHTASTIC_EXCLUDE_POSITIONDB` |
| `nodeTelemetry` | `meshtastic_DeviceMetrics` | `MESHTASTIC_EXCLUDE_TELEMETRYDB` |
| `nodeEnvironment` | `meshtastic_EnvironmentMetrics` | `MESHTASTIC_EXCLUDE_ENVIRONMENTDB` |
| `nodeStatus` | `meshtastic_StatusMessage` | `MESHTASTIC_EXCLUDE_STATUSDB` |
Defaults are ON (i.e., maps **excluded**) for STM32WL only — see `src/mesh/mesh-pb-constants.h`. On every other arch all four maps are present. When excluded, the map member is absent and the corresponding accessors return `false`.
All four maps are guarded by **`mutable concurrency::Lock satelliteMutex`** — concurrent access from receive threads, the phone API state machine, and the renderer is the rule, not the exception.
### Accessor convention
**Never hand out pointers into the maps.** Use the copy-out accessors on `NodeDB`:
```cpp
bool copyNodePosition(NodeNum, meshtastic_PositionLite &out) const;
bool copyNodeTelemetry(NodeNum, meshtastic_DeviceMetrics &out) const;
bool copyNodeEnvironment(NodeNum, meshtastic_EnvironmentMetrics &out) const;
bool copyNodeStatus(NodeNum, meshtastic_StatusMessage &out) const;
```
Each takes the lock, copies the value if present, returns `false` if the entry is absent or the DB is excluded. Pass-by-out-param is deliberate — pointer-style accessors would invite UAF and lock-leak bugs across the renderer. The "has any X" convenience predicates (`hasValidPosition` etc.) are implemented in terms of these.
Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which dispatches on `which_variant` for device vs environment metrics) — these own the lock and the eviction hooks.
### Eviction
Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly.
### Sync flow: thin NodeInfo + post-COMPLETE_ID replay (no opt-in)
There is no capability flag and no special "gradient" nonce. The **default** sync flow is:
1. Config / module-config / channel / metadata segments (same as before).
2. `STATE_SEND_OWN_NODEINFO`**our own** NodeInfo, still bundled with our position and device_metrics (because the replay snapshot excludes our own NodeNum). Emitted via `ConvertToNodeInfo(lite)`.
3. `STATE_SEND_OTHER_NODEINFOS` — every other peer's NodeInfo, **always thin** (no `position`, no `device_metrics`). Emitted via `ConvertToNodeInfoThin(lite)`.
4. `STATE_SEND_FILEMANIFEST``STATE_SEND_COMPLETE_ID` — the phone sees `config_complete_id` and treats sync as done.
5. `STATE_SEND_PACKETS` — live mesh packets, with a trailing replay drain interleaved. The replay drain walks four cached satellite stores in order (positions → telemetry → environment → status) and emits each cached entry as an ordinary `MeshPacket` on the matching portnum (`POSITION_APP`, `TELEMETRY_APP` device + environment variants, `NODE_STATUS_APP`). These are indistinguishable on the wire from live mesh traffic, so clients need no special handling — any code that already updates UI on `POSITION_APP` etc. works.
`PhoneAPI::sendConfigComplete()` arms `replayPhase = REPLAY_PHASE_POSITIONS` for default/full sync and `SPECIAL_NONCE_ONLY_NODES`, while `SPECIAL_NONCE_ONLY_CONFIG` skips replay. The drain runs inside `STATE_SEND_PACKETS` via `popReplayPacket()`, lower priority than live traffic. When all four phases drain, `replayPhase` flips back to `REPLAY_PHASE_IDLE` and the snapshot vectors get `shrink_to_fit`ed.
STM32WL and any other build with all four `MESHTASTIC_EXCLUDE_*DB` flags set produces zero replay packets — `popReplayPacket` advances through each phase in microseconds without emitting anything.
Special nonces that still mean something:
- `SPECIAL_NONCE_ONLY_CONFIG` (69420) — skip node sync entirely, just config.
- `SPECIAL_NONCE_ONLY_NODES` (69421) — skip config segments, jump straight to `STATE_SEND_OWN_NODEINFO`. Still gets the post-COMPLETE_ID replay drain.
There are no other reserved nonces; everything else is a fresh random `want_config_id` from the client.
### v24 → v25 migration
The legacy migration code lives in **`src/mesh/NodeDBLegacyMigration.cpp`**, not in `NodeDB.cpp`. It owns the `meshtastic_NodeDatabase_Legacy` callback and `NodeDB::migrateLegacyNodeDatabase()`. The legacy proto descriptor is `protobufs/meshtastic/deviceonly_legacy.proto` (only included by the migration TU). The boot path peeks the file's leading version tag, runs the migration if `version < 25`, then re-saves in v25 layout. The legacy descriptor is scheduled for removal once `DEVICESTATE_MIN_VER` is bumped.
### Read-site rules of thumb
- Never `node->position.X` / `node->device_metrics.X` — those fields no longer exist. Pull from the satellite map via `copyNodePosition` / `copyNodeTelemetry`.
- Never `node->user.long_name``long_name`, `short_name`, `public_key`, `hw_model`, `role`, `macaddr` (gone), `is_licensed`, `is_unmessagable` are flat on `NodeInfoLite`.
- Never `node->is_favorite` / `node->is_ignored` / `node->via_mqtt` / `node->is_key_manually_verified` — use the bitfield helpers.
- Never assume `nodeDB->getMeshNode(num)->position.time` — call `copyNodePosition` and check the return.
- Don't lock `satelliteMutex` yourself in renderer code; the copy-out accessors already do.
Unit tests for the conversion layer live in `test/test_type_conversions/test_main.cpp` (Unity) — bitfield round-trips, `long_name` truncation, thin-vs-full conversions. Add cases there when extending the schema.
## Project Structure
```
@@ -80,21 +238,46 @@ firmware/
│ │ ├── NodeDB.* # Node database management
│ │ ├── Router.* # Packet routing
│ │ ├── Channels.* # Channel management
│ │ ├── CryptoEngine.* # AES-CTR (channels) + X25519 ECDH→AES-256-CCM (PKI for DMs/admin)
│ │ ├── *Interface.* # Radio interface implementations
│ │ ├── api/ # WiFi/Ethernet server APIs (ServerAPI, PacketAPI)
│ │ ├── http/ # HTTP server (WebServer, ContentHandler)
│ │ ├── wifi/ # WiFi support (WiFiAPClient)
│ │ ├── eth/ # Ethernet support (ethClient)
│ │ ├── udp/ # UDP multicast
│ │ ├── compression/ # Message compression (unishox2)
│ │ └── generated/ # Protobuf generated code
│ ├── modules/ # Feature modules (Position, Telemetry, etc.)
│ │ └── Telemetry/ # Telemetry subsystem
│ │ └── Sensor/ # 50+ I2C sensor drivers
│ ├── gps/ # GPS handling
│ ├── graphics/ # Display drivers and UI
├── platform/ # Platform-specific code
│ ├── input/ # Input device handling
── concurrency/ # Threading utilities
│ └── niche/ # Specialized UIs (InkHUD e-ink framework)
│ ├── platform/ # Platform-specific code (esp32, nrf52, rp2xx0, stm32wl, portduino)
── input/ # Input device handling (InputBroker, keyboards, buttons)
│ ├── detect/ # I2C hardware auto-detection (80+ device types)
│ ├── motion/ # Accelerometer drivers (BMA423, BMI270, MPU6050, etc.)
│ ├── mqtt/ # MQTT bridge client
│ ├── power/ # Power HAL
│ ├── nimble/ # BLE via NimBLE
│ ├── buzz/ # Audio/notification (buzzer, RTTTL)
│ ├── serialization/ # JSON serialization, COBS encoding
│ ├── watchdog/ # Hardware watchdog thread
│ ├── concurrency/ # Threading utilities (OSThread, Lock)
│ ├── PowerFSM.* # Power finite state machine
│ └── Observer.h # Observer/Observable event pattern
├── variants/ # Hardware variant definitions
│ ├── esp32/ # ESP32 variants
│ ├── esp32s3/ # ESP32-S3 variants
│ ├── nrf52/ # nRF52 variants
── rp2xxx/ # RP2040/RP2350 variants
│ ├── esp32c3/ # ESP32-C3 variants
── esp32c6/ # ESP32-C6 variants
│ ├── nrf52840/ # nRF52 variants
│ ├── rp2040/ # RP2040/RP2350 variants
│ ├── stm32/ # STM32WL variants
│ └── native/ # Linux/Portduino variants
├── protobufs/ # Protocol buffer definitions
├── boards/ # Custom PlatformIO board definitions
├── test/ # Unit tests (12 test suites)
└── bin/ # Build and utility scripts
```
@@ -105,6 +288,9 @@ firmware/
- Follow existing code style - run `trunk fmt` before commits
- Prefer `LOG_DEBUG`, `LOG_INFO`, `LOG_WARN`, `LOG_ERROR` for logging
- Use `assert()` for invariants that should never fail
- C++17 features are available (`std::optional`, structured bindings, `if constexpr`, etc.)
- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
### Naming Conventions
@@ -118,70 +304,151 @@ firmware/
#### Module System
Modules inherit from `MeshModule` or `ProtobufModule<T>` and implement:
Modules use a three-tier class hierarchy:
- `handleReceivedProtobuf()` - Process incoming packets
- `allocReply()` - Generate response packets
- `runOnce()` - Periodic task execution (returns next run interval in ms)
1. **`MeshModule`** - Base class. Implement `wantPacket()` and `handleReceived()`. Returns `ProcessMessage::STOP` or `ProcessMessage::CONTINUE`.
2. **`SinglePortModule`** - Handles a single portnum. Simplified `wantPacket()` that checks `decoded.portnum`.
3. **`ProtobufModule<T>`** - Template for protobuf-based modules. Handles encoding/decoding automatically.
Most modules also inherit from **`OSThread`** for periodic tasks (the "mixin" pattern):
```cpp
class MyModule : public ProtobufModule<meshtastic_MyMessage>
class MyModule : public ProtobufModule<meshtastic_MyMessage>, private concurrency::OSThread
{
public:
MyModule();
protected:
virtual bool handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshtastic_MyMessage *msg) override;
virtual int32_t runOnce() override;
virtual meshtastic_MeshPacket *allocReply() override; // Generate response packets
virtual int32_t runOnce() override; // Periodic task (returns next interval in ms)
virtual bool alterReceivedProtobuf(meshtastic_MeshPacket &mp, meshtastic_MyMessage *msg); // Modify in-flight
virtual bool wantUIFrame(); // Request a UI display frame
};
```
Modules are registered in `src/modules/Modules.cpp` guarded by `MESHTASTIC_EXCLUDE_*` flags.
#### Observer/Observable Pattern
Event-driven communication between subsystems uses `src/Observer.h`:
```cpp
// Observable emits events
Observable<const meshtastic::Status *> newStatus;
newStatus.notifyObservers(&status);
// Observer receives events via callback
CallbackObserver<MyClass, const meshtastic::Status *> statusObserver =
CallbackObserver<MyClass, const meshtastic::Status *>(this, &MyClass::handleStatusUpdate);
```
#### Configuration Access
- `config.*` - Device configuration (LoRa, position, power, etc.)
- `moduleConfig.*` - Module-specific configuration
- `channels.*` - Channel configuration and management
- `owner` - Device owner info
- `myNodeInfo` - Local node info
#### Default Values
Use the `Default` class helpers in `src/mesh/Default.h`:
- `Default::getConfiguredOrDefaultMs(configured, default)` - Returns ms, using default if configured is 0
- `Default::getConfiguredOrDefault(configured, default)` - Generic configured/default getter
- `Default::getConfiguredOrMinimumValue(configured, min)` - Enforces minimum values
- `Default::getConfiguredOrDefaultMsScaled(configured, default, numNodes)` - Scales based on network size
#### Thread Safety
- Use `concurrency::Lock` for mutex protection
- Use `concurrency::Lock` and `concurrency::LockGuard` for mutex protection
- Radio SPI access uses `SPILock`
- Prefer `OSThread` for background tasks
### Hardware Detection
`src/detect/ScanI2C` automatically enumerates 80+ I2C device types at boot including displays, sensors, RTCs, keyboards, PMUs, and touch controllers. This drives automatic initialization of the correct drivers.
### Graphics/UI System
Multiple display driver families in `src/graphics/`:
- **OLED**: SSD1306, SH1106, ST7567
- **TFT**: TFTDisplay (LovyanGFX-based)
- **E-Ink**: EInkDisplay2, EInkDynamicDisplay, EInkParallelDisplay
**InkHUD** (`src/graphics/niche/InkHUD/`) is an event-driven e-ink UI framework:
- Applet-based architecture — modular display tiles
- Read-only, static display optimized for minimal refreshes and low power
- Configured per-variant via `nicheGraphics.h`
- Separate PlatformIO config: `src/graphics/niche/InkHUD/PlatformioConfig.ini`
### Input System
`src/input/InputBroker` is the centralized input event dispatcher. Supports multiple input sources: buttons, keyboards (BBQ10, Cardputer, TCA8418), touch screens, rotary encoders, and matrix keyboards.
### Power Management
`src/PowerFSM.*` implements a finite state machine with states: `stateON`, `statePOWER`, `stateSERIAL`, `stateDARK`. Key events: `EVENT_PRESS`, `EVENT_WAKE_TIMER`, `EVENT_LOW_BATTERY`, `EVENT_RECEIVED_MSG`, `EVENT_SHUTDOWN`. Conditionally excluded with `MESHTASTIC_EXCLUDE_POWER_FSM` (falls back to `FakeFsm`).
### Motion Sensors
`src/motion/AccelerometerThread` provides background motion monitoring with automatic screen wake and double-tap button press detection. Supports 10+ accelerometer/gyroscope chips (BMA423, BMI270, MPU6050, LIS3DH, LSM6DS3, STK8XXX, QMA6100P, ICM20948, BMX160).
### Telemetry Sensor Library
`src/modules/Telemetry/Sensor/` contains 50+ I2C sensor drivers organized by category:
- **Power monitoring**: INA219/226/260/3221, MAX17048
- **Environmental**: BME280/680, SCD4X (CO₂), SEN5X (particulate)
- **Humidity/Temperature**: SHT3X/4X, AHT10, MCP9808, MLX90614
- **Light**: BH1750, TSL2561/2591, VEML7700, LTR390UV, OPT3001
- **Air quality**: PMSA003I, SFA30
- **Specialized**: CGRadSens (radiation), NAU7802 (weight scale)
### API/Networking
`src/mesh/api/` provides a template-based `ServerAPI` for client communication over WiFi (`WiFiServerAPI`) and Ethernet (`ethServerAPI`). Default port: **4403**. HTTP server in `src/mesh/http/`. JSON serialization in `src/serialization/MeshPacketSerializer`.
### Hardware Variants
Each hardware variant has:
- `variant.h` - Pin definitions and hardware capabilities
- `platformio.ini` - Build configuration
- Optional: `pins_arduino.h`, `rfswitch.h`
- Optional: `pins_arduino.h`, `rfswitch.h`, `nicheGraphics.h` (for InkHUD variants)
Key defines in variant.h:
```cpp
#define USE_SX1262 // Radio chip selection
#define HAS_GPS 1 // Hardware capabilities
#define HAS_SCREEN 1 // Display present
#define LORA_CS 36 // Pin assignments
#define SX126X_DIO1 14 // Radio-specific pins
```
### Protobuf Messages
- Defined in `protobufs/meshtastic/*.proto`
- Generated code in `src/mesh/generated/`
- Defined in `protobufs/meshtastic/*.proto` (~32 proto files)
- Generated code in `src/mesh/generated/meshtastic/`
- Regenerate with `bin/regen-protos.sh`
- Message types prefixed with `meshtastic_`
- Nanopb `.options` files control field sizes and encoding
### Conditional Compilation
```cpp
#if !MESHTASTIC_EXCLUDE_GPS // Feature exclusion
#if !MESHTASTIC_EXCLUDE_WIFI // Network feature exclusion
#if !MESHTASTIC_EXCLUDE_BLUETOOTH // BLE exclusion
#if !MESHTASTIC_EXCLUDE_POWER_FSM // Power FSM exclusion
#ifdef ARCH_ESP32 // Architecture-specific
#ifdef ARCH_NRF52 // Nordic platform
#ifdef ARCH_RP2040 // Raspberry Pi Pico
#ifdef ARCH_PORTDUINO // Linux native
#if defined(USE_SX1262) // Radio-specific
#ifdef HAS_SCREEN // Hardware capability
#if USERPREFS_EVENT_MODE // User preferences
@@ -189,10 +456,27 @@ Key defines in variant.h:
## Build System
## Agent Tooling Baseline
Mirror counterpart: `AGENTS.md` under **Agent Tooling Baseline**.
To reduce avoidable agent mistakes, assume these tools are available (or install them before significant repo work):
- **Required CLI basics**: `bash`, `git`, `find`, `grep`, `sed`, `awk`, `xargs`
- **Strongly recommended**: `rg` (ripgrep) for fast file/text search, `jq` for JSON processing
- **Build/test tools**: `python3`, `pip`, virtualenv (`python3 -m venv`), `platformio` (`pio`)
- **Containerized native testing**: `docker` (fallback for non-Linux hosts; macOS can also build natively via `pio run -e native-macos`)
Fallback expectations for agents:
- If `rg` is unavailable, use `find` + `grep` instead of failing.
- For native tests on hosts without Linux deps, prefer `./bin/test-native-docker.sh`.
- The simulator helper script is `./bin/test-simulator.sh`.
Uses **PlatformIO** with custom scripts:
- `bin/platformio-pre.py` - Pre-build script
- `bin/platformio-custom.py` - Custom build logic
- `bin/platformio-custom.py` - Custom build logic, manifest generation
Build commands:
@@ -200,23 +484,41 @@ Build commands:
pio run -e tbeam # Build specific target
pio run -e tbeam -t upload # Build and upload
pio run -e native # Build native/Linux version
pio run -e native-macos # Build headless macOS meshtasticd (Homebrew prereqs in variants/native/portduino/platformio.ini)
```
### Build Manifest
`bin/platformio-custom.py` emits a build manifest with metadata:
- `hasMui`, `hasInkHud` - UI capability flags (overridable via `custom_meshtastic_has_mui`, `custom_meshtastic_has_ink_hud`)
- Architecture normalization (e.g., `esp32s3``esp32-s3` for API compatibility)
## Common Tasks
### Adding a New Module
1. Create `src/modules/MyModule.cpp` and `.h`
2. Inherit from appropriate base class
3. Register in `src/modules/Modules.cpp`
4. Add protobuf messages if needed in `protobufs/`
2. Inherit from appropriate base class (`MeshModule`, `SinglePortModule`, or `ProtobufModule<T>`)
3. Mix in `concurrency::OSThread` if periodic work is needed
4. Register in `src/modules/Modules.cpp` guarded by `#if !MESHTASTIC_EXCLUDE_MYMODULE`
5. Add protobuf messages if needed in `protobufs/meshtastic/`
6. Add test suite in `test/test_mymodule/` if applicable
### Adding a New Hardware Variant
1. Create directory under `variants/<arch>/<name>/`
2. Add `variant.h` with pin definitions
3. Add `platformio.ini` with build config
4. Reference common configs with `extends`
2. Add `variant.h` with pin definitions and hardware capability defines
3. Add `platformio.ini` with build config — use `extends` to reference common base (e.g., `esp32s3_base`)
4. Set `custom_meshtastic_support_level = 1` (PR builds) or `2` (merge builds)
5. For e-ink displays, add `nicheGraphics.h` for InkHUD configuration
### Adding a New Telemetry Sensor
1. Create driver in `src/modules/Telemetry/Sensor/` following existing sensor pattern
2. Register I2C address in `src/detect/ScanI2C` for auto-detection
3. Integrate with the appropriate telemetry module (Environment, Health, Power, AirQuality)
4. Add proto fields in `protobufs/meshtastic/telemetry.proto` if new data types are needed
### Modifying Configuration Defaults
@@ -305,9 +607,194 @@ Most workflows can be triggered manually via `workflow_dispatch` for testing.
## Testing
- Unit tests in `test/` directory
- Run with `pio test -e native`
- Use `bin/test-simulator.sh` for simulation testing
### Native unit tests (C++)
Unit tests in `test/` directory with 12 test suites:
- `test_crypto/` - Cryptography
- `test_mqtt/` - MQTT integration
- `test_radio/` - Radio interface
- `test_mesh_module/` - Module framework
- `test_meshpacket_serializer/` - Packet serialization
- `test_transmit_history/` - Retransmission tracking
- `test_atak/` - ATAK integration
- `test_default/` - Default configuration
- `test_http_content_handler/` - HTTP handling
- `test_serial/` - Serial communication
Run with: `pio test -e native`
Simulation testing: `bin/test-simulator.sh`
Quick entry point for new test modules: `test/README.md` (native unit-test authoring guide, skeleton, pitfalls, and setup checklist).
### Hardware-in-the-loop tests (`mcp-server/tests/`)
Separate pytest suite that exercises real USB-connected Meshtastic devices. See the **MCP Server & Hardware Test Harness** section below for invocation, tier layout, and agent usage rules.
## MCP Server & Hardware Test Harness
The `mcp-server/` directory houses a firmware-aware [MCP](https://modelcontextprotocol.io/) server plus a pytest-based integration suite. AI agents that speak MCP get a well-defined tool surface for flashing, configuring, and inspecting physical Meshtastic devices — use it instead of hand-rolling `pio` or `meshtastic --port` calls where possible. `mcp-server/README.md` is the operator-facing setup doc; this section is the agent-facing usage contract.
The repo registers the server via `.mcp.json` at the repo root — Claude Code picks it up automatically once `mcp-server/.venv/` is built (`cd mcp-server && python3 -m venv .venv && .venv/bin/pip install -e '.[test]'`).
### When to use which surface
| Goal | Tool |
| ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| Find a connected device | `mcp__meshtastic__list_devices` |
| Read a live node's config/state | `mcp__meshtastic__device_info`, `list_nodes`, `get_config` |
| Mutate a device (owner, region, channels, reboot) | `set_owner`, `set_config`, `set_channel_url`, `reboot`, `shutdown`, `factory_reset` — all require `confirm=True` |
| Flash firmware to a variant | `pio_flash` (any arch) or `erase_and_flash` (ESP32 factory install) |
| Stream serial logs while debugging | `serial_open``serial_read` loop → `serial_close` |
| Administer `userPrefs.jsonc` build-time constants | `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest` |
| Run the regression suite | `./mcp-server/run-tests.sh` (or `/test` slash command) |
| Diagnose a specific device | `/diagnose [role]` slash command (read-only) |
| Triage a flaky test | `/repro <node-id> [count]` slash command |
**One MCP call per port at a time.** `SerialInterface` holds an exclusive OS-level lock on the serial port for its lifetime. If a `serial_*` session is open on `/dev/cu.usbmodem101`, calling `device_info` on the same port will fail fast pointing at the active session. Sequence calls: open → read/mutate → close, then next device. Never parallelize tool calls on the same port.
### MCP tool surface (43 tools)
Grouped by purpose. Full argument shapes in `mcp-server/README.md`; a few high-value signatures are called out here.
- **Discovery & metadata**: `list_devices`, `list_boards`, `get_board`
- **Build & flash**: `build`, `clean`, `pio_flash`, `erase_and_flash` (ESP32 only), `update_flash` (ESP32 OTA), `touch_1200bps`
- **Serial sessions** (long-running, 10k-line ring buffer): `serial_open`, `serial_read`, `serial_list`, `serial_close`
- **Device reads**: `device_info`, `list_nodes`
- **Device writes**: `set_owner`, `get_config`, `set_config`, `get_channel_url`, `set_channel_url`, `send_text`, `send_input_event` (inject a button/key press via the firmware's InputBroker), `set_debug_log_api`; destructive/power-state writes require `confirm=True`: `reboot`, `shutdown`, `factory_reset`
- **userPrefs admin** (build-time constants, not runtime config): `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest`, `userprefs_testing_profile`
- **Vendor escape hatches**: `esptool_chip_info`, `esptool_erase_flash`, `esptool_raw`, `nrfutil_dfu`, `nrfutil_raw`, `picotool_info`, `picotool_load`, `picotool_raw`
- **USB power control** (via `uhubctl`, per-port PPPS toggle): `uhubctl_list` (read-only), `uhubctl_power(action='on'|'off', confirm=True)`, `uhubctl_cycle(delay_s, confirm=True)`. Target by raw `(location, port)` or by `role` (`"nrf52"`, `"esp32s3"`); role lookup checks `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` + `_PORT_<ROLE>` env vars first, falls back to VID auto-detection.
- **Observability** (UI tier + operator ad-hoc): `capture_screen(role, ocr=True)` — grabs a USB-webcam frame of the device OLED and optionally OCRs it. Requires `mcp-server[ui]` extras (`opencv-python-headless`, `easyocr`) and `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` env var; falls through to a 1×1 black PNG `NullBackend` when unconfigured.
`confirm=True` is a tool-level gate on top of whatever permission prompt your MCP host shows. **Don't bypass it** by asking the host to auto-approve — it exists specifically because MCP hosts sometimes remember "always allow this tool" and that's dangerous for `factory_reset`, `erase_and_flash`, `uhubctl_power(action='off')`, and `uhubctl_cycle`.
**TCP / native-host nodes.** Setting `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` makes `list_devices` surface a `meshtasticd` daemon (e.g. the `native-macos` build) as a synthetic `tcp://host:port` entry, and `connect()` routes through `meshtastic.tcp_interface.TCPInterface` instead of `SerialInterface`. Every read/write/admin tool that flows through `connect()` works against the daemon transparently. USB-only tools (`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`, `serial_open`, `esptool_*`, `nrfutil_*`, `picotool_*`) raise a clear `ConnectionError` when handed a `tcp://` port; `pio_flash` against a `native*` env raises a `FlashError` (no upload step — use `build` and run the binary directly). The pytest harness still assumes USB-attached devices per role; TCP-aware fixtures are deferred. See `mcp-server/README.md` § "TCP / native-host nodes".
### Hardware test suite (`mcp-server/run-tests.sh`)
The wrapper auto-detects connected devices (VID → role map: `0x239A``nrf52`, `0x303A`/`0x10C4``esp32s3`), maps each role to a PlatformIO env (`nrf52``rak4631`, `esp32s3``heltec-v3`, overridable via `MESHTASTIC_MCP_ENV_<ROLE>`), then invokes pytest. Zero pre-flight config needed from the operator.
Suite tiers (collected + run in this order via `pytest_collection_modifyitems`):
1. `tests/unit/` — pure Python (boards parse, pio wrapper, userPrefs parse, testing profile, uhubctl parser). No hardware.
2. `tests/test_00_bake.py` — flashes each detected device with current `userPrefs.jsonc` merged with the session's test profile. Has its own skip-if-already-baked check comparing region + primary channel to the session profile; skips cheaply on warm devices.
3. `tests/mesh/` — multi-device mesh: bidirectional send, broadcast delivery, direct-with-ACK, mesh formation within 60s. Parametrized `[nrf52->esp32s3]` and `[esp32s3->nrf52]`. Includes `test_peer_offline_recovery` which uses uhubctl to physically power off one peer mid-conversation (requires uhubctl; skips without).
4. `tests/telemetry/``DEVICE_METRICS_APP` broadcast timing.
5. `tests/monitor/` — boot-log panic check.
6. `tests/recovery/``uhubctl` power-cycle round-trip + NVS persistence across hard reset. Requires `uhubctl` installed and a PPPS-capable hub; entire tier auto-skips otherwise.
7. `tests/ui/` — input-broker-driven screen navigation with camera + OCR evidence.
8. `tests/fleet/` — PSK seed session isolation.
9. `tests/admin/` — channel URL roundtrip, owner persistence across reboot.
10. `tests/provisioning/` — region + modem + slot bake, admin key presence, `UNSET` region blocks TX, userPrefs survive factory reset.
Invocation patterns:
```bash
./mcp-server/run-tests.sh # full suite (auto-bake-if-needed)
./mcp-server/run-tests.sh --force-bake # reflash before testing
./mcp-server/run-tests.sh --assume-baked # skip bake (caller vouches for device state)
./mcp-server/run-tests.sh tests/mesh # one tier
./mcp-server/run-tests.sh tests/mesh/test_direct_with_ack.py # one file
./mcp-server/run-tests.sh -k telemetry # name filter
```
**No hardware detected?** The wrapper auto-narrows to `tests/unit/` only and prints `detected hub : (none)` in the pre-flight header. Agents interpreting the output should call this out explicitly — a 52-test green run without hardware is qualitatively different from a 12-unit-test green run.
**Artifacts every run produces:**
- `mcp-server/tests/report.html` — self-contained pytest-html. Each test gets a `Meshtastic debug` section with the tail of firmware log + device state dump. **Open this first** on failures; it's the canonical evidence source.
- `mcp-server/tests/junit.xml` — CI-parseable.
- `mcp-server/tests/reportlog.jsonl` — pytest-reportlog stream (`$report_type` keyed JSONL). Consumed by the live TUI.
- `mcp-server/tests/fwlog.jsonl` — firmware log mirror from the `meshtastic.log.line` pubsub topic. Populated by the `_firmware_log_stream` autouse session fixture.
### Live TUI (`meshtastic-mcp-test-tui`)
A Textual-based live view that wraps `run-tests.sh`. Tails reportlog for per-test state, streams firmware logs, polls device state at startup + post-run (gated out of the active run because `hub_devices` holds exclusive port locks). Key bindings:
| Key | Action |
| --- | ------------------------------------------------------------------------------------------------------------ |
| `r` | re-run focused test (leaf → that node id; internal node → directory or `-k`) |
| `f` | filter tree by substring |
| `d` | failure detail modal (pulls `longrepr` + captured stdout from the reportlog) |
| `g` | export reproducer bundle (tar.gz with README, test_report.json, time-filtered fwlog, devices.json, env.json) |
| `l` | toggle firmware log pane |
| `x` | tool coverage modal |
| `c` | cross-run history sparkline |
| `q` | quit (SIGINT → SIGTERM → SIGKILL escalation, 5-s windows each) |
Launch:
```bash
cd mcp-server
.venv/bin/meshtastic-mcp-test-tui # full suite
.venv/bin/meshtastic-mcp-test-tui tests/mesh # args pass through to pytest
```
The plain CLI stays primary; the TUI is for operators who want a live dashboard. Both consume the same `run-tests.sh`.
### Slash commands (Claude Code + Copilot)
Three AI-assisted workflows wrap the test harness. Claude Code operators get `/test`, `/diagnose`, `/repro`; Copilot operators get `/mcp-test`, `/mcp-diagnose`, `/mcp-repro`. Bodies:
- `.claude/commands/{test,diagnose,repro}.md`
- `.github/prompts/mcp-{test,diagnose,repro}.prompt.md`
`.claude/commands/README.md` is the index.
House rules for agents running these prompts:
- **Interpret failures, don't just echo them.** Pull firmware log tails from `report.html` and classify each failure as transient / environmental / regression. Use the exact format in `.claude/commands/test.md`.
- **No destructive writes without operator approval.** Any skill that could reflash, factory-reset, or reboot a device must describe the action and stop. The operator authorizes.
- **Sequential MCP calls per port.** See above.
- **"Unknown" is a valid classification.** If evidence doesn't support a root cause, say so and list what would disambiguate. Do not invent.
### Key fixtures (test authors + agents debugging)
`mcp-server/tests/conftest.py` provides:
- **`_session_userprefs`** (autouse session) — snapshots `userPrefs.jsonc` at session start, merges the session test profile via `userprefs.merge_active(test_profile)`, restores at teardown. Four layers of safety: pytest teardown + `atexit` + sidecar file (`userPrefs.jsonc.mcp-session-bak`) + startup self-heal in `run-tests.sh`. **Do not edit `userPrefs.jsonc` from inside a test.**
- **`_firmware_log_stream`** (autouse session) — subscribes to `meshtastic.log.line` pubsub on every connected `SerialInterface` and mirrors lines to `tests/fwlog.jsonl`. Drives the TUI firmware-log pane.
- **`_debug_log_buffer`** (autouse per-test) — captures last 200 firmware log lines + device state for attachment to the pytest-html `Meshtastic debug` section on failure.
- **`hub_devices`** (session) — `dict[role, SerialInterface]` with session-long exclusive port locks. Reason the TUI's device poller is gated to startup + post-run only.
- **`baked_mesh`** — parametrized mesh-pair fixture; depends on `test_00_bake`. `pytest_generate_tests` in `conftest.py` auto-generates `[nrf52->esp32s3]` and `[esp32s3->nrf52]` variants.
- **`test_profile`** — session-scoped dict: region, primary channel, admin key, PSK seed. Derived from `MESHTASTIC_MCP_SEED` (defaults to `mcp-<user>-<host>`).
### Firmware integration points tied to the test harness
Two firmware changes exist specifically so the test harness works reliably. **Keep these in mind when touching related code.**
- **`src/mesh/StreamAPI.cpp` + `StreamAPI.h`** — `emitLogRecord` uses a dedicated `fromRadioScratchLog` + `txBufLog` pair and a `concurrency::Lock streamLock`. Before this fix, `debug_log_api_enabled=true` would tear `FromRadio` protobufs on the serial transport because `emitTxBuffer` and `emitLogRecord` shared a single scratch buffer. The conftest enables the log stream session-wide; without this fix the device would corrupt its own FromRadio replies mid-session.
- **`src/mesh/PhoneAPI.cpp`** — `ToRadio` `Heartbeat(nonce=1)` triggers `nodeInfoModule->sendOurNodeInfo(NODENUM_BROADCAST, true, 0, true)` for serial clients, mirroring the pre-existing behavior for TCP/UDP clients in `PacketAPI.cpp`. The mesh tests rely on this to force a NodeInfo broadcast right after connect so the peer discovers them before the test's first assertion.
If you're modifying `StreamAPI`, `PhoneAPI`, `NodeInfoModule`, or `userPrefs` flow, run `./mcp-server/run-tests.sh` at minimum before asking for review.
### Recovery playbooks
| Symptom | First check | Fix |
| --------------------------------------------------------------------------------- | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `userPrefs.jsonc` dirty after test run | `git status --porcelain userPrefs.jsonc` | If non-empty, re-run `./mcp-server/run-tests.sh` once — the pre-flight self-heal restores from sidecar. If still dirty, `git checkout userPrefs.jsonc`. |
| Port busy / wedged CP2102 on macOS | `lsof /dev/cu.usbserial-0001` | Kill the holder. USB replug if the kernel still reports busy. Often a stale `pio device monitor` or zombie `meshtastic_mcp` process. |
| nRF52 appears unresponsive | `list_devices` shows VID `0x239A` but `device_info` times out | `touch_1200bps(port=...)` drops it into the DFU bootloader → `pio_flash` re-installs. |
| Device fully wedged (Guru Meditation, frozen CDC, no DFU) | `list_devices` shows the VID but every admin call times out | `uhubctl_cycle(role="nrf52", confirm=True)` hard-power-cycles the port via USB hub PPPS. `baked_single`'s auto-recovery hook does this once automatically if uhubctl is installed. Falls back to physical replug if no PPPS hub. |
| Multiple MCP server processes | `ps aux \| grep meshtastic_mcp` shows >1 | Kill all but the one your MCP host spawned. Zombies hold ports and break tests. |
| Mesh formation fails, one side sees peer but other doesn't | `/diagnose` (or `list_nodes` on both sides) | Asymmetric NodeInfo. `test_direct_with_ack` has a heal path; `/repro` it a few times. If persistent, both devices' clocks may be out of sync with their NodeInfo cooldown. |
| "role not present on hub" in skip reasons | `list_devices` | Expected if a device is unplugged. Reconnect before re-running the tier. |
| Entire `tests/recovery/` tier skipped | `command -v uhubctl` | Expected if `uhubctl` isn't on PATH. Install via `brew install uhubctl` (macOS) or `apt install uhubctl` (Debian/Ubuntu). Also skips if no hub advertises PPPS. |
| Entire `tests/ui/` tier skipped ("firmware not baked with USERPREFS_UI_TEST_LOG") | reportlog.jsonl for the skip reason | Re-run with `--force-bake` so the UI-log macro gets compiled into the fresh firmware. First run after the Round-3 landing always re-bakes. |
| `tests/ui/` runs but captures are all 1×1 black PNGs | `MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3` | Env var not set → `NullBackend`. Point a USB webcam at the heltec-v3 OLED and set the device index; `.venv/bin/python -c "import cv2; [print(i, cv2.VideoCapture(i).read()[0]) for i in range(5)]"` discovers it. |
| Tests fail only on first attempt then pass on rerun | — | State leak from a prior session. Run with `--force-bake` to reset to a known state. |
### Never do these without asking
- `factory_reset` — wipes node identity; regenerates PKI keypair. Mesh peers will reject old DMs until re-exchange. Legitimate only when the operator explicitly wants it.
- `erase_and_flash` — full chip erase; destroys all on-device state.
- `esptool_erase_flash` / `esptool_raw` write/erase — bypasses pio's safety chain.
- `set_config` on `lora.region` — changes regulatory domain; requires physical-location context the operator has and the agent doesn't.
- `reboot` / `shutdown` mid-test — breaks fixture invariants.
- `push -f`, `rebase -i`, `reset --hard`, or any history-rewriting git operation.
- Clicking computer-use tools on web links in Mail/Messages/PDFs — open URLs via the claude-in-chrome MCP so the extension's link-safety checks apply.
## Resources
+64
View File
@@ -0,0 +1,64 @@
---
mode: agent
description: Device health report via the meshtastic MCP tools (Copilot equivalent of the Claude Code /diagnose slash command)
---
# `/mcp-diagnose` — device health report
Equivalent of `.claude/commands/diagnose.md`. Use when the operator asks to "check the devices", "what's the mesh looking like", "is nrf52 alive", etc.
This prompt assumes the meshtastic MCP server is registered with your VS Code Copilot agent. If it isn't, fall back to running `./mcp-server/run-tests.sh tests/unit` plus a short `device_info` script via the terminal.
## What to do
1. **Enumerate hardware** via the `list_devices` MCP tool (with `include_unknown=True`). For each entry where `likely_meshtastic=True`, capture `port`, `vid`, `pid`, `description`.
2. **Apply the operator's filter** (if any):
- No filter → every likely-meshtastic device.
- `nrf52``vid == 0x239a`
- `esp32s3``vid == 0x303a` or `vid == 0x10c4`
- A `/dev/cu.*` path → only that port.
- Anything else → substring match on port.
3. **For each selected device, in sequence (don't parallelize — SerialInterface holds an exclusive port lock):**
- `device_info(port=<p>)``my_node_num`, `long_name`, `short_name`, `firmware_version`, `hw_model`, `region`, `num_nodes`, `primary_channel`
- `list_nodes(port=<p>)` → peer count, which peers have `publicKey`, SNR/RSSI distribution
- `get_config(section="lora", port=<p>)` → region, preset, channel_num, tx_power, hop_limit
- If anything looks off (can't connect, `num_nodes` wrong, missing `firmware_version`), open a short firmware-log window: `serial_open(port=<p>, env=<inferred>)`, wait 3 seconds, `serial_read(session_id, max_lines=100)`, `serial_close(session_id)`. Infer env from VID (0x239a → `rak4631`, 0x303a/0x10c4 → `heltec-v3`) unless an `MESHTASTIC_MCP_ENV_<ROLE>` env var overrides it.
4. **Hub health** (call once, not per-device): `uhubctl_list()` — enumerates every USB hub the host sees. Cross-reference each Meshtastic device's VID to find which hub + port it's on. Flag in the report if:
- No hub advertises `ppps=true``tests/recovery/` can't run; hard-recovery via `uhubctl_cycle` isn't available.
- A Meshtastic device is on a non-PPPS hub → note it; moving to a PPPS hub unlocks auto-recovery.
- `uhubctl_list` raises `ConfigError: uhubctl not found` → report as "uhubctl not installed"; don't treat as a device fault.
5. **Render per-device report** as a compact block:
```text
[nrf52 @ /dev/cu.usbmodem1101] fw=2.7.23.bce2825, hw=RAK4631
owner : Meshtastic 40eb / 40eb
region/band : US, channel 88, LONG_FAST
tx_power : 30 dBm, hop_limit=3
peers : 1 (esp32s3 0x433c2428, pubkey ✓, SNR 6.0 / RSSI -24 dBm)
primary ch : McpTest
hub : 1-1.3 port 2 (PPPS, uhubctl-controllable)
firmware : no panics in last 3s
```
Flag abnormalities inline with `⚠︎ <short reason>` — missing pubkey on a known peer, region UNSET, mismatched channel name, device on non-PPPS hub, etc.
6. **Cross-device correlation** (when >1 device selected):
- Do both see each other in `nodesByNum`?
- Do `region`, `channel_num`, `modem_preset` match across devices?
- Do the primary channel names match? (Different name → different PSK → no decode.)
7. **Suggest next steps only for recognizable failure modes**, never speculatively:
- Stale PKI one-way → "`/mcp-test tests/mesh/test_direct_with_ack.py` — the test's retry+nodeinfo-ping heals this."
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
- Device unreachable, DFU reachable → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds and the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
- CP2102-wedged-driver on macOS → see `run-tests.sh` notes.
## Hard constraints
- **Read-only.** No `set_config`, no `reboot`, no `factory_reset`, no `flash`. If the operator wants mutation, they'll escalate explicitly.
- **Open/query/close per device.** Never hold multiple SerialInterfaces to the same port. The port lock is exclusive.
- **Don't infer env beyond the VID map** — if the operator has an unusual board, ask them which env to use rather than guessing.
+68
View File
@@ -0,0 +1,68 @@
---
mode: agent
description: Re-run a specific test N times to triage flakes; diff firmware logs between passes and failures (Copilot equivalent of the Claude Code /repro slash command)
---
# `/mcp-repro` — flakiness triage for one test
Equivalent of `.claude/commands/repro.md`. Use when the operator says "that one test is flaky — dig in", "repro the direct_with_ack failure", "why does X sometimes fail?".
## What to do
1. **Parse the operator's input** into two pieces:
- **Test identifier** — either a pytest node id (has `::` or starts with `tests/`) or a `-k`-style filter (plain substring like `direct_with_ack`).
- **Count** — integer, default `5`, cap at `20`. If the operator asks for 50, negotiate down and explain (airtime + USB wear).
2. **Sanity-check the hub** via the `list_devices` MCP tool. If the test name references `nrf52` or `esp32s3` and the matching VID isn't present, stop and report — re-running won't help.
3. **Loop** N times. Each iteration:
```bash
./mcp-server/run-tests.sh <test-id> --tb=short -p no:cacheprovider
```
`-p no:cacheprovider` keeps pytest from caching anything between iterations. Capture: exit code, duration, and (on failure) the `Meshtastic debug` firmware-log section from `mcp-server/tests/report.html`.
4. **Tally** results as you go:
```text
attempt 1: PASS (42s)
attempt 2: FAIL (128s) ← fw log captured
attempt 3: PASS (39s)
attempt 4: FAIL (121s)
attempt 5: PASS (41s)
--------------------------------------------------
pass rate: 3/5 (60%) | mean duration: 74s
```
5. **On mixed outcomes, diff the firmware logs** between one representative pass and one representative fail. Focus on:
- Error-level lines present only in failures (`PKI_UNKNOWN_PUBKEY`, `Alloc an err=`, `Skip send`, `No suitable channel`, `NAK`)
- Timing around the assertion point (broadcast sent? ACK received? retry fired?)
- Device-state fields that changed between attempts
Surface the top 3 differences as a compact "passes when / fails when" table with uptime timestamps. Don't dump full logs.
6. **Classify** the flake into one of:
- **LoRa airtime collision** — pass rate improves with fewer concurrent transmitters. Suggest a `time.sleep` gap or retry bump in the test body.
- **PKI key staleness** — first attempt fails, subsequent ones pass; existing retry-loop pattern in `test_direct_with_ack.py` is the fix.
- **NodeInfo cooldown** — `Skip send NodeInfo since we sent it <600s ago` in fail-only logs; needs a `broadcast_nodeinfo_ping()` warmup.
- **Hardware-specific** — one direction consistently fails, firmware versions differ, CP2102 driver wedged, etc. For a device wedged past `touch_1200bps`, recommend `uhubctl_cycle(role=..., confirm=True)` to hard-power-cycle its hub port (requires `uhubctl` installed).
- **Device went dark mid-run** — fails from some iteration onward and never recovers; firmware log stops arriving. Almost always a Guru crash with frozen CDC. Recommend `uhubctl_cycle` before the next iteration; escalate to replug if that also fails.
- **Unknown** — say so. Don't invent a root cause.
7. **Report back** with:
- Pass rate + mean duration.
- Classification + the specific log evidence for it.
- A concrete next step (tighter assertion, more retries, open `/mcp-diagnose`, file a bug, nothing).
## Examples
- `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip[esp32s3->nrf52] 10` — 10 runs of that parametrized case.
- `broadcast_delivers` — no `::`, no `tests/`; treat as `-k broadcast_delivers`; runs every match 5 times.
- `tests/telemetry/test_device_telemetry_broadcast.py 3` — shorter count for a slow test.
## Notes
- If the FIRST attempt fails and the rest pass, that's a state-leak signature — suggest starting from `--force-bake` or a clean device state rather than chasing the first-failure firmware logs.
- If ALL N fail, this isn't a flake — it's a regression. Say so, stop iterating, escalate to `/mcp-test` for full-suite context.
- Don't rebuild firmware during triage. Flakes that only reproduce under different firmware belong in a separate session with a plan.
+57
View File
@@ -0,0 +1,57 @@
---
mode: agent
description: Run the mcp-server test suite and interpret results (Copilot equivalent of the Claude Code /test slash command)
---
# `/mcp-test` — mcp-server test runner with interpretation
Equivalent of the Claude Code `/test` slash command in `.claude/commands/test.md`. Use this when the operator asks you to "run the tests", "check the mcp test suite", "run the mesh tests", etc.
## What to do
1. **Invoke the wrapper** from the firmware repo root:
```bash
./mcp-server/run-tests.sh [pytest-args]
```
If the operator specified a subset (e.g. "just the mesh tests"), pass it through as `tests/mesh` or a pytest `-k filter`. If they said nothing, use the wrapper's defaults (full suite with pytest-html report).
The wrapper auto-detects connected Meshtastic devices, maps each to its PlatformIO env, exports the required env vars, and invokes pytest. Zero pre-flight config needed from the operator.
2. **Read the pre-flight header** (first few lines of wrapper output). The `detected hub :` line lists role → port → env mappings. If it reads `(none)`, the wrapper narrowed to `tests/unit` only — call that out explicitly so the operator knows hardware tiers were skipped.
3. **On pass**: one-line summary like `N passed, M skipped in <duration>`. Don't enumerate test names. DO mention any non-placeholder SKIPs and name the cause:
- `"role not present on hub"` → device unplugged; operator should reconnect.
- `"firmware not baked with USERPREFS_UI_TEST_LOG"` → tests/ui skipped; the UI-log compile macro isn't in the baked firmware. Suggest `--force-bake`.
- `"uhubctl not installed"` → tests/recovery + `test_peer_offline_recovery` skipped. Suggest `brew install uhubctl` / `apt install uhubctl`.
- `"no PPPS-capable hubs detected"` → tests/recovery skipped because the attached hub doesn't support per-port power switching; won't run on that setup.
- `"opencv-python-headless is not installed"` → tests/ui auto-deselected by `run-tests.sh`. Suggest `pip install -e 'mcp-server/.[ui]'`.
4. **On failure**: open `mcp-server/tests/report.html` (pytest-html output, self-contained) and extract the `Meshtastic debug` section for each failed test. That section includes a firmware log stream (last 200 lines) and device state dump. For each failure, summarise:
- test name
- one-line assertion message
- the specific firmware log lines that explain why (look for `PKI_UNKNOWN_PUBKEY`, `Skip send NodeInfo`, `Error=`, `Guru Meditation`, `assertion failed`, `No suitable channel`)
- for UI-tier failures also check `mcp-server/tests/ui_captures/<session>/<test>/transcript.md` (per-step frame + OCR)
5. **Classify each failure** as one of:
- **Transient flake** — LoRa collision, first-attempt NAK with self-heal pattern, timing-sensitive assertion. Suggest `/mcp-repro <test-id>` to confirm.
- **Environmental** — device unreachable, port busy, CP2102 driver wedged on macOS. Suggest recovery in escalation order: (a) replug USB, (b) `touch_1200bps` + `pio_flash` for nRF52 DFU, (c) `uhubctl_cycle(role=..., confirm=True)` for a device wedged past DFU (needs `uhubctl` installed; `baked_single` does this once automatically when available). Also check `git status userPrefs.jsonc`.
- **Regression** — same assertion fails repeatedly on re-runs, firmware log shows novel errors. Identify the firmware module likely responsible.
6. **Do NOT run destructive recovery automatically**. If a failure looks like it needs a reflash, factory*reset, `uhubctl_cycle`, or replug — \_describe the steps* and let the operator decide. Never burn airtime or flash cycles without approval.
## Arguments convention
Operators generally invoke this prompt either with no arguments (full suite) or with a specific subset. Examples:
- `tests/mesh` — one tier
- `tests/mesh/test_direct_with_ack.py::test_direct_with_ack_roundtrip` — one test
- `--force-bake` — reflash devices first
- `-k telemetry` — name-filter
## Side-effects to confirm in your summary
- `userPrefs.jsonc` should be clean after a successful run. The session fixture in `mcp-server/tests/conftest.py` (`_session_userprefs`) snapshots and restores. Check `git status --porcelain userPrefs.jsonc` and report if it's non-empty.
- `mcp-server/tests/report.html` and `junit.xml` regenerate on every run.
- The wrapper prints a warning if a `.mcp-session-bak` sidecar was left over from a crashed prior session and auto-restores from it — mention that if it happened.
+138
View File
@@ -0,0 +1,138 @@
# New Meshtastic Module
Guide for developing a new Meshtastic firmware module.
## Module Hierarchy
Choose the appropriate base class:
1. **`MeshModule`** — Raw base class. Override `wantPacket()` and `handleReceived()`. Returns `ProcessMessage::STOP` or `ProcessMessage::CONTINUE`.
2. **`SinglePortModule`** — Handles a single `meshtastic_PortNum`. Constructor takes `(name, portNum)`. Simplified `wantPacket()` checking `decoded.portnum`. Use `allocDataPacket()` to create outgoing packets.
3. **`ProtobufModule<T>`** — Template for protobuf-encoded modules. Constructor takes `(name, portNum, fields)`. Override `handleReceivedProtobuf()`. Use `allocDataProtobuf(payload)` to create outgoing packets.
Most modules also mix in `concurrency::OSThread` for periodic background tasks.
## Implementation Pattern
```cpp
// src/modules/MyModule.h
#pragma once
#include "ProtobufModule.h"
#include "concurrency/OSThread.h"
class MyModule : public ProtobufModule<meshtastic_MyMessage>, private concurrency::OSThread
{
public:
MyModule();
protected:
// Process incoming protobuf packet. Return true to stop further processing.
virtual bool handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshtastic_MyMessage *msg) override;
// Generate response packet (optional)
virtual meshtastic_MeshPacket *allocReply() override;
// Periodic task — return next run interval in ms, or disable()
virtual int32_t runOnce() override;
// Modify packet in-flight before delivery (optional)
virtual bool alterReceivedProtobuf(meshtastic_MeshPacket &mp, meshtastic_MyMessage *msg);
// Request a UI display frame (optional)
virtual bool wantUIFrame();
};
```
## Registration
Register in `src/modules/Modules.cpp` inside `setupModules()`:
```cpp
#if !MESHTASTIC_EXCLUDE_MYMODULE
new MyModule();
#endif
```
If other code needs to reference the module instance:
```cpp
#if !MESHTASTIC_EXCLUDE_MYMODULE
myModule = new MyModule();
#endif
```
And declare the global in the header:
```cpp
extern MyModule *myModule;
```
Some modules also conditionally instantiate based on `moduleConfig`:
```cpp
#if !MESHTASTIC_EXCLUDE_MYMODULE
if (moduleConfig.has_my_module && moduleConfig.my_module.enabled) {
new MyModule();
}
#endif
```
## Conditional Compilation
Add a `MESHTASTIC_EXCLUDE_MYMODULE` guard. This allows the module to be excluded from constrained builds. The flag name must follow the pattern: `MESHTASTIC_EXCLUDE_` + uppercase module name.
## Protobuf Messages (if needed)
1. Define messages in `protobufs/meshtastic/` (e.g., `mymodule.proto`)
2. Add a `.options` file for nanopb field size constraints
3. Regenerate with `bin/regen-protos.sh`
4. Generated code appears in `src/mesh/generated/meshtastic/`
5. Assign a `meshtastic_PortNum` if the module uses a new port number
## Timing and Defaults
Use `Default` class helpers for configurable intervals:
```cpp
int32_t MyModule::runOnce()
{
uint32_t interval = Default::getConfiguredOrDefaultMs(moduleConfig.my_module.update_interval,
default_my_module_interval);
// ... do work ...
return interval;
}
```
On public/default channels, enforce minimums with `Default::getConfiguredOrMinimumValue()`.
## Observer Pattern
Subscribe to system events:
```cpp
CallbackObserver<MyModule, const meshtastic::Status *> statusObserver =
CallbackObserver<MyModule, const meshtastic::Status *>(this, &MyModule::handleStatusUpdate);
```
## Testing
Add test suite in `test/test_mymodule/`:
```text
test/
└── test_mymodule/
└── test_main.cpp
```
Run with: `pio test -e native`
## Checklist
- [ ] Header and implementation files in `src/modules/`
- [ ] Inherit from appropriate base class (MeshModule / SinglePortModule / ProtobufModule)
- [ ] Mix in OSThread if periodic work is needed
- [ ] Register in `src/modules/Modules.cpp` with `MESHTASTIC_EXCLUDE_` guard
- [ ] Add protobuf definitions if needed (`protobufs/meshtastic/`)
- [ ] Use `Default::getConfiguredOrDefaultMs()` for timing
- [ ] Respect bandwidth limits on public channels
- [ ] Add test suite in `test/`
+149
View File
@@ -0,0 +1,149 @@
# New Telemetry Sensor
Guide for adding a new I2C telemetry sensor driver to Meshtastic firmware.
## Overview
Telemetry sensors live in `src/modules/Telemetry/Sensor/`. There are 50+ existing drivers organized by measurement type. Each sensor integrates with one of the telemetry modules:
- **EnvironmentTelemetryModule** — Temperature, humidity, pressure, gas, light
- **AirQualityTelemetryModule** — Particulate matter, VOCs
- **PowerTelemetryModule** — Voltage, current, power monitoring
- **HealthTelemetryModule** — Heart rate, SpO2, body temperature
## Sensor Driver Pattern
Each sensor has a `.h` and `.cpp` file pair following this pattern:
```cpp
// src/modules/Telemetry/Sensor/MySensor.h
#pragma once
#include "TelemetrySensor.h"
#include <MySensorLibrary.h> // Arduino/PlatformIO library
class MySensor : virtual public TelemetrySensor
{
private:
MySensorLibrary sensor;
public:
MySensor() : TelemetrySensor(meshtastic_TelemetrySensorType_MY_SENSOR, "MySensor") {}
// Initialize sensor hardware. Return true on success.
virtual void setup() override;
// Read sensor data into the telemetry protobuf. Return true on success.
virtual bool getMetrics(meshtastic_Telemetry *measurement) override;
};
```
```cpp
// src/modules/Telemetry/Sensor/MySensor.cpp
#include "MySensor.h"
#include "TelemetrySensor.h"
void MySensor::setup()
{
sensor.begin();
// Configure sensor parameters...
}
bool MySensor::getMetrics(meshtastic_Telemetry *measurement)
{
// Read from hardware
float value = sensor.readValue();
// Populate the appropriate protobuf variant
measurement->variant.environment_metrics.temperature = value;
// ... other fields ...
return true;
}
```
## I2C Address Registration
Register the sensor's I2C address(es) in `src/detect/ScanI2C` so it's auto-detected at boot:
1. Add a `DeviceType` enum entry in `src/detect/ScanI2C.h`
2. Add the I2C address mapping in `src/detect/ScanI2CTwoWire.cpp`
The scan runs at boot and populates a device map that telemetry modules use to decide which sensors to initialize.
## Protobuf Fields
If the sensor provides data not covered by existing telemetry fields:
1. Add fields to the appropriate message in `protobufs/meshtastic/telemetry.proto`:
- `EnvironmentMetrics` — Environmental measurements
- `AirQualityMetrics` — Air quality data
- `PowerMetrics` — Power/energy data
- `HealthMetrics` — Health/biometric data
2. Add a `.options` constraint if needed (field sizes for nanopb)
3. Regenerate: `bin/regen-protos.sh`
## Sensor Type Enum
Add the sensor to `meshtastic_TelemetrySensorType` enum in `protobufs/meshtastic/telemetry.proto`:
```protobuf
enum TelemetrySensorType {
// ... existing entries ...
MY_SENSOR = XX;
}
```
## Integration with Telemetry Module
Wire the sensor into the appropriate telemetry module. For environment sensors, this is typically in `src/modules/Telemetry/EnvironmentTelemetry.cpp`:
1. Include the sensor header
2. Add initialization in `setupSensor()` guarded by detection results
3. Call `getMetrics()` in the measurement collection path
Example pattern from existing sensors:
```cpp
#include "Sensor/MySensor.h"
MySensor mySensor;
// In setup:
if (nodeTelemetrySensorsMap[meshtastic_TelemetrySensorType_MY_SENSOR].first > 0) {
mySensor.setup();
}
// In measurement collection:
if (nodeTelemetrySensorsMap[meshtastic_TelemetrySensorType_MY_SENSOR].first > 0) {
mySensor.getMetrics(&measurement);
}
```
## Library Dependencies
If the sensor needs an external library, add it to the `lib_deps` in the relevant base platformio.ini configs:
```ini
lib_deps =
${env.lib_deps}
mysensorlibrary@^1.0.0
```
Or use a conditional dependency if it's platform-specific.
## Unit Conversions
If the sensor reports values in non-standard units, use `src/modules/Telemetry/UnitConversions.h` for conversion helpers (e.g., Celsius ↔ Fahrenheit, hPa ↔ inHg).
## Checklist
- [ ] Create `src/modules/Telemetry/Sensor/MySensor.h` and `.cpp`
- [ ] Inherit from `TelemetrySensor` base class
- [ ] Implement `setup()` and `getMetrics()` methods
- [ ] Add `meshtastic_TelemetrySensorType` enum entry in `telemetry.proto`
- [ ] Add I2C address to `src/detect/ScanI2C` for auto-detection
- [ ] Add protobuf fields in `telemetry.proto` if new data types needed
- [ ] Regenerate protos: `bin/regen-protos.sh`
- [ ] Wire into the appropriate telemetry module (Environment/AirQuality/Power/Health)
- [ ] Add library dependency if external library required
- [ ] Test on hardware or native build
+178
View File
@@ -0,0 +1,178 @@
# New Hardware Variant
Guide for adding a new Meshtastic hardware variant to the firmware.
## Directory Structure
Create under `variants/<arch>/<name>/`:
```text
variants/
├── esp32/ # ESP32
├── esp32s3/ # ESP32-S3
├── esp32c3/ # ESP32-C3
├── esp32c6/ # ESP32-C6
├── nrf52840/ # nRF52840
├── rp2040/ # RP2040/RP2350
├── stm32/ # STM32WL
└── native/ # Linux/Portduino
```
Each variant needs at minimum:
- `variant.h` — Pin definitions and hardware capabilities
- `platformio.ini` — Build configuration
Optional files:
- `pins_arduino.h` — Arduino pin mapping overrides
- `rfswitch.h` — RF switch control for multi-band radios
- `nicheGraphics.h` — InkHUD e-ink configuration
## variant.h Template
```cpp
// Pin definitions
#define I2C_SDA 21
#define I2C_SCL 22
// LoRa radio
#define USE_SX1262 // Radio chip: USE_SX1262, USE_SX1268, USE_SX1280, USE_RF95, USE_LLCC68, USE_LR1110, USE_LR1120, USE_LR1121
#define LORA_CS 18
#define LORA_SCK 5
#define LORA_MOSI 27
#define LORA_MISO 19
#define LORA_DIO1 33 // SX126x: DIO1, SX128x: DIO1, RF95: IRQ
#define LORA_RESET 23
#define LORA_BUSY 32 // SX126x/SX128x only
#define SX126X_DIO2_AS_RF_SWITCH // Common for SX1262 boards
// GPS
#define HAS_GPS 1
#define GPS_RX_PIN 34
#define GPS_TX_PIN 12
// #define PIN_GPS_EN 47 // Optional GPS enable pin
// #define GPS_BAUDRATE 9600 // Override default 9600
// Display
#define HAS_SCREEN 1
// #define USE_SSD1306 // OLED type
// #define USE_SH1106 // Alternative OLED
// #define USE_ST7789 // TFT type
// #define SCREEN_WIDTH 128
// #define SCREEN_HEIGHT 64
// LEDs
#define LED_PIN 2 // Status LED (optional)
// #define HAS_NEOPIXEL 1 // WS2812 support
// Buttons
#define BUTTON_PIN 38
// #define BUTTON_PIN_ALT 0 // Secondary button
// Power management
// #define HAS_AXP192 1 // AXP192 PMU (T-Beam v1.0)
// #define HAS_AXP2101 1 // AXP2101 PMU (T-Beam v1.2+)
// #define BATTERY_PIN 35 // ADC battery voltage pin
// #define ADC_MULTIPLIER 2.0 // Voltage divider ratio
// Optional I2C devices
// #define HAS_RTC 1 // Real-time clock
// #define HAS_TELEMETRY 1 // Enable telemetry sensor support
// #define HAS_SENSOR 1 // I2C sensors present
```
## platformio.ini Template
```ini
[env:my_variant]
extends = esp32s3_base ; Use architecture-specific base
board = esp32-s3-devkitc-1 ; PlatformIO board definition (or custom in boards/)
board_level = extra ; Build level: extra, or omit for default
custom_meshtastic_support_level = 1 ; 1 = PR builds, 2 = merge builds only
build_flags =
${esp32s3_base.build_flags}
-D MY_VARIANT_SPECIFIC_FLAG=1
-I variants/esp32s3/my_variant ; Include path for variant.h
upload_speed = 921600
```
### Common Base Configs
- `esp32_base` / `esp32-common.ini` — ESP32
- `esp32s3_base` — ESP32-S3
- `esp32c3_base` — ESP32-C3
- `esp32c6_base` — ESP32-C6
- `nrf52840_base` / `nrf52.ini` — nRF52840
- `rp2040_base` — RP2040/RP2350
### Support Levels
- `custom_meshtastic_support_level = 1` — Built on every PR (actively supported)
- `custom_meshtastic_support_level = 2` — Built only on merge to main branches
- `board_level = extra` — Only built on full releases
## Build Manifest Metadata
`bin/platformio-custom.py` emits UI capability flags in the build manifest:
- `custom_meshtastic_has_mui = true/false` — Override MUI detection
- `custom_meshtastic_has_ink_hud = true/false` — Override InkHUD detection
- Architecture names are normalized (e.g., `esp32s3``esp32-s3`)
## InkHUD E-Ink Variants
For e-ink display variants using the InkHUD framework, add `nicheGraphics.h`:
```cpp
// nicheGraphics.h — InkHUD configuration for this variant
#define INKHUD // Enable InkHUD
// Configure display, applets, and refresh behavior per device
```
InkHUD has its own PlatformIO config: `src/graphics/niche/InkHUD/PlatformioConfig.ini`
## I2C Device Detection
If the variant has I2C devices, ensure `src/detect/ScanI2C` will detect them. The auto-detection system handles 80+ device types including displays, sensors, RTCs, keyboards, PMUs, and touch controllers at boot.
## Custom Board Definitions
If the PlatformIO board doesn't exist, create a custom board JSON in `boards/`:
```json
{
"build": {
"arduino": { "ldscript": "esp32s3_out.ld" },
"core": "esp32",
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"mcu": "esp32s3",
"variant": "esp32s3"
},
"connectivity": ["wifi", "bluetooth"],
"frameworks": ["arduino", "espidf"],
"name": "My Custom Board",
"upload": {
"flash_size": "8MB",
"maximum_ram_size": 327680,
"maximum_size": 8388608
},
"url": "https://example.com",
"vendor": "MyVendor"
}
```
## Checklist
- [ ] Create `variants/<arch>/<name>/variant.h` with pin definitions
- [ ] Create `variants/<arch>/<name>/platformio.ini` extending correct base
- [ ] Set `custom_meshtastic_support_level` (1 or 2)
- [ ] Verify radio chip define matches hardware (`USE_SX1262`, etc.)
- [ ] Set hardware capability flags (`HAS_GPS`, `HAS_SCREEN`, etc.)
- [ ] Add custom board JSON in `boards/` if needed
- [ ] Test build: `pio run -e my_variant`
- [ ] For e-ink: add `nicheGraphics.h` with InkHUD config
+8 -3
View File
@@ -32,10 +32,15 @@ jobs:
shell: bash
working-directory: meshtasticd
run: |
# Build-tools (notably platformio) come from the Meshtastic project
# on the OpenSUSE Build Service:
# https://build.opensuse.org/project/show/network:Meshtastic:build-tools
echo 'deb http://download.opensuse.org/repositories/network:/Meshtastic:/build-tools/xUbuntu_24.04/ /' \
| sudo tee /etc/apt/sources.list.d/network:Meshtastic:build-tools.list
curl -fsSL https://download.opensuse.org/repositories/network:Meshtastic:build-tools/xUbuntu_24.04/Release.key \
| gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/network_Meshtastic_build-tools.gpg >/dev/null
sudo apt-get update -y --fix-missing
sudo apt-get install -y software-properties-common build-essential devscripts equivs
sudo add-apt-repository ppa:meshtastic/build-tools -y
sudo apt-get update -y --fix-missing
sudo apt-get install -y build-essential devscripts equivs
sudo mk-build-deps --install --remove --tool='apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends --yes' debian/control
- name: Import GPG key
+51
View File
@@ -0,0 +1,51 @@
name: Build MacOS Binary
on:
workflow_call:
inputs:
macos_ver:
required: false
default: "26" # ARM64
type: string
permissions:
contents: read
jobs:
build-MacOS:
runs-on: macos-${{ inputs.macos_ver }}
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
submodules: recursive
- name: Install deps
shell: bash
run: |
brew update
brew install platformio yaml-cpp libuv openssl@3 libusb argp-standalone pkg-config ulfius
- name: Get release version string
run: |
echo "long=$(./bin/buildinfo.py long)" >> $GITHUB_OUTPUT
id: version
- name: Build for MacOS
run: |
platformio run -e native-macos
env:
PKG_VERSION: ${{ steps.version.outputs.long }}
# Errors in this step should not fail the entire workflow while MacOS support is in development.
continue-on-error: true
- name: List output files
run: ls -lah .pio/build/native-macos/
- name: Store binaries as an artifact
uses: actions/upload-artifact@v7
with:
name: firmware-macos-${{ inputs.macos_ver }}-${{ steps.version.outputs.long }}
overwrite: true
path: |
.pio/build/native-macos/meshtasticd
+3 -1
View File
@@ -73,7 +73,9 @@ jobs:
- name: Sanitize platform string
id: sanitize_platform
# Replace slashes with underscores
run: echo "cleaned_platform=${{ inputs.platform }}" | sed 's/\//_/g' >> $GITHUB_OUTPUT
env:
plat: ${{ inputs.platform }}
run: echo "cleaned_platform=${plat}" | sed 's/\//_/g' >> $GITHUB_OUTPUT
- name: Docker login
if: ${{ inputs.push }}
+22
View File
@@ -43,6 +43,15 @@ jobs:
push: true
secrets: inherit
docker-debian-riscv64:
uses: ./.github/workflows/docker_build.yml
with:
distro: debian
platform: linux/riscv64
runs-on: ubuntu-24.04-arm
push: true
secrets: inherit
docker-alpine-amd64:
uses: ./.github/workflows/docker_build.yml
with:
@@ -70,16 +79,27 @@ jobs:
push: true
secrets: inherit
docker-alpine-riscv64:
uses: ./.github/workflows/docker_build.yml
with:
distro: alpine
platform: linux/riscv64
runs-on: ubuntu-24.04-arm
push: true
secrets: inherit
docker-manifest:
needs:
# Debian
- docker-debian-amd64
- docker-debian-arm64
- docker-debian-armv7
- docker-debian-riscv64
# Alpine
- docker-alpine-amd64
- docker-alpine-arm64
- docker-alpine-armv7
- docker-alpine-riscv64
runs-on: ubuntu-24.04
steps:
- name: Checkout code
@@ -162,6 +182,7 @@ jobs:
meshtastic/meshtasticd@${{ needs.docker-debian-amd64.outputs.digest }}
meshtastic/meshtasticd@${{ needs.docker-debian-arm64.outputs.digest }}
meshtastic/meshtasticd@${{ needs.docker-debian-armv7.outputs.digest }}
meshtastic/meshtasticd@${{ needs.docker-debian-riscv64.outputs.digest }}
- name: Docker meta (Alpine)
id: meta_alpine
@@ -182,3 +203,4 @@ jobs:
meshtastic/meshtasticd@${{ needs.docker-alpine-amd64.outputs.digest }}
meshtastic/meshtasticd@${{ needs.docker-alpine-arm64.outputs.digest }}
meshtastic/meshtasticd@${{ needs.docker-alpine-armv7.outputs.digest }}
meshtastic/meshtasticd@${{ needs.docker-alpine-riscv64.outputs.digest }}
+20 -2
View File
@@ -116,6 +116,20 @@ jobs:
build_location: local
secrets: inherit
MacOS:
strategy:
fail-fast: false
matrix:
macos_ver:
- "26" # ARM64
# - '26-intel' # x86_64
- "15" # ARM64
# - '15-intel' # x86_64
uses: ./.github/workflows/build_macos_bin.yml
with:
macos_ver: ${{ matrix.macos_ver }}
# secrets: inherit
package-pio-deps-native-tft:
if: ${{ github.repository == 'meshtastic/firmware' && github.event_name == 'workflow_dispatch' }}
uses: ./.github/workflows/package_pio_deps.yml
@@ -286,6 +300,7 @@ jobs:
- gather-artifacts
- build-debian-src
- package-pio-deps-native-tft
# - MacOS
steps:
- name: Checkout
uses: actions/checkout@v6
@@ -301,10 +316,12 @@ jobs:
id: release_notes
run: |
chmod +x ./bin/generate_release_notes.py
NOTES=$(./bin/generate_release_notes.py ${{ needs.version.outputs.long }})
NOTES=$(./bin/generate_release_notes.py ${{ needs.version.outputs.long }} --compare-ref HEAD 2>release_notes.log)
echo "notes<<EOF" >> $GITHUB_OUTPUT
echo "$NOTES" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
echo "### Release note range" >> $GITHUB_STEP_SUMMARY
cat release_notes.log >> $GITHUB_STEP_SUMMARY
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -316,6 +333,7 @@ jobs:
prerelease: true
name: Meshtastic Firmware ${{ needs.version.outputs.long }} Alpha
tag_name: v${{ needs.version.outputs.long }}
target_commitish: ${{ github.sha }}
body: ${{ steps.release_notes.outputs.notes }}
- name: Download source deb
@@ -466,7 +484,7 @@ jobs:
- name: Generate release notes
run: |
chmod +x ./bin/generate_release_notes.py
./bin/generate_release_notes.py ${{ needs.version.outputs.long }} > ./publish/release_notes.md
./bin/generate_release_notes.py ${{ needs.version.outputs.long }} --compare-ref HEAD > ./publish/release_notes.md
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+7 -1
View File
@@ -86,7 +86,13 @@ jobs:
run: sed -i 's/-DBUILD_EPOCH=$UNIX_TIME/#-DBUILD_EPOCH=$UNIX_TIME/' platformio.ini
- name: PlatformIO Tests
run: platformio test -e coverage -v --junit-output-path testreport.xml
run: |
set -o pipefail
# Filter out SKIPPED summary rows for hardware variants that can't run on the
# native host. They flood the log and make it harder to spot real failures.
# The JUnit XML is written directly to testreport.xml before the pipe, so
# the test artifact is unaffected.
platformio test -e coverage -v --junit-output-path testreport.xml 2>&1 | grep -v "[[:space:]]SKIPPED$"
- name: Save test results
if: always() # run this step even if previous step failed
+12
View File
@@ -47,6 +47,10 @@ data/boot/logo.*
managed_components/*
arduino-lib-builder*
dependencies.lock
# JLink / RTT debug artifacts (nRF SoCs)
flash.jlink
rtt_*.txt
idf_component.yml
CMakeLists.txt
/sdkconfig.*
@@ -54,3 +58,11 @@ CMakeLists.txt
# PYTHONPATH used by the Nix shell
.python3
.claude/scheduled_tasks.lock
userPrefs.jsonc.mcp-session-bak
# Fake-NodeDB fixture pipeline (bin/regen-fake-nodedbs.sh)
# JSONL seeds are committed (test/fixtures/nodedb/seed_v25_*.jsonl);
# compiled .proto outputs are ephemeral build artifacts.
build/fixtures/
bin/_generated/
+11
View File
@@ -0,0 +1,11 @@
{
"mcpServers": {
"meshtastic": {
"command": "./mcp-server/.venv/bin/python",
"args": ["-m", "meshtastic_mcp"],
"env": {
"MESHTASTIC_FIRMWARE_ROOT": "."
}
}
}
}
+27 -1
View File
@@ -1,2 +1,28 @@
[bandit]
skips = B101
# Rule IDs: https://bandit.readthedocs.io/en/latest/plugins/index.html
#
# B101 assert_used
# pytest assertions + internal invariants; required for pytest.
# B110 try_except_pass
# best-effort cleanup paths (atexit handlers, pubsub unsubscribe,
# session-end file close, socket shutdown). Logging inside the
# except block would be worse than the silent pass — teardown is
# already at end-of-session and the surrounding caller has context.
# B112 try_except_continue
# defensive loops over flaky sources (pubsub handlers, device
# re-enumeration polls). One failed iteration shouldn't abort the loop.
# B404 import_subprocess
# mcp-server wraps PlatformIO, esptool, nrfutil, picotool, and the
# pytest test-runner — subprocess is a load-bearing import here, not
# a smell. The "consider possible security implications" advisory is
# redundant given the file-level review already applied.
# B603 subprocess_without_shell_equals_true
# all subprocess calls use a static argv list; `shell=False` is the
# default and we never string-interpolate user input into the command.
# B606 start_process_with_no_shell
# same invariant as B603 — running a binary via argv list (not
# `shell=True`) is the safe pattern bandit is asking for.
#
# Higher-severity checks (B102 exec_used, B301 pickle, B307 eval,
# B602 shell=True, etc.) remain enabled.
skips = B101,B110,B112,B404,B603,B606
+17 -10
View File
@@ -4,29 +4,29 @@ cli:
plugins:
sources:
- id: trunk
ref: v1.7.6
ref: v1.10.0
uri: https://github.com/trunk-io/plugins
lint:
enabled:
- checkov@3.2.517
- renovate@43.110.9
- prettier@3.8.1
- trufflehog@3.94.3
- checkov@3.2.529
- renovate@43.150.0
- prettier@3.8.3
- trufflehog@3.95.3
- yamllint@1.38.0
- bandit@1.9.4
- trivy@0.69.3
- trivy@0.70.0
- taplo@0.10.0
- ruff@0.15.9
- ruff@0.15.13
- isort@8.0.1
- markdownlint@0.48.0
- oxipng@10.1.0
- oxipng@10.1.1
- svgo@4.0.1
- actionlint@1.7.12
- flake8@7.3.0
- hadolint@2.14.0
- shfmt@3.6.0
- shellcheck@0.11.0
- black@26.3.1
- black@26.5.1
- git-diff-check
- gitleaks@8.30.1
- clang-format@16.0.3
@@ -34,9 +34,16 @@ lint:
- linters: [ALL]
paths:
- bin/**
# Fake-NodeDB fixture JSONL files contain deterministic synthetic
# public_key_hex (64-char hex) values that gitleaks misidentifies as
# generic-api-key. These are not secrets — they're test fixtures
# produced by bin/gen-fake-nodedb-seed.py with a fixed RNG seed.
- linters: [gitleaks]
paths:
- test/fixtures/nodedb/seed_v25_*.jsonl
runtimes:
enabled:
- python@3.10.8
- python@3.14.4
- go@1.21.0
- node@22.16.0
actions:
+4 -4
View File
@@ -1,10 +1,10 @@
{
// See http://go.microsoft.com/fwlink/?LinkId=827846
// for the documentation about the extensions.json format
"recommendations": [
"platformio.platformio-ide"
"Jason2866.esp-decoder",
"pioarduino.pioarduino-ide"
],
"unwantedRecommendations": [
"ms-vscode.cpptools-extension-pack"
"ms-vscode.cpptools-extension-pack",
"platformio.platformio-ide"
]
}
+144
View File
@@ -0,0 +1,144 @@
# Agent instructions
This repository is the [Meshtastic](https://meshtastic.org) firmware — a C++17 embedded codebase targeting ESP32 / nRF52 / RP2040 / STM32WL / Linux-Portduino LoRa mesh radios — plus a Python MCP server in `mcp-server/` that AI agents use to flash, configure, and test connected devices.
## Primary instruction file
**Read `.github/copilot-instructions.md` first.** That file is the canonical agent-facing document for this repo. It covers project layout, coding conventions (naming, module framework, Observer pattern, thread safety), the build system, CI/CD, the native C++ test suite, and — most importantly for automation work — the **MCP Server & Hardware Test Harness** section. Read it top-to-bottom before starting any non-trivial change.
This file (`AGENTS.md`) is a short pointer + quick reference for agents that don't read `.github/copilot-instructions.md` by default.
## Quick command reference
| Action | Command |
| -------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| Build a firmware variant | `pio run -e <env>` (e.g. `pio run -e rak4631`, `pio run -e heltec-v3`) |
| Build native macOS host binary | `pio run -e native-macos` (Homebrew prereqs + CH341 LoRa setup in `variants/native/portduino/platformio.ini`) |
| Clean + rebuild | `pio run -e <env> -t clean && pio run -e <env>` |
| Flash a device | `pio run -e <env> -t upload --upload-port <port>` (or use the `pio_flash` MCP tool) |
| Run firmware unit tests (native) | `pio test -e native` |
| Run MCP hardware tests | `./mcp-server/run-tests.sh` |
| Live TUI test runner | `mcp-server/.venv/bin/meshtastic-mcp-test-tui` |
| Format before commit | `trunk fmt` |
| Regenerate protobuf bindings | `bin/regen-protos.sh` |
| Generate CI matrix | `./bin/generate_ci_matrix.py all [--level pr]` |
## MCP server (device + test automation)
The `mcp-server/` package exposes ~32 MCP tools for device discovery, building, flashing, serial monitoring, and live-node administration. Tools are grouped as:
- **Discovery**: `list_devices`, `list_boards`, `get_board`
- **Build & flash**: `build`, `clean`, `pio_flash`, `erase_and_flash` (ESP32 factory), `update_flash` (ESP32 OTA), `touch_1200bps`
- **Serial sessions**: `serial_open`, `serial_read`, `serial_list`, `serial_close`
- **Device reads**: `device_info`, `list_nodes`
- **Device writes** (require `confirm=True`): `set_owner`, `get_config`, `set_config`, `get_channel_url`, `set_channel_url`, `send_text`, `reboot`, `shutdown`, `factory_reset`, `set_debug_log_api`
- **userPrefs admin**: `userprefs_get`, `userprefs_set`, `userprefs_reset`, `userprefs_manifest`, `userprefs_testing_profile`
- **Vendor escape hatches**: `esptool_*`, `nrfutil_*`, `picotool_*`
Setup: `cd mcp-server && python3 -m venv .venv && .venv/bin/pip install -e '.[test]'`. The repo registers the server via `.mcp.json` — Claude Code picks it up automatically.
See `mcp-server/README.md` for argument shapes and the **MCP Server & Hardware Test Harness** section of `.github/copilot-instructions.md` for agent usage rules (tool surface, fixture contract, firmware integration points, recovery playbooks).
## Slash commands (AI-assisted workflows)
Three test-and-diagnose workflows exist as slash commands:
- **`/test` (Claude Code) / `/mcp-test` (Copilot)** — run the hardware test suite and interpret failures
- **`/diagnose` / `/mcp-diagnose`** — read-only device health report
- **`/repro` / `/mcp-repro`** — flakiness triage: re-run one test N times, diff firmware logs between passes and failures
Bodies live in `.claude/commands/` and `.github/prompts/` respectively. `.claude/commands/README.md` is the index.
## Encryption at a glance
Two layers, both in `src/mesh/CryptoEngine.cpp`:
- **Channel (symmetric)** — **AES-CTR** with a channel-wide PSK (AES-128 or AES-256). Nonce = packet_id ‖ from_node ‖ block_counter. No AEAD; integrity is soft (channel-hash filter). The well-known default PSK lives in `src/mesh/Channels.h`; a 1-byte PSK is a short-form index into it.
- **Per-peer PKI** — **X25519 ECDH** (Curve25519, 32-byte keys) → SHA-256 → **AES-256-CCM** with an 8-byte MAC. Fresh 32-bit `extraNonce` per packet, sent in the clear alongside the MAC. 12-byte wire overhead (`MESHTASTIC_PKC_OVERHEAD`). Used for DMs. Also used for remote admin (`src/modules/AdminModule.cpp`), where AdminMessage authorization is gated by `config.security.admin_key[0..2]`. Disabled entirely in Ham mode (`user.is_licensed=true`).
Key rotation to never trigger casually: only the **full** factory reset (`factory_reset_device`, `eraseBleBonds=true`) wipes `security.private_key` and regenerates the keypair — every peer holds the old public key, so DMs silently fail PKI decrypt until NodeInfo re-exchanges. The **partial** config reset (`factory_reset_config`) preserves the private key and doesn't invalidate peer relationships. Explicitly blanking `security.private_key` via admin also triggers regen. See the **Encryption & Key Management** section of `.github/copilot-instructions.md` for the full spec (nonce layout, send/receive selection logic including infrastructure-portnum exceptions, admin-key + session-passkey authorization, `is_managed` scope, key-rotation hazards).
## House rules
- **No destructive device operations without operator approval.** `factory_reset`, `erase_and_flash`, `reboot`, `shutdown`, history-rewriting git ops — describe the action and stop. Operator authorizes.
- **One MCP call per serial port at a time.** The port lock is exclusive; concurrent calls deadlock. Sequence: open → read/mutate → close, then next device.
- **`userPrefs.jsonc` is session state during tests.** The `_session_userprefs` fixture snapshots + restores it; never edit it from inside a test.
- **Don't speculate about firmware root causes.** When evidence doesn't support a classification, say "unknown" and list what would disambiguate.
- **Run `trunk fmt` before proposing a commit.** The `trunk_check` CI gate will reject unformatted code.
- **`confirm=True` on destructive MCP tools is a real gate, not a formality.** Don't bypass it via auto-approve settings.
- **Keep code comments minimal — one or two lines, max.** Comment only when the _why_ isn't obvious from the code; never restate what the next line does. No multi-paragraph block comments explaining straightforward changes. The diff and commit message carry the rationale; the code carries the behavior.
- **Use `Throttle` for time-based rate limiting, not raw `millis()` math.** `src/mesh/Throttle.h` provides `Throttle::isWithinTimespanMs(lastMs, intervalMs)` (returns true while inside the cooldown) and `Throttle::execute(&lastMs, intervalMs, func)` (function-pointer form that updates the timestamp on fire). Use these for any "did N ms pass since X" check — raw `millis() > lastMs + N` is rollover-unsafe (breaks after ~49.7 days) and inconsistent with the rest of the codebase. The helpers compute `now - lastMs` with unsigned subtraction, which wraps correctly.
## Typical agent workflows
### Flashing a device
1. `list_devices` → find the port + likely VID
2. `list_boards` → confirm the env, or use the known default for the hardware
3. `pio_flash(env=..., port=..., confirm=True)` for any arch, or `erase_and_flash(env=..., port=..., confirm=True)` for an ESP32 factory install
### Inspecting live node state
1. `device_info(port=...)` — short summary (node num, firmware version, region, peer count)
2. `list_nodes(port=...)` — full peer table (SNR, RSSI, pubkey presence, last_heard)
3. `get_config(section="lora", port=...)` — LoRa settings for cross-device comparison
Sequence these; don't parallelize on the same port.
### Testing a firmware change
1. Build locally: `pio run -e <env>`
2. Flash the test device: `pio_flash(env=..., port=..., confirm=True)`
3. Run the suite: `./mcp-server/run-tests.sh tests/<tier>` or `/test tests/<tier>`
4. On failure, open `mcp-server/tests/report.html``Meshtastic debug` section for the firmware log tail + device state dump
5. Iterate
### Debugging a flaky test
1. `/repro <test-node-id> [count]` — re-runs the test N times, diffs firmware logs between passes and failures
2. If the first attempt always fails and the rest pass, that's a state-leak pattern → suggest `--force-bake` or a clean device state, don't chase the first failure
3. If all N fail, this isn't a flake — it's a regression. Stop iterating and escalate to `/test` for full-suite context.
## Where to look
| Path | What's there |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `src/` | Firmware C++ source (`mesh/`, `modules/`, `platform/`, `graphics/`, `gps/`, `motion/`, `mqtt/`, …) |
| `src/mesh/` | Core: NodeDB, Router, Channels, CryptoEngine, radio interfaces, StreamAPI, PhoneAPI |
| `src/modules/` | Feature modules; `Telemetry/Sensor/` has 50+ I2C sensor drivers |
| `variants/` | 200+ hardware variant definitions (`variant.h` + `platformio.ini` per board) |
| `protobufs/` | `.proto` definitions; regenerate with `bin/regen-protos.sh` |
| `test/` | Firmware unit tests (12 suites; `pio test -e native`) |
| `mcp-server/` | Python MCP server + pytest hardware integration tests |
| `mcp-server/tests/` | Tiered pytest suite: `unit/`, `mesh/`, `telemetry/`, `monitor/`, `recovery/`, `ui/`, `fleet/`, `admin/`, `provisioning/` |
| `.claude/commands/` | Claude Code slash command bodies |
| `.github/prompts/` | Copilot prompt bodies (mirrors of the Claude Code ones) |
| `.github/copilot-instructions.md` | **Primary agent instructions — read this** |
| `.github/workflows/` | CI pipelines |
| `.mcp.json` | MCP server registration for Claude Code |
## Recovery one-liners
- **`userPrefs.jsonc` dirty after a test run?** Re-run `./mcp-server/run-tests.sh` once (pre-flight self-heals from the sidecar). If still dirty: `git checkout userPrefs.jsonc`.
- **nRF52 not responding?** `mcp__meshtastic__touch_1200bps(port=...)` drops it into the DFU bootloader, then `pio_flash` re-installs.
- **Device fully wedged (no DFU)?** `mcp__meshtastic__uhubctl_cycle(role="nrf52", confirm=True)` hard-power-cycles it via USB hub PPPS. Needs `uhubctl` installed (`brew install uhubctl` / `apt install uhubctl`); on Linux without udev rules, permission errors fail fast, so use `sudo uhubctl` yourself or configure udev access.
- **Port busy?** `lsof <port>` to find the holder. Usually a stale `pio device monitor` or zombie `meshtastic_mcp` process. Kill it.
- **Multiple MCP servers running?** `ps aux | grep meshtastic_mcp` — zombies hold ports. Kill all but the one your host spawned.
- **macOS: `LIBUSB_ERROR_BUSY` on a CH341 LoRa adapter?** A third-party WCH `CH34xVCPDriver` is claiming interface 0. Find the bundle ID with `ioreg -p IOUSB -l -w 0 | grep -B2 -A30 0x5512`, then `sudo kmutil unload -b <bundleID>`. Apple's bundled CH34x kext targets the CH340 UART (PID 0x7523), not the SPI bridge — it's never the culprit.
## Environment variables (test harness)
| Var | Purpose |
| ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `MESHTASTIC_MCP_ENV_<ROLE>` | Override PlatformIO env for a role (e.g. `MESHTASTIC_MCP_ENV_NRF52=rak4631-dap`). Default map: `nrf52→rak4631`, `esp32s3→heltec-v3`. |
| `MESHTASTIC_MCP_SEED` | PSK seed for the session test profile. Defaults to `mcp-<user>-<host>`. |
| `MESHTASTIC_MCP_FLASH_LOG` | File path to tee pio/esptool/nrfutil/picotool output. `run-tests.sh` sets this to `tests/flash.log` so the TUI can stream live flash progress. |
| `MESHTASTIC_MCP_TCP_HOST` | `host` or `host:port` of a `meshtasticd` daemon (e.g. the `native-macos` build). Surfaces it in `list_devices` as `tcp://host:port` so `connect()`-based tools target it transparently. Default port 4403. |
| `MESHTASTIC_UHUBCTL_BIN` | Absolute path to `uhubctl` binary. Default: PATH lookup. |
| `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` | Pin a role to a specific uhubctl hub location (e.g. `1-1.3`). Wins over VID auto-detection — use when multiple devices share a VID. |
| `MESHTASTIC_UHUBCTL_PORT_<ROLE>` | Pin a role to a specific hub port number. Required alongside `LOCATION_<ROLE>`. |
| `MESHTASTIC_UI_CAMERA_BACKEND` | Camera backend for UI tier + `capture_screen` tool: `opencv` / `ffmpeg` / `null` / `auto` (default). |
| `MESHTASTIC_UI_CAMERA_DEVICE` | Generic camera device (index or path). Used by the UI tier when no per-role var is set. |
| `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` | Per-role camera pinning (e.g. `MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0` for the OLED-bearing heltec-v3). |
| `MESHTASTIC_UI_OCR_BACKEND` | OCR engine selection: `easyocr` / `pytesseract` / `null` / `auto` (default). |
| `MESHTASTIC_UI_TUI_CAMERA` | Set to `1` to mount the live camera-feed panel in `meshtastic-mcp-test-tui`. |
+3 -1
View File
@@ -3,15 +3,17 @@
# trunk-ignore-all(hadolint/DL3008): Do not pin apt package versions
# trunk-ignore-all(hadolint/DL3013): Do not pin pip package versions
FROM python:3.14-slim-trixie AS builder
FROM debian:trixie AS builder
ARG PIO_ENV=native
ENV DEBIAN_FRONTEND=noninteractive
ENV TZ=Etc/UTC
# Install Dependencies
ENV PIP_ROOT_USER_ACTION=ignore
ENV PIP_BREAK_SYSTEM_PACKAGES=1
RUN apt-get update && apt-get install --no-install-recommends -y \
curl wget g++ zip git ca-certificates pkg-config \
python3-pip python3-grpc-tools \
libgpiod-dev libyaml-cpp-dev libbluetooth-dev libi2c-dev libuv1-dev \
libusb-1.0-0-dev libulfius-dev liborcania-dev libssl-dev \
libx11-dev libinput-dev libxkbcommon-x11-dev libsqlite3-dev libsdl2-dev \
+10 -3
View File
@@ -3,12 +3,19 @@
# trunk-ignore-all(hadolint/DL3018): Do not pin apk package versions
# trunk-ignore-all(hadolint/DL3013): Do not pin pip package versions
FROM python:3.14-alpine3.22 AS builder
# Ensure the Alpine version is updated in both stages of the container!
FROM alpine:3.23 AS builder
ARG PIO_ENV=native
ENV PIP_ROOT_USER_ACTION=ignore
# Enable Alpine community repository (for 'py3-grpcio-tools')
RUN echo "https://dl-cdn.alpinelinux.org/alpine/v$(cut -d. -f1,2 /etc/alpine-release)/community" >> /etc/apk/repositories
# Install Dependencies
ENV PIP_ROOT_USER_ACTION=ignore
ENV PIP_BREAK_SYSTEM_PACKAGES=1
RUN apk --no-cache add \
bash g++ libstdc++-dev linux-headers zip git ca-certificates libbsd-dev \
py3-pip py3-grpcio-tools \
libgpiod-dev yaml-cpp-dev bluez-dev \
libusb-dev i2c-tools-dev libuv-dev openssl-dev pkgconf argp-standalone \
libx11-dev libinput-dev libxkbcommon-dev sqlite-dev sdl2-dev \
@@ -60,4 +67,4 @@ EXPOSE 4403
CMD [ "sh", "-cx", "meshtasticd --fsdir=/var/lib/meshtasticd" ]
HEALTHCHECK NONE
HEALTHCHECK NONE
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env python3
"""Post-process protoc-generated Python files to live under a local namespace.
Called by bin/regen-py-protos.sh. Walks the generated *_pb2.py files in the
target directory and rewrites every `meshtastic` reference (imports, dotted
attribute access) to use the new namespace (e.g., `meshtastic_v25`).
Why: the .proto files declare `package meshtastic;`, so protoc emits
`from meshtastic import mesh_pb2 as ...` lines. That would shadow the PyPI
`meshtastic` package which other parts of the mcp-server depend on. Renaming
to a local namespace keeps both available.
Usage:
_rewrite_proto_namespace.py <generated_dir> <new_namespace>
"""
from __future__ import annotations
import pathlib
import re
import sys
def rewrite(dir_path: pathlib.Path, new_ns: str) -> int:
# Standard protoc import forms:
# from meshtastic.X_pb2 import ... (rare, for direct symbol pulls)
# from meshtastic import X_pb2 as ... (common, the cross-file ref)
# import meshtastic.X_pb2 (also possible)
pattern_dotted_from = re.compile(r"^from meshtastic\.", re.MULTILINE)
pattern_bare_from = re.compile(r"^from meshtastic import ", re.MULTILINE)
pattern_dotted_import = re.compile(r"^import meshtastic\.", re.MULTILINE)
count = 0
for p in dir_path.glob("*.py"):
text = p.read_text(encoding="utf-8")
new = pattern_dotted_from.sub(f"from {new_ns}.", text)
new = pattern_bare_from.sub(f"from {new_ns} import ", new)
new = pattern_dotted_import.sub(f"import {new_ns}.", new)
# NOTE: we deliberately leave `meshtastic/X.proto` source-filename
# references inside descriptor strings alone. The descriptor pool is
# keyed by source filename (independent of Python package layout), so
# those don't collide with the PyPI package's descriptors.
if new != text:
p.write_text(new, encoding="utf-8")
count += 1
return count
def main(argv: list[str]) -> int:
if len(argv) != 2:
print("usage: _rewrite_proto_namespace.py <generated_dir> <new_namespace>", file=sys.stderr)
return 2
dir_path = pathlib.Path(argv[0])
new_ns = argv[1]
if not dir_path.is_dir():
print(f"directory not found: {dir_path}", file=sys.stderr)
return 2
n = rewrite(dir_path, new_ns)
print(f"rewrote {n} file(s) in {dir_path} → namespace {new_ns}", file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+1 -1
View File
@@ -38,4 +38,4 @@ cp bin/device-install.* $OUTDIR/
cp bin/device-update.* $OUTDIR/
echo "Copying manifest"
cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json || true
cp $BUILDDIR/$basename.mt.json $OUTDIR/$basename.mt.json
+2 -1
View File
@@ -31,5 +31,6 @@ basename=meshtasticd-$1-$VERSION
pio pkg install --environment "$PIO_ENV" || platformioFailed
pio run --environment "$PIO_ENV" || platformioFailed
cp "$BUILDDIR/meshtasticd" "$OUTDIR/meshtasticd_linux_$(uname -m)"
os_name=$(uname -s | tr '[:upper:]' '[:lower:]')
cp "$BUILDDIR/meshtasticd" "$OUTDIR/meshtasticd_${os_name}_$(uname -m)"
cp bin/native-install.* $OUTDIR/
-10
View File
@@ -204,16 +204,6 @@ HostMetrics:
# UserStringCommand: cat /sys/firmware/devicetree/base/serial-number # Command to execute, to send the results as the userString
StoreAndForward:
# Enabled: true # Enable Store and Forward++, true by default
# DBPath: /var/lib/meshtasticd/ # Path to the S&F++ Sqlite DB
# Stratum0: false # Specify if this node is a Stratum 0 node, the controller node.
# InitialSync: 10 # Number of messages to
# Hops: 3 # Number of hops to use for SF++ messages
# AnnounceInterval: 5 # Interval in minutes between announcing tip of chain hash
# MaxChainLength: 1000 # Maximum number of messages to store in a chain
Config:
# DisplayMode: TWOCOLOR # uncomment to force BaseUI
# DisplayMode: COLOR # uncomment to force MUI
@@ -0,0 +1,31 @@
# For use with Armbian luckfox-pico-max
# Waveshare LoRa HAT for Raspberry Pi Pico
# https://www.waveshare.com/wiki/Pico-LoRa-SX1262
Meta:
name: luckfox-pico-max-ws-raspberry-pi-pico-hat
support: community
compatible:
- luckfox-pico-max # Armbian
Lora:
Module: sx1262
DIO2_AS_RF_SWITCH: true
DIO3_TCXO_VOLTAGE: true
spidev: spidev0.0
Busy: # GPIO1_C7 / GP2
pin: 55
gpiochip: 1
line: 23
CS: # GPIO1_C6 / GP3
pin: 54
gpiochip: 1
line: 22
Reset: # GPIO1_D1 / GP15
pin: 57
gpiochip: 1
line: 25
IRQ: # GPIO2_A2 / GP20
pin: 66
gpiochip: 2
line: 2
+18
View File
@@ -0,0 +1,18 @@
# Station G3 motherboard with a Raspberry Pi Zero 2W as the MCU daughterboard.
# Verify spidev / I2C device paths for your OS — they may differ.
Meta:
name: Station G3
support: community
compatible:
- raspberry-pi
Lora:
Module: sx1262
IRQ: 22 # BCM pin — wiki spec
Reset: 16 # BCM pin — wiki spec
Busy: 24 # BCM pin — wiki spec
# CS: 8 # BCM 8 = SPI0 CE0 (default); uncomment only to override
DIO2_AS_RF_SWITCH: true
DIO3_TCXO_VOLTAGE: true
spidev: spidev0.0
# SX126X_MAX_POWER: 19 # matches Station G2 firmware cap; raise carefully per PA jumper mode
File diff suppressed because it is too large Load Diff
+439
View File
@@ -0,0 +1,439 @@
#!/usr/bin/env python3
"""Deterministic seed-data generator for the fake NodeDB fixture pipeline.
Writes a JSONL file describing N fake-but-realistic Meshtastic peers.
The output is hand-editable and committed; a sibling compile step
(bin/seed-json-to-proto.py) turns it into a binary `meshtastic_NodeDatabase`
v25 protobuf with fresh "now-relative" timestamps.
Determinism contract:
Same --seed -> byte-identical JSONL output, regardless of wall clock.
All timestamps are stored as `*_offset_sec` (seconds before "now"); the
compile step resolves them to absolute epochs at compile time.
Structural fields covered:
* NodeInfoLite header: num, long_name, short_name, hw_model, role,
public_key, snr, channel, hops_away, next_hop, bitfield flags
* PositionLite: lat/long Gaussian around --centroid, altitude, source
* DeviceMetrics: battery/voltage/util/uptime
* EnvironmentMetrics: temp/humidity/pressure/iaq
* StatusMessage: error_code (usually zero)
Active-board allow-list:
hw_model values are restricted to the intersection of
(a) variants with `custom_meshtastic_support_level = 1` in
variants/*/*/platformio.ini, AND
(b) values present in the `HardwareModel` enum in mesh.proto.
See HW_MODEL_WEIGHTS below. Deprecated boards (legacy TLORA / Heltec V1-2 /
classic TBEAM / TBEAM_V0P7 / Nano G1 / etc.) and fuzzer-only sentinels
(PORTDUINO, ANDROID_SIM, DIY_V1, ...) are excluded.
Active-role allow-list:
Excludes ROUTER_CLIENT (deprecated v2.3.15) and REPEATER (deprecated v2.7.11).
"""
from __future__ import annotations
import argparse
import datetime as _dt
import json
import math
import pathlib
import random
import sys
# --------------------------------------------------------------------------
# Active-board allow-list (intersection of tier-1 variants + HardwareModel enum).
# Refresh by running:
# for f in $(find variants -name 'platformio.ini' | xargs grep -lE 'custom_meshtastic_support_level = 1'); do
# grep custom_meshtastic_hw_model_slug $f | awk -F= '{print $2}' | tr -d ' ';
# done | sort -u | comm -12 - <(python3 -c "from meshtastic.protobuf.mesh_pb2 import HardwareModel; print('\\n'.join(HardwareModel.keys()))" | sort)
# --------------------------------------------------------------------------
HW_MODEL_WEIGHTS: dict[str, float] = {
"HELTEC_V3": 14.0,
"T_DECK": 9.0,
"HELTEC_V4": 8.0,
"RAK4631": 8.0,
"HELTEC_MESH_POCKET": 6.0,
"TRACKER_T1000_E": 5.0,
"HELTEC_MESH_NODE_T114": 5.0,
"T_DECK_PRO": 5.0,
"LILYGO_TBEAM_S3_CORE": 4.0,
"HELTEC_WIRELESS_PAPER": 4.0,
"HELTEC_WSL_V3": 3.0,
"T_ECHO": 3.0,
"HELTEC_WIRELESS_TRACKER": 3.0,
"HELTEC_WIRELESS_TRACKER_V2": 2.0,
"HELTEC_VISION_MASTER_E290": 2.0,
"HELTEC_MESH_SOLAR": 2.0,
"SEEED_WIO_TRACKER_L1": 2.0,
"T_LORA_PAGER": 1.5,
"HELTEC_VISION_MASTER_E213": 1.5,
"T_ECHO_PLUS": 1.0,
"MUZI_BASE": 1.0,
"WISMESH_TAP_V2": 1.0,
"THINKNODE_M2": 1.0,
"THINKNODE_M5": 1.0,
"TLORA_T3_S3": 1.0,
# Long tail (uniform low weight across remaining tier-1 boards):
"HELTEC_V4_R8": 0.3,
"HELTEC_VISION_MASTER_T190": 0.3,
"HELTEC_HT62": 0.3,
"HELTEC_MESH_NODE_T096": 0.3,
"M5STACK_C6L": 0.3,
"MINI_EPAPER_S3": 0.3,
"MUZI_R1_NEO": 0.3,
"NOMADSTAR_METEOR_PRO": 0.3,
"RAK3312": 0.3,
"RAK3401": 0.3,
"SEEED_SOLAR_NODE": 0.3,
"SEEED_WIO_TRACKER_L1_EINK": 0.3,
"SENSECAP_INDICATOR": 0.3,
"TBEAM_1_WATT": 0.3,
"THINKNODE_M1": 0.3,
"THINKNODE_M3": 0.3,
"THINKNODE_M6": 0.3,
"T_ECHO_LITE": 0.3,
"WISMESH_TAG": 0.3,
"WISMESH_TAP": 0.3,
"XIAO_NRF52_KIT": 0.3,
"CROWPANEL": 0.3,
}
# Non-deprecated roles only.
ROLE_WEIGHTS: dict[str, float] = {
"CLIENT": 75.0,
"CLIENT_MUTE": 5.0,
"ROUTER": 7.0,
"TRACKER": 3.0,
"SENSOR": 2.0,
"CLIENT_HIDDEN": 2.0,
"ROUTER_LATE": 2.0,
"CLIENT_BASE": 2.0,
"TAK": 1.0,
"TAK_TRACKER": 0.5,
"LOST_AND_FOUND": 0.5,
}
# Name pools — 60 firsts × 60 lasts = 3600 combinations.
FIRSTS = [
"Quick", "Brave", "Silent", "Wild", "Lone", "Bright", "Red", "Blue",
"Green", "Black", "White", "Iron", "Steel", "Copper", "Silver", "Gold",
"Stone", "River", "Forest", "Mountain", "Canyon", "Desert", "Storm", "Sky",
"Solar", "Lunar", "Dawn", "Dusk", "Misty", "Frosty", "Sunny", "Shady",
"Happy", "Sleepy", "Drowsy", "Sneaky", "Sharp", "Smooth", "Rough", "Loud",
"Soft", "Slow", "Fast", "Tall", "Short", "Old", "New", "Tiny",
"Giant", "Hidden", "Lost", "Found", "Wandering", "Roving", "Drifting", "Floating",
"Burning", "Frozen", "Whispering", "Howling",
]
LASTS = [
"Phoenix", "Lion", "Bear", "Wolf", "Hawk", "Eagle", "Fox", "Lynx",
"Cougar", "Coyote", "Raven", "Owl", "Crow", "Falcon", "Heron", "Crane",
"Otter", "Badger", "Bison", "Elk", "Moose", "Stag", "Doe", "Hare",
"Marmot", "Mole", "Beaver", "Squirrel", "Mustang", "Bronco", "Pony", "Colt",
"Cobra", "Viper", "Mamba", "Adder", "Gecko", "Iguana", "Tortoise", "Turtle",
"Salmon", "Trout", "Bass", "Pike", "Shark", "Whale", "Dolphin", "Seal",
"Cactus", "Yucca", "Sage", "Juniper", "Pine", "Cedar", "Aspen", "Oak",
"Bluff", "Mesa", "Arroyo", "Ridge",
]
# Brief callsign pool for licensed-looking suffixes.
CALLSIGN_PREFIXES = ["KX", "WD", "N5", "KE", "AB", "W5", "K1", "KQ", "AE", "NM"]
# Only emojis that fit in 4 UTF-8 bytes (no variation selectors). short_name's
# nanopb max_size:5 (incl. NUL) limits content to 4 bytes. ❄️ / ☀️ would be
# 6 bytes due to U+FE0F variation selector — explicitly excluded.
EMOJI_SHORTNAMES = ["🦊", "🐺", "🦅", "🐢", "🌵", "🔥", "🌙",
"🌊", "🗻", "🌲", "🦌", "🐝", "🦂", "🦉",
"🦇", "🦋"]
# --------------------------------------------------------------------------
# Helpers
# --------------------------------------------------------------------------
NUM_RESERVED = 4 # firmware reserves 0..3 (per NodeDB constants)
NUM_MAX_EXCLUSIVE = 0x80000000 # restrict to positive int32 range for readability
def _weighted_choice(rng: random.Random, weights: dict[str, float]) -> str:
"""Deterministic weighted pick. Uses sorted keys so dict order is fixed."""
keys = sorted(weights.keys())
totals = [weights[k] for k in keys]
return rng.choices(keys, weights=totals, k=1)[0]
def _gen_long_name(rng: random.Random, is_licensed: bool) -> str:
base = f"{rng.choice(FIRSTS)} {rng.choice(LASTS)}"
if is_licensed:
prefix = rng.choice(CALLSIGN_PREFIXES)
# Two trailing alpha chars after the digit; keep within 25 - len(base) - 1
suffix = f" {prefix}{rng.randint(0,9)}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}{rng.choice('ABCDEFGHIJKLMNOPQRSTUVWXYZ')}"
# nanopb max_size:25 means C string fits 24 bytes + NUL.
if len(base) + len(suffix) <= 24:
base = base + suffix
# Hard cap to 24 chars (nanopb max_size:25 minus NUL).
return base[:24]
def _gen_short_name(rng: random.Random, long_name: str) -> str:
# 10% emoji-only short_name
if rng.random() < 0.10:
return rng.choice(EMOJI_SHORTNAMES)
first_char = long_name[0].upper() if long_name else "X"
alphanums = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
return first_char + "".join(rng.choices(alphanums, k=3))
def _gen_hops_away(rng: random.Random) -> int:
# Geometric-ish: 0→55%, 1→25%, 2→12%, 3→5%, 4→2%, 5+→1%
r = rng.random()
if r < 0.55:
return 0
if r < 0.80:
return 1
if r < 0.92:
return 2
if r < 0.97:
return 3
if r < 0.99:
return 4
return rng.randint(5, 7)
def _gen_position(
rng: random.Random,
centroid_lat: float,
centroid_lon: float,
spread_km: float,
last_heard_offset_sec: int,
) -> dict:
# 1 deg ≈ 111 km at the equator; we use this as a flat approximation.
lat = centroid_lat + rng.gauss(0.0, spread_km / 111.0)
lon = centroid_lon + rng.gauss(0.0, spread_km / 111.0)
altitude = max(0, round(rng.gauss(1376.0, 250.0))) # T or C valley floor + relief
# Position was reported up to 300s before last_heard.
time_offset_sec = last_heard_offset_sec + rng.randint(0, 300)
return {
"latitude": round(lat, 6),
"longitude": round(lon, 6),
"altitude": altitude,
"time_offset_sec": time_offset_sec,
"location_source": "LOC_INTERNAL",
}
def _gen_telemetry(rng: random.Random) -> dict:
# 5% plugged-in (battery_level == 101); rest uniform [10..100].
if rng.random() < 0.05:
battery_level = 101
voltage = 4.20
else:
battery_level = rng.randint(10, 100)
voltage = round(3.3 + (battery_level / 100.0) * 0.9, 3)
# Beta distributions for low/right-skewed metrics; randomly draw via gammavariate.
def _beta(a: float, b: float) -> float:
x = rng.gammavariate(a, 1.0)
y = rng.gammavariate(b, 1.0)
return x / (x + y)
channel_utilization = round(_beta(2.0, 15.0) * 100.0, 2)
air_util_tx = round(_beta(1.5, 20.0) * 10.0, 3)
uptime_seconds = int(rng.expovariate(1.0 / 86400.0))
return {
"battery_level": battery_level,
"voltage": voltage,
"channel_utilization": channel_utilization,
"air_util_tx": air_util_tx,
"uptime_seconds": uptime_seconds,
}
def _gen_environment(rng: random.Random) -> dict:
return {
"temperature": round(rng.gauss(22.0, 8.0), 2),
"relative_humidity": round(min(100.0, max(0.0, rng.gauss(55.0, 20.0))), 2),
"barometric_pressure": round(rng.gauss(1013.0, 8.0), 2),
"iaq": int(min(500, max(0, round(rng.gauss(50.0, 30.0))))),
}
def _gen_status(rng: random.Random) -> dict:
# `StatusMessage` (mesh.proto:1445) has a single free-form `string status`.
# Most peers report a healthy short status; occasional alert string.
healthy = ["OK", "online", "active", "running", "ready", "nominal"]
alert = ["low-batt", "no-gps", "weak-signal", "rebooted", "offline-soon"]
if rng.random() < 0.92:
return {"status": rng.choice(healthy)}
return {"status": rng.choice(alert)}
def _gen_node(
rng: random.Random,
num: int,
centroid_lat: float,
centroid_lon: float,
spread_km: float,
coverage: dict[str, float],
last_heard_mean_sec: int,
last_heard_max_sec: int,
) -> dict:
is_licensed = rng.random() < 0.05
long_name = _gen_long_name(rng, is_licensed)
short_name = _gen_short_name(rng, long_name)
hw_model = _weighted_choice(rng, HW_MODEL_WEIGHTS)
role = _weighted_choice(rng, ROLE_WEIGHTS)
has_public_key = rng.random() < 0.92
public_key_hex = (
"".join(f"{rng.randint(0,255):02x}" for _ in range(32)) if has_public_key else ""
)
snr = round(max(-20.0, min(12.0, rng.gauss(6.0, 4.0))), 2)
channel = 0 if rng.random() < 0.90 else rng.randint(1, 7)
hops_away = _gen_hops_away(rng)
next_hop = rng.randint(0, 255) if hops_away > 0 else 0
last_heard_offset_sec = int(min(rng.expovariate(1.0 / last_heard_mean_sec), last_heard_max_sec))
bitfield = {
"has_user": True,
"is_favorite": rng.random() < 0.08,
"is_muted": rng.random() < 0.03,
"via_mqtt": rng.random() < 0.12,
"is_ignored": rng.random() < 0.01,
"is_licensed": is_licensed,
"has_is_unmessagable": True,
"is_unmessagable": rng.random() < 0.02,
"is_key_manually_verified": rng.random() < 0.04,
}
node: dict = {
"num": f"0x{num:08x}",
"long_name": long_name,
"short_name": short_name,
"hw_model": hw_model,
"role": role,
"public_key_hex": public_key_hex,
"snr": snr,
"channel": channel,
"hops_away": hops_away,
"next_hop": next_hop,
"last_heard_offset_sec": last_heard_offset_sec,
"bitfield": bitfield,
"position": (
_gen_position(rng, centroid_lat, centroid_lon, spread_km, last_heard_offset_sec)
if rng.random() < coverage["position"]
else None
),
"telemetry": _gen_telemetry(rng) if rng.random() < coverage["telemetry"] else None,
"environment": _gen_environment(rng) if rng.random() < coverage["environment"] else None,
"status": _gen_status(rng) if rng.random() < coverage["status"] else None,
}
return node
def _parse_my_node_num(s: str | None) -> int | None:
if s is None:
return None
s = s.strip()
if s.startswith("0x") or s.startswith("0X"):
return int(s, 16)
return int(s)
def main(argv: list[str]) -> int:
p = argparse.ArgumentParser(
description="Deterministic JSONL seed for the fake NodeDB fixture.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
p.add_argument("--count", type=int, required=True, help="Number of fake nodes to emit.")
p.add_argument("--seed", type=int, required=True, help="Deterministic seed.")
p.add_argument("--out", required=True, help="Output JSONL path.")
p.add_argument(
"--centroid",
default="33.1284,-107.2528",
help="LAT,LON centroid (default: Truth or Consequences, NM).",
)
p.add_argument("--spread-km", type=float, default=60.0, help="Gaussian std-dev in km.")
p.add_argument("--position-coverage", type=float, default=0.85)
p.add_argument("--telemetry-coverage", type=float, default=0.70)
p.add_argument("--environment-coverage", type=float, default=0.25)
p.add_argument("--status-coverage", type=float, default=0.40)
p.add_argument("--my-node-num", default=None, help="Exclude this NodeNum from generated set (hex or dec).")
p.add_argument("--last-heard-mean-sec", type=int, default=3600)
p.add_argument("--last-heard-max-sec", type=int, default=7 * 86400)
args = p.parse_args(argv)
if args.count <= 0:
print("--count must be positive", file=sys.stderr)
return 2
try:
centroid_lat, centroid_lon = (float(s) for s in args.centroid.split(","))
except ValueError:
print(f"--centroid must be LAT,LON; got {args.centroid!r}", file=sys.stderr)
return 2
my_node_num = _parse_my_node_num(args.my_node_num)
rng = random.Random(args.seed)
# 1) Generate a unique deterministic set of NodeNums.
nums: set[int] = set()
while len(nums) < args.count:
n = rng.randrange(NUM_RESERVED, NUM_MAX_EXCLUSIVE)
if my_node_num is not None and n == my_node_num:
continue
nums.add(n)
ordered_nums = sorted(nums) # sort to fix output order independent of set hash
# 2) Per-node generation (in num order, single RNG continues).
coverage = {
"position": args.position_coverage,
"telemetry": args.telemetry_coverage,
"environment": args.environment_coverage,
"status": args.status_coverage,
}
nodes = [
_gen_node(
rng,
n,
centroid_lat,
centroid_lon,
args.spread_km,
coverage,
args.last_heard_mean_sec,
args.last_heard_max_sec,
)
for n in ordered_nums
]
# 3) Write JSONL.
out_path = pathlib.Path(args.out)
out_path.parent.mkdir(parents=True, exist_ok=True)
# `generated_at_iso` is informational; it does NOT affect determinism because
# we derive it from the seed, not from wall clock. (Same seed -> same string.)
generated_at = _dt.datetime.fromtimestamp(args.seed, tz=_dt.timezone.utc).isoformat().replace("+00:00", "Z")
meta = {
"_meta": {
"version": 25,
"seed": args.seed,
"count": args.count,
"centroid": [centroid_lat, centroid_lon],
"spread_km": args.spread_km,
"generated_at_iso": generated_at,
"my_node_num_excluded": (None if my_node_num is None else f"0x{my_node_num:08x}"),
"coverage": coverage,
"last_heard_mean_sec": args.last_heard_mean_sec,
"last_heard_max_sec": args.last_heard_max_sec,
}
}
with out_path.open("w", encoding="utf-8") as f:
# `ensure_ascii=False` so emoji short_names survive. `sort_keys=True` for
# determinism (insertion order varies by Python version otherwise).
f.write(json.dumps(meta, ensure_ascii=False, sort_keys=True) + "\n")
for node in nodes:
f.write(json.dumps(node, ensure_ascii=False, sort_keys=True) + "\n")
print(f"wrote {args.count} nodes to {out_path} ({out_path.stat().st_size} bytes)", file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+54 -29
View File
@@ -1,25 +1,31 @@
#!/usr/bin/env python3
"""
Generate release notes from merged PRs on develop and master branches.
Categorizes PRs into Enhancements and Bug Fixes/Maintenance sections.
"""
"""Generate release notes from the actual release commit range."""
import subprocess
import re
import argparse
import json
import re
import subprocess
import sys
from datetime import datetime
def get_last_release_tag():
"""Get the most recent release tag."""
def get_last_release_tag(compare_ref, exclude_tag=None):
"""Get the most recent version tag merged into compare_ref."""
result = subprocess.run(
["git", "describe", "--tags", "--abbrev=0"],
["git", "tag", "--merged", compare_ref, "--sort=-version:refname", "v*"],
capture_output=True,
text=True,
check=True,
)
return result.stdout.strip()
for line in result.stdout.splitlines():
candidate = line.strip()
if not candidate:
continue
if exclude_tag and candidate == exclude_tag:
continue
return candidate
raise subprocess.CalledProcessError(result.returncode, result.args, output=result.stdout, stderr=result.stderr)
def get_tag_date(tag):
@@ -33,18 +39,18 @@ def get_tag_date(tag):
return result.stdout.strip()
def get_merged_prs_since_tag(tag, branch):
"""Get all merged PRs since the given tag on the specified branch."""
# Get commits since tag on the branch - look for PR numbers in parentheses
def get_merged_prs_in_range(tag, compare_ref):
"""Get all merged PRs in the git range between tag and compare_ref."""
result = subprocess.run(
[
"git",
"log",
f"{tag}..origin/{branch}",
f"{tag}..{compare_ref}",
"--oneline",
],
capture_output=True,
text=True,
check=True,
)
prs = []
@@ -65,6 +71,25 @@ def get_merged_prs_since_tag(tag, branch):
return prs
def parse_args():
"""Parse CLI arguments."""
parser = argparse.ArgumentParser(
description="Generate release notes from the actual release commit range."
)
parser.add_argument("new_version", help="Version that will be tagged for this release")
parser.add_argument(
"--base-tag",
dest="base_tag",
help="Existing version tag to diff from. Defaults to the latest version tag merged into the compare ref.",
)
parser.add_argument(
"--compare-ref",
default="HEAD",
help="Git ref to diff to. Defaults to HEAD.",
)
return parser.parse_args()
def get_pr_details(pr_number):
"""Get PR details from GitHub API via gh CLI."""
try:
@@ -268,28 +293,28 @@ def get_new_contributors(pr_details_list, tag, repo="meshtastic/firmware"):
def main():
if len(sys.argv) < 2:
print("Usage: generate_release_notes.py <new_version>", file=sys.stderr)
sys.exit(1)
new_version = sys.argv[1]
args = parse_args()
new_version = args.new_version
compare_ref = args.compare_ref
current_tag = f"v{new_version}"
# Get last release tag
try:
last_tag = get_last_release_tag()
last_tag = args.base_tag or get_last_release_tag(compare_ref, exclude_tag=current_tag)
except subprocess.CalledProcessError:
print("Error: Could not find last release tag", file=sys.stderr)
sys.exit(1)
# Collect PRs from both branches
all_pr_numbers = set()
print(
f"Resolved release note range: {last_tag}..{compare_ref}",
file=sys.stderr,
)
for branch in ["develop", "master"]:
try:
prs = get_merged_prs_since_tag(last_tag, branch)
all_pr_numbers.update(prs)
except Exception as e:
print(f"Warning: Could not get PRs from {branch}: {e}", file=sys.stderr)
try:
all_pr_numbers = set(get_merged_prs_in_range(last_tag, compare_ref))
except subprocess.CalledProcessError as e:
print(f"Error: Could not get PRs for range {last_tag}..{compare_ref}: {e}", file=sys.stderr)
sys.exit(1)
# Get details for all PRs
enhancements = []
@@ -87,6 +87,12 @@
</screenshots>
<releases>
<release version="2.7.24" date="2026-05-08">
<url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.24</url>
</release>
<release version="2.7.23" date="2026-04-14">
<url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.23</url>
</release>
<release version="2.7.22" date="2026-04-06">
<url type="details">https://github.com/meshtastic/firmware/releases?q=tag%3Av2.7.22</url>
</release>
+8 -4
View File
@@ -293,9 +293,12 @@ if ("HAS_TFT", 1) in env.get("CPPDEFINES", []):
board_arch = infer_architecture(env.BoardConfig())
should_skip_manifest = board_arch is None
# For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
mtjson_deps = [] if should_skip_manifest else ["buildprog"]
if not should_skip_manifest and platform.name == "espressif32":
# Most platforms can generate the manifest as part of the default 'buildprog' target.
# Typically this passes success/failure properly.
mtjson_deps = ["buildprog"]
if platform.name == "espressif32":
# On ESP32, we need to explicitly depend upon the binary to prevent fake-success upon failure.
mtjson_deps = ["$BUILD_DIR/${PROGNAME}.bin"]
# Build littlefs image as part of mtjson target
# Equivalent to `pio run -t buildfs`
target_lfs = env.DataToBin(
@@ -309,7 +312,8 @@ if should_skip_manifest:
env.AddCustomTarget(
name="mtjson",
dependencies=mtjson_deps,
# For host/native envs, avoid depending on 'buildprog' (some targets don't define it)
dependencies=[],
actions=[skip_manifest],
title="Meshtastic Manifest (skipped)",
description="mtjson generation is skipped for native environments",
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env bash
# Regenerate the fake-NodeDB fixtures: produces 250 / 500 / 1000 / 2000-node
# JSONL seed files + their compiled v25 protobufs.
#
# Layout:
# test/fixtures/nodedb/seed_v25_<N>.jsonl — COMMITTED, hand-editable.
# build/fixtures/nodedb/nodes_v25_<N>.proto — .gitignored, build artifact.
# Drop into /prefs/nodes.proto.
#
# Daily use: ./bin/regen-fake-nodedbs.sh
# - Recompiles protos from committed seeds (fresh wall-clock timestamps).
# Intentional seed bump: REGEN_SEEDS=yes ./bin/regen-fake-nodedbs.sh
# - Overwrites the committed JSONL files with freshly-seeded data.
set -euo pipefail
cd "$(dirname "$0")/.."
# 1) Make sure the Python protobuf bindings exist (in-tree generation; .gitignored).
if [[ ! -d bin/_generated/meshtastic ]]; then
echo "regenerating Python protobuf bindings (one-time)..."
./bin/regen-py-protos.sh
fi
# 2) Pick a Python interpreter that has the meshtastic deps installed.
# Prefer the mcp-server venv (most likely to be set up by the operator).
PY="python3"
for cand in mcp-server/.venv/bin/python3 .venv/bin/python3; do
if [[ -x "$cand" ]]; then
PY="$cand"
break
fi
done
# 3) Pinned seeds per size — bump only when you intentionally want different
# structural data committed. Parallel arrays so the script works on
# macOS bash 3.2 (no `declare -A`).
SIZES=(250 500 1000 2000)
SEEDS=(20260511 20260512 20260513 20260514)
REGEN_SEEDS="${REGEN_SEEDS:-no}"
mkdir -p build/fixtures/nodedb test/fixtures/nodedb
for i in 0 1 2 3; do
n="${SIZES[$i]}"
seed="${SEEDS[$i]}"
jsonl=$(printf "test/fixtures/nodedb/seed_v25_%04d.jsonl" "$n")
proto=$(printf "build/fixtures/nodedb/nodes_v25_%04d.proto" "$n")
if [[ "$REGEN_SEEDS" == "yes" || ! -f "$jsonl" ]]; then
$PY bin/gen-fake-nodedb-seed.py \
--count "$n" \
--seed "$seed" \
--out "$jsonl" \
--centroid 33.1284,-107.2528 \
--spread-km 60 \
--position-coverage 0.85 \
--telemetry-coverage 0.70 \
--environment-coverage 0.25 \
--status-coverage 0.40
echo " seed: $jsonl ($(wc -c < "$jsonl") bytes)"
fi
$PY bin/seed-json-to-proto.py --in "$jsonl" --out "$proto"
echo " proto: $proto ($(wc -c < "$proto") bytes)"
done
echo ""
echo "Done. To load on Portduino native:"
echo " cp build/fixtures/nodedb/nodes_v25_1000.proto ~/.portduino/default/prefs/nodes.proto"
echo ""
echo "To push to a hardware device:"
echo " Use the mcp-server tool: push_fake_nodedb(size=1000, target=\"hardware\", port=\"/dev/cu.usbmodemXXXX\", confirm=True)"
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env bash
# Regenerate Python protobuf bindings from the in-tree `protobufs/` submodule
# into `bin/_generated/`. Called by bin/regen-fake-nodedbs.sh; also useful as
# a standalone refresh after any change to a .proto file.
#
# Output is .gitignored — bindings are a build artifact.
#
# Namespace rewrite:
# The .proto files declare `package meshtastic;`, which makes protoc emit
# imports like `from meshtastic import mesh_pb2`. That conflicts with the
# PyPI `meshtastic` package (which the mcp-server relies on for its
# SerialInterface/BLEInterface transport). We post-process the generated
# files to live under `meshtastic_v25` instead — both the directory layout
# and all internal imports — so they coexist cleanly with the PyPI package.
set -euo pipefail
cd "$(dirname "$0")/.."
if ! command -v protoc >/dev/null 2>&1; then
echo "ERROR: protoc not found in PATH." >&2
echo " macOS: brew install protobuf" >&2
echo " Ubuntu/Debian: apt install protobuf-compiler" >&2
exit 1
fi
OUT=bin/_generated
LOCAL_NS=meshtastic_v25
rm -rf "$OUT"
mkdir -p "$OUT"
# 1) Generate from the in-tree protos. nanopb.proto first so its descriptor
# is available for the [(nanopb).*] options on other messages.
protoc \
--proto_path=protobufs \
--python_out="$OUT" \
protobufs/nanopb.proto \
protobufs/meshtastic/*.proto
# 2) Move the generated `meshtastic/` directory to `meshtastic_v25/`.
mv "$OUT/meshtastic" "$OUT/$LOCAL_NS"
# 3) Rewrite internal imports: any reference to `meshtastic.X_pb2` or
# `from meshtastic import X_pb2` becomes `meshtastic_v25.*`.
python3 bin/_rewrite_proto_namespace.py "$OUT/$LOCAL_NS" "$LOCAL_NS"
# 4) Make the package importable.
touch "$OUT/__init__.py"
touch "$OUT/$LOCAL_NS/__init__.py"
echo "regenerated Python protobuf bindings -> $OUT/$LOCAL_NS/ (namespace: $LOCAL_NS)" >&2
+342
View File
@@ -0,0 +1,342 @@
#!/usr/bin/env python3
"""Compile a committed seed JSONL into a binary meshtastic_NodeDatabase v25 proto.
The input is produced by `bin/gen-fake-nodedb-seed.py`. Timestamps in the JSONL
are stored as `*_offset_sec` (seconds before "now"); this script resolves them
to absolute epochs using `--now-epoch` (default: current wall clock).
Output is a raw `pb_encode`-compatible binary that can be dropped at
`/prefs/nodes.proto` on the device (Portduino prefs dir or hardware via
XModem) and loaded by `NodeDB::loadFromDisk` at boot.
Wire format reference:
protobufs/meshtastic/deviceonly.proto (NodeDatabase, NodeInfoLite, sat entries)
src/mesh/NodeDB.h:467-484 (bitfield bit positions)
src/mesh/NodeDB.cpp:1523-1524 (pb_decode entry point)
"""
from __future__ import annotations
import argparse
import json
import pathlib
import sys
import time
from typing import Any
# Prefer the in-tree generated Python protobuf bindings (bin/_generated/meshtastic_v25/)
# because the firmware branch's protos (v25 NodeDatabase satellite arrays, slim
# NodeInfoLite) are typically newer than what the PyPI `meshtastic` package
# ships. Run `bin/regen-py-protos.sh` to (re)generate.
#
# Namespace note: the local bindings live under `meshtastic_v25` (NOT `meshtastic`)
# to avoid shadowing the PyPI `meshtastic` package — bin/regen-py-protos.sh
# post-processes the protoc output to rename the package.
_HERE = pathlib.Path(__file__).resolve().parent
_LOCAL_PROTO_DIR = _HERE / "_generated"
if _LOCAL_PROTO_DIR.is_dir():
sys.path.insert(0, str(_LOCAL_PROTO_DIR))
try:
from meshtastic_v25.deviceonly_pb2 import ( # type: ignore[import-not-found]
NodeDatabase,
NodeInfoLite,
NodePositionEntry,
NodeTelemetryEntry,
NodeEnvironmentEntry,
NodeStatusEntry,
PositionLite,
)
from meshtastic_v25.mesh_pb2 import HardwareModel, Position, StatusMessage # type: ignore[import-not-found]
from meshtastic_v25.config_pb2 import Config # type: ignore[import-not-found]
from meshtastic_v25.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics # type: ignore[import-not-found]
except ImportError as local_err:
# Fall back to the PyPI package if in-tree bindings haven't been generated.
# Will fail the v25 assertion below if the PyPI package predates the
# satellite-DB schema, but at least gives a clear "run regen-py-protos.sh"
# error message instead of an opaque ImportError.
try:
from meshtastic.protobuf.deviceonly_pb2 import (
NodeDatabase,
NodeInfoLite,
NodePositionEntry,
NodeTelemetryEntry,
NodeEnvironmentEntry,
NodeStatusEntry,
PositionLite,
)
from meshtastic.protobuf.mesh_pb2 import HardwareModel, Position, StatusMessage
from meshtastic.protobuf.config_pb2 import Config
from meshtastic.protobuf.telemetry_pb2 import DeviceMetrics, EnvironmentMetrics
except ImportError as pypi_err:
print(
"ERROR: could not import meshtastic protobuf bindings.\n"
" In-tree generation: run `bin/regen-py-protos.sh` (requires protoc).\n"
" PyPI fallback: `pip install meshtastic` (may lag firmware branch).\n"
f" local error (meshtastic_v25): {local_err}\n"
f" pypi error (meshtastic.protobuf): {pypi_err}",
file=sys.stderr,
)
sys.exit(1)
# Fail loudly if bindings predate v25 (no satellite arrays).
assert (
hasattr(NodeDatabase, "DESCRIPTOR")
and "positions" in NodeDatabase.DESCRIPTOR.fields_by_name
), (
"Loaded meshtastic bindings are older than v25 (NodeDatabase.positions missing). "
"Run `bin/regen-py-protos.sh` against the in-tree protobufs/ submodule."
)
# ---------------------------------------------------------------------------
# Bitfield bit positions (mirror src/mesh/NodeDB.h:467-484).
# ---------------------------------------------------------------------------
BIT_IS_KEY_MANUALLY_VERIFIED = 0
BIT_IS_MUTED = 1
BIT_VIA_MQTT = 2
BIT_IS_FAVORITE = 3
BIT_IS_IGNORED = 4
BIT_HAS_USER = 5
BIT_IS_LICENSED = 6
BIT_IS_UNMESSAGABLE = 7
BIT_HAS_IS_UNMESSAGABLE = 8
BITFIELD_LAYOUT = (
# JSON key bit position
("is_key_manually_verified", BIT_IS_KEY_MANUALLY_VERIFIED),
("is_muted", BIT_IS_MUTED),
("via_mqtt", BIT_VIA_MQTT),
("is_favorite", BIT_IS_FAVORITE),
("is_ignored", BIT_IS_IGNORED),
("has_user", BIT_HAS_USER),
("is_licensed", BIT_IS_LICENSED),
("is_unmessagable", BIT_IS_UNMESSAGABLE),
("has_is_unmessagable", BIT_HAS_IS_UNMESSAGABLE),
)
def _pack_bitfield(bf: dict[str, bool]) -> int:
out = 0
for key, shift in BITFIELD_LAYOUT:
if bf.get(key, False):
out |= (1 << shift)
return out
def _validate_node(node: dict[str, Any]) -> None:
"""Friendly errors so hand-editors get clear feedback."""
if "num" not in node or not isinstance(node["num"], str):
raise ValueError(f"node missing/invalid 'num' (must be hex string): {node!r}")
if "long_name" not in node:
raise ValueError(f"node {node['num']}: missing 'long_name'")
if len(node["long_name"]) > 24:
raise ValueError(
f"node {node['num']}: long_name {node['long_name']!r} is "
f"{len(node['long_name'])} chars; max 24 (nanopb max_size:25 minus NUL)"
)
if "short_name" in node:
# short_name max_size:5 (incl. NUL) → 4 bytes of content.
# Char count is irrelevant — emojis with variation selectors (e.g. ❄️ = 6 B)
# would slip past a `len(str) > 4` check. Always measure bytes.
b = node["short_name"].encode("utf-8")
if len(b) > 4:
raise ValueError(
f"node {node['num']}: short_name {node['short_name']!r} is "
f"{len(b)} bytes UTF-8; max 4 (nanopb max_size:5 minus NUL)"
)
pk = node.get("public_key_hex", "")
if pk and len(pk) != 64:
raise ValueError(
f"node {node['num']}: public_key_hex must be 64 hex chars or empty; "
f"got {len(pk)} chars"
)
if pk:
try:
bytes.fromhex(pk)
except ValueError as e:
raise ValueError(f"node {node['num']}: public_key_hex is not valid hex: {e}")
def _resolve_time(
node: dict[str, Any],
field_absolute: str,
field_offset: str,
now_epoch: int,
) -> int:
"""If `field_absolute` is set, use it; else compute `now_epoch - offset`."""
if field_absolute in node and node[field_absolute] is not None:
return int(node[field_absolute])
offset = node.get(field_offset, 0)
return max(0, int(now_epoch) - int(offset))
def _build_node_info_lite(node: dict[str, Any], now_epoch: int) -> NodeInfoLite:
_validate_node(node)
info = NodeInfoLite()
info.num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
info.long_name = node.get("long_name", "")
info.short_name = node.get("short_name", "")
# Enum lookups will raise ValueError on unknown names — that's exactly what we want.
info.hw_model = HardwareModel.Value(node.get("hw_model", "UNSET"))
info.role = Config.DeviceConfig.Role.Value(node.get("role", "CLIENT"))
pk_hex = node.get("public_key_hex", "")
if pk_hex:
info.public_key = bytes.fromhex(pk_hex)
info.snr = float(node.get("snr", 0.0))
info.channel = int(node.get("channel", 0))
if "hops_away" in node:
# `optional uint32 hops_away = 9;` — in Python protobuf, assigning the
# field implicitly sets HasField("hops_away") to True. No has_hops_away
# setter exists (unlike the C++ nanopb-generated header).
info.hops_away = int(node["hops_away"])
info.next_hop = int(node.get("next_hop", 0))
info.last_heard = _resolve_time(node, "last_heard", "last_heard_offset_sec", now_epoch)
info.bitfield = _pack_bitfield(node.get("bitfield", {}))
return info
def _build_position_entry(num: int, pos: dict[str, Any], now_epoch: int) -> NodePositionEntry:
entry = NodePositionEntry()
entry.num = num
pl = PositionLite()
# Firmware stores lat/long as int32 in 1e-7 degrees.
pl.latitude_i = int(round(float(pos["latitude"]) * 1e7))
pl.longitude_i = int(round(float(pos["longitude"]) * 1e7))
pl.altitude = int(pos.get("altitude", 0))
pl.time = _resolve_time(pos, "time", "time_offset_sec", now_epoch)
pl.location_source = Position.LocSource.Value(pos.get("location_source", "LOC_UNSET"))
entry.position.CopyFrom(pl)
return entry
def _build_telemetry_entry(num: int, tel: dict[str, Any]) -> NodeTelemetryEntry:
entry = NodeTelemetryEntry()
entry.num = num
dm = DeviceMetrics()
if "battery_level" in tel:
dm.battery_level = int(tel["battery_level"])
if "voltage" in tel:
dm.voltage = float(tel["voltage"])
if "channel_utilization" in tel:
dm.channel_utilization = float(tel["channel_utilization"])
if "air_util_tx" in tel:
dm.air_util_tx = float(tel["air_util_tx"])
if "uptime_seconds" in tel:
dm.uptime_seconds = int(tel["uptime_seconds"])
entry.device_metrics.CopyFrom(dm)
return entry
def _build_environment_entry(num: int, env: dict[str, Any]) -> NodeEnvironmentEntry:
entry = NodeEnvironmentEntry()
entry.num = num
em = EnvironmentMetrics()
if "temperature" in env:
em.temperature = float(env["temperature"])
if "relative_humidity" in env:
em.relative_humidity = float(env["relative_humidity"])
if "barometric_pressure" in env:
em.barometric_pressure = float(env["barometric_pressure"])
if "iaq" in env:
em.iaq = int(env["iaq"])
entry.environment_metrics.CopyFrom(em)
return entry
def _build_status_entry(num: int, status: dict[str, Any]) -> NodeStatusEntry:
# `StatusMessage` (mesh.proto:1445) has a single `string status` field.
entry = NodeStatusEntry()
entry.num = num
sm = StatusMessage()
if "status" in status:
sm.status = str(status["status"])
entry.status.CopyFrom(sm)
return entry
def compile_jsonl_to_proto(jsonl_path: pathlib.Path, now_epoch: int) -> bytes:
"""Read a seed JSONL and return the encoded NodeDatabase bytes."""
lines = jsonl_path.read_text(encoding="utf-8").splitlines()
if not lines:
raise ValueError(f"{jsonl_path} is empty")
meta_line = lines[0]
meta_obj = json.loads(meta_line)
meta = meta_obj.get("_meta", {})
version = meta.get("version")
if version != 25:
raise ValueError(
f"{jsonl_path}: meta version is {version!r}; this compiler "
f"requires version=25. Regenerate the seed with the matching tooling."
)
db = NodeDatabase()
db.version = 25
for ln, raw in enumerate(lines[1:], start=2):
raw = raw.strip()
if not raw:
continue
try:
node = json.loads(raw)
except json.JSONDecodeError as e:
raise ValueError(f"{jsonl_path}:{ln} JSON parse error: {e}")
num = int(node["num"], 16) if isinstance(node["num"], str) else int(node["num"])
# Header
info = _build_node_info_lite(node, now_epoch)
db.nodes.append(info)
# Satellites (nullable)
if node.get("position"):
db.positions.append(_build_position_entry(num, node["position"], now_epoch))
if node.get("telemetry"):
db.telemetry.append(_build_telemetry_entry(num, node["telemetry"]))
if node.get("environment"):
db.environment.append(_build_environment_entry(num, node["environment"]))
if node.get("status"):
db.status.append(_build_status_entry(num, node["status"]))
return db.SerializeToString()
def main(argv: list[str]) -> int:
p = argparse.ArgumentParser(
description="Compile a seed JSONL into a binary v25 NodeDatabase proto.",
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
)
p.add_argument("--in", dest="in_path", required=True, help="Input seed JSONL.")
p.add_argument("--out", required=True, help="Output binary .proto path.")
p.add_argument(
"--now-epoch",
type=int,
default=None,
help="Pin 'now' to this Unix epoch (for byte-identical CI). Default: time.time().",
)
args = p.parse_args(argv)
in_path = pathlib.Path(args.in_path)
if not in_path.is_file():
print(f"input not found: {in_path}", file=sys.stderr)
return 2
now_epoch = args.now_epoch if args.now_epoch is not None else int(time.time())
try:
encoded = compile_jsonl_to_proto(in_path, now_epoch)
except ValueError as e:
print(f"ERROR: {e}", file=sys.stderr)
return 3
out_path = pathlib.Path(args.out)
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_bytes(encoded)
print(
f"compiled {in_path} -> {out_path} ({len(encoded)} bytes, now_epoch={now_epoch})",
file=sys.stderr,
)
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+118
View File
@@ -0,0 +1,118 @@
#!/bin/bash
# Script to show commits in develop that are not in master
# with their associated PR info and commit hashes
#
# Usage:
# ./show-unmerged-prs.sh # Show all unmerged commits
# ./show-unmerged-prs.sh --bugfix # Show only bugfix-labeled PRs
set -e
REPO="firmware"
OWNER="meshtastic"
BASE_BRANCH="master"
HEAD_BRANCH="develop"
LIMIT=100
FILTER_LABEL=""
# Parse arguments
for arg in "$@"; do
case $arg in
--bugfix)
FILTER_LABEL="bugfix"
shift
;;
--feature)
FILTER_LABEL="feature"
shift
;;
--help)
echo "Usage: $0 [OPTIONS]"
echo "Options:"
echo " --bugfix Show only PRs labeled with 'bugfix'"
echo " --feature Show only PRs labeled with 'feature'"
echo " --help Show this help message"
exit 0
;;
esac
done
if [ -n "$FILTER_LABEL" ]; then
echo "Fetching commits in $HEAD_BRANCH that are not in $BASE_BRANCH (filtered by label: $FILTER_LABEL)..."
else
echo "Fetching commits in $HEAD_BRANCH that are not in $BASE_BRANCH..."
fi
echo ""
# Check if gh CLI is available
if ! command -v gh &> /dev/null; then
echo "ERROR: GitHub CLI (gh) not found. Please install it first."
echo "Visit: https://cli.github.com/"
exit 1
fi
# Get commits in develop that are not in master
# For each commit, try to find associated PR
git fetch origin develop master 2>/dev/null || true
# Use git to get the list of commits
commits=$(git log --pretty=format:"%H|%s" origin/master..origin/develop | head -n $LIMIT)
count=0
displayed=0
echo "Commits in $HEAD_BRANCH not in $BASE_BRANCH:"
echo "=============================================="
echo ""
while IFS='|' read -r hash subject; do
((count++))
# Try to find the PR for this commit
# Extract PR number, title, description, and labels
pr_response=$(gh api -X GET "/repos/$OWNER/$REPO/commits/$hash/pulls" \
-H "Accept: application/vnd.github.v3+json" 2>/dev/null | \
jq -r '.[0] | "\(.number)|\(.title)|\(.body // "No description")|\(.labels | map(.name) | join(","))"' 2>/dev/null || echo "||||")
if [ -z "$pr_response" ] || [ "$pr_response" = "||||" ]; then
# If no PR found, skip if filter is active, otherwise show the commit
if [ -z "$FILTER_LABEL" ]; then
((displayed++))
echo "[$displayed] Commit: $hash"
echo " Subject: $subject"
echo " PR: Not found in GitHub"
echo ""
fi
else
IFS='|' read -r pr_num pr_title pr_desc pr_labels <<< "$pr_response"
# Check if filter matches
if [ -n "$FILTER_LABEL" ]; then
# Only show if the label is in the labels list
if ! echo "$pr_labels" | grep -q "$FILTER_LABEL"; then
continue
fi
fi
((displayed++))
echo "[$displayed] PR #$pr_num - $pr_title"
echo " Commit: $hash"
if [ -n "$pr_desc" ] && [ "$pr_desc" != "No description" ]; then
# Truncate description to 200 chars
desc_short="${pr_desc:0:200}"
[ ${#pr_desc} -gt 200 ] && desc_short+="..."
echo " Description: $desc_short"
fi
if [ -n "$pr_labels" ] && [ "$pr_labels" != "" ]; then
echo " Labels: $pr_labels"
fi
echo ""
fi
done <<< "$commits"
echo ""
if [ -n "$FILTER_LABEL" ]; then
echo "Done. Showing $displayed PRs with label '$FILTER_LABEL' from $displayed commits checked."
else
echo "Done. Showing $displayed commits from $HEAD_BRANCH not in $BASE_BRANCH."
fi
+1 -1
View File
@@ -7,7 +7,7 @@
"extra_flags": [
"-D CDEBYTE_EORA_S3",
"-D ARDUINO_USB_CDC_ON_BOOT=1",
"-D ARDUINO_USB_MODE=0",
"-D ARDUINO_USB_MODE=1",
"-D ARDUINO_RUNNING_CORE=1",
"-D ARDUINO_EVENT_RUNNING_CORE=1",
"-D BOARD_HAS_PSRAM"
+42
View File
@@ -0,0 +1,42 @@
{
"build": {
"arduino": {
"ldscript": "esp32s3_out.ld",
"memory_type": "qio_opi"
},
"core": "esp32",
"extra_flags": [
"-D BOARD_HAS_PSRAM",
"-D ARDUINO_USB_CDC_ON_BOOT=0",
"-D ARDUINO_USB_MODE=0",
"-D ARDUINO_RUNNING_CORE=1",
"-D ARDUINO_EVENT_RUNNING_CORE=0"
],
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"psram_type": "qio_opi",
"hwids": [["0x303A", "0x1001"]],
"mcu": "esp32s3",
"variant": "ELECROW-ThinkNode-M7"
},
"connectivity": ["wifi", "bluetooth", "lora"],
"debug": {
"default_tool": "esp-builtin",
"onboard_tools": ["esp-builtin"],
"openocd_target": "esp32s3.cfg"
},
"frameworks": ["arduino", "espidf"],
"name": "ELECROW ThinkNode M7",
"upload": {
"flash_size": "8MB",
"maximum_ram_size": 524288,
"maximum_size": 8388608,
"use_1200bps_touch": true,
"wait_for_upload_port": true,
"require_upload_port": true,
"speed": 921600
},
"url": "https://www.elecrow.com",
"vendor": "ELECROW"
}
+1 -1
View File
@@ -6,7 +6,7 @@
"core": "esp32",
"extra_flags": [
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1",
"-DBOARD_HAS_PSRAM"
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+54
View File
@@ -0,0 +1,54 @@
{
"build": {
"arduino": {
"ldscript": "nrf52840_s140_v6.ld"
},
"core": "nRF5",
"cpu": "cortex-m4",
"extra_flags": "-DNRF52840_XXAA",
"f_cpu": "64000000L",
"hwids": [
["0x239A", "0x4405"],
["0x239A", "0x0029"],
["0x239A", "0x002A"],
["0x2886", "0x1667"]
],
"usb_product": "HT-n5262",
"mcu": "nrf52840",
"variant": "heltec_mesh_node_t1",
"variants_dir": "variants",
"bsp": {
"name": "adafruit"
},
"softdevice": {
"sd_flags": "-DS140",
"sd_name": "s140",
"sd_version": "6.1.1",
"sd_fwid": "0x00B6"
},
"bootloader": {
"settings_addr": "0xFF000"
}
},
"connectivity": ["bluetooth"],
"debug": {
"jlink_device": "nRF52840_xxAA",
"onboard_tools": ["jlink"],
"svd_path": "nrf52840.svd",
"openocd_target": "nrf52840-mdk-rs"
},
"frameworks": ["arduino"],
"name": "Heltec Mesh Node T1",
"upload": {
"maximum_ram_size": 248832,
"maximum_size": 815104,
"speed": 115200,
"protocol": "nrfutil",
"protocols": ["jlink", "nrfjprog", "nrfutil", "stlink"],
"use_1200bps_touch": true,
"require_upload_port": true,
"wait_for_upload_port": true
},
"url": "https://heltec.org",
"vendor": "Heltec"
}
+43
View File
@@ -0,0 +1,43 @@
{
"build": {
"arduino": {
"ldscript": "esp32s3_out.ld",
"partitions": "default_16MB.csv",
"memory_type": "qio_opi"
},
"core": "esp32",
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"psram_type": "opi",
"hwids": [["0x303A", "0x1001"]],
"mcu": "esp32s3",
"variant": "heltec_v4_r8"
},
"connectivity": ["wifi", "bluetooth", "lora"],
"debug": {
"default_tool": "esp-builtin",
"onboard_tools": ["esp-builtin"],
"openocd_target": "esp32s3.cfg"
},
"frameworks": ["arduino", "espidf"],
"name": "heltec_wifi_lora_32 v4 r8 (16 MB FLASH, 8 MB PSRAM)",
"upload": {
"flash_size": "16MB",
"maximum_ram_size": 327680,
"maximum_size": 16777216,
"use_1200bps_touch": true,
"wait_for_upload_port": true,
"require_upload_port": true,
"speed": 921600
},
"url": "https://heltec.org/",
"vendor": "heltec"
}
+1 -1
View File
@@ -9,7 +9,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -9,7 +9,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -9,7 +9,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DHELTEC_WIRELESS_TRACKER",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -7,7 +7,7 @@
"core": "esp32",
"extra_flags": [
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+26
View File
@@ -0,0 +1,26 @@
{
"build": {
"cpu": "cortex-m33",
"f_cpu": "128000000L",
"mcu": "nrf54l15",
"zephyr": {
"variant": "nrf54l15dk/nrf54l15/cpuapp"
}
},
"connectivity": ["bluetooth"],
"debug": {
"default_tools": ["jlink"],
"jlink_device": "nRF54L15_M33",
"svd_path": "nrf54l15.svd"
},
"frameworks": ["zephyr"],
"name": "Nordic nRF54L15-DK (PCA10156)",
"upload": {
"maximum_ram_size": 262144,
"maximum_size": 1572864,
"protocol": "jlink",
"protocols": ["jlink"]
},
"url": "https://www.nordicsemi.com/Products/nRF54L15",
"vendor": "Nordic Semiconductor"
}
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=0"
],
+41
View File
@@ -0,0 +1,41 @@
{
"build": {
"arduino": {
"ldscript": "esp32s3_out.ld",
"memory_type": "qio_opi"
},
"core": "esp32",
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=0"
],
"f_cpu": "240000000L",
"f_flash": "80000000L",
"flash_mode": "qio",
"hwids": [["0x303A", "0x1001"]],
"mcu": "esp32s3",
"variant": "station-g3"
},
"connectivity": ["wifi", "bluetooth", "lora"],
"debug": {
"default_tool": "esp-builtin",
"onboard_tools": ["esp-builtin"],
"openocd_target": "esp32s3.cfg"
},
"frameworks": ["arduino", "espidf"],
"name": "BQ Station G3",
"upload": {
"flash_size": "16MB",
"maximum_ram_size": 327680,
"maximum_size": 16777216,
"use_1200bps_touch": true,
"wait_for_upload_port": true,
"require_upload_port": true,
"speed": 921600
},
"url": "",
"vendor": "BQ Consulting"
}
+1 -1
View File
@@ -9,7 +9,7 @@
"-DBOARD_HAS_PSRAM",
"-DLILYGO_TBEAM_1W",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -8,7 +8,7 @@
"extra_flags": [
"-DBOARD_HAS_PSRAM",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -8,7 +8,7 @@
"-DBOARD_HAS_PSRAM",
"-DLILYGO_TBEAM_S3_CORE",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+1 -1
View File
@@ -7,7 +7,7 @@
"extra_flags": [
"-DLILYGO_T3S3_V1",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1",
"-DBOARD_HAS_PSRAM"
+1 -1
View File
@@ -10,7 +10,7 @@
"-DBOARD_HAS_PSRAM",
"-DUNPHONE_SPIN=9",
"-DARDUINO_USB_CDC_ON_BOOT=1",
"-DARDUINO_USB_MODE=0",
"-DARDUINO_USB_MODE=1",
"-DARDUINO_RUNNING_CORE=1",
"-DARDUINO_EVENT_RUNNING_CORE=1"
],
+12
View File
@@ -1,3 +1,15 @@
meshtasticd (2.7.24.0) unstable; urgency=medium
* Version 2.7.24
-- GitHub Actions <github-actions[bot]@users.noreply.github.com> Fri, 08 May 2026 10:44:12 +0000
meshtasticd (2.7.23.0) unstable; urgency=medium
* Version 2.7.23
-- GitHub Actions <github-actions[bot]@users.noreply.github.com> Tue, 14 Apr 2026 12:29:48 +0000
meshtasticd (2.7.22.0) unstable; urgency=medium
* Version 2.7.22
+1
View File
@@ -1,4 +1,5 @@
#!/usr/bin/bash
set -e
export DEBEMAIL="jbennett@incomsystems.biz"
export PLATFORMIO_LIBDEPS_DIR=pio/libdeps
export PLATFORMIO_PACKAGES_DIR=pio/packages
+7
View File
@@ -0,0 +1,7 @@
# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x640000,
app1, app, ota_1, 0x650000,0x640000,
spiffs, data, spiffs, 0xc90000,0x360000,
coredump, data, coredump,0xFF0000,0x10000,
1 # Name Type SubType Offset Size Flags
2 nvs data nvs 0x9000 0x5000
3 otadata data ota 0xe000 0x2000
4 app0 app ota_0 0x10000 0x640000
5 app1 app ota_1 0x650000 0x640000
6 spiffs data spiffs 0xc90000 0x360000
7 coredump data coredump 0xFF0000 0x10000
+7
View File
@@ -0,0 +1,7 @@
# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x5000,
otadata, data, ota, 0xe000, 0x2000,
app0, app, ota_0, 0x10000, 0x330000,
app1, app, ota_1, 0x340000,0x330000,
spiffs, data, spiffs, 0x670000,0x180000,
coredump, data, coredump,0x7F0000,0x10000,
1 # Name Type SubType Offset Size Flags
2 nvs data nvs 0x9000 0x5000
3 otadata data ota 0xe000 0x2000
4 app0 app ota_0 0x10000 0x330000
5 app1 app ota_1 0x340000 0x330000
6 spiffs data spiffs 0x670000 0x180000
7 coredump data coredump 0x7F0000 0x10000
+3 -14
View File
@@ -70,17 +70,6 @@ def esp32_create_combined_bin(source, target, env):
env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_combined_bin)
esp32_kind = env.GetProjectOption("custom_esp32_kind")
if esp32_kind == "esp32":
# Free up some IRAM by removing auxiliary SPI flash chip drivers.
# Wrapped stub symbols are defined in src/platform/esp32/iram-quirk.c.
env.Append(
LINKFLAGS=[
"-Wl,--wrap=esp_flash_chip_gd",
"-Wl,--wrap=esp_flash_chip_issi",
"-Wl,--wrap=esp_flash_chip_winbond",
]
)
else:
# For newer ESP32 targets, using newlib nano works better.
env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])
# Enable Newlib Nano formatting to save space
# ...but allow printf float support (compromise)
env.Append(LINKFLAGS=["--specs=nano.specs", "-u", "_printf_float"])
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env python3
# trunk-ignore-all(ruff/F821)
# trunk-ignore-all(flake8/F821): For SConstruct imports
# force linker response file instead of command line arguments
Import("env")
def wrap_with_tempfile(command_key):
command = env.get(command_key)
if not command or not isinstance(command, str):
return
if "TEMPFILE(" in command:
return
env.Replace(**{command_key: "${TEMPFILE('%s')}" % command})
# Force SCons to spill long commands into response files on this target.
env.Replace(MAXLINELENGTH=8192)
for key in ("LINKCOM", "CXXLINKCOM", "SHLINKCOM", "SHCXXLINKCOM"):
wrap_with_tempfile(key)
+140
View File
@@ -0,0 +1,140 @@
#!/usr/bin/env python3
# trunk-ignore-all(ruff/F821)
# trunk-ignore-all(flake8/F821): For SConstruct imports
#
# post:extra_scripts/nrf54l15_linker.py
#
# Fix for Zephyr two-pass link on nRF54L15:
# platformio-build.py registers env.Depends("$PROG_PATH", final_ld_script) but
# the SCons dependency chain is broken (final_ld_script Command never runs).
# This script adds a PreAction on the final firmware binary that runs the gcc
# preprocessing command directly (extracted from build.ninja) to generate
# zephyr/linker.cmd before the link step.
#
# PlatformIO bundles an old Ninja that can't handle multi-output depslog rules,
# so we parse the COMMAND line from build.ninja and run just the gcc -E part,
# skipping the cmake_transform_depfile step (only needed for Ninja deps tracking).
import os
import re
import subprocess
Import("env")
if env.get("PIOENV") != "nrf54l15dk":
pass # Only for the nrf54l15dk environment
else:
def _extract_gcc_command(ninja_build):
"""Parse build.ninja to find the gcc -E command that generates linker.cmd.
The rule format depends on the host:
Windows (CMake's RunCMake wraps every command):
COMMAND = cmd.exe /C "cd /D DIR && arm-none-eabi-gcc.exe ... -o linker.cmd && cmake.exe -E cmake_transform_depfile ..."
POSIX (Linux/macOS — no wrapper):
COMMAND = cd DIR && arm-none-eabi-gcc ... -o linker.cmd && cmake -E cmake_transform_depfile ...
Returns (gcc_cmd_string, cwd_path) or raises RuntimeError.
"""
in_rule = False
with open(ninja_build, "r", encoding="utf-8", errors="replace") as f:
for line in f:
# Detect start of the linker.cmd custom command rule
if not in_rule:
if "build zephyr/linker.cmd" in line and "CUSTOM_COMMAND" in line:
in_rule = True
continue
stripped = line.strip()
if not stripped.startswith("COMMAND = "):
continue
command_val = stripped[len("COMMAND = ") :]
# On Windows the value is wrapped in `cmd.exe /C "..."` — strip
# the wrapper. On POSIX hosts the inner sequence is the value
# itself (no quoting layer).
m = re.search(r'/C\s+"(.*)"\s*$', command_val)
inner = m.group(1) if m else command_val
parts = inner.split(" && ")
cwd = None
gcc_cmd = None
for part in parts:
part = part.strip()
if part.startswith("cd /D "): # Windows form
cwd = part[len("cd /D ") :]
elif part.startswith("cd "): # POSIX form
cwd = part[len("cd ") :]
elif "arm-none-eabi-gcc" in part:
gcc_cmd = part
if not gcc_cmd:
raise RuntimeError(
"nRF54L15 linker fix: arm-none-eabi-gcc command not found in:\n%s"
% inner[:400]
)
return gcc_cmd, cwd
raise RuntimeError(
"nRF54L15 linker fix: 'build zephyr/linker.cmd' rule not found in build.ninja"
)
def _generate_linker_cmd(target, source, env):
"""Generate zephyr/linker.cmd via direct gcc invocation before the final link."""
build_dir = env.subst("$BUILD_DIR")
zephyr_dir = os.path.join(build_dir, "zephyr")
linker_cmd = os.path.join(zephyr_dir, "linker.cmd")
if os.path.exists(linker_cmd):
return # Already present — nothing to do
ninja_build = os.path.join(build_dir, "build.ninja")
if not os.path.exists(ninja_build):
raise RuntimeError(
"nRF54L15 linker fix: build.ninja not found at %s\n"
"Run a full build first so CMake generates the Ninja files."
% ninja_build
)
gcc_cmd, cwd = _extract_gcc_command(ninja_build)
run_cwd = cwd if cwd else zephyr_dir
print(
"==> nRF54L15: Generating zephyr/linker.cmd (LINKER_ZEPHYR_FINAL) via GCC"
)
# gcc_cmd comes verbatim from our own build.ninja (never user input) and
# contains Windows-style paths with spaces that cannot be safely argv-split
# with shlex, so we run it via the platform shell. nosec/nosemgrep below
# acknowledge this deliberate, scoped use of shell=True.
result = subprocess.run( # nosec B602
gcc_cmd,
shell=True, # nosemgrep: python.lang.security.audit.subprocess-shell-true.subprocess-shell-true
cwd=run_cwd,
capture_output=True,
text=True,
)
if result.returncode != 0:
print("GCC stdout:", result.stdout[:2000])
print("GCC stderr:", result.stderr[:2000])
raise RuntimeError(
"nRF54L15 linker fix: GCC failed to generate linker.cmd (rc=%d)"
% result.returncode
)
if not os.path.exists(linker_cmd):
raise RuntimeError(
"nRF54L15 linker fix: GCC returned 0 but linker.cmd was not created at %s"
% linker_cmd
)
print("==> linker.cmd generated successfully")
# Use PIOMAINPROG (set by ZephyrBuildProgram) to get the exact SCons node
prog = env.get("PIOMAINPROG")
if prog:
env.AddPreAction(prog, _generate_linker_cmd)
else:
print(
"[nrf54l15_linker] WARNING: PIOMAINPROG not set, falling back to $PROG_PATH"
)
env.AddPreAction(env.subst("$PROG_PATH"), _generate_linker_cmd)
+35
View File
@@ -0,0 +1,35 @@
.venv/
__pycache__/
*.py[cod]
*.egg-info/
.pytest_cache/
.mypy_cache/
dist/
build/
# Persistent device-log capture (recorder + Datadog cursor).
# Cross-session JSONL streams written by the autouse Recorder singleton
# (see src/meshtastic_mcp/recorder/). Lives outside tests/ so the pytest
# fixture truncate doesn't touch it.
.mtlog/
# Test harness artifacts
tests/report.html
tests/junit.xml
tests/reportlog.jsonl
tests/fwlog.jsonl
# Subprocess-output tee from pio/esptool/nrfutil/picotool (live flash
# progress for the TUI; also a post-run diagnostic for plain CLI runs).
tests/flash.log
tests/tool_coverage.json
tests/.coverage
htmlcov/
# Persistent run counter for meshtastic-mcp-test-tui header.
tests/.tui-runs
# Cross-run history (TUI duration sparkline).
tests/.history/
# Reproducer bundles (TUI `x` export on failed tests).
tests/reproducers/
# UI-tier camera captures + per-test transcripts. Regenerated every run;
# left on disk for human review between runs.
tests/ui_captures/
+412
View File
@@ -0,0 +1,412 @@
# Meshtastic MCP Server
An [MCP](https://modelcontextprotocol.io) server for working with the Meshtastic firmware repo and connected devices. Lets Claude Code / Claude Desktop:
- Discover USB-connected Meshtastic devices
- Enumerate PlatformIO board variants (166+) with Meshtastic metadata
- Build, clean, flash, erase-and-flash (factory), and OTA-update firmware
- Read serial logs via `pio device monitor` (with board-specific exception decoders)
- Trigger 1200bps touch-reset for bootloader entry (nRF52, ESP32-S3, RP2040)
- Query and administer a running node via the [`meshtastic` Python API](https://github.com/meshtastic/python): owner name, config (LocalConfig + ModuleConfig), channels, messaging, reboot/shutdown/factory-reset
- Call `esptool`, `nrfutil`, `picotool` directly when PlatformIO doesn't cover the operation
## Design principle
**PlatformIO first.** Its `pio run -t upload` knows the correct protocol, offsets, and post-build chain for every variant in `variants/`. Direct vendor-tool wrappers (`esptool_*`, `nrfutil_*`, `picotool_*`) exist as escape hatches for operations pio doesn't cover (blank-chip erase, DFU `.zip` packages, BOOTSEL-mode inspection).
## Prerequisites
- Python ≥ 3.11
- [PlatformIO Core](https://platformio.org/install/cli) — `pio` on `$PATH` or at `~/.platformio/penv/bin/pio`
- The Meshtastic firmware repo checked out somewhere (set via `MESHTASTIC_FIRMWARE_ROOT`)
- Optional: `esptool`, `nrfutil`, `picotool` on `$PATH` (or under the firmware venv at `.venv/bin/`) if you want to use the direct-tool wrappers
## Install
```bash
cd <firmware-repo>/mcp-server
python3 -m venv .venv
.venv/bin/pip install -e .
```
Verify:
```bash
MESHTASTIC_FIRMWARE_ROOT=<firmware-repo> .venv/bin/python -m meshtastic_mcp
```
The server blocks on stdin (that's correct — it speaks MCP over stdio). Ctrl-C to exit.
## Register with Claude Code
Edit `~/.claude/settings.json` (global) or `<firmware-repo>/.claude/settings.local.json` (project-only):
```json
{
"mcpServers": {
"meshtastic": {
"command": "<firmware-repo>/mcp-server/.venv/bin/python",
"args": ["-m", "meshtastic_mcp"],
"env": {
"MESHTASTIC_FIRMWARE_ROOT": "<firmware-repo>"
}
}
}
}
```
Replace `<firmware-repo>` with the absolute path, e.g. `/Users/you/GitHub/firmware`. Restart Claude Code after editing.
## Register with Claude Desktop
Same `mcpServers` block, but in `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows).
## Tools (43)
### Discovery & metadata
| Tool | What it does |
| -------------- | ------------------------------------------------------------------------------------------ |
| `list_devices` | USB/serial port listing, flags likely-Meshtastic candidates |
| `list_boards` | PlatformIO envs with `custom_meshtastic_*` metadata; filters by arch/supported/query/level |
| `get_board` | Full env dict incl. raw pio config |
### Build & flash
| Tool | What it does |
| ----------------- | -------------------------------------------------------------------- |
| `build` | `pio run -e <env>` (+ mtjson target) |
| `clean` | `pio run -e <env> -t clean` |
| `pio_flash` | `pio run -e <env> -t upload --upload-port <port>` — any architecture |
| `erase_and_flash` | ESP32 full factory flash via `bin/device-install.sh` |
| `update_flash` | ESP32 OTA app-partition update via `bin/device-update.sh` |
| `touch_1200bps` | 1200-baud open/close to trigger USB CDC bootloader entry |
### Serial log sessions
Backed by long-running `pio device monitor` subprocesses with a 10k-line ring buffer per session and board-specific filters (`esp32_exception_decoder` auto-selected when you pass `env=`).
| Tool | What it does |
| -------------- | ------------------------------------------------------------------ |
| `serial_open` | Start a monitor session; returns `session_id` |
| `serial_read` | Cursor-based pull; reports `dropped` if lines aged out of the ring |
| `serial_list` | All active sessions |
| `serial_close` | Terminate a session |
### Device reads
| Tool | What it does |
| ------------- | --------------------------------------------------------------------------- |
| `device_info` | my_node_num, long/short name, firmware version, region, channel, node count |
| `list_nodes` | Full node database with position, SNR, RSSI, last_heard, battery |
_The tool tables below document 38 currently registered MCP server tools._
### Device writes
| Tool | What it does |
| ------------------- | -------------------------------------------------------------------------- |
| `set_owner` | Long name + optional short name (≤4 chars) |
| `get_config` | One section or all (LocalConfig + ModuleConfig) |
| `set_config` | Dot-path field write: `lora.region`=`"US"`, `device.role`=`"ROUTER"`, etc. |
| `get_channel_url` | Primary-only or include_all=admin URL |
| `set_channel_url` | Import channels from a Meshtastic URL |
| `set_debug_log_api` | Enable or disable debug logging for the Meshtastic Python API client |
| `send_text` | Broadcast or direct text message |
| `reboot` | `localNode.reboot(secs)` — requires `confirm=True` |
| `shutdown` | `localNode.shutdown(secs)` — requires `confirm=True` |
| `factory_reset` | `localNode.factoryReset(full?)` — requires `confirm=True` |
### Direct hardware tools (escape hatches)
| Tool | What it does |
| --------------------- | --------------------------------------------------------- |
| `esptool_chip_info` | Read chip, MAC, crystal, flash size |
| `esptool_erase_flash` | Full-chip erase (destructive) |
| `esptool_raw` | Pass-through; confirm=True required for write/erase/merge |
| `nrfutil_dfu` | DFU-flash a `.zip` package |
| `nrfutil_raw` | Pass-through |
| `picotool_info` | Read Pico BOOTSEL-mode info |
| `picotool_load` | Load a UF2 |
| `picotool_raw` | Pass-through |
### USB power control (uhubctl)
| Tool | What it does |
| --------------- | ----------------------------------------------------------- |
| `uhubctl_list` | Enumerate USB hubs + attached-device VID/PID (read-only) |
| `uhubctl_power` | Drive a hub port `on` or `off`; `off` requires confirm=True |
| `uhubctl_cycle` | Off → wait `delay_s` → on; confirm=True required |
Target a port by explicit `(location, port)` (raw uhubctl syntax like
`location="1-1.3", port=2`) or by `role` (`"nrf52"`, `"esp32s3"`). Role
lookup checks `MESHTASTIC_UHUBCTL_LOCATION_<ROLE>` +
`MESHTASTIC_UHUBCTL_PORT_<ROLE>` env vars first, then auto-detects via VID
against `uhubctl`'s output.
Requires [`uhubctl`](https://github.com/mvp/uhubctl) on PATH:
```bash
brew install uhubctl # macOS
apt install uhubctl # Debian/Ubuntu
```
Modern macOS + PPPS-capable hubs generally work without root. On Linux
without udev rules, or on old macOS with driver quirks, you may need
`sudo`. If uhubctl returns a permission error the MCP tool raises a
clear `UhubctlError` pointing at the
[udev-rules / sudo fallback](https://github.com/mvp/uhubctl#linux-usb-permissions)
rather than auto-`sudo`'ing mid-run.
## Safety
- **All destructive flash/admin tools require `confirm=True`** as a tool-level gate, on top of any permission prompt from Claude.
- **Serial port is exclusive.** If a `serial_*` session is active on a port, `device_info`/admin tools on the same port will fail fast with a pointer at the active `session_id`. Close the session first.
- **Flash confirmation by architecture**: `erase_and_flash` / `update_flash` error if the env's architecture isn't ESP32 — use `pio_flash` for nRF52/RP2040/STM32.
## Environment variables
| Var | Default | Purpose |
| -------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `MESHTASTIC_FIRMWARE_ROOT` | walks up from cwd for `platformio.ini` | Pin the firmware repo |
| `MESHTASTIC_PIO_BIN` | `~/.platformio/penv/bin/pio``$PATH` `pio``platformio` | Override `pio` location |
| `MESHTASTIC_ESPTOOL_BIN` | `<firmware>/.venv/bin/esptool``$PATH` | Override esptool |
| `MESHTASTIC_NRFUTIL_BIN` | `$PATH` | Override nrfutil |
| `MESHTASTIC_PICOTOOL_BIN` | `$PATH` | Override picotool |
| `MESHTASTIC_MCP_SEED` | `mcp-<user>-<host>` | PSK seed for test-harness session (CI override) |
| `MESHTASTIC_MCP_FLASH_LOG` | `<mcp-server>/tests/flash.log` | Tee target for pio/esptool/nrfutil subprocess output (TUI tails it) |
| `MESHTASTIC_MCP_TCP_HOST` | unset | `host` or `host:port` of a `meshtasticd` daemon to surface as a TCP device (see "TCP / native-host nodes" below) |
## TCP / native-host nodes
The `native-macos` and `native` PlatformIO envs build a headless `meshtasticd`
binary that runs on the host (Apple Silicon / Intel macOS, or Linux Portduino).
The daemon exposes the meshtastic TCP API on port `4403` rather than a USB
serial endpoint — point the MCP server at it via `MESHTASTIC_MCP_TCP_HOST`:
```bash
# 1. Build + run a daemon on this host (see variants/native/portduino/platformio.ini
# for full Homebrew prereqs and CH341 LoRa-adapter setup).
pio run -e native-macos
~/.meshtasticd/meshtasticd
# 2. Point the MCP server at it.
export MESHTASTIC_MCP_TCP_HOST=localhost # or host:port, default port 4403
```
**First-run gotcha — MAC address.** `meshtasticd` derives its MAC from the
USB adapter's serial-number / product strings. Many cheap CH341 dongles
(MeshStick included — VID 0x1A86 / PID 0x5512) ship with `iSerialNumber=0`
and `iProduct=0`, so the daemon aborts on boot with `*** Blank MAC Address
not allowed!`. Set the MAC explicitly in `config.yaml`:
```yaml
# Under General:
MACAddress: 02:CA:FE:BA:BE:01
```
Use a locally-administered address (first byte's second-LSB set, e.g.
`02:*` / `06:*` / `0A:*` / `0E:*`) to avoid colliding with a real OUI.
There is also a `--hwid AA:BB:CC:DD:EE:FF` CLI flag visible in
`meshtasticd --help`, but it is **currently broken** in
`MAC_from_string()` (`src/platform/portduino/PortduinoGlue.cpp`): the
function strips colons from its parameter but then reads bytes from the
global `portduino_config.mac_address`, so `--hwid` is silently overridden
when `MACAddress:` is also set, and crashes the daemon (uncaught
`std::invalid_argument: stoi: no conversion`) when it isn't. Use the YAML
form until that's fixed upstream.
`list_devices` will surface the daemon as `tcp://localhost:4403` with
`likely_meshtastic=True`, so `device_info`, `list_nodes`, `get_config`,
`set_config`, `set_owner`, `send_text`, `userprefs_*`, and the admin RPCs
auto-select it when no `port` is passed. Pass `port="tcp://other-host:9999"`
explicitly to target a different daemon.
**Tools that don't apply to a TCP/native node** (no USB hardware to operate
on) raise a clear `ConnectionError` rather than failing mysteriously:
`pio_flash`, `erase_and_flash`, `update_flash`, `touch_1200bps`,
`serial_open` (use info/admin tools directly), and the vendor escape hatches
`esptool_*`, `nrfutil_*`, `picotool_*`. `pio_flash` against a `native*` env
similarly raises — there's no upload step; use `build` and run the binary
directly.
The pytest harness in `tests/` still assumes USB-attached devices per role —
TCP-aware fixtures are not part of this surface yet.
## Hardware Test Suite
`mcp-server/tests/` holds a pytest-based integration suite that exercises
real USB-connected Meshtastic devices against the MCP server surface. Separate
from the native C++ unit tests in the firmware repo's top-level `test/`
directory — this one validates the device-facing behavior end-to-end.
### Invocation
```bash
./mcp-server/run-tests.sh # full suite (auto-detect + auto-bake-if-needed)
./mcp-server/run-tests.sh --force-bake # reflash devices before testing
./mcp-server/run-tests.sh --assume-baked # skip the bake step (caller vouches for state)
./mcp-server/run-tests.sh tests/mesh # one tier
./mcp-server/run-tests.sh tests/mesh/test_traceroute.py # one file
./mcp-server/run-tests.sh -k telemetry # pytest name filter
```
The wrapper auto-detects connected devices (VID `0x239A``nrf52` → env
`rak4631`; `0x303A` or `0x10C4``esp32s3` → env `heltec-v3`), exports
`MESHTASTIC_MCP_ENV_<ROLE>` env vars, and invokes pytest. Overrides via
per-role env vars: `MESHTASTIC_MCP_ENV_NRF52=heltec-mesh-node-t114 ./run-tests.sh`.
No hardware connected? The wrapper narrows to `tests/unit/` only and says so
in the pre-flight header.
### Tiers (run in this order)
- **`bake`** (`tests/test_00_bake.py`) — flashes both hub roles with the
session's test profile. Has a skip-if-already-baked check (region + channel
match); `--force-bake` overrides.
- **`unit`** — pure Python, no hardware. boards / PIO wrapper /
userPrefs-parse / testing-profile fixtures.
- **`mesh`** — 2-device mesh: formation, broadcast delivery, direct+ACK,
traceroute, bidirectional. Parametrized over both directions. Includes
`test_peer_offline_recovery` which uses uhubctl to power-cycle one peer
mid-conversation and verifies the mesh recovers (skips without uhubctl).
- **`telemetry`** — periodic telemetry broadcast + on-demand request/reply
(`TELEMETRY_APP` with `wantResponse=True`).
- **`monitor`** — boot log has no panic markers within 60 s of reboot.
- **`recovery`** — `uhubctl` power-cycle round-trip: verifies the hub port
can be toggled off/on, the device re-enumerates with the same
`my_node_num`, and NVS-resident config (region, channel, modem preset)
survives a hard reset. Requires `uhubctl` on PATH; skips cleanly otherwise.
- **`ui`** — input-broker-driven screen navigation (`AdminMessage.send_input_event`
injection → `Screen::handleInputEvent` → frame transition). Parametrized
on the screen-bearing role (heltec-v3 OLED). Captures images via USB
webcam + OCRs them for HTML-report evidence. Requires `pip install -e '.[ui]'`
and `MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=<index>`; tier is auto-deselected
if `cv2` isn't importable.
- **`fleet`** — PSK-seed isolation: two labs with different seeds never
overlap.
- **`admin`** — owner persistence across reboot, channel URL round-trip,
`lora.hop_limit` persistence.
- **`provisioning`** — region/channel baking, userPrefs survive
`factory_reset(full=False)`.
#### UI tier setup
The `tests/ui/` tier drives the on-device OLED via the firmware's existing
`AdminMessage.send_input_event` RPC (no firmware changes required) and
verifies transitions via a macro-gated log line + camera + OCR. Summary:
1. Install extras: `pip install -e 'mcp-server/.[ui]'` — pulls in
`opencv-python-headless`, `numpy`, `easyocr`, `Pillow`. First easyocr
run downloads ~100 MB of models to `~/.EasyOCR/`; an autouse session
fixture pre-warms the reader so per-test OCR is <100 ms after that.
2. Point a USB webcam at the heltec-v3 OLED. Discover its index:
```bash
.venv/bin/python -c "import cv2; [print(i, cv2.VideoCapture(i).read()[0]) for i in range(5)]"
```
3. Export the per-role device env var:
```bash
export MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0
```
4. Run:
```bash
./run-tests.sh tests/ui -v
```
Captures land under `tests/ui_captures/<session_seed>/<test_id>/`, one
PNG + `.ocr.txt` per `frame_capture()` call, with a per-test
`transcript.md` stepping through event → frame → OCR. The HTML report
embeds the full image strip inline (pass or fail).
On macOS, `cv2.VideoCapture(0)` triggers the TCC Camera permission prompt
on first use. Pre-grant Terminal (or your IDE's terminal) before running.
The `OpenCVBackend` fails fast on 10 consecutive black frames so a silent
permission denial surfaces as a clear error, not an empty PNG strip.
No camera? Set `MESHTASTIC_UI_CAMERA_BACKEND=null` (or leave the device var
unset). Tests still exercise the event-injection path and log assertions;
captures just become 1×1 black PNGs.
### Artifacts (regenerated every run, under `tests/`)
- `report.html` — self-contained pytest-html report. Each test gets a
**Meshtastic debug** section attached on failure with a 200-line firmware
log tail + device-state dump. Open this first on failures.
- `junit.xml` — CI-parseable.
- `reportlog.jsonl` — `pytest-reportlog` event stream; consumed by the TUI.
- `fwlog.jsonl` — firmware log mirror (`meshtastic.log.line` pubsub → JSONL).
- `flash.log` — tee of all pio / esptool / nrfutil / picotool subprocess
output during the run (driven by `MESHTASTIC_MCP_FLASH_LOG`).
### Live TUI
```bash
.venv/bin/meshtastic-mcp-test-tui
.venv/bin/meshtastic-mcp-test-tui tests/mesh # pytest args pass through
```
Textual-based wrapper over `run-tests.sh` with a live test tree, tier
counters, pytest output pane, firmware-log pane, and a device-status strip.
Key bindings: `r` re-run focused, `f` filter, `d` failure detail, `g` open
`report.html`, `x` export reproducer bundle, `l` cycle fw-log filter, `q`
quit (SIGINT → SIGTERM → SIGKILL escalation).
Set `MESHTASTIC_UI_TUI_CAMERA=1` to mount a bottom-of-screen **UI camera**
panel. Left side: the latest capture PNG rendered as Unicode half-blocks
(via `rich-pixels`, works in any terminal — no kitty/sixel required).
Right side: live transcript tail ("step 3 — frame 4/8 name=nodelist_nodes
— OCR: Nodes 2/2") so you can see every event-injection and its result
as each UI test runs. Requires the `[ui]` extras for image rendering; the
transcript alone works without them.
### Slash commands
Three AI-assisted workflows are wired up for Claude Code operators
(`.claude/commands/`) and Copilot operators (`.github/prompts/`):
`/test` (run + interpret), `/diagnose` (read-only health report), `/repro`
(flake triage, N-times re-run with log diff).
### House rules (for human + agent contributors)
- Session-scoped fixtures in `tests/conftest.py` snapshot + restore
`userPrefs.jsonc`; **never edit `userPrefs.jsonc` from inside a test**.
Use the `test_profile` / `no_region_profile` fixtures for ephemeral
overrides.
- `SerialInterface` holds an **exclusive port lock**; sequence calls
open → mutate → close, then next device. No parallel calls to the
same port.
- Directed PKI-encrypted sends need **bilateral NodeInfo warmup** —
both sides must hold the other's current pubkey. See
`tests/mesh/_receive.py::nudge_nodeinfo_port` and the three directed-
send tests (`test_direct_with_ack`, `test_traceroute`,
`test_telemetry_request_reply`) for the canonical pattern.
## Layout
```text
mcp-server/
├── pyproject.toml
├── README.md
└── src/meshtastic_mcp/
├── __main__.py # entry: python -m meshtastic_mcp
├── server.py # FastMCP app + @app.tool() registrations (thin)
├── config.py # firmware_root, pio_bin, esptool_bin, etc.
├── pio.py # subprocess wrapper (timeouts, JSON, tail_lines)
├── devices.py # list_devices (findPorts + comports)
├── boards.py # list_boards / get_board (pio project config parse + cache)
├── flash.py # build, clean, flash, erase_and_flash, update_flash, touch_1200bps
├── serial_session.py # SerialSession + reader thread + ring buffer
├── registry.py # session registry + per-port locks
├── connection.py # connect(port) ctx mgr — SerialInterface + port lock
├── info.py # device_info, list_nodes
├── admin.py # set_owner, get/set_config, channels, send_text, reboot/shutdown/factory_reset
└── hw_tools.py # esptool / nrfutil / picotool wrappers
```
## Troubleshooting
- **"Could not locate Meshtastic firmware root"** — set `MESHTASTIC_FIRMWARE_ROOT`.
- **"Could not find `pio`"** — install PlatformIO or set `MESHTASTIC_PIO_BIN`.
- **"Port is held by serial session ..."** — call `serial_close(session_id)` or `serial_list` to find it.
- **`factory.bin` not found after build** — the env may not be ESP32; only ESP32 envs produce a `.factory.bin`.
- **`touch_1200bps` reported `new_port: null`** — the device may not have 1200bps-reset stdio, or the bootloader re-uses the same port name. Check `list_devices` manually.
+54
View File
@@ -0,0 +1,54 @@
[project]
name = "meshtastic-mcp"
version = "0.1.0"
description = "MCP server for Meshtastic firmware development: device discovery, PlatformIO tooling, flashing, serial monitoring, and device administration via the meshtastic Python API."
readme = "README.md"
requires-python = ">=3.11"
license = { text = "GPL-3.0-only" }
authors = [{ name = "thebentern" }]
dependencies = ["mcp>=1.2", "pyserial>=3.5", "meshtastic>=2.7.8"]
[project.optional-dependencies]
dev = ["pytest>=7"]
test = [
"pytest>=8",
"pytest-html>=4",
"pytest-reportlog>=0.4",
"pytest-timeout>=2.3",
"coverage[toml]>=7",
"pyyaml>=6",
# textual is required by the `meshtastic-mcp-test-tui` script (see
# `src/meshtastic_mcp/cli/test_tui.py`). Bundled into `test` rather than a
# separate `[tui]` extra because v1 expects test operators are the only
# consumers; revisit if install cost pushes back.
"textual>=0.50",
]
# UI test tier + `capture_screen` MCP tool. Optional because the ML OCR
# model alone is ~100 MB and camera hardware is user-supplied.
# pip install -e '.[ui]' — full (OpenCV + easyocr)
# pip install -e '.[ui-min]' — image capture only, no OCR
ui = [
"opencv-python-headless>=4.9",
"numpy>=1.26",
"easyocr>=1.7",
"Pillow>=10.0",
# Renders the latest camera capture as Unicode half-blocks in the TUI
# (MESHTASTIC_UI_TUI_CAMERA=1). Terminal-agnostic — no kitty / sixel
# dependency. Pure Python, tiny.
"rich-pixels>=3.0",
]
ui-min = ["opencv-python-headless>=4.9", "numpy>=1.26"]
[project.scripts]
meshtastic-mcp = "meshtastic_mcp.__main__:main"
# Live TUI wrapping run-tests.sh — shells out to the same script the plain
# CLI uses, tails pytest-reportlog for per-test state, and polls the device
# list at startup + post-run (port lock forces it to stay idle during the run).
meshtastic-mcp-test-tui = "meshtastic_mcp.cli.test_tui:main"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/meshtastic_mcp"]
+270
View File
@@ -0,0 +1,270 @@
#!/usr/bin/env bash
# mcp-server hardware test runner.
#
# Auto-detects connected Meshtastic devices, maps each to its PlatformIO env
# via the same role table the pytest fixtures use, exports the right
# MESHTASTIC_MCP_ENV_* env vars, and invokes pytest.
#
# Usage:
# ./run-tests.sh # full suite, default pytest args
# ./run-tests.sh tests/mesh # subset (any pytest args pass through)
# ./run-tests.sh --force-bake # override one default with another
# MESHTASTIC_MCP_ENV_NRF52=foo ./run-tests.sh # override env per role
# MESHTASTIC_MCP_SEED=ci-run-42 ./run-tests.sh # override PSK seed
#
# If zero supported devices are detected, only the unit tier runs.
#
# Also restores `userPrefs.jsonc` from the session-backup sidecar if a prior
# run exited abnormally (belt to conftest.py's atexit suspenders).
set -euo pipefail
# cd to the script's directory so relative paths resolve consistently no
# matter where the user invoked from.
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
cd "$SCRIPT_DIR"
VENV_PY="$SCRIPT_DIR/.venv/bin/python"
if [[ ! -x $VENV_PY ]]; then
echo "error: $VENV_PY not found or not executable." >&2
echo " Bootstrap the venv first:" >&2
echo " cd $SCRIPT_DIR && python3 -m venv .venv && .venv/bin/pip install -e '.[test]'" >&2
exit 2
fi
# Resolve firmware root the same way conftest.py does (this script sits in
# mcp-server/, firmware repo root is one level up).
FIRMWARE_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
USERPREFS_PATH="$FIRMWARE_ROOT/userPrefs.jsonc"
USERPREFS_SIDECAR="$USERPREFS_PATH.mcp-session-bak"
# ---------- Pre-flight: recover stale userPrefs.jsonc from prior crash ----
# If conftest.py's atexit hook didn't fire (SIGKILL, kernel panic, OS
# restart), the sidecar is the ground truth. Self-heal before running so we
# don't bake the previous run's dirty state into this run's firmware.
if [[ -f $USERPREFS_SIDECAR ]]; then
echo "[pre-flight] found $USERPREFS_SIDECAR from a prior abnormal exit;" >&2
echo " restoring userPrefs.jsonc before starting." >&2
cp "$USERPREFS_SIDECAR" "$USERPREFS_PATH"
rm -f "$USERPREFS_SIDECAR"
fi
# If userPrefs.jsonc has uncommitted changes BEFORE the run starts, that's
# worth warning about — tests will snapshot this dirty state and restore to
# it at the end, which may not be what the operator wants.
if command -v git >/dev/null 2>&1; then
cd "$FIRMWARE_ROOT"
# Capture the git status into a local first — SC2312 flags command
# substitution inside `[[ -n ... ]]` because the exit code of `git
# status` is masked. A two-step assignment makes the failure path
# explicit (non-git, missing file) and keeps the bracket test clean.
_git_status_porcelain="$(git status --porcelain userPrefs.jsonc 2>/dev/null || true)"
if [[ -n $_git_status_porcelain ]]; then
echo "[pre-flight] warning: userPrefs.jsonc has uncommitted changes." >&2
echo " Tests will snapshot THIS state and restore to it" >&2
echo " at teardown. If that's not intended, run:" >&2
echo " git checkout userPrefs.jsonc" >&2
echo " and re-invoke." >&2
fi
cd "$SCRIPT_DIR"
fi
# ---------- Seed default --------------------------------------------------
# Per-machine default so repeated runs from the same operator land on the
# same PSK (makes --assume-baked valid across invocations). Operator can
# override with an explicit env var if they want isolation (e.g. CI).
if [[ -z ${MESHTASTIC_MCP_SEED-} ]]; then
WHO="$(whoami 2>/dev/null || echo anon)"
HOST="$(hostname -s 2>/dev/null || echo host)"
export MESHTASTIC_MCP_SEED="mcp-${WHO}-${HOST}"
fi
# ---------- Flash progress log --------------------------------------------
# pio.py / hw_tools.py tee subprocess output (pio run -t upload, esptool,
# nrfutil, picotool) to this file line-by-line as it arrives when this env
# var is set. The TUI tails it so the operator sees live flash progress
# instead of 3 minutes of silence during `test_00_bake.py`. Plain CLI users
# also benefit — the log is a post-run diagnostic even without the TUI.
# Truncate at session start so each run gets a clean log.
export MESHTASTIC_MCP_FLASH_LOG="$SCRIPT_DIR/tests/flash.log"
: >"$MESHTASTIC_MCP_FLASH_LOG"
# ---------- Detect connected hardware -------------------------------------
# In-process call to the same Python API the test fixtures use, so the
# script never drifts from what pytest sees. Returns a JSON object
# {role: port, ...}.
ROLES_JSON="$(
"$VENV_PY" - <<'PY'
import json
import sys
sys.path.insert(0, "src")
from meshtastic_mcp import devices
# Role → canonical VID map. Kept in sync with
# `tests/conftest.py::hub_profile` defaults; if that changes, this must too.
ROLE_BY_VID = {
0x239A: "nrf52", # Adafruit / RAK nRF52 native USB (app + DFU)
0x303A: "esp32s3", # Espressif native USB (ESP32-S3)
0x10C4: "esp32s3", # CP2102 USB-UART (common on Heltec/LilyGO ESP32 boards)
}
out: dict[str, str] = {}
for dev in devices.list_devices(include_unknown=True):
vid_raw = dev.get("vid") or ""
try:
if isinstance(vid_raw, str) and vid_raw.startswith("0x"):
vid = int(vid_raw, 16)
else:
vid = int(vid_raw)
except (TypeError, ValueError):
continue
role = ROLE_BY_VID.get(vid)
# First port wins per role — matches hub_devices fixture semantics.
if role and role not in out:
out[role] = dev["port"]
json.dump(out, sys.stdout)
PY
)"
# ---------- Map role → pio env --------------------------------------------
# Honor MESHTASTIC_MCP_ENV_<ROLE> operator overrides; fall back to the
# same defaults hardcoded in tests/conftest.py::_DEFAULT_ROLE_ENVS.
resolve_env() {
local role="$1"
local default="$2"
local upper
upper="$(echo "$role" | tr '[:lower:]' '[:upper:]')"
local var="MESHTASTIC_MCP_ENV_${upper}"
eval "local override=\${$var:-}"
if [[ -n $override ]]; then
echo "$override"
else
echo "$default"
fi
}
NRF52_PORT="$(echo "$ROLES_JSON" | "$VENV_PY" -c 'import json,sys; print(json.loads(sys.stdin.read()).get("nrf52", ""))')"
ESP32S3_PORT="$(echo "$ROLES_JSON" | "$VENV_PY" -c 'import json,sys; print(json.loads(sys.stdin.read()).get("esp32s3", ""))')"
DETECTED=""
if [[ -n $NRF52_PORT ]]; then
NRF52_ENV="$(resolve_env nrf52 rak4631)"
export MESHTASTIC_MCP_ENV_NRF52="$NRF52_ENV"
DETECTED="${DETECTED} nrf52 @ ${NRF52_PORT} -> env=${NRF52_ENV}\n"
fi
if [[ -n $ESP32S3_PORT ]]; then
ESP32S3_ENV="$(resolve_env esp32s3 heltec-v3)"
export MESHTASTIC_MCP_ENV_ESP32S3="$ESP32S3_ENV"
DETECTED="${DETECTED} esp32s3 @ ${ESP32S3_PORT} -> env=${ESP32S3_ENV}\n"
fi
# ---------- Pre-flight summary --------------------------------------------
# Surface what pytest is about to do with respect to the bake phase: the
# operator should see "will verify + bake if needed" by default, so a
# 3-minute flash appearing mid-run isn't a surprise. Detection of the
# explicit overrides is best-effort — we just scan $@ for the known flags.
_bake_mode="auto (verify + bake if needed)"
for _arg in "$@"; do
case "$_arg" in
--assume-baked) _bake_mode="skip (--assume-baked)" ;;
--force-bake) _bake_mode="force (--force-bake)" ;;
*) ;; # any other arg: pass-through; bake mode unchanged
esac
done
echo "mcp-server test runner"
echo " firmware root : $FIRMWARE_ROOT"
echo " seed : $MESHTASTIC_MCP_SEED"
echo " bake : $_bake_mode"
if [[ -n $DETECTED ]]; then
echo " detected hub :"
printf "%b" "$DETECTED"
else
echo " detected hub : (none)"
fi
echo
# ---------- Invoke pytest -------------------------------------------------
# If no devices detected, only the unit tier would produce meaningful
# PASS/FAIL — every hardware test would SKIP with "role not present". We
# narrow to tests/unit explicitly so the summary reads as "no hardware,
# unit suite only" instead of "big skip count looks suspicious".
if [[ -z $DETECTED && $# -eq 0 ]]; then
echo "[pre-flight] no supported devices detected; running unit tier only."
echo
exec "$VENV_PY" -m pytest tests/unit -v --report-log=tests/reportlog.jsonl
fi
# Default pytest args when the user passed none. Power users can invoke
# `./run-tests.sh tests/mesh -v --tb=long` and skip all of these defaults.
#
# NOTE: `--assume-baked` is DELIBERATELY omitted here. `tests/test_00_bake.py`
# has an internal skip-if-already-baked check (`_bake_role`: query device_info,
# compare region + primary_channel to the session profile, skip on match).
# So the fast path is ~8-10 s of verification overhead when the devices are
# already baked — negligible next to the 2-6 min suite runtime. Letting
# test_00_bake.py run means a fresh device, a re-seeded session, or a post-
# factory-reset device gets flashed automatically instead of silently
# skipping half the hardware tests with "not baked with session profile"
# errors. Power users who know their hardware is current and want to shave
# those seconds can pass `--assume-baked` explicitly.
if [[ $# -eq 0 ]]; then
set -- tests/ \
--html=tests/report.html --self-contained-html \
--junitxml=tests/junit.xml \
-v --tb=short
fi
# UI tier requires opencv-python-headless (and ideally easyocr). If it's
# not installed, auto-deselect tests/ui so operators without the [ui]
# extra still get a green run. Printed in yellow; silent when cv2 is
# present.
_cv2_ok=0
if "$VENV_PY" -c "import cv2" >/dev/null 2>&1; then
_cv2_ok=1
fi
_running_ui=0
for _arg in "$@"; do
case "$_arg" in
*tests/ui* | tests/) _running_ui=1 ;;
*) ;;
esac
done
if [[ $_running_ui -eq 1 && $_cv2_ok -eq 0 ]]; then
printf '\033[33m[pre-flight] tests/ui tier detected, but opencv-python-headless is not installed — deselecting.\033[0m\n'
printf ' install with: .venv/bin/pip install -e "mcp-server/.[ui]"\n'
echo
set -- "$@" --ignore=tests/ui
fi
# Recovery tier needs `uhubctl` on PATH — it power-cycles devices via USB
# hub PPPS. The tier's conftest already skips cleanly, so this is just a
# friendly heads-up before the skip happens. `baked_single`'s auto-
# recovery hook also benefits from having uhubctl available across the
# whole suite.
if ! command -v uhubctl >/dev/null 2>&1; then
printf "\033[33m[pre-flight] uhubctl not found on PATH — recovery tier will skip, and\n"
printf " wedged-device auto-recovery is disabled.\033[0m\n"
printf " install with: brew install uhubctl (macOS) or apt install uhubctl (Debian/Ubuntu).\n"
echo
fi
# Always emit `tests/reportlog.jsonl` (unless the operator explicitly passed
# their own `--report-log=...`). Consumers — notably the
# `meshtastic-mcp-test-tui` TUI — tail the reportlog for live per-test state.
# Appending here means power-user invocations like `./run-tests.sh tests/mesh`
# also produce it, not just the all-defaults invocation.
_has_report_log=0
for _arg in "$@"; do
case "$_arg" in
--report-log | --report-log=*) _has_report_log=1 ;;
*) ;; # any other arg: no-op; loop continues
esac
done
if [[ $_has_report_log -eq 0 ]]; then
set -- "$@" --report-log=tests/reportlog.jsonl
fi
exec "$VENV_PY" -m pytest "$@"
+217
View File
@@ -0,0 +1,217 @@
{
"title": "Meshtastic Firmware — Recorder Stream",
"description": "Live view of `.mtlog/` streams shipped by `mtlog_to_datadog.py`. Heap, packet volume, log levels, errors. One row per port.",
"widgets": [
{
"definition": {
"title": "Free heap (bytes)",
"type": "timeseries",
"show_legend": true,
"requests": [
{
"queries": [
{
"name": "free_heap",
"data_source": "metrics",
"query": "avg:mesh.local.heap_free_bytes{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
],
"yaxis": { "label": "bytes" }
}
},
{
"definition": {
"title": "Heap slope (bytes/min) — last 1h",
"type": "query_value",
"precision": 0,
"requests": [
{
"queries": [
{
"name": "slope",
"data_source": "metrics",
"query": "derivative(avg:mesh.local.heap_free_bytes{service:meshtastic-firmware})",
"aggregator": "avg"
}
],
"response_format": "scalar"
}
],
"conditional_formats": [
{ "comparator": "<", "value": -100, "palette": "white_on_red" },
{ "comparator": "<", "value": 0, "palette": "white_on_yellow" },
{ "comparator": ">=", "value": 0, "palette": "white_on_green" }
]
}
},
{
"definition": {
"title": "Total heap (bytes)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "total_heap",
"data_source": "metrics",
"query": "avg:mesh.local.heap_total_bytes{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Battery level (%)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "battery",
"data_source": "metrics",
"query": "avg:mesh.device.battery_level{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
],
"yaxis": { "min": "0", "max": "105" }
}
},
{
"definition": {
"title": "Air utilization (TX %)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "airutil",
"data_source": "metrics",
"query": "avg:mesh.device.air_util_tx{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Channel utilization (%)",
"type": "timeseries",
"requests": [
{
"queries": [
{
"name": "chutil",
"data_source": "metrics",
"query": "avg:mesh.device.channel_utilization{service:meshtastic-firmware} by {port}"
}
],
"response_format": "timeseries",
"display_type": "line"
}
]
}
},
{
"definition": {
"title": "Log volume by level",
"type": "timeseries",
"show_legend": true,
"requests": [
{
"response_format": "timeseries",
"display_type": "bars",
"queries": [
{
"name": "log_count",
"data_source": "logs",
"indexes": ["*"],
"compute": { "aggregation": "count" },
"search": { "query": "service:meshtastic-firmware" },
"group_by": [
{
"facet": "@level",
"limit": 10,
"sort": { "order": "desc", "aggregation": "count" }
}
]
}
]
}
]
}
},
{
"definition": {
"title": "Recent ERROR / CRIT firmware logs",
"type": "list_stream",
"requests": [
{
"response_format": "event_list",
"query": {
"data_source": "logs_stream",
"query_string": "service:meshtastic-firmware (status:error OR @level:ERROR OR @level:CRIT)",
"indexes": [],
"sort": { "column": "timestamp", "order": "desc" }
},
"columns": [
{ "field": "timestamp", "width": "auto" },
{ "field": "host", "width": "auto" },
{ "field": "@port", "width": "auto" },
{ "field": "@level", "width": "auto" },
{ "field": "@thread", "width": "auto" },
{ "field": "message", "width": "stretch" }
]
}
]
}
},
{
"definition": {
"title": "Recorder marker events",
"type": "list_stream",
"requests": [
{
"response_format": "event_list",
"query": {
"data_source": "logs_stream",
"query_string": "service:meshtastic-firmware @level:MARK",
"indexes": [],
"sort": { "column": "timestamp", "order": "desc" }
},
"columns": [
{ "field": "timestamp", "width": "auto" },
{ "field": "host", "width": "auto" },
{ "field": "message", "width": "stretch" }
]
}
]
}
}
],
"template_variables": [
{
"name": "port",
"prefix": "port",
"available_values": [],
"default": "*"
},
{ "name": "host", "prefix": "host", "available_values": [], "default": "*" }
],
"layout_type": "ordered",
"notify_list": [],
"reflow_type": "auto"
}
+389
View File
@@ -0,0 +1,389 @@
#!/usr/bin/env python3
"""Forward selected recorder JSONL streams to Datadog.
Reads `.mtlog/logs.jsonl` and `.mtlog/telemetry.jsonl`, ships logs to the
Logs Intake API and telemetry numerics to the Metrics v2 series API.
Resumes from `.mtlog/.dd-cursor.json` so a daemon restart doesn't
duplicate rows already shipped from the current live files.
This forwarder does not currently backfill rotated `.jsonl.gz` archives.
If the recorder rotates before this process drains the live file, or the
forwarder is down across a rotation, those older rows are skipped.
Usage:
DD_API_KEY=... ./scripts/mtlog_to_datadog.py --tail
./scripts/mtlog_to_datadog.py --once # catch up + exit
./scripts/mtlog_to_datadog.py --since 3600 # backfill last hour from start
Default `DD_SITE` is `us5.datadoghq.com` — the team's Datadog instance.
Override via `DD_SITE=...` env var or `--site` flag for one-offs.
The forwarder is a separate process by design — a Datadog outage or
auth failure must not backpressure the recorder. We exit non-zero on
fatal config errors (missing API key) and keep retrying on transient
network/HTTP errors.
"""
from __future__ import annotations
import argparse
import json
import os
import socket
import sys
import time
from pathlib import Path
from typing import Any, Iterator
try:
import requests
except ImportError:
print(
"requests is required. Install it in the mcp-server venv: "
"uv pip install requests",
file=sys.stderr,
)
sys.exit(2)
_DEFAULT_LOG_DIR = Path(__file__).resolve().parents[1] / ".mtlog"
_LOG_INTAKE_TPL = "https://http-intake.logs.{site}/api/v2/logs"
_METRICS_TPL = "https://api.{site}/api/v2/series"
_LOG_BATCH = 50
_METRICS_BATCH = 100
_MAX_RETRIES = 5
_RETRY_BASE_S = 1.5
# --- streaming JSONL with byte-position cursor -------------------------
class _StreamReader:
"""Reads a single rotating JSONL with cursor-based resume.
This tails only the live `.jsonl` file. The recorder rotates files
(live `.jsonl` → `.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz`), which means
the live file shrinks abruptly. We detect that via inode change OR live
size < cursor position, and reset the live-file cursor to 0.
"""
def __init__(self, path: Path, cursor: dict[str, Any]):
self.path = path
self.cursor = cursor
def _state(self) -> tuple[int, int]:
"""Return (inode, size) for the live file. (0, 0) if missing."""
try:
st = self.path.stat()
return (st.st_ino, st.st_size)
except FileNotFoundError:
return (0, 0)
def iter_new_records(self) -> Iterator[dict[str, Any]]:
ino, size = self._state()
last_ino = self.cursor.get("ino")
last_pos = int(self.cursor.get("pos") or 0)
if ino == 0:
return
if last_ino is not None and last_ino != ino:
# Rotation happened. Start over.
last_pos = 0
if last_pos > size:
# Live file truncated/shrunk under us — recorder rotated.
last_pos = 0
try:
with self.path.open("r", encoding="utf-8") as fh:
fh.seek(last_pos)
for line in fh:
line = line.rstrip("\n")
if not line:
continue
try:
yield json.loads(line)
except json.JSONDecodeError:
continue
last_pos = fh.tell()
except FileNotFoundError:
return
self.cursor["ino"] = ino
self.cursor["pos"] = last_pos
def _load_cursor(path: Path) -> dict[str, Any]:
if not path.exists():
return {}
try:
return json.loads(path.read_text())
except (OSError, json.JSONDecodeError):
return {}
def _save_cursor(path: Path, data: dict[str, Any]) -> None:
tmp = path.with_suffix(".json.tmp")
tmp.write_text(json.dumps(data, separators=(",", ":")))
tmp.replace(path)
# --- Datadog clients ---------------------------------------------------
class _DDSession:
"""Pool one HTTPS session, share retry logic."""
def __init__(self, api_key: str, site: str, hostname: str) -> None:
self.api_key = api_key
self.site = site
self.hostname = hostname
self.session = requests.Session()
self.session.headers.update(
{
"DD-API-KEY": api_key,
"Content-Type": "application/json",
}
)
def _post(self, url: str, payload: Any) -> bool:
for attempt in range(_MAX_RETRIES):
try:
resp = self.session.post(url, json=payload, timeout=30)
except requests.RequestException as e:
_wait_retry(attempt, f"network error: {e}")
continue
if 200 <= resp.status_code < 300:
return True
if resp.status_code in (408, 429, 500, 502, 503, 504):
_wait_retry(
attempt,
f"HTTP {resp.status_code} retrying",
)
continue
print(
f"datadog refused: {resp.status_code} {resp.text[:200]}",
file=sys.stderr,
)
return False
return False
def send_logs(self, records: list[dict[str, Any]]) -> int:
if not records:
return 0
url = _LOG_INTAKE_TPL.format(site=self.site)
sent = 0
for i in range(0, len(records), _LOG_BATCH):
batch = records[i : i + _LOG_BATCH]
if self._post(url, batch):
sent += len(batch)
return sent
def send_metrics(self, series: list[dict[str, Any]]) -> int:
if not series:
return 0
url = _METRICS_TPL.format(site=self.site)
sent = 0
for i in range(0, len(series), _METRICS_BATCH):
batch = series[i : i + _METRICS_BATCH]
if self._post(url, {"series": batch}):
sent += len(batch)
return sent
def _wait_retry(attempt: int, reason: str) -> None:
wait = _RETRY_BASE_S * (2**attempt)
print(
f" retry {attempt + 1}/{_MAX_RETRIES} in {wait:.1f}s ({reason})",
file=sys.stderr,
)
time.sleep(wait)
# --- record → datadog payload ------------------------------------------
def _log_record_to_dd(rec: dict[str, Any], host: str) -> dict[str, Any]:
line = rec.get("line") or ""
tags = [
f"role:{rec.get('role')}",
f"port:{rec.get('port')}",
]
level = rec.get("level")
if level:
tags.append(f"level:{level}")
tag = rec.get("tag")
if tag:
tags.append(f"thread:{tag}")
return {
"ddsource": "meshtastic-firmware",
"service": "meshtastic-firmware",
"hostname": host,
"message": line,
"ddtags": ",".join(t for t in tags if t and "None" not in t),
"timestamp": int((rec.get("ts") or time.time()) * 1000),
"level": level,
}
def _telemetry_record_to_metrics(
rec: dict[str, Any], host: str
) -> list[dict[str, Any]]:
fields = rec.get("fields") or {}
if not isinstance(fields, dict):
return []
variant = rec.get("variant") or "unknown"
ts = int(rec.get("ts") or time.time())
out: list[dict[str, Any]] = []
tags = []
if rec.get("port"):
tags.append(f"port:{rec['port']}")
if rec.get("role"):
tags.append(f"role:{rec['role']}")
if rec.get("from_node"):
tags.append(f"from_node:{rec['from_node']}")
tags.append(f"variant:{variant}")
for field, value in fields.items():
if not isinstance(value, (int, float)) or isinstance(value, bool):
continue
metric = f"mesh.{variant}.{_metric_safe(field)}"
out.append(
{
"metric": metric,
"type": 3, # GAUGE
"points": [{"timestamp": ts, "value": float(value)}],
"tags": tags,
"resources": [{"type": "host", "name": host}],
}
)
return out
def _metric_safe(name: str) -> str:
# Lowercase, replace non-alnum with underscore for safe metric names.
return "".join(c.lower() if c.isalnum() else "_" for c in name)
# --- main loop ---------------------------------------------------------
def run(
log_dir: Path,
*,
once: bool,
since_seconds: float | None,
poll_interval: float,
dd: _DDSession,
) -> int:
cursor_path = log_dir / ".dd-cursor.json"
cursors = _load_cursor(cursor_path)
# `--since` overrides cursor: rewind to (now-since) timestamp.
# We can't seek by timestamp directly (cursor is byte position), so
# we just reset cursors to 0 and let the time filter in iter_new
# drop older records.
cutoff_ts: float | None = None
if since_seconds is not None:
cursors = {}
cutoff_ts = time.time() - since_seconds
sent_total = {"logs": 0, "telemetry": 0}
while True:
# logs.jsonl → DD logs
log_cursor = cursors.setdefault("logs", {})
log_batch: list[dict[str, Any]] = []
for rec in _StreamReader(log_dir / "logs.jsonl", log_cursor).iter_new_records():
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
continue
log_batch.append(_log_record_to_dd(rec, dd.hostname))
if log_batch:
n = dd.send_logs(log_batch)
sent_total["logs"] += n
print(f"logs: sent {n}/{len(log_batch)}")
# telemetry.jsonl → DD metrics
telem_cursor = cursors.setdefault("telemetry", {})
metric_series: list[dict[str, Any]] = []
for rec in _StreamReader(
log_dir / "telemetry.jsonl", telem_cursor
).iter_new_records():
if cutoff_ts and (rec.get("ts") or 0) < cutoff_ts:
continue
metric_series.extend(_telemetry_record_to_metrics(rec, dd.hostname))
if metric_series:
n = dd.send_metrics(metric_series)
sent_total["telemetry"] += n
print(f"telemetry: sent {n}/{len(metric_series)} metric points")
_save_cursor(cursor_path, cursors)
if once:
print(f"done. logs={sent_total['logs']} metrics={sent_total['telemetry']}")
return 0
time.sleep(poll_interval)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--log-dir",
default=str(_DEFAULT_LOG_DIR),
help="Path to .mtlog/ (default: mcp-server/.mtlog)",
)
mode = parser.add_mutually_exclusive_group()
mode.add_argument("--once", action="store_true", help="Catch up then exit")
mode.add_argument(
"--tail",
action="store_true",
help="Daemon: poll forever (default)",
)
parser.add_argument(
"--since",
type=float,
default=None,
help="Backfill last N seconds. Resets cursor.",
)
parser.add_argument(
"--poll-interval",
type=float,
default=5.0,
help="Seconds between tail polls (default 5)",
)
parser.add_argument(
"--site",
default=os.environ.get("DD_SITE", "us5.datadoghq.com"),
help=(
"Datadog site. Default is the team's instance (us5.datadoghq.com). "
"Override via DD_SITE env var or this flag."
),
)
parser.add_argument(
"--host",
default=socket.gethostname(),
help="Hostname tag (default: socket.gethostname())",
)
args = parser.parse_args(argv)
api_key = os.environ.get("DD_API_KEY")
if not api_key:
print("DD_API_KEY env var required.", file=sys.stderr)
return 2
log_dir = Path(args.log_dir)
if not log_dir.exists():
print(
f"log dir {log_dir} does not exist — start the mcp-server first.",
file=sys.stderr,
)
return 2
dd = _DDSession(api_key=api_key, site=args.site, hostname=args.host)
once = args.once and not args.tail
return run(
log_dir,
once=once,
since_seconds=args.since,
poll_interval=args.poll_interval,
dd=dd,
)
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,3 @@
"""Meshtastic MCP server — device discovery, PlatformIO tooling, and device admin."""
__version__ = "0.1.0"
+11
View File
@@ -0,0 +1,11 @@
"""Entry point for `python -m meshtastic_mcp`."""
from meshtastic_mcp.server import app
def main() -> None:
app.run()
if __name__ == "__main__":
main()
+417
View File
@@ -0,0 +1,417 @@
"""Device administration: owner, config, channels, messaging, admin actions.
All operations use the same `connect()` context manager so port selection,
port-busy detection, and cleanup are handled uniformly.
Config writes use a dot-path: the first segment names a section (e.g.
`"lora"` in LocalConfig or `"mqtt"` in LocalModuleConfig), remaining segments
walk protobuf fields. Enum fields accept their string names (`"US"` for
`lora.region`) so callers don't need to know the numeric values.
"""
from __future__ import annotations
from typing import Any
from google.protobuf import descriptor as pb_descriptor
from google.protobuf import json_format
from meshtastic.protobuf import localonly_pb2
from .connection import connect
class AdminError(RuntimeError):
pass
LOCAL_CONFIG_SECTIONS = {f.name for f in localonly_pb2.LocalConfig.DESCRIPTOR.fields}
MODULE_CONFIG_SECTIONS = {
f.name for f in localonly_pb2.LocalModuleConfig.DESCRIPTOR.fields
}
def _require_confirm(confirm: bool, operation: str) -> None:
if not confirm:
raise AdminError(f"{operation} is destructive and requires confirm=True.")
def _message_to_dict(msg: Any) -> dict[str, Any]:
# `including_default_value_fields` was renamed to
# `always_print_fields_with_no_presence` in protobuf 5.26+. Pick whichever
# kwarg the installed version accepts so we work against both.
kwargs: dict[str, Any] = {"preserving_proto_field_name": True}
import inspect
sig = inspect.signature(json_format.MessageToDict)
if "always_print_fields_with_no_presence" in sig.parameters:
kwargs["always_print_fields_with_no_presence"] = False
elif "including_default_value_fields" in sig.parameters:
kwargs["including_default_value_fields"] = False
return json_format.MessageToDict(msg, **kwargs)
# ---------- owner ----------------------------------------------------------
def set_owner(
long_name: str,
short_name: str | None = None,
port: str | None = None,
) -> dict[str, Any]:
if short_name is not None and len(short_name) > 4:
raise AdminError("short_name must be 4 characters or fewer")
with connect(port=port) as iface:
iface.localNode.setOwner(long_name=long_name, short_name=short_name)
return {
"ok": True,
"long_name": long_name,
"short_name": short_name,
}
# ---------- config reads ---------------------------------------------------
def _section_container(node, section: str) -> tuple[Any, str]:
"""Return (container_message, parent_name) for a section name.
Parent is 'localConfig' or 'moduleConfig' so callers know where to call
writeConfig() after mutating.
"""
if section in LOCAL_CONFIG_SECTIONS:
return getattr(node.localConfig, section), "localConfig"
if section in MODULE_CONFIG_SECTIONS:
return getattr(node.moduleConfig, section), "moduleConfig"
raise AdminError(
f"Unknown config section: {section!r}. "
f"Valid sections: {sorted(LOCAL_CONFIG_SECTIONS | MODULE_CONFIG_SECTIONS)}"
)
def get_config(section: str | None = None, port: str | None = None) -> dict[str, Any]:
"""Read one or all config sections.
`section` may be any name in LocalConfig (device, lora, position, power,
network, display, bluetooth, security) or LocalModuleConfig (mqtt, serial,
telemetry, ...). Omit `section` or pass `"all"` for everything.
"""
with connect(port=port) as iface:
node = iface.localNode
if section in (None, "all"):
lc = _message_to_dict(node.localConfig)
mc = _message_to_dict(node.moduleConfig)
return {
"config": {
"localConfig": lc,
"moduleConfig": mc,
}
}
container, _parent = _section_container(node, section)
return {"config": {section: _message_to_dict(container)}}
# ---------- config writes --------------------------------------------------
def _coerce_enum(field: pb_descriptor.FieldDescriptor, value: Any) -> int:
"""Accept an enum value as either its int or its string name."""
enum_type = field.enum_type
if isinstance(value, bool):
raise AdminError(f"{field.name}: expected enum {enum_type.name}, got bool")
if isinstance(value, int):
if enum_type.values_by_number.get(value) is None:
raise AdminError(
f"{field.name}: {value} is not a valid {enum_type.name} value"
)
return value
if isinstance(value, str):
upper = value.upper()
ev = enum_type.values_by_name.get(upper)
if ev is None:
valid = sorted(enum_type.values_by_name.keys())
raise AdminError(
f"{field.name}: {value!r} is not a valid {enum_type.name}. "
f"Valid: {valid}"
)
return ev.number
raise AdminError(
f"{field.name}: expected enum {enum_type.name}, got {type(value).__name__}"
)
def _coerce_scalar(field: pb_descriptor.FieldDescriptor, value: Any) -> Any:
t = field.type
FT = pb_descriptor.FieldDescriptor
if t == FT.TYPE_ENUM:
return _coerce_enum(field, value)
if t == FT.TYPE_BOOL:
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.strip().lower() in ("true", "yes", "1", "on")
if isinstance(value, int):
return bool(value)
if t in (
FT.TYPE_INT32,
FT.TYPE_INT64,
FT.TYPE_UINT32,
FT.TYPE_UINT64,
FT.TYPE_SINT32,
FT.TYPE_SINT64,
FT.TYPE_FIXED32,
FT.TYPE_FIXED64,
):
return int(value)
if t in (FT.TYPE_FLOAT, FT.TYPE_DOUBLE):
return float(value)
if t == FT.TYPE_STRING:
return str(value)
if t == FT.TYPE_BYTES:
if isinstance(value, (bytes, bytearray)):
return bytes(value)
return str(value).encode("utf-8")
raise AdminError(
f"{field.name}: unsupported field type {t}. Use raw protobuf for this field."
)
def _walk_to_field(
root_msg: Any, path_segments: list[str]
) -> tuple[Any, pb_descriptor.FieldDescriptor]:
"""Walk `root_msg` by field names until the leaf; return (parent_msg, leaf_field_descriptor)."""
msg = root_msg
for i, name in enumerate(path_segments):
desc = msg.DESCRIPTOR
field = desc.fields_by_name.get(name)
if field is None:
trail = ".".join(path_segments[:i] or ["<root>"])
valid = [f.name for f in desc.fields]
raise AdminError(f"No field {name!r} in {trail}. Valid: {valid}")
is_last = i == len(path_segments) - 1
if is_last:
return msg, field
if field.type != pb_descriptor.FieldDescriptor.TYPE_MESSAGE:
raise AdminError(
f"{'.'.join(path_segments[:i+1])} is a scalar; cannot descend into it"
)
msg = getattr(msg, name)
# path_segments was empty
raise AdminError("Empty config path")
def set_config(path: str, value: Any, port: str | None = None) -> dict[str, Any]:
"""Set a single config field by dot-path and write it to the device.
Examples:
set_config("lora.region", "US")
set_config("lora.modem_preset", "LONG_FAST")
set_config("device.role", "ROUTER")
set_config("mqtt.enabled", True)
set_config("mqtt.address", "mqtt.example.com")
"""
segments = [s for s in path.split(".") if s]
if not segments:
raise AdminError("path cannot be empty")
section = segments[0]
with connect(port=port) as iface:
node = iface.localNode
container, parent_name = _section_container(node, section)
# Treat the section as the root; the rest of the path walks into it.
leaf_parent, field = _walk_to_field(container, segments[1:] or [])
# Use `is_repeated` (modern upb protobuf API) rather than the
# deprecated `label == LABEL_REPEATED` check — the C-extension
# FieldDescriptor in protobuf >= 5.x doesn't expose `.label` at
# all, and `is_repeated` is the supported replacement that works
# across both the pure-python and upb backends.
if field.is_repeated:
raise AdminError(
f"{path!r} is a repeated field; v1 only supports scalar sets. "
"Use the raw meshtastic CLI for now."
)
old_raw = getattr(leaf_parent, field.name)
coerced = _coerce_scalar(field, value)
try:
setattr(leaf_parent, field.name, coerced)
except (TypeError, ValueError) as exc:
raise AdminError(f"{path}: {exc}") from exc
node.writeConfig(section)
# Stringify enums for the response (so the caller can see the change in
# the same vocabulary they used to set it).
if field.type == pb_descriptor.FieldDescriptor.TYPE_ENUM:
try:
old_display = field.enum_type.values_by_number[old_raw].name
new_display = field.enum_type.values_by_number[coerced].name
except Exception:
old_display, new_display = old_raw, coerced
else:
old_display, new_display = old_raw, coerced
return {
"ok": True,
"path": path,
"section": section,
"parent": parent_name,
"old_value": old_display,
"new_value": new_display,
}
# ---------- channels -------------------------------------------------------
def get_channel_url(
include_all: bool = False, port: str | None = None
) -> dict[str, Any]:
with connect(port=port) as iface:
url = iface.localNode.getURL(includeAll=include_all)
return {"url": url}
def set_channel_url(url: str, port: str | None = None) -> dict[str, Any]:
with connect(port=port) as iface:
# setURL replaces the channel set from the URL's contents. It does not
# return a count; we infer by counting non-DISABLED channels after.
iface.localNode.setURL(url)
channels = iface.localNode.channels or []
active = sum(1 for c in channels if getattr(c, "role", 0) != 0)
return {"ok": True, "channels_imported": active}
# ---------- messaging ------------------------------------------------------
def send_text(
text: str,
to: str | int | None = None,
channel_index: int = 0,
want_ack: bool = False,
port: str | None = None,
) -> dict[str, Any]:
destination = to if to is not None else "^all"
with connect(port=port) as iface:
packet = iface.sendText(
text,
destinationId=destination,
wantAck=want_ack,
channelIndex=channel_index,
)
packet_id = getattr(packet, "id", None)
return {"ok": True, "packet_id": packet_id, "destination": destination}
# ---------- diagnostics ----------------------------------------------------
def set_debug_log_api(enabled: bool, port: str | None = None) -> dict[str, Any]:
"""Toggle `config.security.debug_log_api_enabled` on the local node.
When enabled, firmware emits log lines as protobuf `LogRecord` messages
over the StreamAPI instead of raw text. meshtastic-python surfaces them
on pubsub topic `meshtastic.log.line`, which flows through the SAME
SerialInterface our tests already hold open — no `pio device monitor`
needed, no port-contention with admin/info calls.
Firmware gate: `src/SerialConsole.cpp` (`usingProtobufs &&
config.security.debug_log_api_enabled`). Setting persists in NVS; it
survives reboot. `factory_reset(full=False)` clears it unless it's
re-applied after reset.
Previously-documented concurrency hazard (emitLogRecord sharing the
main packet-emission buffers) has been fixed — see `StreamAPI.h`
where the log path now owns dedicated `fromRadioScratchLog` /
`txBufLog` buffers, and `StreamAPI::emitTxBuffer` +
`StreamAPI::emitLogRecord` both serialize their `stream->write`
calls via `streamLock`. Leaving the flag on under traffic is safe.
"""
with connect(port=port) as iface:
sec = iface.localNode.localConfig.security
sec.debug_log_api_enabled = bool(enabled)
iface.localNode.writeConfig("security")
return {"ok": True, "debug_log_api_enabled": bool(enabled)}
# ---------- admin actions --------------------------------------------------
def reboot(
port: str | None = None, confirm: bool = False, seconds: int = 10
) -> dict[str, Any]:
_require_confirm(confirm, "reboot")
with connect(port=port) as iface:
iface.localNode.reboot(secs=seconds)
return {"ok": True, "rebooting_in_s": seconds}
def shutdown(
port: str | None = None, confirm: bool = False, seconds: int = 10
) -> dict[str, Any]:
_require_confirm(confirm, "shutdown")
with connect(port=port) as iface:
iface.localNode.shutdown(secs=seconds)
return {"ok": True, "shutting_down_in_s": seconds}
def send_input_event(
event_code: int | str,
kb_char: int = 0,
touch_x: int = 0,
touch_y: int = 0,
port: str | None = None,
) -> dict[str, Any]:
"""Inject an InputBroker event (button press / key / gesture) into the UI.
Wraps `AdminMessage.send_input_event` (handled in firmware at
src/modules/AdminModule.cpp::handleSendInputEvent). Local-only — no PKI
warmup needed since the admin message is addressed to `my_node_num`.
`event_code` accepts an int, a case-insensitive name
(`"RIGHT"` / `"input_broker_right"`), or an `InputEventCode`. The
firmware-side enum lives in src/input/InputBroker.h and is mirrored in
`meshtastic_mcp.input_events`.
"""
from meshtastic.protobuf import admin_pb2 # type: ignore[import-untyped]
from .input_events import coerce_event_code
code = coerce_event_code(event_code)
if not 0 <= kb_char <= 255:
raise ValueError(f"kb_char out of u8 range: {kb_char}")
if not 0 <= touch_x <= 65535:
raise ValueError(f"touch_x out of u16 range: {touch_x}")
if not 0 <= touch_y <= 65535:
raise ValueError(f"touch_y out of u16 range: {touch_y}")
with connect(port=port) as iface:
msg = admin_pb2.AdminMessage()
msg.send_input_event.event_code = code
msg.send_input_event.kb_char = kb_char
msg.send_input_event.touch_x = touch_x
msg.send_input_event.touch_y = touch_y
iface.localNode._sendAdmin(msg)
return {"ok": True, "event_code": code, "kb_char": kb_char}
def factory_reset(
port: str | None = None, confirm: bool = False, full: bool = False
) -> dict[str, Any]:
"""Tell the node to factory-reset its config.
Works around a meshtastic-python 2.7.8 bug: `Node.factoryReset(full=True)`
internally does `p.factory_reset_config = True` where the field is
int32. protobuf 5.x rejects bool→int assignment as a TypeError. We build
the AdminMessage directly with int values (1=non-full, 2=full) and call
`_sendAdmin` to sidestep the SDK bug entirely.
"""
_require_confirm(confirm, "factory_reset")
from meshtastic.protobuf import admin_pb2 # type: ignore[import-untyped]
with connect(port=port) as iface:
msg = admin_pb2.AdminMessage()
msg.factory_reset_config = 2 if full else 1
iface.localNode._sendAdmin(msg)
return {"ok": True, "full": full}
+159
View File
@@ -0,0 +1,159 @@
"""Board / PlatformIO env enumeration.
Parses `pio project config --json-output` — a nested list of
`[section_name, [[key, value], ...]]` pairs — into a dict keyed by env name,
extracting the `custom_meshtastic_*` metadata the firmware variants expose.
The parsed config is cached and invalidated when `platformio.ini`'s mtime
changes, so subsequent calls don't pay the 12s pio startup cost.
"""
from __future__ import annotations
import threading
from typing import Any
from . import config, pio
_CACHE_LOCK = threading.Lock()
_CACHE: dict[str, Any] = {"mtime": None, "envs": None}
def _parse_bool(value: Any) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.strip().lower() in ("true", "yes", "1", "on")
return bool(value)
def _parse_int(value: Any) -> int | None:
try:
return int(value)
except (TypeError, ValueError):
return None
def _parse_tags(value: Any) -> list[str]:
if value is None:
return []
if isinstance(value, list):
return [str(v).strip() for v in value if str(v).strip()]
return [t.strip() for t in str(value).replace(",", " ").split() if t.strip()]
def _env_record(env_name: str, items: list[list[Any]]) -> dict[str, Any]:
"""Build a normalized dict for one env section."""
d = dict(items)
return {
"env": env_name,
"architecture": d.get("custom_meshtastic_architecture"),
"hw_model": _parse_int(d.get("custom_meshtastic_hw_model")),
"hw_model_slug": d.get("custom_meshtastic_hw_model_slug"),
"display_name": d.get("custom_meshtastic_display_name"),
"actively_supported": _parse_bool(
d.get("custom_meshtastic_actively_supported")
),
"support_level": _parse_int(d.get("custom_meshtastic_support_level")),
"board_level": d.get("board_level"), # "pr", "extra", or None
"tags": _parse_tags(d.get("custom_meshtastic_tags")),
"images": _parse_tags(d.get("custom_meshtastic_images")),
"board": d.get("board"),
"upload_speed": _parse_int(d.get("upload_speed")),
"upload_protocol": d.get("upload_protocol"),
"monitor_speed": _parse_int(d.get("monitor_speed")),
"monitor_filters": d.get("monitor_filters") or [],
"_raw": d, # Full dict for get_board
}
def _load_all() -> dict[str, dict[str, Any]]:
"""Parse `pio project config` into `{env_name: record}`."""
raw = pio.run_json(["project", "config"], timeout=pio.TIMEOUT_PROJECT_CONFIG)
result: dict[str, dict[str, Any]] = {}
for section_name, items in raw:
if not isinstance(section_name, str) or not section_name.startswith("env:"):
continue
env_name = section_name.split(":", 1)[1]
result[env_name] = _env_record(env_name, items)
return result
def _get_cached() -> dict[str, dict[str, Any]]:
root = config.firmware_root()
platformio_ini = root / "platformio.ini"
try:
mtime = platformio_ini.stat().st_mtime
except FileNotFoundError:
mtime = None
with _CACHE_LOCK:
if _CACHE["envs"] is not None and _CACHE["mtime"] == mtime:
return _CACHE["envs"]
envs = _load_all()
_CACHE["envs"] = envs
_CACHE["mtime"] = mtime
return envs
def invalidate_cache() -> None:
with _CACHE_LOCK:
_CACHE["envs"] = None
_CACHE["mtime"] = None
def _public_record(rec: dict[str, Any]) -> dict[str, Any]:
"""Strip the `_raw` field for list outputs."""
return {k: v for k, v in rec.items() if not k.startswith("_")}
def list_boards(
architecture: str | None = None,
actively_supported_only: bool = False,
query: str | None = None,
board_level: str | None = None, # "release" | "pr" | "extra"
) -> list[dict[str, Any]]:
"""Enumerate PlatformIO envs with Meshtastic metadata.
Filters are cumulative (AND). `board_level="release"` means envs with no
explicit `board_level` set (the default release targets).
"""
envs = _get_cached()
q = query.lower().strip() if query else None
out = []
for rec in envs.values():
if architecture and rec.get("architecture") != architecture:
continue
if actively_supported_only and not rec.get("actively_supported"):
continue
if board_level is not None:
rec_level = rec.get("board_level")
if board_level == "release":
if rec_level not in (None, ""):
continue
elif rec_level != board_level:
continue
if q:
display = (rec.get("display_name") or "").lower()
env_name = rec.get("env", "").lower()
slug = (rec.get("hw_model_slug") or "").lower()
if q not in display and q not in env_name and q not in slug:
continue
out.append(_public_record(rec))
out.sort(key=lambda r: (r.get("architecture") or "", r.get("env")))
return out
def get_board(env: str) -> dict[str, Any]:
"""Full metadata for one env, including the raw pio config dict."""
envs = _get_cached()
rec = envs.get(env)
if rec is None:
raise KeyError(
f"Unknown env: {env!r}. Use list_boards() to see available envs."
)
public = _public_record(rec)
public["raw_config"] = rec["_raw"]
return public
+286
View File
@@ -0,0 +1,286 @@
"""Cross-platform USB-webcam capture for UI tests + the `capture_screen` tool.
Backends:
- `opencv` — cv2.VideoCapture (AVFoundation on macOS, V4L2 on Linux).
- `ffmpeg` — subprocess shelling out to the system `ffmpeg` binary. Slower
per frame, but zero Python deps beyond stdlib.
- `null` — no-op stub returning a 1×1 black PNG. Used when no camera is
configured; keeps code paths alive without forcing every operator to
hook up hardware.
Environment variables (read at `get_camera()` call time):
- `MESHTASTIC_UI_CAMERA_BACKEND` — one of `opencv` / `ffmpeg` / `null` /
`auto` (default). `auto` picks opencv if `cv2` imports, else ffmpeg if
`ffmpeg --version` resolves, else null.
- `MESHTASTIC_UI_CAMERA_DEVICE` — generic default (index or path).
- `MESHTASTIC_UI_CAMERA_DEVICE_<ROLE>` — per-role override, e.g.
`MESHTASTIC_UI_CAMERA_DEVICE_ESP32S3=0` for the OLED-bearing heltec-v3.
Role suffix is uppercased before lookup.
Dependencies land in the optional `[ui]` extra; imports are lazy so clients
without `opencv-python-headless` installed can still import this module.
"""
from __future__ import annotations
import io
import os
import shutil
import subprocess
import sys
import time
import warnings
from pathlib import Path
from typing import Protocol
class CameraError(RuntimeError):
"""Raised when a camera backend fails to initialize or capture."""
class CameraBackend(Protocol):
name: str
def capture(self) -> bytes:
"""Return one PNG-encoded frame."""
...
def close(self) -> None: ...
# ---------- OpenCV backend -------------------------------------------------
class OpenCVBackend:
name = "opencv"
def __init__(self, device: int | str, warmup_frames: int = 5) -> None:
try:
import cv2 # type: ignore[import-untyped] # noqa: PLC0415
except ImportError as exc:
raise CameraError(
"opencv backend requested but `cv2` is not installed. "
"Install the mcp-server [ui] extra: pip install -e '.[ui]'"
) from exc
self._cv2 = cv2
device_arg: int | str
if isinstance(device, str) and device.isdigit():
device_arg = int(device)
else:
device_arg = device
self._cap = cv2.VideoCapture(device_arg)
if not self._cap.isOpened():
raise CameraError(
f"cv2.VideoCapture({device_arg!r}) failed to open. "
"On macOS check TCC Camera permission; on Linux check /dev/video* and v4l2 access."
)
# Drop the first few frames — auto-exposure + white-balance settle.
for _ in range(warmup_frames):
self._cap.read()
# Detect a stuck black-frame camera early rather than silently
# producing all-black captures.
ok, frame = self._cap.read()
if not ok or frame is None:
self._cap.release()
raise CameraError(f"camera {device_arg!r} opened but returned no frames")
def capture(self) -> bytes:
cv2 = self._cv2
ok, frame = self._cap.read()
if not ok or frame is None:
raise CameraError("cv2.VideoCapture.read() returned no frame")
success, buf = cv2.imencode(".png", frame)
if not success:
raise CameraError("cv2.imencode('.png', ...) failed")
return bytes(buf)
def close(self) -> None:
try:
self._cap.release()
except Exception: # noqa: BLE001
pass
# ---------- ffmpeg subprocess backend --------------------------------------
class FfmpegBackend:
name = "ffmpeg"
def __init__(self, device: int | str) -> None:
if shutil.which("ffmpeg") is None:
raise CameraError("ffmpeg backend requested but `ffmpeg` is not on PATH")
self._device = str(device)
# Platform-specific -f flag:
# macOS → avfoundation (index like "0")
# Linux → v4l2 (device like "/dev/video0" or "0")
if sys.platform == "darwin":
self._input_format = "avfoundation"
self._input_spec = self._device # bare index for avfoundation
else:
self._input_format = "v4l2"
self._input_spec = (
self._device
if self._device.startswith("/dev/")
else f"/dev/video{self._device}"
)
def capture(self) -> bytes:
cmd = [
"ffmpeg",
"-hide_banner",
"-loglevel",
"error",
"-f",
self._input_format,
"-i",
self._input_spec,
"-frames:v",
"1",
"-f",
"image2pipe",
"-vcodec",
"png",
"-",
]
try:
out = subprocess.run(
cmd, capture_output=True, check=True, timeout=15 # noqa: S603
)
except subprocess.CalledProcessError as exc:
raise CameraError(
f"ffmpeg capture failed (rc={exc.returncode}): {exc.stderr.decode(errors='replace')[:200]}"
) from exc
except subprocess.TimeoutExpired as exc:
raise CameraError("ffmpeg capture timed out after 15s") from exc
return out.stdout
def close(self) -> None:
pass # stateless — each capture spawns a new process
# ---------- Null backend ---------------------------------------------------
# A tiny valid 1×1 transparent PNG so callers always get a decodable image.
_BLACK_1X1_PNG = bytes.fromhex(
"89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c489"
"0000000d49444154789c6300010000000500010d0a2db40000000049454e44ae426082"
)
class NullBackend:
name = "null"
def capture(self) -> bytes:
return _BLACK_1X1_PNG
def close(self) -> None:
pass
# ---------- Factory --------------------------------------------------------
def _resolve_device(role: str | None) -> str | None:
if role:
specific = os.environ.get(f"MESHTASTIC_UI_CAMERA_DEVICE_{role.upper()}")
if specific:
return specific
return os.environ.get("MESHTASTIC_UI_CAMERA_DEVICE")
def get_camera(role: str | None = None) -> CameraBackend:
"""Return a CameraBackend for the given device role (e.g. `"esp32s3"`).
Falls back to `NullBackend` if no camera is configured or the selected
backend fails to init — tests should treat captures as best-effort
evidence, not a blocker.
"""
backend = os.environ.get("MESHTASTIC_UI_CAMERA_BACKEND", "auto").lower()
device = _resolve_device(role)
if backend in ("null", "none") or device is None:
return NullBackend()
if backend == "auto":
# Prefer opencv if importable; fall back to ffmpeg; else null.
try:
import cv2 # type: ignore[import-untyped] # noqa: F401,PLC0415
backend = "opencv"
except ImportError:
backend = "ffmpeg" if shutil.which("ffmpeg") else "null"
if backend == "opencv":
try:
return OpenCVBackend(device)
except CameraError as exc:
warnings.warn(
f"camera backend {backend!r} failed to initialize for device "
f"{device!r}: {exc}; falling back to null backend",
RuntimeWarning,
stacklevel=2,
)
return NullBackend()
if backend == "ffmpeg":
try:
return FfmpegBackend(device)
except CameraError as exc:
warnings.warn(
f"camera backend {backend!r} failed to initialize for device "
f"{device!r}: {exc}; falling back to null backend",
RuntimeWarning,
stacklevel=2,
)
return NullBackend()
if backend == "null":
return NullBackend()
raise CameraError(f"unknown MESHTASTIC_UI_CAMERA_BACKEND: {backend!r}")
def save_capture(png_bytes: bytes, path: Path) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(png_bytes)
def capture_to_file(role: str | None, path: Path) -> dict[str, object]:
"""One-shot: open camera, capture, write PNG, close. Returns metadata."""
started = time.monotonic()
cam = get_camera(role)
try:
data = cam.capture()
finally:
cam.close()
save_capture(data, path)
return {
"backend": cam.name,
"path": str(path),
"bytes": len(data),
"elapsed_s": round(time.monotonic() - started, 3),
}
def _is_png(data: bytes) -> bool:
return data.startswith(b"\x89PNG\r\n\x1a\n")
# Exposed so callers can sanity-check a capture without a full PIL import.
__all__ = [
"CameraBackend",
"CameraError",
"FfmpegBackend",
"NullBackend",
"OpenCVBackend",
"capture_to_file",
"get_camera",
"save_capture",
]
# Keep `io` import used (pyflakes is picky) via a small guard used at import
# time to normalize stdin/stdout if a subclass ever needs it.
_ = io.BytesIO # noqa: SLF001
@@ -0,0 +1,6 @@
"""Command-line entry points that sit alongside the MCP server.
Modules here are loaded on-demand by `[project.scripts]` entries in
`pyproject.toml`. They are NOT imported by `meshtastic_mcp.server` or the
admin/info tool surface — the MCP server stays pure stdio JSON-RPC.
"""
@@ -0,0 +1,73 @@
"""Flash progress log tailer for ``meshtastic-mcp-test-tui``.
``pio.py`` / ``hw_tools.py`` tee subprocess output (``pio run -t upload``,
``esptool erase_flash``, ``nrfutil dfu``, etc.) to ``tests/flash.log``
line-by-line as it arrives — controlled by the ``MESHTASTIC_MCP_FLASH_LOG``
env var that ``run-tests.sh`` sets. The TUI tails that file so the operator
sees live flash progress in the pytest pane instead of 3 minutes of silence
during ``test_00_bake``.
Separate from ``_fwlog.py`` because that one parses JSONL, this one
streams plain text lines. Same daemon-thread + EOF-backoff structure.
"""
from __future__ import annotations
import pathlib
import threading
import time
from typing import Callable
class FlashLogTailer(threading.Thread):
"""Tail a plain-text log file, publish each stripped line via ``post``.
``post`` is invoked with a single ``str`` for every new line. Lines are
stripped of trailing newlines; empty lines after stripping are dropped.
The file may not exist yet when this thread starts — it's truncated by
``run-tests.sh`` at session start, but if the tailer races the shell,
we tolerate FileNotFoundError for up to ``wait_s`` seconds.
"""
def __init__(
self,
path: pathlib.Path,
post: Callable[[str], None],
stop: threading.Event,
*,
wait_s: float = 30.0,
) -> None:
super().__init__(daemon=True, name="flashlog-tail")
self._path = path
self._post = post
self._stop = stop
self._wait_s = wait_s
def run(self) -> None:
deadline = time.monotonic() + self._wait_s
while not self._path.is_file():
if self._stop.is_set() or time.monotonic() > deadline:
return
time.sleep(0.1)
try:
fh = self._path.open("r", encoding="utf-8", errors="replace")
except OSError:
return
try:
while not self._stop.is_set():
line = fh.readline()
if not line:
time.sleep(0.05)
continue
line = line.rstrip("\r\n")
if not line:
continue
try:
self._post(line)
except Exception:
# A post failure (e.g. closed app) is terminal for this
# thread but we still want to close the file handle.
return
finally:
fh.close()
@@ -0,0 +1,96 @@
"""Firmware log tail worker for ``meshtastic-mcp-test-tui``.
Complements v1's reportlog-tail worker. ``tests/conftest.py`` owns a
session-scoped autouse fixture (``_firmware_log_stream``) that mirrors
every ``meshtastic.log.line`` pubsub event to ``tests/fwlog.jsonl`` —
one JSON object per line:
{"ts": 1729100000.123, "port": "/dev/cu.usbmodem1101", "line": "..."}
The TUI tails that file from a worker thread; each new line becomes a
:class:`FirmwareLogLine` message posted to the App. Same pattern as the
reportlog tail worker — truncate on launch, tolerate missing file for
30 s, back off at EOF.
Kept in its own module so the (large) ``test_tui.py`` stays focused on
the Textual App shell.
"""
from __future__ import annotations
import json
import pathlib
import threading
import time
from typing import Any, Callable
class FirmwareLogTailer(threading.Thread):
"""Tail ``tests/fwlog.jsonl``, publish parsed records via ``post``.
``post`` is the App's ``post_message`` (or any callable that accepts a
single payload arg). We pass parsed dicts rather than constructing
Textual Message objects here — keeps this module free of the
textual dependency so it's unit-testable in a bare venv.
Parameters
----------
path:
Path to ``tests/fwlog.jsonl``. The file may not exist yet at
startup — pytest only creates it once the session fixture runs.
post:
Callable invoked with a dict ``{"ts", "port", "line"}`` for every
new line parsed from the file.
stop:
An event the App sets to signal shutdown.
wait_s:
How long to poll for the file's creation before giving up. Default
30 s; pytest collection on a cold cache can be slow.
"""
def __init__(
self,
path: pathlib.Path,
post: Callable[[dict[str, Any]], None],
stop: threading.Event,
*,
wait_s: float = 30.0,
) -> None:
super().__init__(daemon=True, name="fwlog-tail")
self._path = path
self._post = post
self._stop = stop
self._wait_s = wait_s
def run(self) -> None:
deadline = time.monotonic() + self._wait_s
while not self._path.is_file():
if self._stop.is_set() or time.monotonic() > deadline:
return
time.sleep(0.1)
try:
fh = self._path.open("r", encoding="utf-8")
except OSError:
return
try:
while not self._stop.is_set():
line = fh.readline()
if not line:
time.sleep(0.05)
continue
line = line.strip()
if not line:
continue
try:
record = json.loads(line)
except json.JSONDecodeError:
continue
# Defensive: require the three fields we rely on.
if not isinstance(record, dict):
continue
if "line" not in record:
continue
self._post(record)
finally:
fh.close()
@@ -0,0 +1,127 @@
"""Cross-run history for ``meshtastic-mcp-test-tui``.
Persists one JSON object per pytest run to
``mcp-server/tests/.history/runs.jsonl``. The TUI reads the last N
entries on launch to render a duration sparkline in the header — a
quick read on whether the suite is slowing down over time.
Schema (keep small; the file can grow for months):
{"run": 42, "ts": 1729100000.0, "duration_s": 387.2,
"passed": 52, "failed": 0, "skipped": 23, "exit_code": 0,
"seed": "mcp-user-host"}
"""
from __future__ import annotations
import json
import pathlib
import time
from dataclasses import asdict, dataclass
from typing import Iterable
# Sparkline glyphs, low → high. 8 levels is the Unicode convention.
_SPARK_BLOCKS = "▁▂▃▄▅▆▇█"
@dataclass
class RunRecord:
run: int
ts: float
duration_s: float
passed: int
failed: int
skipped: int
exit_code: int
seed: str
class HistoryStore:
"""Append-only JSONL store with bounded read.
Writes are fsynced after each append (the file is tiny; fsync cost
is negligible and protects against truncation on a crash).
"""
def __init__(self, path: pathlib.Path, *, keep_last: int = 50) -> None:
self._path = path
self._keep_last = keep_last
def append(self, record: RunRecord) -> None:
try:
self._path.parent.mkdir(parents=True, exist_ok=True)
with self._path.open("a", encoding="utf-8") as fh:
fh.write(json.dumps(asdict(record)) + "\n")
fh.flush()
except Exception:
# Non-fatal: history is cosmetic.
pass
def read_recent(self) -> list[RunRecord]:
"""Return the last ``keep_last`` records in chronological order."""
if not self._path.is_file():
return []
try:
lines = self._path.read_text(encoding="utf-8").splitlines()
except OSError:
return []
out: list[RunRecord] = []
# Parse tail-first so we don't waste work on a huge history.
for line in lines[-self._keep_last :]:
line = line.strip()
if not line:
continue
try:
raw = json.loads(line)
except json.JSONDecodeError:
continue
try:
out.append(RunRecord(**raw))
except TypeError:
# Schema drift; skip the record rather than crash.
continue
return out
def record_run(
self,
*,
run: int,
duration_s: float,
passed: int,
failed: int,
skipped: int,
exit_code: int,
seed: str,
) -> RunRecord:
rec = RunRecord(
run=run,
ts=time.time(),
duration_s=float(duration_s),
passed=int(passed),
failed=int(failed),
skipped=int(skipped),
exit_code=int(exit_code),
seed=seed,
)
self.append(rec)
return rec
def sparkline(values: Iterable[float], *, width: int = 20) -> str:
"""Render a Unicode block-character sparkline from the last ``width`` values.
Returns an empty string for empty input so the header handles
"no history yet" gracefully.
"""
buf = [v for v in values if v >= 0][-width:]
if not buf:
return ""
lo, hi = min(buf), max(buf)
if hi - lo < 1e-9:
return _SPARK_BLOCKS[len(_SPARK_BLOCKS) // 2] * len(buf)
n = len(_SPARK_BLOCKS) - 1
out = []
for v in buf:
idx = int(round((v - lo) / (hi - lo) * n))
out.append(_SPARK_BLOCKS[max(0, min(n, idx))])
return "".join(out)
@@ -0,0 +1,214 @@
"""Reproducer bundle builder for ``meshtastic-mcp-test-tui``.
When the operator presses ``x`` on a failed test leaf, we package the
minimum viable failure context into a tarball under
``mcp-server/tests/reproducers/``:
::
repro-<ts>-<short_nodeid>.tar.gz
├── README.md human-readable overview
├── test_report.json the failing TestReport event from reportlog
├── fwlog.jsonl firmware log filtered to the failure window
├── devices.json per-device device_info + lora config snapshot
└── env.json seed, run #, pytest version, platform, hostname
Separate module so the logic can be unit-tested without Textual. The
TUI glue is thin — one key binding calls :func:`build_reproducer_bundle`
with the focused test's state and shows the path in a modal.
"""
from __future__ import annotations
import io
import json
import pathlib
import platform
import re
import socket
import tarfile
import time
from dataclasses import dataclass
from typing import Any, Iterable
@dataclass
class ReproContext:
"""Everything :func:`build_reproducer_bundle` needs. Shaped to map
cleanly onto the state the TUI already tracks — no extra data
collection required at export time."""
nodeid: str
longrepr: str
sections: list[tuple[str, str]]
start_ts: float | None
stop_ts: float | None
seed: str
run_number: int
exit_code: int | None
fwlog_path: pathlib.Path
output_dir: pathlib.Path
extra_device_rows: list[dict[str, Any]] # [{role, port, info, ...}, ...]
def _short_nodeid(nodeid: str) -> str:
"""Collapse a pytest nodeid into a filename-safe slug (<= 60 chars)."""
# Drop the file path prefix; keep test name + parametrization.
tail = nodeid.split("::", 1)[-1] if "::" in nodeid else nodeid
slug = re.sub(r"[^A-Za-z0-9_.\-]", "_", tail)
return slug[:60].strip("_.-") or "test"
def _filtered_fwlog(
fwlog_path: pathlib.Path,
start_ts: float | None,
stop_ts: float | None,
*,
pad_s: float = 5.0,
) -> bytes:
"""Return fwlog.jsonl lines whose ``ts`` lies in [start-pad, stop+pad]."""
if not fwlog_path.is_file():
return b""
if start_ts is None or stop_ts is None:
# Without a time window, include the whole file — rare; happens
# when a test fails in setup before pytest emitted a start ts.
try:
return fwlog_path.read_bytes()
except OSError:
return b""
lo, hi = start_ts - pad_s, stop_ts + pad_s
out = io.BytesIO()
try:
with fwlog_path.open("r", encoding="utf-8") as fh:
for line in fh:
stripped = line.strip()
if not stripped:
continue
try:
record = json.loads(stripped)
except json.JSONDecodeError:
continue
ts = record.get("ts")
if not isinstance(ts, (int, float)):
continue
if lo <= ts <= hi:
out.write(line.encode("utf-8"))
except OSError:
return b""
return out.getvalue()
def _readme(ctx: ReproContext) -> str:
t = time.strftime("%Y-%m-%d %H:%M:%S %Z", time.localtime())
return f"""# Reproducer bundle
Exported by `meshtastic-mcp-test-tui` on {t}.
## Failing test
- **nodeid:** `{ctx.nodeid}`
- **seed:** `{ctx.seed}`
- **run #:** {ctx.run_number}
- **suite exit code (at export time):** {ctx.exit_code if ctx.exit_code is not None else "in progress"}
## Files in this archive
| File | Contents |
|---|---|
| `test_report.json` | The pytest-reportlog `TestReport` event for the failing test — includes `longrepr`, captured `sections` (stdout/stderr/log), `duration`, `location`, `keywords`. |
| `fwlog.jsonl` | Firmware log lines (from `meshtastic.log.line` pubsub) filtered to [start5s, stop+5s] around the test's run window. Each line is `{{ts, port, line}}`. |
| `devices.json` | Per-device snapshot at export time: `device_info` + `lora` config per detected role. |
| `env.json` | Python version, platform, hostname, seed, run number. |
## How to triage
1. Open `test_report.json` and read `longrepr` + `sections` — most failures explain themselves there.
2. If the failure is a mesh/telemetry assertion, `fwlog.jsonl` is where the answer usually lives. Grep for `Error=`, `NAK`, `PKI_UNKNOWN_PUBKEY`, `Skip send`, `Guru Meditation`, or the uptime timestamps around the assertion event.
3. Compare `devices.json` against the expected state (e.g. `num_nodes >= 2`, `primary_channel == "McpTest"`, `region == "US"`). If fields disagree with the seed-derived USERPREFS profile, the device probably wasn't baked with this session's profile.
## Reproducing locally
```bash
cd mcp-server
MESHTASTIC_MCP_SEED='{ctx.seed}' .venv/bin/pytest '{ctx.nodeid}' --tb=long -v
```
"""
def build_reproducer_bundle(ctx: ReproContext) -> pathlib.Path:
"""Build a tarball under ``ctx.output_dir`` and return its path.
Parent dirs are created as needed. Errors during optional sections
(devices, env) are swallowed — the bundle is still useful without
them; refusing to export because the device poller had a hiccup
would be worse than the export missing a file.
"""
ctx.output_dir.mkdir(parents=True, exist_ok=True)
ts = int(time.time())
slug = _short_nodeid(ctx.nodeid)
archive_path = ctx.output_dir / f"repro-{ts}-{slug}.tar.gz"
with tarfile.open(archive_path, "w:gz") as tar:
def _add(name: str, data: bytes) -> None:
info = tarfile.TarInfo(name=name)
info.size = len(data)
info.mtime = ts
tar.addfile(info, io.BytesIO(data))
# README
_add("README.md", _readme(ctx).encode("utf-8"))
# test_report.json — reconstruct from the fields the TUI stashes.
test_report = {
"nodeid": ctx.nodeid,
"outcome": "failed",
"longrepr": ctx.longrepr,
"sections": [list(s) for s in ctx.sections],
"start": ctx.start_ts,
"stop": ctx.stop_ts,
}
_add(
"test_report.json",
json.dumps(test_report, indent=2, default=str).encode("utf-8"),
)
# fwlog.jsonl (filtered)
_add("fwlog.jsonl", _filtered_fwlog(ctx.fwlog_path, ctx.start_ts, ctx.stop_ts))
# devices.json
try:
devices_payload = json.dumps(
ctx.extra_device_rows or [], indent=2, default=str
)
except Exception:
devices_payload = "[]"
_add("devices.json", devices_payload.encode("utf-8"))
# env.json
try:
from importlib.metadata import version as _pkg_version
pytest_version = _pkg_version("pytest")
except Exception:
pytest_version = "unknown"
env_payload = {
"seed": ctx.seed,
"run": ctx.run_number,
"exit_code": ctx.exit_code,
"export_ts": ts,
"python": platform.python_version(),
"pytest": pytest_version,
"platform": f"{platform.system()} {platform.release()} {platform.machine()}",
"hostname": socket.gethostname(),
}
_add("env.json", json.dumps(env_payload, indent=2).encode("utf-8"))
return archive_path
def iter_entries(archive_path: pathlib.Path) -> Iterable[str]:
"""Yield member names — used by callers that want to confirm the bundle shape."""
with tarfile.open(archive_path, "r:gz") as tar:
for m in tar.getmembers():
yield m.name
@@ -0,0 +1,83 @@
"""UI-capture transcript tailer for ``meshtastic-mcp-test-tui``.
Watches ``tests/ui_captures/<session_seed>/`` for new transcript lines
(one per ``frame_capture()`` call from the UI tier) and posts them to
the TUI. Enabled by ``MESHTASTIC_UI_TUI_CAMERA=1``.
Design mirrors ``_flashlog.py``:
- Daemon thread, cooperative stop via ``threading.Event``.
- Tolerates the captures directory not existing yet (UI tier hasn't run).
- Per-file seek state so we only forward genuinely-new lines.
"""
from __future__ import annotations
import pathlib
import threading
import time
from typing import Callable
class UiCaptureTailer(threading.Thread):
"""Recursively watch a captures root for new `transcript.md` lines.
Invokes ``post(test_id, line)`` for each new line, where ``test_id``
is derived from the path — the sanitized nodeid directory name.
"""
def __init__(
self,
root: pathlib.Path,
post: Callable[[str, str], None],
stop: threading.Event,
*,
poll_interval: float = 0.5,
) -> None:
super().__init__(daemon=True, name="uicap-tail")
self._root = root
self._post = post
self._stop = stop
self._poll_interval = poll_interval
# path → byte offset we've already read through
self._offsets: dict[pathlib.Path, int] = {}
def run(self) -> None:
while not self._stop.is_set():
try:
self._scan_once()
except Exception:
# Best-effort tailer — never bring down the TUI because a
# directory vanished mid-scan.
pass
time.sleep(self._poll_interval)
def _scan_once(self) -> None:
if not self._root.is_dir():
return
for transcript in self._root.rglob("transcript.md"):
test_id = transcript.parent.name
offset = self._offsets.get(transcript, 0)
try:
size = transcript.stat().st_size
except OSError:
continue
if size < offset:
# File truncated / rewritten — reset and re-emit.
offset = 0
if size == offset:
continue
try:
with transcript.open("rb") as fh:
fh.seek(offset)
chunk = fh.read(size - offset).decode("utf-8", errors="replace")
except OSError:
continue
for line in chunk.splitlines():
line = line.rstrip()
if not line or line.startswith("#"):
continue
try:
self._post(test_id, line)
except Exception:
return
self._offsets[transcript] = size
File diff suppressed because it is too large Load Diff
+148
View File
@@ -0,0 +1,148 @@
"""Resolves the firmware repo root and the binaries we invoke.
Everything that needs a path (the firmware root, `pio`, `esptool`, etc.) goes
through this module so the rest of the package never calls `shutil.which` or
parses environment variables directly.
"""
from __future__ import annotations
import os
import shutil
from pathlib import Path
from typing import Iterable
class ConfigError(RuntimeError):
"""Raised when a required path or binary cannot be resolved."""
def firmware_root() -> Path:
"""Resolve the root of the Meshtastic firmware repo.
Resolution order:
1. `MESHTASTIC_FIRMWARE_ROOT` env var.
2. Walk up from `cwd` looking for a directory with `platformio.ini`.
"""
env = os.environ.get("MESHTASTIC_FIRMWARE_ROOT")
if env:
root = Path(env).expanduser().resolve()
if not (root / "platformio.ini").is_file():
raise ConfigError(
f"MESHTASTIC_FIRMWARE_ROOT={env!r} does not contain platformio.ini"
)
return root
cur = Path.cwd().resolve()
for candidate in (cur, *cur.parents):
if (candidate / "platformio.ini").is_file():
return candidate
raise ConfigError(
"Could not locate Meshtastic firmware root. Set MESHTASTIC_FIRMWARE_ROOT "
"to the directory containing platformio.ini."
)
def _first_existing(paths: Iterable[Path]) -> Path | None:
for p in paths:
if p and p.is_file() and os.access(p, os.X_OK):
return p
return None
def pio_bin() -> Path:
"""Resolve the `pio` binary.
Order: MESHTASTIC_PIO_BIN → ~/.platformio/penv/bin/pio (PlatformIO keeps
this one current) → `pio` on PATH → `platformio` on PATH.
"""
env = os.environ.get("MESHTASTIC_PIO_BIN")
if env:
p = Path(env).expanduser()
if p.is_file() and os.access(p, os.X_OK):
return p
raise ConfigError(f"MESHTASTIC_PIO_BIN={env!r} is not an executable file")
penv = Path.home() / ".platformio" / "penv" / "bin" / "pio"
if penv.is_file() and os.access(penv, os.X_OK):
return penv
for name in ("pio", "platformio"):
w = shutil.which(name)
if w:
return Path(w)
raise ConfigError(
"Could not find `pio`. Install PlatformIO (https://platformio.org/install/cli) "
"or set MESHTASTIC_PIO_BIN."
)
def _hw_tool(env_var: str, names: tuple[str, ...], install_hint: str) -> Path:
"""Shared resolver for esptool / nrfutil / picotool.
Prefers the firmware repo's own `.venv/bin/<name>` (esptool lives there),
then PATH.
"""
env = os.environ.get(env_var)
if env:
p = Path(env).expanduser()
if p.is_file() and os.access(p, os.X_OK):
return p
raise ConfigError(f"{env_var}={env!r} is not an executable file")
try:
venv_bin = firmware_root() / ".venv" / "bin"
except ConfigError:
venv_bin = None
for name in names:
if venv_bin is not None:
p = venv_bin / name
if p.is_file() and os.access(p, os.X_OK):
return p
for name in names:
w = shutil.which(name)
if w:
return Path(w)
raise ConfigError(
f"Could not find `{names[0]}`. {install_hint} "
f"Or set {env_var} to an absolute path."
)
def esptool_bin() -> Path:
return _hw_tool(
"MESHTASTIC_ESPTOOL_BIN",
("esptool", "esptool.py"),
"Install via `pip install esptool`.",
)
def nrfutil_bin() -> Path:
return _hw_tool(
"MESHTASTIC_NRFUTIL_BIN",
("nrfutil", "adafruit-nrfutil"),
"Install via `pip install adafruit-nrfutil` or download Nordic nRF Util.",
)
def picotool_bin() -> Path:
return _hw_tool(
"MESHTASTIC_PICOTOOL_BIN",
("picotool",),
"Install via `brew install picotool` or build from https://github.com/raspberrypi/picotool.",
)
def uhubctl_bin() -> Path:
return _hw_tool(
"MESHTASTIC_UHUBCTL_BIN",
("uhubctl",),
"Install via `brew install uhubctl` (macOS) or `apt install uhubctl` "
"(Debian/Ubuntu). On Linux without the udev rules, or on older macOS "
"with certain hubs, you may need to run via `sudo`: "
"https://github.com/mvp/uhubctl#linux-usb-permissions",
)
+229
View File
@@ -0,0 +1,229 @@
"""Context manager for meshtastic interface connections (serial + TCP).
Every info/admin tool goes through `connect(port)` so we have a single place
that:
- auto-selects the port when one likely_meshtastic device is present,
- fails fast if a serial_session is already holding the port,
- guarantees `.close()` is called, even on exception.
Two transports:
- Serial: USB-attached firmware on `/dev/cu.*` / `/dev/ttyUSB*` / `COM*`.
- TCP: a `meshtasticd` daemon (e.g. the native macOS / Linux Portduino
headless build) addressed as `tcp://host[:port]` (default port 4403).
Surfaced by `devices.list_devices()` when `MESHTASTIC_MCP_TCP_HOST` is
set, so `resolve_port(None)` auto-selects it like a USB candidate.
Both `SerialInterface` and `TCPInterface` block on construction waiting for
the node database; that's fine for v1 since every tool is a short-lived
request.
"""
from __future__ import annotations
from contextlib import contextmanager
from typing import Iterator
from . import devices, registry
DEFAULT_TCP_PORT = 4403
TCP_SCHEME = "tcp://"
TCP_HOST_ENV = "MESHTASTIC_MCP_TCP_HOST"
class ConnectionError(RuntimeError):
pass
def is_tcp_port(port: str | None) -> bool:
return bool(port) and port.startswith(TCP_SCHEME)
def parse_tcp_port(port: str) -> tuple[str, int]:
"""Parse `tcp://host[:port]` → (host, port). Defaults to 4403.
Validates host shape (non-empty, no path separators) and port range
(1..65535). Raises `ConnectionError` on malformed input — never lets
a raw `ValueError` bubble up to a tool surface.
"""
if not port.startswith(TCP_SCHEME):
raise ConnectionError(
f"Invalid TCP endpoint {port!r}: expected '{TCP_SCHEME}host[:port]'."
)
rest = port[len(TCP_SCHEME) :]
if ":" in rest:
host, port_str = rest.rsplit(":", 1)
try:
tcp_port = int(port_str)
except ValueError as e:
raise ConnectionError(
f"Invalid TCP endpoint {port!r}: port {port_str!r} is not an integer."
) from e
else:
host, tcp_port = rest, DEFAULT_TCP_PORT
if not host:
raise ConnectionError(f"Invalid TCP endpoint {port!r}: empty host.")
if any(c in host for c in ("/", "\\")):
raise ConnectionError(
f"Invalid TCP endpoint {port!r}: host {host!r} contains a path "
"separator. TCP hostnames cannot contain '/' or '\\' — did you "
"pass a serial port path or a Windows drive path by mistake?"
)
if not (1 <= tcp_port <= 65535):
raise ConnectionError(
f"Invalid TCP endpoint {port!r}: port {tcp_port} out of range "
"(must be 1..65535)."
)
return host, tcp_port
def normalize_tcp_endpoint(endpoint: str) -> str:
r"""Normalize `host`, `host:port`, or `tcp://host[:port]` → canonical
`tcp://host:port` form. One place that owns the lock-key shape.
Defers all validation to `parse_tcp_port`, so path-like inputs
(`/dev/cu.foo`, `C:\Windows\…`), empty hosts, non-integer ports,
and out-of-range ports raise `ConnectionError` here too.
"""
if endpoint.startswith(TCP_SCHEME):
canonical = endpoint
elif ":" in endpoint:
canonical = f"{TCP_SCHEME}{endpoint}"
else:
canonical = f"{TCP_SCHEME}{endpoint}:{DEFAULT_TCP_PORT}"
host, port = parse_tcp_port(canonical)
return f"{TCP_SCHEME}{host}:{port}"
def reject_if_tcp(port: str | None, tool_name: str) -> None:
"""Raise if `port` is a TCP endpoint — for tools that need real USB
hardware (flash, bootloader, vendor escape hatches, serial monitor).
Only checks the explicit arg; auto-selection via env var is the caller's
responsibility to handle if it matters.
"""
if is_tcp_port(port):
raise ConnectionError(
f"{tool_name} is not applicable to TCP/native nodes ({port}). "
"This tool requires USB-attached hardware."
)
def resolve_port(port: str | None) -> str:
"""Pick a port: explicit > sole likely_meshtastic candidate > error.
A `tcp://` string passes through (after canonicalization). When `port`
is None and no USB candidates are present, `MESHTASTIC_MCP_TCP_HOST`
is consulted via `devices.list_devices()`.
"""
if port:
if is_tcp_port(port):
return normalize_tcp_endpoint(port)
return port
candidates = [d for d in devices.list_devices() if d["likely_meshtastic"]]
if not candidates:
raise ConnectionError(
"No Meshtastic devices detected. Plug one in, set "
f"{TCP_HOST_ENV}=<host[:port]> for a meshtasticd daemon, "
"or pass `port` explicitly. Run `list_devices` with "
"include_unknown=True to see all serial ports."
)
if len(candidates) > 1:
ports = ", ".join(c["port"] for c in candidates)
raise ConnectionError(
f"Multiple Meshtastic devices detected ({ports}). "
"Specify `port` explicitly."
)
return candidates[0]["port"]
@contextmanager
def connect(port: str | None = None, timeout_s: float = 8.0) -> Iterator:
"""Open a meshtastic interface (serial or TCP) and always close it.
For serial: raises `ConnectionError` immediately if another serial
session holds the port (a `pio device monitor` in `serial_sessions/`).
For TCP: no exclusive-access requirement, so the serial-session check
is skipped — but the `port_lock` still serializes parallel `connect()`
calls to the same daemon endpoint.
`timeout_s` is plumbed through to both `SerialInterface(timeout=...)`
and `TCPInterface(timeout=...)`. The meshtastic library uses the value
as the reply-wait deadline for `localNode.waitForConfig()` during
construction and for any subsequent admin RPC. `int()`-converted at
the boundary because the upstream API expects whole seconds.
"""
resolved = resolve_port(port)
timeout = int(timeout_s)
if is_tcp_port(resolved):
from meshtastic.tcp_interface import (
TCPInterface, # type: ignore[import-untyped]
)
host, tcp_port = parse_tcp_port(resolved)
lock = registry.port_lock(resolved)
if not lock.acquire(blocking=False):
raise ConnectionError(
f"TCP endpoint {resolved} is busy — another device operation "
"is in flight. Retry shortly."
)
iface = None
try:
iface = TCPInterface(
hostname=host,
portNumber=tcp_port,
connectNow=True,
noProto=False,
timeout=timeout,
)
yield iface
finally:
if iface is not None:
try:
iface.close()
except Exception:
pass
try:
lock.release()
except RuntimeError:
pass
return
from meshtastic.serial_interface import (
SerialInterface, # type: ignore[import-untyped]
)
active = registry.active_session_for_port(resolved)
if active is not None:
raise ConnectionError(
f"Port {resolved} is held by serial session {active.id}. "
"Call `serial_close` first."
)
lock = registry.port_lock(resolved)
if not lock.acquire(blocking=False):
raise ConnectionError(
f"Port {resolved} is busy — another device operation is in flight. "
"Retry shortly."
)
iface = None
try:
iface = SerialInterface(
devPath=resolved,
connectNow=True,
noProto=False,
timeout=timeout,
)
yield iface
finally:
if iface is not None:
try:
iface.close()
except Exception:
pass
try:
lock.release()
except RuntimeError:
pass
+135
View File
@@ -0,0 +1,135 @@
"""USB/serial + TCP device discovery.
Combines the canonical `meshtastic.util.findPorts()` allowlist/blocklist with
the richer metadata (`serial.tools.list_ports.comports()`) so callers see
VID/PID, descriptions, and manufacturer strings alongside the "is this likely
a Meshtastic device" signal.
If `MESHTASTIC_MCP_TCP_HOST=<host[:port]>` is set, a synthetic entry for the
`meshtasticd` daemon at that endpoint is prepended to the result, so
`resolve_port(None)` auto-selects it like a USB candidate.
"""
from __future__ import annotations
import os
from typing import Any
from serial.tools import list_ports
def _to_hex(value: int | None) -> str | None:
if value is None:
return None
return f"0x{value:04x}"
def _tcp_endpoint_from_env() -> dict[str, Any] | None:
"""Synthesize a TCP device entry from MESHTASTIC_MCP_TCP_HOST, if set.
If the env var is malformed (non-integer port, path-like host, etc.),
return an entry with `likely_meshtastic=False` and the parser error in
the description, rather than raising — `list_devices` is the diagnostic
tool a user reaches for when their env var isn't working, so it must
not crash on misconfiguration.
"""
host = os.environ.get("MESHTASTIC_MCP_TCP_HOST")
if not host:
return None
# Lazy import to avoid a circular dependency (connection imports devices).
from . import connection
try:
port = connection.normalize_tcp_endpoint(host)
description = "meshtasticd (TCP)"
likely = True
except connection.ConnectionError as e:
# Surface the raw env-var value plus the parser's reason so the
# user can see exactly what they set and why it was rejected.
# Don't double the scheme if the user already prefixed `tcp://`.
port = host if host.startswith(connection.TCP_SCHEME) else f"tcp://{host}"
description = f"meshtasticd (TCP) — invalid MESHTASTIC_MCP_TCP_HOST: {e}"
likely = False
return {
"port": port,
"vid": None,
"pid": None,
"description": description,
"manufacturer": None,
"product": None,
"serial_number": None,
"likely_meshtastic": likely,
"blacklisted": False,
}
def list_devices(include_unknown: bool = False) -> list[dict[str, Any]]:
"""Return enriched info for serial ports, flagging Meshtastic candidates.
`likely_meshtastic` is True when the port's USB VID matches the Meshtastic
allowlist (`0x239a` Adafruit/RAK, `0x303a` Espressif). When no allowlisted
ports are present, ports whose VID is NOT in the blocklist (J-Link, ST-LINK,
PPK2, etc.) are surfaced as `likely_meshtastic=False` candidates.
With `include_unknown=False` (default), we return only ports that are
plausibly Meshtastic. With `include_unknown=True`, every serial port the
OS knows about is returned (useful for debugging "why isn't my board
detected").
"""
# Import lazily so the module loads even without the `meshtastic` package
# (useful for introspection / schema generation).
from meshtastic import util as mt_util # type: ignore[import-untyped]
meshtastic_ports: set[str] = set(mt_util.findPorts(eliminate_duplicates=True))
whitelist = getattr(mt_util, "whitelistVids", {})
blacklist = getattr(mt_util, "blacklistVids", {})
results: list[dict[str, Any]] = []
for info in list_ports.comports():
port_path = info.device
vid = info.vid
in_whitelist = vid is not None and vid in whitelist
in_blacklist = vid is not None and vid in blacklist
likely = port_path in meshtastic_ports and in_whitelist
# If no allowlisted ports were found, findPorts falls back to
# everything-not-in-blacklist; treat those as plausible candidates
# but not "likely".
fallback_candidate = port_path in meshtastic_ports and not in_whitelist
if not likely and not fallback_candidate and not include_unknown:
continue
results.append(
{
"port": port_path,
"vid": _to_hex(vid),
"pid": _to_hex(info.pid),
"description": info.description or None,
"manufacturer": info.manufacturer or None,
"product": info.product or None,
"serial_number": info.serial_number or None,
"likely_meshtastic": likely,
"blacklisted": in_blacklist,
}
)
# Append the TCP endpoint (if env var set) and sort everything together.
tcp_entry = _tcp_endpoint_from_env()
if tcp_entry is not None:
results.append(tcp_entry)
# Stable ordering: likely_meshtastic first; within rank, TCP wins over
# USB (explicit env-var configuration takes precedence over USB
# enumeration); then by port path. A misconfigured TCP entry has
# likely_meshtastic=False and lands among the other ignored entries —
# it does NOT pre-empt real USB devices at the top of the list.
results.sort(
key=lambda r: (
not r["likely_meshtastic"],
not r["port"].startswith("tcp://"),
r["port"],
)
)
return results
+382
View File
@@ -0,0 +1,382 @@
"""Fake NodeDB fixture push — Portduino file copy + hardware XModem upload.
The fixture pipeline is two-stage:
1. `bin/gen-fake-nodedb-seed.py` produces a deterministic JSONL describing N
fake-but-realistic peers. Committed under `test/fixtures/nodedb/`.
2. `bin/seed-json-to-proto.py` compiles JSONL → binary v25 NodeDatabase
protobuf with fresh wall-clock timestamps.
This module exposes `push_fake_nodedb(...)`, the MCP tool that:
- target="portduino": compiles the JSONL into the device's prefs dir on
the local filesystem (`~/.portduino/<config>/prefs/nodes.proto`).
- target="hardware": compiles to a temp file, then streams it over the
XModem protocol (via the meshtastic SerialInterface/BLEInterface +
`meshtastic.xmodempacket` pubsub topic) to `/prefs/nodes.proto` on the
device. Triggers a reboot so the firmware loads the new state on next
boot.
XModem wire details (mirrors firmware impl at src/xmodem.cpp:115-260):
* 128-byte chunks; final chunk padded to 128 B with 0x1A (SUB) bytes.
* CRC16-CCITT (poly 0x1021, init 0x0000).
* SOH/seq=0 carries the destination filename in `buffer.bytes`. ACK if
`FSCom.open(filename, FILE_O_WRITE)` succeeds; NAK otherwise.
* SOH/seq≥1 carries a 128-byte chunk. ACK = advance; NAK = retransmit.
* EOT after the last chunk flushes + closes the file on-device.
Hardware push requires `confirm=True` (mirrors factory_reset / erase_and_flash
in the .github/copilot-instructions.md "never do these without asking" list).
"""
from __future__ import annotations
import dataclasses
import hashlib
import pathlib
import queue
import shutil
import subprocess
import sys
import tempfile
import time
from typing import Any, Literal
from .connection import connect, is_tcp_port
# Resolve repo root so the tool works regardless of mcp-server cwd.
_REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
_SEED_DIR = _REPO_ROOT / "test" / "fixtures" / "nodedb"
_COMPILE_SCRIPT = _REPO_ROOT / "bin" / "seed-json-to-proto.py"
_DEFAULT_NODES_FILENAME = "/prefs/nodes.proto"
_XMODEM_CHUNK = 128
_XMODEM_SUB = 0x1A
_ACK_TIMEOUT_INIT_S = 5.0
_ACK_TIMEOUT_CHUNK_S = 2.0
_MAX_CHUNK_RETRIES = 5
_VALID_SIZES = (250, 500, 1000, 2000)
class FixtureError(RuntimeError):
"""Raised for any fixture-push failure (compile, transport, ack timeout, …)."""
# ---------------------------------------------------------------------------
# CRC16-CCITT (poly 0x1021, init 0x0000). Matches the firmware's `crc16_ccitt`.
# Hand-rolled to avoid the optional `crcmod` dep.
# ---------------------------------------------------------------------------
def _crc16_ccitt(data: bytes, *, init: int = 0x0000) -> int:
crc = init
for b in data:
crc ^= b << 8
for _ in range(8):
if crc & 0x8000:
crc = ((crc << 1) ^ 0x1021) & 0xFFFF
else:
crc = (crc << 1) & 0xFFFF
return crc
# ---------------------------------------------------------------------------
# Compile step — shells out to bin/seed-json-to-proto.py so the MCP module
# doesn't have to duplicate the proto-encoding logic.
# ---------------------------------------------------------------------------
def _compile_proto(jsonl_path: pathlib.Path, out_path: pathlib.Path) -> None:
if not _COMPILE_SCRIPT.is_file():
raise FixtureError(f"compile script missing at {_COMPILE_SCRIPT}")
cmd = [
sys.executable,
str(_COMPILE_SCRIPT),
"--in",
str(jsonl_path),
"--out",
str(out_path),
]
try:
subprocess.run(cmd, check=True, capture_output=True, text=True)
except subprocess.CalledProcessError as exc:
raise FixtureError(
f"seed-json-to-proto.py failed (exit {exc.returncode}):\n"
f" stdout: {exc.stdout}\n stderr: {exc.stderr}"
) from exc
def _resolve_seed_jsonl(size: int, custom: str | None) -> pathlib.Path:
if custom is not None:
p = pathlib.Path(custom).expanduser().resolve()
if not p.is_file():
raise FixtureError(f"custom_seed_jsonl not found: {p}")
return p
p = _SEED_DIR / f"seed_v25_{size:04d}.jsonl"
if not p.is_file():
raise FixtureError(
f"missing committed seed at {p}. "
f"Run `./bin/regen-fake-nodedbs.sh` to generate it."
)
return p
# ---------------------------------------------------------------------------
# Portduino push — file copy into ~/.portduino/<config>/prefs/
# ---------------------------------------------------------------------------
def _portduino_prefs_dir(config_name: str) -> pathlib.Path:
home = pathlib.Path.home()
return home / ".portduino" / config_name / "prefs"
def _push_portduino(
size: int,
jsonl: pathlib.Path,
portduino_config: str,
backup_existing: bool,
) -> dict[str, Any]:
prefs = _portduino_prefs_dir(portduino_config)
prefs.mkdir(parents=True, exist_ok=True)
target = prefs / "nodes.proto"
backed_up_to: str | None = None
if backup_existing and target.is_file():
ts = int(time.time())
backup = prefs / f"nodes.proto.bak.{ts}"
shutil.move(str(target), str(backup))
backed_up_to = str(backup)
_compile_proto(jsonl, target)
raw = target.read_bytes()
return {
"transport": "portduino",
"path": str(target),
"bytes": len(raw),
"sha256": hashlib.sha256(raw).hexdigest(),
"jsonl_source": str(jsonl),
"backed_up_to": backed_up_to,
}
# ---------------------------------------------------------------------------
# Hardware push — XModem over BLE/serial via the meshtastic Python interface.
# ---------------------------------------------------------------------------
@dataclasses.dataclass
class _AckEvent:
control: int
seq: int
def _wait_for_response(q: "queue.Queue[_AckEvent]", timeout_s: float) -> _AckEvent:
try:
return q.get(timeout=timeout_s)
except queue.Empty as exc:
raise FixtureError(
f"XModem response timeout after {timeout_s:.1f}s — device not responding"
) from exc
def _push_hardware(
size: int,
jsonl: pathlib.Path,
port: str | None,
reboot_after: bool,
) -> dict[str, Any]:
# Lazy imports so the module loads even when the meshtastic deps aren't
# available (e.g. CI in a Python env without the package installed).
try:
from meshtastic.protobuf import mesh_pb2, xmodem_pb2
from pubsub import pub
except ImportError as exc: # pragma: no cover — dep missing
raise FixtureError(
f"hardware push requires the meshtastic + pypubsub packages: {exc}"
) from exc
if is_tcp_port(port):
raise FixtureError(
"hardware push over TCP/portduino is not supported — use "
"target='portduino' to drop the fixture directly into the prefs dir."
)
# Compile the fixture to a temp file with fresh timestamps.
with tempfile.NamedTemporaryFile(suffix=".proto", delete=False) as tf:
proto_path = pathlib.Path(tf.name)
try:
_compile_proto(jsonl, proto_path)
payload = proto_path.read_bytes()
finally:
proto_path.unlink(missing_ok=True)
sha256 = hashlib.sha256(payload).hexdigest()
total_bytes = len(payload)
# Subscribe to XModem responses BEFORE we open the interface, so we don't
# race the first ACK that arrives during the SOH/seq=0 handshake.
#
# NB: the signature MUST declare every kwarg pypubsub will see for this
# topic, or pubsub locks the topic spec to a smaller set (whichever
# subscribe arrives first) and then *rejects* the meshtastic library's
# publish call with `SenderUnknownMsgDataError: unknown ... interface`.
# The meshtastic lib publishes both `packet=` and `interface=`
# (mesh_interface.py:1389-1395), so both must appear here.
response_q: "queue.Queue[_AckEvent]" = queue.Queue()
def _on_xmodem(packet: Any = None, interface: Any = None, **_kw: Any) -> None:
if packet is None:
return
response_q.put(_AckEvent(control=int(packet.control), seq=int(packet.seq)))
pub.subscribe(_on_xmodem, "meshtastic.xmodempacket")
chunks_sent = 0
retried = 0
rebooted = False
XMC = xmodem_pb2.XModem.Control
try:
with connect(port=port) as iface:
# 1) Send the filename (SOH, seq=0).
init_pkt = xmodem_pb2.XModem(
control=XMC.Value("SOH"),
seq=0,
buffer=_DEFAULT_NODES_FILENAME.encode("utf-8"),
)
iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=init_pkt))
ack = _wait_for_response(response_q, _ACK_TIMEOUT_INIT_S)
if ack.control != XMC.Value("ACK"):
raise FixtureError(
f"device refused filename {_DEFAULT_NODES_FILENAME!r} "
f"(got control={ack.control}, expected ACK). "
f"Filesystem full or permissions issue?"
)
# 2) Stream the payload in 128 B chunks.
for offset in range(0, total_bytes, _XMODEM_CHUNK):
chunk = payload[offset : offset + _XMODEM_CHUNK]
if len(chunk) < _XMODEM_CHUNK:
# Pad final chunk to 128 B with SUB. The trailing 0x1A bytes
# become part of the file on-device, but nanopb ignores
# bytes past the end of the top-level message.
chunk = chunk + bytes([_XMODEM_SUB] * (_XMODEM_CHUNK - len(chunk)))
seq = ((offset // _XMODEM_CHUNK) + 1) % 256
# Retry loop on NAK / timeout.
attempts = 0
while True:
pkt = xmodem_pb2.XModem(
control=XMC.Value("SOH"),
seq=seq,
buffer=chunk,
crc16=_crc16_ccitt(chunk),
)
iface._sendToRadio(mesh_pb2.ToRadio(xmodemPacket=pkt))
ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
if ack.control == XMC.Value("ACK"):
chunks_sent += 1
break
if ack.control == XMC.Value("NAK"):
attempts += 1
retried += 1
if attempts >= _MAX_CHUNK_RETRIES:
# Abort: send CAN so the firmware removes the half-
# written file via FSCom.remove(filename).
iface._sendToRadio(
mesh_pb2.ToRadio(
xmodemPacket=xmodem_pb2.XModem(
control=XMC.Value("CAN")
)
)
)
raise FixtureError(
f"chunk seq={seq} NAK'd {attempts} times; "
f"aborted transfer (file removed on-device)."
)
continue # retry the same chunk
raise FixtureError(
f"unexpected XModem control={ack.control} on seq={seq}"
)
# 3) Tell the device we're done.
iface._sendToRadio(
mesh_pb2.ToRadio(
xmodemPacket=xmodem_pb2.XModem(control=XMC.Value("EOT"))
)
)
ack = _wait_for_response(response_q, _ACK_TIMEOUT_CHUNK_S)
if ack.control != XMC.Value("ACK"):
raise FixtureError(f"EOT not ACKed (got control={ack.control})")
# 4) Reboot so loadFromDisk picks up the new file.
if reboot_after:
iface.localNode.reboot(secs=1)
rebooted = True
finally:
try:
pub.unsubscribe(_on_xmodem, "meshtastic.xmodempacket")
except Exception:
pass
return {
"transport": "hardware",
"port": port,
"filename_on_device": _DEFAULT_NODES_FILENAME,
"bytes": total_bytes,
"chunks_sent": chunks_sent,
"retried": retried,
"sha256": sha256,
"jsonl_source": str(jsonl),
"rebooted": rebooted,
}
# ---------------------------------------------------------------------------
# Public entry point — registered as an MCP tool in server.py.
# ---------------------------------------------------------------------------
def push_fake_nodedb(
size: int,
target: Literal["portduino", "hardware"] = "portduino",
*,
port: str | None = None,
portduino_config: str = "default",
backup_existing: bool = True,
confirm: bool = False,
reboot_after: bool = True,
custom_seed_jsonl: str | None = None,
) -> dict[str, Any]:
"""Compile a fresh-timestamp NodeDatabase fixture and push it to a device.
Args:
size: 250, 500, 1000, or 2000 — selects which committed seed JSONL to use.
target: "portduino" (file copy to ~/.portduino/<config>/prefs/) or
"hardware" (XModem upload to /prefs/nodes.proto + reboot).
port: required for target="hardware". Serial path (e.g. /dev/cu.usbmodemXXXX)
or BLE identifier. TCP endpoints are rejected — use target="portduino"
instead.
portduino_config: which Portduino instance dir under ~/.portduino/. Default "default".
backup_existing: portduino only. Move nodes.proto -> nodes.proto.bak.<ts>
if present, so you can roll back.
confirm: required True for target="hardware" (writes flash + reboots).
reboot_after: hardware only. If True, send a 1-second reboot after the
final ACK so loadFromDisk picks up the new file at next boot.
custom_seed_jsonl: override the committed JSONL. Use to push a hand-edited
test scenario.
Returns:
dict with transport, bytes, sha256, etc. — depends on target.
"""
if size not in _VALID_SIZES:
raise FixtureError(
f"size must be one of {_VALID_SIZES}; got {size!r}. "
f"Add a new committed seed if you need a different cardinality."
)
jsonl = _resolve_seed_jsonl(size, custom_seed_jsonl)
if target == "portduino":
return _push_portduino(size, jsonl, portduino_config, backup_existing)
if target == "hardware":
if not confirm:
raise FixtureError(
"hardware push writes flash and triggers a reboot — pass confirm=True."
)
if not port:
raise FixtureError(
"target='hardware' requires a port (e.g. /dev/cu.usbmodemXXXX)."
)
return _push_hardware(size, jsonl, port, reboot_after)
raise FixtureError(f"unknown target {target!r}; expected 'portduino' or 'hardware'")
+507
View File
@@ -0,0 +1,507 @@
"""Build, clean, flash, and bootloader-entry operations.
Design: pio is the preferred path for every architecture via `flash()`. For
ESP32 factory flashes we shell out to `bin/device-install.sh` (which knows
about partition offsets and the OTA/littlefs partitions); for ESP32 OTA
updates we use `bin/device-update.sh`. Both scripts require the build
artifacts to exist, so these tools build first if needed.
"""
from __future__ import annotations
import subprocess
import threading
import time
from pathlib import Path
from typing import Any
import serial
from . import boards, config, connection, devices, pio, userprefs
# Meshtastic variants use both `esp32s3` and `esp32-s3` style names across
# variants/*/platformio.ini (no consistency enforced). Accept both spellings.
ESP32_ARCHES = {
"esp32",
"esp32s2",
"esp32-s2",
"esp32s3",
"esp32-s3",
"esp32c3",
"esp32-c3",
"esp32c6",
"esp32-c6",
}
class FlashError(RuntimeError):
pass
def _require_confirm(confirm: bool, operation: str) -> None:
if not confirm:
raise FlashError(
f"{operation} is destructive and requires confirm=True. "
"This will overwrite firmware on the device."
)
def _reject_native_env(env: str, operation: str) -> None:
"""`native*` envs build a host executable, not firmware — there's no
upload step. The user wants `build` (or just runs the binary directly).
"""
if env.startswith("native"):
raise FlashError(
f"{operation} is not applicable for env {env!r}: native envs "
"produce a host executable, not flashable firmware. Use `build` "
"instead, then run the resulting binary directly."
)
def _artifacts_for(env: str) -> list[Path]:
build_dir = config.firmware_root() / ".pio" / "build" / env
if not build_dir.is_dir():
return []
patterns = (
"firmware*.bin",
"firmware*.uf2",
"firmware*.hex",
"firmware*.zip",
"firmware*.elf",
"*.mt.json",
"littlefs-*.bin",
)
out: list[Path] = []
for pat in patterns:
out.extend(sorted(build_dir.glob(pat)))
return out
def _factory_bin_for(env: str) -> Path | None:
build_dir = config.firmware_root() / ".pio" / "build" / env
if not build_dir.is_dir():
return None
matches = sorted(build_dir.glob("firmware-*.factory.bin"))
return matches[0] if matches else None
def _firmware_bin_for(env: str) -> Path | None:
"""Return the OTA-update firmware binary (app partition only)."""
build_dir = config.firmware_root() / ".pio" / "build" / env
if not build_dir.is_dir():
return None
# device-update.sh expects firmware-<env>-<version>.bin (not .factory.bin)
matches = sorted(
p
for p in build_dir.glob("firmware-*.bin")
if not p.name.endswith(".factory.bin")
)
return matches[0] if matches else None
def _userprefs_summary(active: dict[str, str]) -> dict[str, Any]:
"""Compact summary of which USERPREFS_* are baked into the build."""
return {"count": len(active), "keys": sorted(active.keys())}
def build(
env: str,
with_manifest: bool = True,
userprefs_overrides: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Run `pio run -e <env>` and return artifact paths.
`userprefs_overrides` (optional): dict of `USERPREFS_<KEY>: value` to inject
into userPrefs.jsonc for this build only. File is restored byte-for-byte
on exit. Use `userprefs_set()` for persistent changes.
`build_flags` (optional): dict of `-D<NAME>=<VALUE>` macros to set for
this build only via `PLATFORMIO_BUILD_FLAGS`. Common useful flag:
`{"DEBUG_HEAP": 1}` enables per-thread leak detection + `[heap N]`
prefix on every log line. Combines with the recorder so heap shows
up at log cadence (much higher resolution than the ~60 s LocalStats
packet) — see `recorder/parsers.py:_HEAP_PREFIX_RE`. Bool values
expand to bare `-D<NAME>` (presence-only flags).
"""
args = ["run", "-e", env]
if with_manifest:
args.extend(["-t", "mtjson"])
extra_env = _build_flags_env(build_flags) if build_flags else None
with userprefs.temporary_overrides(userprefs_overrides) as effective:
result = pio.run(
args,
timeout=pio.TIMEOUT_BUILD,
check=False,
extra_env=extra_env,
)
return {
"exit_code": result.returncode,
"artifacts": [str(p) for p in _artifacts_for(env)],
"stdout_tail": pio.tail_lines(result.stdout, 200),
"stderr_tail": pio.tail_lines(result.stderr, 200),
"duration_s": round(result.duration_s, 2),
"userprefs": _userprefs_summary(effective),
"build_flags": dict(build_flags) if build_flags else None,
}
def _build_flags_env(build_flags: dict[str, Any]) -> dict[str, str]:
"""Translate `{"DEBUG_HEAP": 1, "FOO": "bar"}` → `{"PLATFORMIO_BUILD_FLAGS":
"-DDEBUG_HEAP=1 -DFOO=bar"}`. Bool True → bare `-D<NAME>`; False/None drop
the flag entirely. Other types stringify."""
parts: list[str] = []
for key, value in build_flags.items():
if value is False or value is None:
continue
if value is True:
parts.append(f"-D{key}")
else:
parts.append(f"-D{key}={value}")
if not parts:
return {}
return {"PLATFORMIO_BUILD_FLAGS": " ".join(parts)}
def clean(env: str) -> dict[str, Any]:
"""Run `pio run -e <env> -t clean`."""
result = pio.run(["run", "-e", env, "-t", "clean"], timeout=120, check=False)
return {
"exit_code": result.returncode,
"stdout_tail": pio.tail_lines(result.stdout, 200),
"stderr_tail": pio.tail_lines(result.stderr, 200),
"duration_s": round(result.duration_s, 2),
}
def flash(
env: str,
port: str,
confirm: bool = False,
userprefs_overrides: dict[str, Any] | None = None,
build_flags: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""`pio run -e <env> -t upload --upload-port <port>`. All architectures.
`userprefs_overrides` (optional): see `build()` — the rebuild-before-upload
that pio performs will pick up the injected values.
`build_flags` (optional): same shape as `build()` — `PLATFORMIO_BUILD_FLAGS`
is exported for the rebuild-before-upload, so the uploaded firmware
actually carries the flags. Without this propagation, `pio run -t upload`
would relink without the env var and silently drop them. Common use:
`build_flags={"DEBUG_HEAP": 1}` for the leak-hunt path.
"""
_require_confirm(confirm, "flash")
_reject_native_env(env, "flash")
connection.reject_if_tcp(port, "flash")
extra_env = _build_flags_env(build_flags) if build_flags else None
with userprefs.temporary_overrides(userprefs_overrides) as effective:
result = pio.run(
["run", "-e", env, "-t", "upload", "--upload-port", port],
timeout=pio.TIMEOUT_UPLOAD,
check=False,
extra_env=extra_env,
)
return {
"exit_code": result.returncode,
"stdout_tail": pio.tail_lines(result.stdout, 200),
"stderr_tail": pio.tail_lines(result.stderr, 200),
"duration_s": round(result.duration_s, 2),
"userprefs": _userprefs_summary(effective),
"build_flags": dict(build_flags) if build_flags else None,
}
def _check_esp32_env(env: str) -> str:
rec = boards.get_board(env)
arch = rec.get("architecture")
if arch not in ESP32_ARCHES:
raise FlashError(
f"Env {env!r} has architecture {arch!r}, not ESP32. "
"Use `flash` for non-ESP32 boards."
)
return arch
def _run_install_script(script: Path, port: str, binary: Path) -> dict[str, Any]:
"""Invoke bin/device-install.sh or bin/device-update.sh."""
t0 = time.monotonic()
proc = subprocess.run(
[str(script), "-p", port, "-f", str(binary)],
cwd=str(config.firmware_root()),
capture_output=True,
text=True,
timeout=pio.TIMEOUT_UPLOAD,
)
duration = time.monotonic() - t0
return {
"exit_code": proc.returncode,
"stdout_tail": pio.tail_lines(proc.stdout, 200),
"stderr_tail": pio.tail_lines(proc.stderr, 200),
"duration_s": round(duration, 2),
}
def erase_and_flash(
env: str,
port: str,
confirm: bool = False,
skip_build: bool = False,
userprefs_overrides: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""ESP32-only: full erase + factory flash via bin/device-install.sh.
`userprefs_overrides`: baked into the factory.bin via a fresh build. If
overrides are provided we always force a rebuild (skip_build=True errors
in that case) since a cached factory.bin would not reflect the new prefs.
"""
_require_confirm(confirm, "erase_and_flash")
connection.reject_if_tcp(port, "erase_and_flash")
_check_esp32_env(env)
if userprefs_overrides and skip_build:
raise FlashError(
"userprefs_overrides forces a rebuild so the factory.bin reflects "
"the new values; skip_build=True is incompatible."
)
with userprefs.temporary_overrides(userprefs_overrides) as effective:
# If overrides were provided, always build; otherwise only build if
# no factory.bin is present.
factory = _factory_bin_for(env)
if factory is None or userprefs_overrides:
if skip_build:
raise FlashError(
f"No factory.bin found for env {env!r} and skip_build=True. "
"Run `build` first or set skip_build=False."
)
build_args = ["run", "-e", env, "-t", "mtjson"]
build_result = pio.run(build_args, timeout=pio.TIMEOUT_BUILD, check=False)
if build_result.returncode != 0:
return {
"exit_code": build_result.returncode,
"stdout_tail": pio.tail_lines(build_result.stdout, 200),
"stderr_tail": pio.tail_lines(build_result.stderr, 200),
"duration_s": round(build_result.duration_s, 2),
"error": "build failed before erase_and_flash could run",
"userprefs": _userprefs_summary(effective),
}
factory = _factory_bin_for(env)
if factory is None:
raise FlashError(
f"Build succeeded but no factory.bin appeared in .pio/build/{env}/"
)
script = config.firmware_root() / "bin" / "device-install.sh"
if not script.is_file():
raise FlashError(f"device-install.sh not found at {script}")
result = _run_install_script(script, port, factory)
result["userprefs"] = _userprefs_summary(effective)
return result
def update_flash(
env: str,
port: str,
confirm: bool = False,
skip_build: bool = False,
userprefs_overrides: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""ESP32-only: OTA app-partition update via bin/device-update.sh.
`userprefs_overrides`: baked into the firmware.bin via a fresh build. If
overrides are provided we always force a rebuild.
"""
_require_confirm(confirm, "update_flash")
connection.reject_if_tcp(port, "update_flash")
_check_esp32_env(env)
if userprefs_overrides and skip_build:
raise FlashError(
"userprefs_overrides forces a rebuild so the firmware.bin reflects "
"the new values; skip_build=True is incompatible."
)
with userprefs.temporary_overrides(userprefs_overrides) as effective:
firmware = _firmware_bin_for(env)
if firmware is None or userprefs_overrides:
if skip_build:
raise FlashError(
f"No firmware.bin found for env {env!r} and skip_build=True. "
"Run `build` first or set skip_build=False."
)
build_args = ["run", "-e", env, "-t", "mtjson"]
build_result = pio.run(build_args, timeout=pio.TIMEOUT_BUILD, check=False)
if build_result.returncode != 0:
return {
"exit_code": build_result.returncode,
"stdout_tail": pio.tail_lines(build_result.stdout, 200),
"stderr_tail": pio.tail_lines(build_result.stderr, 200),
"duration_s": round(build_result.duration_s, 2),
"error": "build failed before update_flash could run",
"userprefs": _userprefs_summary(effective),
}
firmware = _firmware_bin_for(env)
if firmware is None:
raise FlashError(
f"Build succeeded but no firmware.bin appeared in .pio/build/{env}/"
)
script = config.firmware_root() / "bin" / "device-update.sh"
if not script.is_file():
raise FlashError(f"device-update.sh not found at {script}")
result = _run_install_script(script, port, firmware)
result["userprefs"] = _userprefs_summary(effective)
return result
def _do_1200bps_touch(port: str, settle_ms: int, touch_timeout_s: float = 3.0) -> None:
"""Open port at 1200 baud and close, bounded by a worker thread.
Both the open and the close can block on a busy CDC device — we wrap the
whole thing in a worker so the caller returns in at most `touch_timeout_s`
regardless. The touch is signal-only: the USB configuration change to
1200 baud alone is enough to trip the Adafruit bootloader's reset, so a
worker that's still blocked in the background after timeout has already
delivered the signal.
"""
errors: list[BaseException] = []
def _inner() -> None:
try:
s = serial.Serial(port, 1200)
except serial.SerialException as exc:
if "No such file" in str(exc) or "could not open" in str(exc).lower():
raise
return # other serial errors mid-open are expected during DFU entry
try:
time.sleep(settle_ms / 1000.0)
finally:
try:
s.close()
except Exception:
pass
def _runner() -> None:
try:
_inner()
except BaseException as exc: # re-raised on caller thread after join
errors.append(exc)
worker = threading.Thread(target=_runner, daemon=True)
worker.start()
worker.join(timeout=touch_timeout_s)
if worker.is_alive():
return # signal already delivered; allow daemon worker to finish/exit
if errors:
raise errors[0]
# Adafruit nRF52 bootloader VID/PID (BOTH RAK4631 and most Feather nRF52 boards).
# See https://github.com/adafruit/Adafruit_nRF52_Bootloader
_NRF52_BOOTLOADER_VID = 0x239A
_NRF52_BOOTLOADER_PIDS = {
0x0029, # Adafruit nRF52 bootloader (generic, used by RAK4631)
0x002A, # Adafruit Feather Express bootloader variant
0x4029, # alt seen on some boards
}
def _find_nrf52_bootloader_port() -> dict[str, Any] | None:
"""Return a dict for any currently-enumerated nRF52 bootloader port, or None."""
for d in devices.list_devices(include_unknown=True):
vid_str = d.get("vid")
pid_str = d.get("pid")
if vid_str is None or pid_str is None:
continue
try:
vid = int(vid_str, 16) if isinstance(vid_str, str) else int(vid_str)
pid = int(pid_str, 16) if isinstance(pid_str, str) else int(pid_str)
except ValueError:
continue
if vid == _NRF52_BOOTLOADER_VID and pid in _NRF52_BOOTLOADER_PIDS:
return d
return None
def touch_1200bps(
port: str,
settle_ms: int = 250,
poll_timeout_s: float = 8.0,
retries: int = 2,
) -> dict[str, Any]:
"""Open port at 1200 baud, close immediately — triggers USB CDC bootloader.
Works for: nRF52840 (Adafruit bootloader), ESP32-S3 (native USB download
mode), RP2040 (when built with 1200bps-reset stdio), Arduino Leonardo/Micro.
For nRF52 specifically: after the touch, polls for the Adafruit bootloader
VID/PID (0x239A / 0x0029) for up to `poll_timeout_s` seconds. Adafruit's
bootloader docs note a touch sometimes needs to be repeated, so this
retries up to `retries` times. The returned `new_port` is the bootloader
port (distinct from the app port) — exactly what's needed for `pio run
-t upload` to drive nrfutil.
For non-nRF52 devices (ESP32-S3, RP2040, Arduino), falls back to
"any-new-port appeared" detection.
Returns `{ok, former_port, new_port, new_port_vid_pid, attempts}`.
"""
connection.reject_if_tcp(port, "touch_1200bps")
before_list = devices.list_devices(include_unknown=True)
before_ports = {d["port"] for d in before_list}
attempts = 0
new_port_info: dict[str, Any] | None = None
for attempt in range(1, retries + 1):
attempts = attempt
_do_1200bps_touch(port, settle_ms=settle_ms, touch_timeout_s=3.0)
# Poll for either (a) the nRF52 bootloader VID/PID appearing, or
# (b) a brand-new port appearing that wasn't there before.
deadline = time.monotonic() + poll_timeout_s
while time.monotonic() < deadline:
time.sleep(0.2)
bootloader = _find_nrf52_bootloader_port()
if bootloader is not None:
new_port_info = bootloader
break
current = devices.list_devices(include_unknown=True)
current_paths = {d["port"] for d in current}
added = current_paths - before_ports
if added:
added_record = next((d for d in current if d["port"] in added), None)
if added_record:
new_port_info = added_record
break
if new_port_info is not None:
break
# No bootloader appeared; try touching again (Adafruit recommends
# sometimes requiring two touches for reliability).
if new_port_info is not None:
return {
"ok": True,
"former_port": port,
"new_port": new_port_info["port"],
"new_port_vid_pid": (
new_port_info.get("vid"),
new_port_info.get("pid"),
),
"attempts": attempts,
}
return {
"ok": False,
"former_port": port,
"new_port": None,
"new_port_vid_pid": (None, None),
"attempts": attempts,
}
+248
View File
@@ -0,0 +1,248 @@
"""Direct wrappers around vendor flashing tools: esptool, nrfutil, picotool.
These are escape hatches. Prefer the pio-based tools in flash.py when they
cover the operation — pio knows the correct offsets, protocols, and filters
for every supported board. Use these when pio doesn't: to erase a bricked
ESP32, DFU-flash an nRF52 zip package, or inspect an RP2040's bootloader.
Every destructive `*_raw` subcommand is gated by `confirm=True` so callers
can't accidentally `--write-flash` from freeform args.
"""
from __future__ import annotations
import re
import subprocess
from pathlib import Path
from typing import Any, Sequence
from . import config, connection, pio
_TIMEOUT_SHORT = 30
_TIMEOUT_LONG = 600
class ToolError(RuntimeError):
pass
def _run(
binary: Path,
args: Sequence[str],
*,
timeout: float = _TIMEOUT_LONG,
cwd: Path | None = None,
) -> dict[str, Any]:
# Shared with pio.run(): if `MESHTASTIC_MCP_FLASH_LOG` is set, each line
# of output is tee'd to that file as it arrives so the TUI can show live
# esptool/nrfutil/picotool progress instead of 3 minutes of silence.
full = [str(binary), *args]
try:
rc, stdout, stderr, duration = pio._run_capturing(
full,
cwd=cwd,
timeout=timeout,
tee_header=f"{binary.name} {' '.join(args)}",
)
except subprocess.TimeoutExpired as exc:
raise ToolError(
f"{binary.name} {' '.join(args)} timed out after {timeout}s"
) from exc
return {
"exit_code": rc,
"stdout": stdout,
"stderr": stderr,
"stdout_tail": pio.tail_lines(stdout, 200),
"stderr_tail": pio.tail_lines(stderr, 200),
"duration_s": round(duration, 2),
}
def _require_confirm(confirm: bool, what: str) -> None:
if not confirm:
raise ToolError(f"{what} is destructive and requires confirm=True.")
# ---------- esptool --------------------------------------------------------
ESPTOOL_DESTRUCTIVE = {
"write_flash",
"write-flash",
"erase_flash",
"erase-flash",
"erase_region",
"erase-region",
"merge_bin",
"merge-bin",
}
def _parse_esptool_chip_info(stdout: str) -> dict[str, Any]:
"""Parse `esptool chip_id` / `flash_id` output into structured fields."""
result: dict[str, Any] = {
"chip": None,
"mac": None,
"crystal_mhz": None,
"flash_size": None,
"features": [],
}
for line in stdout.splitlines():
line = line.strip()
if m := re.match(r"Chip is (.+)", line):
result["chip"] = m.group(1).strip()
elif m := re.match(r"MAC: ([0-9a-fA-F:]+)", line):
result["mac"] = m.group(1)
elif m := re.match(r"Crystal is (\d+)MHz", line):
result["crystal_mhz"] = int(m.group(1))
elif m := re.match(r"Detected flash size: (\S+)", line):
result["flash_size"] = m.group(1)
elif m := re.match(r"Features: (.+)", line):
result["features"] = [f.strip() for f in m.group(1).split(",") if f.strip()]
return result
def esptool_chip_info(port: str) -> dict[str, Any]:
connection.reject_if_tcp(port, "esptool_chip_info")
binary = config.esptool_bin()
# `chip_id` prints chip + mac + crystal + features. `flash_id` adds flash.
combined = _run(binary, ["--port", port, "flash_id"], timeout=_TIMEOUT_SHORT)
if combined["exit_code"] != 0:
raise ToolError(
f"esptool failed (exit {combined['exit_code']}):\n{combined['stderr_tail']}"
)
parsed = _parse_esptool_chip_info(combined["stdout"])
return {**parsed, "raw_stdout_tail": combined["stdout_tail"]}
def esptool_erase_flash(port: str, confirm: bool = False) -> dict[str, Any]:
"""Full-chip erase. Leaves the device unbootable until reflashed."""
_require_confirm(confirm, "esptool_erase_flash")
connection.reject_if_tcp(port, "esptool_erase_flash")
binary = config.esptool_bin()
# esptool v5 uses `erase-flash`, older uses `erase_flash`. Try the new name
# first; if it fails with unknown command, retry old.
res = _run(binary, ["--port", port, "erase-flash"], timeout=_TIMEOUT_LONG)
if (
res["exit_code"] != 0
and "unrecognized" in (res["stderr"] or res["stdout"]).lower()
):
res = _run(binary, ["--port", port, "erase_flash"], timeout=_TIMEOUT_LONG)
return res
def esptool_raw(
args: list[str], port: str | None = None, confirm: bool = False
) -> dict[str, Any]:
"""Raw esptool passthrough. Destructive subcommands require confirm=True."""
if not args:
raise ToolError("args must not be empty")
connection.reject_if_tcp(port, "esptool_raw")
# Find the first non-flag arg (the subcommand).
subcommand = next((a for a in args if not a.startswith("-")), None)
if subcommand and subcommand.replace("-", "_") in {
s.replace("-", "_") for s in ESPTOOL_DESTRUCTIVE
}:
_require_confirm(confirm, f"esptool {subcommand}")
binary = config.esptool_bin()
full_args: list[str] = []
if port:
full_args.extend(["--port", port])
full_args.extend(args)
return _run(binary, full_args, timeout=_TIMEOUT_LONG)
# ---------- nrfutil --------------------------------------------------------
NRFUTIL_DESTRUCTIVE = {"dfu", "settings"}
def nrfutil_dfu(port: str, package_path: str, confirm: bool = False) -> dict[str, Any]:
_require_confirm(confirm, "nrfutil_dfu")
connection.reject_if_tcp(port, "nrfutil_dfu")
pkg = Path(package_path).expanduser()
if not pkg.is_file():
raise ToolError(f"Package not found: {pkg}")
binary = config.nrfutil_bin()
return _run(
binary,
["dfu", "serial", "--package", str(pkg), "--port", port, "-b", "115200"],
timeout=_TIMEOUT_LONG,
)
def nrfutil_raw(args: list[str], confirm: bool = False) -> dict[str, Any]:
if not args:
raise ToolError("args must not be empty")
subcommand = next((a for a in args if not a.startswith("-")), None)
if subcommand in NRFUTIL_DESTRUCTIVE:
_require_confirm(confirm, f"nrfutil {subcommand}")
binary = config.nrfutil_bin()
return _run(binary, args, timeout=_TIMEOUT_LONG)
# ---------- picotool -------------------------------------------------------
PICOTOOL_DESTRUCTIVE = {"load", "reboot", "save", "erase"}
def _parse_picotool_info(stdout: str) -> dict[str, Any]:
result: dict[str, Any] = {
"vendor": None,
"product": None,
"serial": None,
"flash_size": None,
"program_name": None,
"program_version": None,
}
for line in stdout.splitlines():
line = line.strip()
if m := re.match(r"Program information:", line):
continue
if m := re.match(r"name:\s*(.+)", line):
result["program_name"] = m.group(1).strip()
elif m := re.match(r"version:\s*(.+)", line):
result["program_version"] = m.group(1).strip()
elif m := re.match(r"vendor:\s*(.+)", line):
result["vendor"] = m.group(1).strip()
elif m := re.match(r"product:\s*(.+)", line):
result["product"] = m.group(1).strip()
elif m := re.match(r"serial number:\s*(.+)", line):
result["serial"] = m.group(1).strip()
elif m := re.match(r"flash size:\s*(.+)", line):
result["flash_size"] = m.group(1).strip()
return result
def picotool_info(port: str | None = None) -> dict[str, Any]:
"""Read device info from a Pico in BOOTSEL mode. `port` is informational
only — picotool auto-detects."""
connection.reject_if_tcp(port, "picotool_info")
binary = config.picotool_bin()
res = _run(binary, ["info", "-a"], timeout=_TIMEOUT_SHORT)
if res["exit_code"] != 0:
raise ToolError(
f"picotool info failed (exit {res['exit_code']}): "
"is the Pico in BOOTSEL mode?\n" + res["stderr_tail"]
)
parsed = _parse_picotool_info(res["stdout"])
return {**parsed, "raw_stdout_tail": res["stdout_tail"]}
def picotool_load(uf2_path: str, confirm: bool = False) -> dict[str, Any]:
_require_confirm(confirm, "picotool_load")
uf2 = Path(uf2_path).expanduser()
if not uf2.is_file():
raise ToolError(f"UF2 not found: {uf2}")
binary = config.picotool_bin()
return _run(binary, ["load", "-x", "-t", "uf2", str(uf2)], timeout=_TIMEOUT_LONG)
def picotool_raw(args: list[str], confirm: bool = False) -> dict[str, Any]:
if not args:
raise ToolError("args must not be empty")
subcommand = next((a for a in args if not a.startswith("-")), None)
if subcommand in PICOTOOL_DESTRUCTIVE:
_require_confirm(confirm, f"picotool {subcommand}")
binary = config.picotool_bin()
return _run(binary, args, timeout=_TIMEOUT_LONG)
+103
View File
@@ -0,0 +1,103 @@
"""Read-only device queries via meshtastic.SerialInterface."""
from __future__ import annotations
from typing import Any
from .connection import connect
def _primary_channel_name(iface) -> str | None:
try:
channels = iface.localNode.channels or []
except AttributeError:
return None
for ch in channels:
role = getattr(ch, "role", None)
# Role enum: 0 DISABLED, 1 PRIMARY, 2 SECONDARY
if role == 1:
name = getattr(getattr(ch, "settings", None), "name", None)
return name or "(default)"
return None
def device_info(port: str | None = None, timeout_s: float = 8.0) -> dict[str, Any]:
"""Return summary info for the connected device."""
with connect(port=port, timeout_s=timeout_s) as iface:
my = iface.myInfo
meta = iface.metadata
local = iface.localNode
# Owner (long/short name) is on the local node's user record
long_name: str | None = None
short_name: str | None = None
hw_model: str | int | None = None
if iface.nodesByNum and my is not None:
local_rec = iface.nodesByNum.get(my.my_node_num, {})
user = local_rec.get("user") or {}
long_name = user.get("longName")
short_name = user.get("shortName")
hw_model = user.get("hwModel")
region = None
if local is not None and local.localConfig is not None:
try:
lora = local.localConfig.lora
# region is an enum; get its string name
region = (
lora.DESCRIPTOR.fields_by_name["region"]
.enum_type.values_by_number[lora.region]
.name
)
except Exception:
region = None
return {
"port": iface.devPath if hasattr(iface, "devPath") else port,
"my_node_num": getattr(my, "my_node_num", None),
"long_name": long_name,
"short_name": short_name,
"firmware_version": getattr(meta, "firmware_version", None),
"hw_model": hw_model,
"region": region,
"num_nodes": len(iface.nodesByNum) if iface.nodesByNum else 0,
"primary_channel": _primary_channel_name(iface),
}
def _node_record(node_dict: dict[str, Any]) -> dict[str, Any]:
user = node_dict.get("user") or {}
position = node_dict.get("position") or None
device_metrics = node_dict.get("deviceMetrics") or {}
return {
"node_num": node_dict.get("num"),
"user": {
"long_name": user.get("longName"),
"short_name": user.get("shortName"),
"hw_model": user.get("hwModel"),
"role": user.get("role"),
},
"position": (
{
"latitude": position.get("latitude"),
"longitude": position.get("longitude"),
"altitude": position.get("altitude"),
"time": position.get("time"),
}
if position
else None
),
"snr": node_dict.get("snr"),
"rssi": node_dict.get("rssi"),
"last_heard": node_dict.get("lastHeard"),
"battery_level": device_metrics.get("batteryLevel"),
"is_favorite": bool(node_dict.get("isFavorite", False)),
}
def list_nodes(port: str | None = None, timeout_s: float = 8.0) -> list[dict[str, Any]]:
"""Return the device's node database."""
with connect(port=port, timeout_s=timeout_s) as iface:
if not iface.nodesByNum:
return []
return [_node_record(n) for n in iface.nodesByNum.values()]
@@ -0,0 +1,67 @@
"""Python mirror of firmware `enum input_broker_event` (src/input/InputBroker.h).
Used by `admin.send_input_event` + `tests/ui/` so callers can say
`InputEventCode.RIGHT` instead of hard-coding 20. Values MUST stay in sync
with the firmware enum — unit test `tests/unit/test_input_event_codes.py`
pins the mapping.
"""
from __future__ import annotations
from enum import IntEnum
class InputEventCode(IntEnum):
"""Button / key / gesture events dispatched by the firmware InputBroker."""
NONE = 0
SELECT = 10
SELECT_LONG = 11
UP_LONG = 12
DOWN_LONG = 13
UP = 17
DOWN = 18
LEFT = 19
RIGHT = 20
CANCEL = 24
BACK = 27
# Auto-incremented values in the C enum (27 + 1, +2, +3):
USER_PRESS = 28
ALT_PRESS = 29
ALT_LONG = 30
SHUTDOWN = 0x9B
GPS_TOGGLE = 0x9E
SEND_PING = 0xAF
FN_F1 = 0xF1
FN_F2 = 0xF2
FN_F3 = 0xF3
FN_F4 = 0xF4
FN_F5 = 0xF5
MATRIXKEY = 0xFE
ANYKEY = 0xFF
def coerce_event_code(value: int | str | InputEventCode) -> int:
"""Accept an int, a case-insensitive name, or an `InputEventCode` and return
the u8 wire value. Raises ValueError on unknown names / out-of-range ints.
"""
if isinstance(value, InputEventCode):
return int(value)
if isinstance(value, int):
if not 0 <= value <= 255:
raise ValueError(f"event_code out of u8 range: {value}")
return value
if isinstance(value, str):
key = value.upper().replace("-", "_")
if key.startswith("INPUT_BROKER_"):
key = key[len("INPUT_BROKER_") :]
try:
return int(InputEventCode[key])
except KeyError as exc:
known = ", ".join(m.name for m in InputEventCode)
raise ValueError(
f"unknown event code name {value!r}; known: {known}"
) from exc
raise TypeError(
f"event_code must be int|str|InputEventCode, got {type(value).__name__}"
)
+410
View File
@@ -0,0 +1,410 @@
"""Read-side queries over the recorder's JSONL streams.
Pure functions over `mcp-server/.mtlog/`. Streaming JSONL reader: never
loads a whole file. Time-bound queries short-circuit as soon as `ts`
exceeds the requested end. The recorder writes monotonically, so a
forward scan is cheap; we don't need an index.
All time arguments accept:
- epoch seconds (int/float)
- relative strings: "-15m", "-2h", "-3d", "now"
- ISO-ish absolute strings: "2026-05-07T14:30:00" (naive timestamps are
treated as UTC)
Tools that return data ALWAYS cap their output (max_lines / max_points
/ max), and report whether more matched than was returned.
"""
from __future__ import annotations
import gzip
import json
import re
import statistics
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Iterator
from .recorder.recorder import get_recorder
_REL_RE = re.compile(r"^\s*-\s*(\d+(?:\.\d+)?)\s*([smhd])\s*$")
_REGEX_PREVIEW_MAX = 100
_REGEX_PREVIEW_TRUNCATE = 97
def _parse_time(value: Any, *, now: float | None = None) -> float:
"""Coerce to epoch seconds. Defaults `now` to `time.time()`."""
if value is None:
return time.time()
if isinstance(value, (int, float)):
return float(value)
if not isinstance(value, str):
raise ValueError(f"invalid time: {value!r}")
s = value.strip().lower()
if s in ("", "now"):
return time.time() if now is None else now
m = _REL_RE.match(s)
if m:
n = float(m.group(1))
unit = m.group(2)
secs = n * {"s": 1, "m": 60, "h": 3600, "d": 86400}[unit]
base = time.time() if now is None else now
return base - secs
# Try ISO 8601. Accept naive (assume UTC) and Z-suffixed.
try:
if s.endswith("z"):
s = s[:-1] + "+00:00"
dt = datetime.fromisoformat(s)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.timestamp()
except ValueError as e:
raise ValueError(f"unparseable time: {value!r}") from e
def _iter_jsonl(path: Path, *, since: float, until: float) -> Iterator[dict[str, Any]]:
"""Stream records in chronological order: rotated archives first
(oldest → newest by lex sort, which is chronological for our
`YYYYMMDD-HHMMSS-uuuuuu-NNNNN` archive naming), then the live file
last. The "keep last N" pop-front logic in the window queries
relies on records arriving in time order across files.
"""
files: list[Path] = []
# Gzipped archives are named "<stem>.YYYYMMDD-HHMMSS-uuuuuu-NNNNN.jsonl.gz".
for archive in sorted(path.parent.glob(f"{path.stem}.*.jsonl.gz")):
files.append(archive)
if path.exists():
files.append(path)
for f in files:
opener = gzip.open if f.suffix == ".gz" else open
try:
with opener(f, "rt", encoding="utf-8") as fh: # type: ignore[arg-type]
for line in fh:
line = line.strip()
if not line:
continue
try:
rec = json.loads(line)
except json.JSONDecodeError:
continue
ts = rec.get("ts")
if not isinstance(ts, (int, float)):
continue
if ts < since:
continue
if ts > until:
# Records are append-monotonic within a file, so
# the rest of this file is also past `until`.
# Archives can still overlap each other, so only
# short-circuit this file, not the whole scan.
break
yield rec
except (FileNotFoundError, OSError):
continue
# -- queries ------------------------------------------------------------
def logs_window(
start: Any = "-15m",
end: Any = "now",
*,
grep: str | None = None,
level: str | None = None,
tag: str | None = None,
port: str | None = None,
max_lines: int = 200,
) -> dict[str, Any]:
"""Recent firmware log lines, filtered.
`level` accepts a single level name or pipe-separated set
("WARN|ERROR|CRIT"). `grep` is a regex (Python re) over the raw
`line` field. Returns the last `max_lines` matches.
"""
s = _parse_time(start)
e = _parse_time(end)
levels = _split_set(level)
if grep:
try:
grep_re = re.compile(grep)
except re.error as exc:
preview = (
grep
if len(grep) <= _REGEX_PREVIEW_MAX
else f"{grep[:_REGEX_PREVIEW_TRUNCATE]}..."
)
raise ValueError(f"invalid grep regex {preview!r}: {exc}") from exc
else:
grep_re = None
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "logs.jsonl", since=s, until=e):
if levels and rec.get("level") not in levels:
continue
if tag and rec.get("tag") != tag:
continue
if port and rec.get("port") != port:
continue
if grep_re and not grep_re.search(rec.get("line") or ""):
continue
matched += 1
out.append(rec)
if len(out) > max_lines:
out.pop(0) # keep the most recent N
return {
"lines": out,
"total_matched": matched,
"dropped": max(0, matched - max_lines),
"window": {"start": s, "end": e},
}
def telemetry_timeline(
window: Any = "1h",
*,
variant: str = "local",
field: str = "free_heap",
port: str | None = None,
max_points: int = 200,
) -> dict[str, Any]:
"""Timeseries of one telemetry field, downsampled.
`field` matches both the protobuf snake_case name (`free_heap`,
`heap_free_bytes`, `battery_level`) and camelCase (`freeHeap`).
Server-side bucket-mean downsamples to ≤ `max_points`. Returns
`slope_per_min` (linear regression slope, units/min) so a leak
detector can read one number.
"""
end = time.time()
if isinstance(window, (int, float)):
# Numeric `window` is a duration in seconds — "last N seconds".
# Without this branch, `_parse_time(-N)` would treat -N as an
# absolute epoch timestamp (i.e., Jan 1 1970 minus N seconds),
# producing a wildly negative `start` and matching nothing.
start = end - float(window)
elif isinstance(window, str) and not window.startswith("-"):
# Bare string like "1h" is sugar for "-1h".
start = _parse_time(f"-{window}", now=end)
else:
start = _parse_time(window, now=end)
base = get_recorder().base_dir
raw: list[tuple[float, float]] = []
field_aliases = _field_aliases(field)
for rec in _iter_jsonl(base / "telemetry.jsonl", since=start, until=end):
if rec.get("variant") != variant:
continue
if port and rec.get("port") != port:
continue
fields = rec.get("fields") or {}
value: Any = None
for alias in field_aliases:
if alias in fields:
value = fields[alias]
break
if not isinstance(value, (int, float)):
continue
raw.append((float(rec["ts"]), float(value)))
if not raw:
return {
"points": [],
"samples": 0,
"min": None,
"max": None,
"slope_per_min": None,
"window": {"start": start, "end": end, "variant": variant, "field": field},
}
points = _downsample(raw, max_points=max_points)
values = [v for _, v in raw]
return {
"points": [{"ts": ts, "value": v} for ts, v in points],
"samples": len(raw),
"min": min(values),
"max": max(values),
"slope_per_min": _slope_per_min(raw),
"window": {"start": start, "end": end, "variant": variant, "field": field},
}
def packets_window(
start: Any = "-5m",
end: Any = "now",
*,
portnum: str | None = None,
from_node: str | None = None,
to_node: str | None = None,
max: int = 200,
) -> dict[str, Any]:
s = _parse_time(start)
e = _parse_time(end)
portnums = _split_set(portnum)
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "packets.jsonl", since=s, until=e):
if portnums and rec.get("portnum") not in portnums:
continue
if from_node and str(rec.get("from_node")) != str(from_node):
continue
if to_node and str(rec.get("to_node")) != str(to_node):
continue
matched += 1
out.append(rec)
if len(out) > max:
out.pop(0)
return {
"packets": out,
"total_matched": matched,
"dropped": matched - max if matched > max else 0,
"window": {"start": s, "end": e},
}
def events_window(
start: Any = "-1h",
end: Any = "now",
*,
kind: str | None = None,
max: int = 200,
) -> dict[str, Any]:
s = _parse_time(start)
e = _parse_time(end)
kinds = _split_set(kind)
base = get_recorder().base_dir
matched = 0
out: list[dict[str, Any]] = []
for rec in _iter_jsonl(base / "events.jsonl", since=s, until=e):
if kinds and rec.get("kind") not in kinds:
continue
matched += 1
out.append(rec)
if len(out) > max:
out.pop(0)
return {
"events": out,
"total_matched": matched,
"dropped": matched - max if matched > max else 0,
"window": {"start": s, "end": e},
}
def export(
start: Any,
end: Any,
dest_dir: str,
*,
streams: list[str] | None = None,
) -> dict[str, Any]:
"""Bundle a slice of each requested stream into `dest_dir`.
For a notebook, a bug report, or a Datadog backfill. Output files
are uncompressed JSONL (callers gzip themselves if they want to).
"""
s = _parse_time(start)
e = _parse_time(end)
selected = streams or ["logs", "telemetry", "packets", "events"]
dest = Path(dest_dir)
dest.mkdir(parents=True, exist_ok=True)
base = get_recorder().base_dir
paths: dict[str, str] = {}
for stream in selected:
src = base / f"{stream}.jsonl"
if not src.exists() and not list(base.glob(f"{stream}.*.jsonl.gz")):
continue
out_path = dest / f"{stream}.jsonl"
n = 0
with out_path.open("w", encoding="utf-8") as fh:
for rec in _iter_jsonl(src, since=s, until=e):
fh.write(json.dumps(rec, separators=(",", ":")) + "\n")
n += 1
paths[stream] = str(out_path)
paths[f"{stream}_count"] = str(n)
return {"dest_dir": str(dest), "paths": paths, "window": {"start": s, "end": e}}
# -- helpers ------------------------------------------------------------
def _split_set(value: str | None) -> set[str] | None:
if not value:
return None
return {v.strip() for v in value.split("|") if v.strip()}
def _field_aliases(field: str) -> list[str]:
"""Accept snake_case OR camelCase, plus a few legacy aliases."""
snake = field
camel = _snake_to_camel(field)
aliases = {snake, camel}
# Old protobuf fields (pre-LocalStats) used different names
legacy = {
"free_heap": ["free_heap", "freeHeap", "heap_free_bytes", "heapFreeBytes"],
"heap_free_bytes": [
"heap_free_bytes",
"heapFreeBytes",
"free_heap",
"freeHeap",
],
"total_heap": ["total_heap", "totalHeap", "heap_total_bytes", "heapTotalBytes"],
"heap_total_bytes": [
"heap_total_bytes",
"heapTotalBytes",
"total_heap",
"totalHeap",
],
}
if field in legacy:
aliases.update(legacy[field])
return list(aliases)
def _snake_to_camel(name: str) -> str:
parts = name.split("_")
return parts[0] + "".join(p.title() for p in parts[1:])
def _downsample(
points: list[tuple[float, float]], *, max_points: int
) -> list[tuple[float, float]]:
if len(points) <= max_points:
return points
# Even-bucket mean. Preserves shape better than nth-sample picking.
n = len(points)
bucket = n / max_points
out: list[tuple[float, float]] = []
i = 0
for k in range(max_points):
end = int((k + 1) * bucket)
end = min(end, n)
if end <= i:
continue
chunk = points[i:end]
ts = chunk[len(chunk) // 2][0]
val = statistics.fmean(v for _, v in chunk)
out.append((ts, val))
i = end
return out
def _slope_per_min(points: list[tuple[float, float]]) -> float | None:
"""Least-squares slope (units per minute). None if too few points."""
if len(points) < 2:
return None
xs = [t for t, _ in points]
ys = [v for _, v in points]
n = len(xs)
mean_x = sum(xs) / n
mean_y = sum(ys) / n
num = sum((xs[i] - mean_x) * (ys[i] - mean_y) for i in range(n))
den = sum((x - mean_x) ** 2 for x in xs)
if den == 0:
return None
slope_per_sec = num / den
return slope_per_sec * 60.0
+147
View File
@@ -0,0 +1,147 @@
"""OCR wrapper for UI tests + the `capture_screen` tool.
Auto-selects a reader in priority order:
1. `easyocr` (deep-learning, high quality on OLED screens — but ~100 MB
model download on first use).
2. `pytesseract` (requires system `tesseract` binary on PATH).
3. `null` — returns `""` with a warning. Tests fall back to log + image
evidence when OCR is unavailable.
Override via `MESHTASTIC_UI_OCR_BACKEND=easyocr|pytesseract|null|auto`
(default `auto`).
`ocr_text(png_bytes) -> str` is the only public entry point. The reader is
constructed lazily on first call and cached, so the easyocr cold-start cost
only hits once per process.
"""
from __future__ import annotations
import functools
import logging
import os
import shutil
import sys
from typing import Callable
log = logging.getLogger(__name__)
def _backend_choice() -> str:
return os.environ.get("MESHTASTIC_UI_OCR_BACKEND", "auto").lower()
@functools.lru_cache(maxsize=1)
def _reader() -> tuple[str, Callable[[bytes], str]]:
"""Return `(backend_name, callable)` for whichever OCR is available."""
choice = _backend_choice()
def _easyocr() -> tuple[str, Callable[[bytes], str]]:
import easyocr # type: ignore[import-untyped] # noqa: PLC0415
import numpy as np # type: ignore[import-untyped] # noqa: PLC0415
reader = easyocr.Reader(["en"], gpu=False, verbose=False)
def _run(png: bytes) -> str:
try:
import cv2 # type: ignore[import-untyped] # noqa: PLC0415
arr = np.frombuffer(png, dtype=np.uint8)
img = cv2.imdecode(arr, cv2.IMREAD_COLOR)
except ImportError:
# Fall back to PIL if cv2 isn't around.
from io import BytesIO # noqa: PLC0415
from PIL import Image # type: ignore[import-untyped] # noqa: PLC0415
img = np.array(Image.open(BytesIO(png)).convert("RGB"))
try:
results = reader.readtext(img, detail=0, paragraph=True)
except Exception as exc: # noqa: BLE001
log.warning("easyocr failed: %s", exc)
return ""
return "\n".join(str(r) for r in results)
return "easyocr", _run
def _pytesseract() -> tuple[str, Callable[[bytes], str]]:
from io import BytesIO # noqa: PLC0415
import pytesseract # type: ignore[import-untyped] # noqa: PLC0415
from PIL import Image # type: ignore[import-untyped] # noqa: PLC0415
if shutil.which("tesseract") is None:
raise ImportError("`tesseract` binary not on PATH")
def _run(png: bytes) -> str:
try:
return str(pytesseract.image_to_string(Image.open(BytesIO(png))))
except Exception as exc: # noqa: BLE001
log.warning("pytesseract failed: %s", exc)
return ""
return "pytesseract", _run
def _null() -> tuple[str, Callable[[bytes], str]]:
log.warning(
"OCR backend is null; install easyocr or tesseract for text extraction"
)
return "null", lambda _png: ""
if choice == "easyocr":
return _easyocr()
if choice == "pytesseract":
return _pytesseract()
if choice == "null":
return _null()
if choice != "auto":
print(
f"[ocr] unknown MESHTASTIC_UI_OCR_BACKEND={choice!r}; falling back to auto",
file=sys.stderr,
)
# auto mode
try:
return _easyocr()
except ImportError:
pass
try:
return _pytesseract()
except ImportError:
pass
return _null()
def ocr_text(png_bytes: bytes) -> str:
"""Run OCR on a PNG-encoded image and return the decoded text (possibly empty)."""
if not png_bytes:
return ""
_, run = _reader()
return run(png_bytes)
def backend_name() -> str:
"""Return the currently-selected backend name, initializing if necessary."""
name, _ = _reader()
return name
def warm() -> None:
"""Run one dummy inference so the easyocr cold-start cost is paid upfront.
Pytest session fixture calls this once so the first real capture doesn't
eat the model-load latency.
"""
# A 64×32 white PNG — decodes clean, no text to extract.
white_png = bytes.fromhex(
"89504e470d0a1a0a0000000d49484452000000400000002008060000007ccac28e"
"0000001c49444154785eedc1010d000000c2a0f74f6d0d370000000000000080"
"0b010000ffff030000000000000049454e44ae426082"
)
try:
ocr_text(white_png)
except Exception as exc: # noqa: BLE001
log.warning("ocr.warm() failed: %s", exc)
__all__ = ["backend_name", "ocr_text", "warm"]
+310
View File
@@ -0,0 +1,310 @@
"""Subprocess wrappers around the `pio` CLI.
Every PlatformIO interaction in this package funnels through `run()` so we
have a single place that owns timeouts, buffer sizes, JSON parsing, and the
"stderr on exit-0 is informational" convention.
`run()` has two execution paths:
* Fast path (default): `subprocess.run(capture_output=True)` — buffered, one
return; fine for sub-second pio calls like `pio --version` or
`pio project config --json-output`.
* Streaming path: when the `MESHTASTIC_MCP_FLASH_LOG` env var is set, each
output line is tee'd to that file as it arrives via a threaded reader.
The TUI tails the file to give live flash progress — otherwise a 3-minute
`pio run -t upload` is completely silent to the operator.
`hw_tools.py` shares the streaming helper via `pio._run_capturing()` so
esptool/nrfutil/picotool output also streams when the env var is set.
"""
from __future__ import annotations
import json
import os
import subprocess
import threading
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Sequence, TextIO
from . import config
# 10 MB matches the reference impl (jl-codes/platformio-mcp). Build output can
# be hundreds of KB; we'd rather keep it in memory than truncate.
_MAX_BUFFER = 10 * 1024 * 1024
# Per-operation defaults (seconds). None = no timeout.
TIMEOUT_DEFAULT = 120
TIMEOUT_PROJECT_CONFIG = 60
TIMEOUT_DEVICE_LIST = 15
TIMEOUT_BUILD = 900
TIMEOUT_UPLOAD = 600
class PioError(RuntimeError):
"""pio exited non-zero."""
def __init__(self, args: Sequence[str], returncode: int, stdout: str, stderr: str):
self.args = list(args)
self.returncode = returncode
self.stdout = stdout
self.stderr = stderr
tail = (stderr or stdout).strip().splitlines()[-20:]
super().__init__(
f"pio {' '.join(args)} failed with exit {returncode}:\n" + "\n".join(tail)
)
class PioTimeout(RuntimeError):
"""pio did not return within the timeout."""
@dataclass
class PioResult:
args: list[str]
returncode: int
stdout: str
stderr: str
duration_s: float
_FLASH_LOG_ENV = "MESHTASTIC_MCP_FLASH_LOG"
def _flash_log_path() -> Path | None:
"""Return the path to tee subprocess output to, or None if streaming off.
Controlled by `MESHTASTIC_MCP_FLASH_LOG`. `run-tests.sh` sets this to
`tests/flash.log`; the TUI tails that file so `pio run -t upload` shows
live progress in the pytest pane.
"""
raw = os.environ.get(_FLASH_LOG_ENV)
if not raw:
return None
return Path(raw)
def _run_capturing(
argv: Sequence[str],
*,
cwd: Path | None = None,
timeout: float | None = None,
tee_header: str | None = None,
extra_env: dict[str, str] | None = None,
) -> tuple[int, str, str, float]:
"""Run a subprocess, capture stdout+stderr, optionally tee to the flash log.
Returns `(returncode, stdout_str, stderr_str, duration_s)`. Raises
`subprocess.TimeoutExpired` on timeout (callers map this to their own
domain-specific error).
`extra_env` merges into the subprocess environment (parent env stays
intact). Used for `PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar.
Fast path: `subprocess.run(capture_output=True)` when no flash log is
configured (unchanged behavior).
Streaming path: `Popen` with line-buffered stdout+stderr pipes; two
reader threads accumulate into result strings AND append each line to
the flash log file. Stdout and stderr stay separate in the return value
(so `stderr_tail` still means stderr), but are interleaved in the log
file in the order they arrived — that's what a human wants to read.
"""
log_path = _flash_log_path()
t0 = time.monotonic()
env = None
if extra_env:
env = {**os.environ, **extra_env}
if log_path is None:
# Fast path — unchanged.
proc = subprocess.run(
list(argv),
cwd=str(cwd) if cwd else None,
capture_output=True,
text=True,
timeout=timeout,
env=env,
)
return (
proc.returncode,
proc.stdout or "",
proc.stderr or "",
time.monotonic() - t0,
)
# Streaming path: line-buffered Popen, threaded readers, tee to file.
# Ensure parent directory exists so the first tee write doesn't fail.
log_path.parent.mkdir(parents=True, exist_ok=True)
log_fh: TextIO | None = None
try:
log_fh = log_path.open("a", encoding="utf-8")
except OSError:
pass
# Append mode: the TUI truncates on startup, the session may produce
# many tee'd commands (erase + flash + factory-reset response), and
# we want all of them chronologically in one log.
proc = subprocess.Popen( # noqa: S603
list(argv),
cwd=str(cwd) if cwd else None,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
bufsize=1, # line-buffered
env=env,
)
stdout_chunks: list[str] = []
stderr_chunks: list[str] = []
log_lock = threading.Lock()
def _append_log(line: str) -> None:
# Hold the lock briefly to serialize interleaved stdout/stderr writes
# so a half-written line from one stream doesn't get garbled by the
# other.
nonlocal log_fh
with log_lock:
if log_fh is None:
return
try:
log_fh.write(line)
log_fh.flush()
except OSError:
# Log file disappeared (umount, operator deleted the dir).
# Don't let that bubble up — the subprocess output is still
# collected in-memory for the return value.
try:
log_fh.close()
except OSError:
pass
log_fh = None
def _tee(stream, sink: list[str]) -> None:
try:
for line in stream:
sink.append(line)
_append_log(line)
except Exception:
pass
# Header line so the operator can tell commands apart in the log.
if tee_header:
_append_log(f"\n--- {tee_header} (start)\n")
assert proc.stdout is not None and proc.stderr is not None
t_out = threading.Thread(
target=_tee, args=(proc.stdout, stdout_chunks), daemon=True
)
t_err = threading.Thread(
target=_tee, args=(proc.stderr, stderr_chunks), daemon=True
)
t_out.start()
t_err.start()
# `Popen.wait` with a timeout is the cleanest way to get TimeoutExpired.
try:
proc.wait(timeout=timeout)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait()
# Drain readers before re-raising so we don't leave threads behind.
t_out.join(timeout=2)
t_err.join(timeout=2)
raise
t_out.join()
t_err.join()
duration = time.monotonic() - t0
if tee_header:
_append_log(f"--- {tee_header} (exit {proc.returncode} in {duration:.1f}s)\n")
try:
return (
proc.returncode,
"".join(stdout_chunks),
"".join(stderr_chunks),
duration,
)
finally:
if log_fh is not None:
try:
log_fh.close()
except OSError:
pass
def run(
args: Sequence[str],
*,
cwd: Path | None = None,
timeout: float | None = TIMEOUT_DEFAULT,
check: bool = True,
extra_env: dict[str, str] | None = None,
) -> PioResult:
"""Invoke `pio <args>` and return captured output.
`cwd` defaults to the firmware root. `check=True` raises `PioError` on
non-zero exit; set `check=False` to inspect `returncode` manually.
`extra_env` merges into the subprocess environment — used for
`PLATFORMIO_BUILD_FLAGS=-DDEBUG_HEAP=1` and similar build-time
toggles that can't be expressed as command-line args.
If `MESHTASTIC_MCP_FLASH_LOG` is set, output is also tee'd to that file
line-by-line as it arrives (for live flash progress in the TUI).
"""
binary = str(config.pio_bin())
work_dir = cwd or config.firmware_root()
full = [binary, *args]
try:
rc, stdout, stderr, duration = _run_capturing(
full,
cwd=work_dir,
timeout=timeout,
tee_header=f"pio {' '.join(args)}",
extra_env=extra_env,
)
except subprocess.TimeoutExpired as exc:
raise PioTimeout(f"pio {' '.join(args)} timed out after {timeout}s") from exc
result = PioResult(
args=list(args),
returncode=rc,
stdout=stdout,
stderr=stderr,
duration_s=duration,
)
if check and result.returncode != 0:
raise PioError(args, result.returncode, result.stdout, result.stderr)
return result
def run_json(
args: Sequence[str],
*,
cwd: Path | None = None,
timeout: float | None = TIMEOUT_DEFAULT,
):
"""Run pio with `--json-output` appended and parse the result."""
full = list(args)
if "--json-output" not in full:
full.append("--json-output")
res = run(full, cwd=cwd, timeout=timeout, check=True)
if not res.stdout.strip():
raise PioError(args, 0, res.stdout, res.stderr or "pio returned empty output")
try:
return json.loads(res.stdout)
except json.JSONDecodeError as exc:
raise PioError(
args, 0, res.stdout[:2000], f"invalid JSON from pio: {exc}"
) from exc
def tail_lines(text: str, n: int = 200) -> str:
"""Last `n` lines of `text`, joined with newlines. Empty string stays empty."""
if not text:
return ""
lines = text.splitlines()
return "\n".join(lines[-n:])
@@ -0,0 +1,19 @@
"""Persistent device-log capture.
Singleton `Recorder` subscribes once to the meshtastic pubsub fan-out
(`meshtastic.log.line`, `meshtastic.receive.*`, `meshtastic.connection.*`)
and appends to four JSONL files under `mcp-server/.mtlog/`. Pubsub is
process-global so a single subscription captures every active interface
(serial / TCP / BLE) without any per-connection bookkeeping.
The recorder is opt-in-by-import: importing this package is a no-op; call
`get_recorder().start()` (which `server.py` does at FastMCP app init) to
begin writing. `pause()` / `resume()` exist for the rare case the user
wants a clean stretch of file (e.g. capturing a known-good baseline).
"""
from __future__ import annotations
from .recorder import Recorder, get_recorder
__all__ = ["Recorder", "get_recorder"]
@@ -0,0 +1,309 @@
"""Best-effort parsers for log lines and telemetry packets.
Two flavors of log line cross our pubsub subscription:
1. Text-mode path (debug_log_api disabled): the meshtastic Python lib
accumulates bytes between protobuf frames and emits the full
firmware-formatted line, e.g.
"INFO | 12:34:56 12345 [Main] Booting"
— level, HH:MM:SS, uptime seconds, thread bracket, then message.
2. LogRecord protobuf path (debug_log_api enabled): the lib calls
`_handleLogLine(record.message)` with ONLY the message body. The
level/source/time fields on the LogRecord are dropped before
pubsub fan-out. We get e.g. just "Booting".
Both arrive on `meshtastic.log.line`. The parser tries to recover a
level + thread when the prefix is present and falls back to level=None
otherwise. Consumers who want level filtering on protobuf-mode hosts
should grep the raw `line` field instead.
Telemetry: `meshtastic.receive.telemetry` packets carry one of several
metric variants in `packet["decoded"]["telemetry"]`. We flatten the
chosen variant into a {field: value} dict so callers don't have to
know the protobuf shape.
"""
from __future__ import annotations
import re
from typing import Any
# Match: LEVEL | HH:MM:SS UPTIME [Thread] message
# HH:MM:SS may be ??:??:?? when RTC isn't valid. The level alternation
# below is the canonical list — DebugConfiguration.h's MESHTASTIC_LOG_LEVEL_*
# macros must stay in sync with these strings.
_LINE_RE = re.compile(
r"""
^
(?P<level>DEBUG|INFO\ |WARN\ |ERROR|CRIT\ |TRACE|HEAP\ )
\s*\|\s*
(?P<clock>(?:\d{2}:\d{2}:\d{2})|(?:\?{2}:\?{2}:\?{2}))
\s+
(?P<uptime>\d+)
\s+
(?:\[(?P<thread>[^\]]+)\]\s+)?
(?P<msg>.*)
$
""",
re.VERBOSE,
)
# DEBUG_HEAP build prepends `[heap N] ` to every message body, AFTER the
# thread bracket. See src/RedirectablePrint.cpp:175.
_HEAP_PREFIX_RE = re.compile(r"^\[heap\s+(?P<heap>\d+)\]\s+(?P<rest>.*)$")
# OSThread leak/free detection. See src/concurrency/OSThread.cpp:89-91.
# Format: "------ Thread NAME leaked heap A -> B (delta) ------"
# "++++++ Thread NAME freed heap A -> B (delta) ++++++"
_THREAD_HEAP_RE = re.compile(
r"""
^[\-+]+\s*
Thread\s+(?P<thread>\S+)\s+
(?P<kind>leaked|freed)\s+heap\s+
(?P<before>-?\d+)\s*->\s*(?P<after>-?\d+)\s+
\((?P<delta>-?\d+)\)
""",
re.VERBOSE,
)
# Power.cpp:908 periodic heap status (DEBUG_HEAP only).
# Format: "Heap status: FREE/TOTAL bytes free (DELTA), running R/N threads"
_HEAP_STATUS_RE = re.compile(
r"""
Heap\s+status:\s+
(?P<free>\d+)\s*/\s*(?P<total>\d+)\s+bytes\s+free
(?:\s+\((?P<delta>-?\d+)\))?
""",
re.VERBOSE,
)
_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
_HEAP_BRACKET_RE = re.compile(r"^heap\s+(?P<heap>\d+)$")
def parse_log_line(line: str) -> dict[str, Any]:
"""Best-effort decompose a raw firmware log line.
Returns a dict with at least `line` (the original, unmodified — ANSI
codes preserved for fidelity). Adds `level`, `tag`, `clock`,
`uptime_s`, and `msg` when the full prefix is present.
Handles two firmware quirks:
- LogRecord.message can carry ANSI color escapes from RedirectablePrint
(the BLE/StreamAPI path inherited the colored body in some builds).
We strip ANSI before regex matching so the prefix survives.
- DEBUG_HEAP injects `[heap N]` after the thread bracket. When NO
thread name is set, the heap takes the thread bracket position —
looks like `[heap 12345] msg`. We detect that shape and move it
out of `tag` and into `heap_free`.
DEBUG_HEAP-build extras (when `[heap N]` is injected): `heap_free`
(bytes), and when a `Thread X leaked|freed heap` line is recognized,
`heap_event` = {kind, thread, before, after, delta}.
Never raises.
"""
out: dict[str, Any] = {"line": line}
if not line:
return out
# Strip ANSI escapes BEFORE any regex matching. The original `line`
# stays in `out["line"]` for fidelity / future grep.
clean = _ANSI_RE.sub("", line)
m = _LINE_RE.match(clean)
msg: str | None = None
if m:
level = m.group("level").rstrip()
out["level"] = level
out["clock"] = m.group("clock")
try:
out["uptime_s"] = int(m.group("uptime"))
except (TypeError, ValueError):
out["uptime_s"] = None
thread = m.group("thread")
if thread:
# If "thread" is actually the heap prefix taking the bracket
# position (DEBUG_HEAP build, no thread set), capture heap
# and leave tag unset.
hb = _HEAP_BRACKET_RE.match(thread.strip())
if hb:
try:
out["heap_free"] = int(hb.group("heap"))
except (TypeError, ValueError):
pass
else:
out["tag"] = thread
msg = m.group("msg")
out["msg"] = msg
else:
# No prefix — bare LogRecord.message body. Inspect the whole
# line for DEBUG_HEAP-style content; the heap-prefix and
# thread-leak patterns can survive on either path.
msg = clean
# DEBUG_HEAP per-line heap prefix: `[heap 92344] message`.
# Sits AFTER the thread bracket and BEFORE the message body, but
# for bare LogRecord lines it's at the start. Match it at the
# head of `msg`.
if msg:
hp = _HEAP_PREFIX_RE.match(msg)
if hp:
try:
out["heap_free"] = int(hp.group("heap"))
except (TypeError, ValueError):
pass
else:
# Strip the prefix from `msg` so a grep on the message
# body doesn't have to know about it.
out["msg"] = hp.group("rest")
msg = hp.group("rest")
# Thread-level leak/free detection.
thr = _THREAD_HEAP_RE.search(msg)
if thr:
try:
out["heap_event"] = {
"kind": thr.group("kind"),
"thread": thr.group("thread"),
"before": int(thr.group("before")),
"after": int(thr.group("after")),
"delta": int(thr.group("delta")),
}
except (TypeError, ValueError):
pass
# Power.cpp periodic "Heap status: F/T bytes free (D), running ..."
hs = _HEAP_STATUS_RE.search(msg)
if hs:
try:
out["heap_free"] = int(hs.group("free"))
out["heap_total"] = int(hs.group("total"))
if hs.group("delta") is not None:
out["heap_delta"] = int(hs.group("delta"))
except (TypeError, ValueError):
pass
return out
# -- Telemetry ----------------------------------------------------------
# Order matters: meshtastic-python decoded packets use the protobuf
# `oneof variant` field name (snake_case) as the dict key.
_TELEMETRY_VARIANTS = (
("device_metrics", "device"),
("local_stats", "local"),
("environment_metrics", "environment"),
("power_metrics", "power"),
("air_quality_metrics", "airQuality"),
("health_metrics", "health"),
("host_metrics", "host"),
)
def extract_telemetry(packet: dict[str, Any]) -> dict[str, Any] | None:
"""Pull the telemetry variant + flat fields out of a `meshtastic.receive.telemetry`
packet. Returns None when the shape isn't what we expect — so the
caller can fall back to a generic packets.jsonl row.
"""
if not isinstance(packet, dict):
return None
decoded = packet.get("decoded")
if not isinstance(decoded, dict):
return None
telem = decoded.get("telemetry")
if not isinstance(telem, dict):
return None
# The Python lib produces dict-of-camelCase keys via MessageToDict.
# Try both camelCase and snake_case to be robust to lib version drift.
for snake, label in _TELEMETRY_VARIANTS:
camel = _snake_to_camel(snake)
for key in (snake, camel):
value = telem.get(key)
if isinstance(value, dict):
return {
"variant": label,
"fields": {k: _scalarize(v) for k, v in value.items()},
"time": telem.get("time"),
}
return None
def _snake_to_camel(name: str) -> str:
parts = name.split("_")
return parts[0] + "".join(p.title() for p in parts[1:])
def _scalarize(value: Any) -> Any:
"""Keep telemetry fields JSON-friendly. Lists/dicts pass through
untouched; bytes -> hex string; protobuf enums occasionally arrive
as ints (fine) or strings (also fine)."""
if isinstance(value, (bytes, bytearray, memoryview)):
return bytes(value).hex()
return value
# -- Generic packet summary ---------------------------------------------
def summarize_packet(
packet: dict[str, Any], *, payload_hex_len: int = 64
) -> dict[str, Any]:
"""Reduce a packet dict to a stable, queryable summary. Drops the
full payload bytes — the recorder records summaries, not pcaps.
"""
if not isinstance(packet, dict):
return {"raw_type": type(packet).__name__}
decoded = packet.get("decoded") if isinstance(packet.get("decoded"), dict) else {}
portnum = decoded.get("portnum") if isinstance(decoded, dict) else None
payload = decoded.get("payload") if isinstance(decoded, dict) else None
payload_hex = None
payload_size = None
if isinstance(payload, (bytes, bytearray, memoryview)):
b = bytes(payload)
payload_size = len(b)
payload_hex = b[:payload_hex_len].hex() if b else ""
elif isinstance(payload, str):
# Some decoded payloads (text messages) come as decoded strings.
payload_size = len(payload)
payload_hex = None # not bytes
return {
"from_node": packet.get("fromId") or packet.get("from"),
"to_node": packet.get("toId") or packet.get("to"),
"portnum": portnum,
"hop_limit": packet.get("hopLimit"),
"want_ack": packet.get("wantAck"),
"rx_rssi": packet.get("rxRssi"),
"rx_snr": packet.get("rxSnr"),
"channel": packet.get("channel"),
"id": packet.get("id"),
"payload_size": payload_size,
"payload_hex_prefix": payload_hex,
}
# -- Interface identification ------------------------------------------
def interface_label(interface: Any) -> dict[str, Any]:
"""Stable identifier for the meshtastic interface that emitted an event.
Used as the `port`/`role` tag on every recorded row. SerialInterface
has `devPath`; TCPInterface has `hostname`+`portNumber`; BLEInterface
has `address`. Falls back to the class name when none of those exist.
"""
if interface is None:
return {"port": None, "role": None}
dev_path = getattr(interface, "devPath", None)
if dev_path:
return {"port": str(dev_path), "role": "serial"}
hostname = getattr(interface, "hostname", None)
if hostname:
port_num = getattr(interface, "portNumber", None)
endpoint = f"tcp://{hostname}:{port_num}" if port_num else f"tcp://{hostname}"
return {"port": endpoint, "role": "tcp"}
address = getattr(interface, "address", None)
if address:
return {"port": str(address), "role": "ble"}
return {"port": type(interface).__name__, "role": None}

Some files were not shown because too many files have changed in this diff Show More