a640d44b5cf39b13be3ceffa3e2ec28b133617f0
597
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
   
|
a640d44b5c |
Add T-Beam BPF - 144-148 Mhz LoRa @ ~37 dBm (5 Watts) (#10558)
* Add T-Beam BPF (144-148 Mhz LoRa) * minor correction to fix compiler warnings * Add T-Beam BPF (144-148 Mhz LoRa) * minor correction to fix compiler warnings * Add ITU regions for this device and make GPS work. * Switch pin after defining it as output Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Lora CS is indeed 1, SD Card CS is 10 * Include the back option. * Fix compilation with pioarduino (USB_MODE) * Default ham to narrow_fast * Default PROFILE_HAM to slot 17 This is an appropriate default in the USA but not the EU. The slot override really should follow the region itself, not the regionprofile. * Fix for ITU 2/3 split * Add ITU region options to MenuAction enum * Add HAS_HAM_2M definition to variant headers for 2M support * Re-add PROFILE_HAM regionprofile Accidentally removed in last merge * Trunk fmt * Initial default slots * Switch to TinyFast Still need to flesh out the default channels * Adjust slotOverrides for TinyFast * RadioLib doesn't accept 15.625 kHz Use 15.6 instead * Set RF95 pins for T-Beam Supreme May cause regressions!! * Remove other-variant changes (BPF-only) These have been moved to other PRs * Remove mismatch guarding (we need a more comprehensive approach) * Add comment back * Add template for PA curve * This is a 5-6W radio!! Add TX_GAIN_LORA * Trunk fmt because NomDeTom hates emdashes --------- Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
 
|
1e982fa78c |
Sign plaintext packets in licensed mode (#10969)
* feat: sign licensed plaintext packets Preserve and publish identity keys in licensed mode so existing XEdDSA signatures can authenticate plaintext traffic. Sign licensed broadcasts and direct messages when they fit, while keeping PKI encryption disabled and normal routing behavior unchanged. * fix(security): persist licensed channel sanitation * fix(security): close licensed migration lifecycle gaps * fix(security): address licensed signing review feedback * test: restore signing globals from Unity teardown * fix(baseui): allow confirmed ham region selection * fix(baseui): guard region picker validation * style: trunk fmt --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Austin <vidplace7@gmail.com> |
||
 
|
45cb8e7500 |
feat: preserve normal radio profiles across event firmware (#11110)
* feat: isolate event radio profiles * fix: preserve identity on degraded config boot * fix: guard event profile storage * style: align event profile storage names * fix: discard event profile on normal boot * refactor: simplify event profile cleanup * fix: limit inactive profile migration to event firmware * fix(event): make event-profile capacity check portable across filesystems The USERPREFS_EVENT_MODE storage preflight called FSCom.totalBytes() and FSCom.usedBytes(), which only exist on ESP32's LittleFS wrapper. Every other backend failed to compile once event mode was enabled: error: 'class InternalFileSystem' has no member named 'totalBytes' (nRF52) error: no member named 'totalBytes' in 'fs::FS' (Portduino) This was not caught earlier because the event block is behind #if USERPREFS_EVENT_MODE, and the existing unit tests only cover the pure helpers, which compile identically either way. Add fsTotalBytes()/fsUsedBytes() to FSCommon and implement them per backend: littlefs v1 traversal for nRF52 (Adafruit_LittleFS) and STM32 (STM32_LittleFS), FSInfo for RP2040, statvfs for Portduino, and the native methods for ESP32 and nRF54L15. The nRF52/STM32 path reports "full" if the traversal errors so the capacity check fails safe. Verified with USERPREFS_EVENT_MODE=1: native-macos test_event_profile_storage passes 5/5, and heltec-v3, rak4631, rak11310 and rak3172 all build clean. * fix(event): use std::filesystem for Portduino capacity, bump native-suite-count Two CI fixes: - native-windows has no <sys/statvfs.h>, so the Portduino branch of fsTotalBytes()/fsUsedBytes() broke the Windows build. Switch to std::filesystem::space(), which is already used under ARCH_PORTDUINO in HostMetrics.cpp and works on both POSIX and MinGW. Errors report "full" so the capacity check still fails safe. - This PR adds test/test_event_profile_storage, taking test/ from 40 to 41 suite directories, which trips the native-suite-count reconciliation check. * fix(event): scope boot-write deferral to the radio profile, address review Review feedback: - Copilot: saveProto()'s boot-write deferral applied to every file, so loadFromDisk()'s recovery writes (e.g. restoring owner fields into devicestate) were silently dropped and never retried, because the ctor CRC baselines are computed after loadFromDisk(). Deferral now applies only to the radio-profile files, via isRadioProfileFile(). - jp-bennett: eventConfigFromStandard() wrapped a struct copy plus one field assignment in a header helper with its own unit test. Inlined at its single call site and removed; the behaviour is covered end-to-end by hardware validation instead (RAK4631 normal -> event -> normal preserves NodeNum and public key while swapping LoRa, and restores the original channel byte-identically). - jp-bennett: the active-backup encrypted-storage migration ran for normal builds too, which changed non-event behaviour and contradicted the PR's stated event-only scope. Scoped to USERPREFS_EVENT_MODE; the adjacent block already covers the inactive standard backup in event builds. New tests (all verified to fail under mutation, not vacuous): - test_event_paths_never_collide_with_standard_or_shared_files: the core safety property. A path-table slip would make event firmware overwrite the user's real config, channels or backup, or capture a shared file like devicestate/nodedb. Nothing asserted this before. - test_only_radio_profile_files_defer_boot_writes: guards the deferral fix above so re-widening it fails loudly. - test_event_reservation_fits_smallest_supported_filesystem: the reservation is compile-time but must fit filesystems as small as 14 KiB (STM32WL) and 28 KiB (nRF52840). Protobuf growth pushing it past those would silently stop event profiles persisting - the exact failure mode confirmed by fault injection on a RAK4631. Verified with USERPREFS_EVENT_MODE both on and off: 7/7 tests pass, and rak4631, heltec-v3, rak11310 and rak3172 all build clean. * fix: preserve event profile storage recovery --------- Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Benjamin Faershtein <benjaminfaershtein@Benjamins-MacBook-Pro-2.local> |
||
 
|
8e104a909b |
fix(admin): persist TAK module config (team color / member role) (#11216)
Setting TAK team/role ACKed and rebooted but stored nothing: handleSetModuleConfig had no tak case, saveToDisk never set has_tak, and handleGetModuleConfig had no TAK_CONFIG case. Add all three, plus a native suite sweeping every ModuleConfig submessage through set -> save -> load -> get and a TAK value-fidelity suite. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
efc9708236 |
Add userpref for the packet signature policy default (#11144)
Co-authored-by: Austin <vidplace7@gmail.com> |
||
|
Thomas GöttgensandGitHub
|
f09f2b1366 | Gate the PKI-decrypt key on signer-proven, not the opportunistic cache (#11116) | ||
|
Ben Meadors
|
7d954e668d |
Merge branch 'develop' into codex/packet-auth-policy
Resolve conflicts against the NodeDB signer/key primitives (#11050) and the admin-key PKI decrypt budget (#11100). - NodeDB: drop this branch's hasSeenXeddsaSigner in favour of develop's isKnownXeddsaSigner. They answer the same question, but develop's reads the dedicated warm signer bit (warmSignerOf) rather than the WarmProtected category, and TrafficManagementModule already depends on it. Keep develop's copyPublicKey/copyPublicKeyAuthoritative, isVerifiedSignerForKey and commitRemoteKey/KeyCommitTrust. - checkXeddsaReceivePolicy: keep this branch's Strict/Balanced/Compatible policy, which is a superset of develop's balanced-only downgrade gate, and call isKnownXeddsaSigner from it. develop's !pki_encrypted term is dropped because the policy returns early for PKI packets before that check. - perhapsDecode: keep develop's key resolution (NodeDB then pending-key, only for real PKI candidates) plus its admin-key token bucket, and re-apply this branch's pkiAttempted flag feeding the DECODE_OPAQUE verdict. Keep both passesRoutingAuthGate and adminKeyFallbackAllowed/Refund. - test_A17: model eviction the way NodeDB actually does it, passing the warm signer bit as well as the XeddsaSigner category, since isKnownXeddsaSigner reads the former. Native suite: 38 suites, 743/743 cases, no sanitizer findings. |
||
|
1872e5b306 |
TMM extend to ephemeral 3rd tier store (#11050)
* Harden TMM NodeInfo direct-response: staleness, key hygiene, throttle The NodeInfo direct-response path answered queries on behalf of other nodes from an unauthenticated, never-expiring cache while suppressing the genuine request (STOP). That let a long-gone or forged identity be served as a fresh-looking, authoritative reply indefinitely. Three fixes: - Staleness gate: refuse to spoof a reply for a node not actually heard within ~6h (PSRAM cache via lastObservedMs; NodeDB fallback via sinceLastSeen, which tolerates last_heard==0 when the clock is unset). A stale entry now falls through so the real request propagates instead of being answered for a dead node. - Key hygiene: mirror NodeDB::updateUser() PKI checks when caching overheard NodeInfo - reject a packet advertising our own public key, and pin keys (drop a NodeInfo whose key mismatches an already-known key from NodeDB or from our own cache). Prevents cache poisoning / key substitution via the spoofed-reply path. - Response throttle: at most one spoofed reply per target node per 30s. Direct responses bypass the per-sender rate limiter (they STOP the request first) and the reply target is attacker-controlled, so this bounds airtime exhaustion and reflected floods. Recorded in a new NodeInfoPayloadEntry.lastResponseMs (PSRAM; self-expiring timestamp compare, no sweep needed). Tests: native NodeDB-fallback staleness (stale drops, fresh serves) plus PSRAM-guarded staleness, throttle, and key-mismatch cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Add TODO(T1-T9) markers for code-review findings on NodeInfo hardening Comment-only. Tags each site flagged in the review of the direct-response throttle/staleness/key-hygiene change so the follow-ups are visible in-context: T1 throttle black-holes distinct requestors; T2 per-target key doesn't bound aggregate TX; T3 non-PSRAM fallback unthrottled; T4 millis-wrap defeats the staleness gate; T5 redundant second O(n) scan for the throttle stamp; T6 duplicated 32-byte key compares; T7 sinceLastSeen() called twice; T8 two 6h constants can desync; T9 clockMs()==0 collides with the 0 sentinel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Address code-review cleanups T6, T7, T8 in NodeInfo direct-response T8: derive kNodeInfoMaxServeAgeSecs from kNodeInfoMaxServeAgeMs so the PSRAM and NodeDB-fallback staleness windows cannot desync. T6: collapse the three duplicated 32-byte key compares in cacheNodeInfoPacket (owner-impersonation + NodeDB pin + cache TOFU pin) into one pubKeysEqual() helper, so the compare lives in one place. T7: compute sinceLastSeen(node) once on the fallback staleness path so the tested age and the logged age cannot diverge (it calls getTime() each time). Remaining review items still marked in-code: T1/T2/T3 (throttle design), T4/T9 (millis-wrap + 0-sentinel), T5 (redundant throttle-stamp scan). Native test_traffic_management suite: 48/48 passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Throttle the NodeDB fallback direct-response path (T3) The per-target NodeInfo response throttle lived only in the PSRAM NodeInfoPayloadEntry, so non-PSRAM boards - which answer from the NodeDB fallback path - emitted spoofed direct replies with no rate limit at all, leaving the airtime/reflection surface the throttle exists to close. Add a single module-global fallback stamp (nodeInfoFallbackLastResponseMs, 4 bytes, guarded by cacheLock) that throttles the fallback path to one spoofed reply per window across all targets. Coarser than the PSRAM per-target throttle, but the fallback path has no per-node slot to stamp, and a global cap still bounds spoofed transmissions - which is the point. Add a native test exercising the fallback throttle window. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Make NodeInfo staleness/throttle wrap- and sentinel-safe (T4, T9) T9: the millis-based fields that use 0 as a "never" sentinel (lastObservedMs, lastResponseMs, nodeInfoFallbackLastResponseMs) could be stamped with a literal 0 during the one-millisecond clockMs()==0 instant at each ~49.7-day wrap, momentarily colliding with the sentinel and disabling the staleness/throttle gate for that entry. Route all such stamps through a new nowStampMs() that maps 0 -> 1; the 1 ms skew is irrelevant to every window these fields feed. T4: the staleness gate's modular age compare is only unambiguous while true age < 2^32 ms (~49.7 days); an entry that lingered unrefreshed that long could wrap to a small age and read as fresh, defeating the gate. Add a NodeInfo eviction pass to the maintenance sweep that drops entries past the serve window, so an entry is removed long before its age can approach the wrap boundary. This also frees slots holding stale identities. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Stamp PSRAM throttle entry by captured index, not a rescan (T5) The post-send throttle stamp did a second full O(n) scan of the PSRAM NodeInfo array under a fresh lock, even though findNodeInfoEntry() had already located the slot at the top of shouldRespondToNodeInfo(). Capture the slot index during that initial lookup and address the entry directly on the stamp path. Because the cache lock is released between the two accesses, the slot could have been evicted or reused, so re-validate node == p->to under the lock before stamping. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Document accepted per-target throttle tradeoffs (T1, T2) Convert the T1/T2 TODO markers into a NOTE: the per-target PSRAM throttle keying and its lack of an aggregate bound are deliberate design choices, not pending work. A distinct requestor being throttled for one window is harmless on a redundant mesh, and keeping the PSRAM path per-target preserves throughput for legitimate multi-target responders (the fallback path already bounds aggregate via its global stamp). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Track signed-provenance of cached NodeInfo public keys Add keySignerProven to NodeInfoPayloadEntry, set when an observed NODEINFO frame's XEdDSA signature was verified (mp.xeddsa_signed). This distinguishes a trust-on-first-use key from one proven to belong to a signer. The flag is monotonic - once proven it stays proven, and the existing key-pin checks forbid the underlying key from changing - so a later unsigned frame cannot downgrade it. A signature can only be verified against a key we already held, so a first-contact key is always TOFU until a later signed frame upgrades it. Groundwork for using the TMM cache as a last-resort public-key source, where this flag serves as a trust tier. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Draw public keys from the TMM NodeInfo cache as a last resort Add TrafficManagementModule::copyPublicKey() and consult it from NodeDB::copyPublicKey() after the hot (NodeInfoLite) and warm (WarmNodeStore) tiers miss. This extends the pool of peers the node can encrypt to: a key the NodeInfo direct-response cache overheard for a node that has since aged out of both NodeDB tiers can still be used. The getter reports whether the key is signer-proven or trust-on-first-use. NodeDB serves TOFU keys here too - the same first-contact trust NodeDB already applies in updateUser() - so the pool actually expands to new long-tail nodes rather than only re-confirming known keys. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Retain keyed NodeInfo entries as a pubkey pool; trust-tier eviction Now that NodeDB::copyPublicKey() draws keys from the NodeInfo cache, the flat 6 h serve-window eviction would discard useful keys the moment a node stops being servable. Split retention: an entry carrying a 32-byte public key is kept for kNodeInfoKeyRetentionMs (7 d), while a keyless entry still expires at the serve window. Both windows stay well under the ~49.7-day millis wrap, preserving the T4 wrap-safety guarantee. Make LRU victim selection trust-tiered to match: a keyless slot is sacrificed before any keyed one, and a trust-on-first-use key before a signer-proven key; within a tier the oldest loses. Mirrors WarmNodeStore's keyed-first admission so the most valuable keys are the stickiest under memory pressure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Test signed-provenance flag and copyPublicKey key-pool source PSRAM-gated tests (run on the ESP32 build, alongside the existing NodeInfo PSRAM tests): - copyPublicKey serves a TOFU key learned from an unsigned NodeInfo and reports signerProven=false - a later signature-verified NodeInfo upgrades provenance to signer-proven while the pinned key bytes stay unchanged - copyPublicKey reports a miss for an uncached node Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Inherit signer provenance from NodeDB when caching a re-found node When the TrafficManagement NodeInfo cache re-caches a node, mark its key signer-proven if NodeDB already knows the node as a verified signer for that same key - even if this particular (unicast/unsigned) frame carried no signature. Previously the flag only upgraded on a frame we verified ourselves, so a node we had already proven elsewhere looked TOFU here. Add NodeDB::isVerifiedSignerForKey(), which checks both tiers - the hot store's signed bitfield and the warm tier's cached signer bit - and requires the key to match so a rotated/mismatched key never inherits a stale verdict. Add WarmNodeStore::isVerifiedSigner() to expose the warm signer bit (the rebase onto develop added the bit itself; this surfaces it for lookups). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Lock NodeInfoPayloadEntry packing with a size assert keySignerProven cost zero bytes: sourceChannel, the two bools, and decodedBitfield are four 1-byte fields that fill a single 4-byte tail word (struct alignment is 4), so the flag consumed former padding rather than growing the 2000-entry PSRAM array. Add a static_assert pinning the entry to sizeof(meshtastic_User) + 20 so a future 5th trailing byte - which would open a fresh word (~8 KB PSRAM across the array) - fails the build instead of silently costing memory, prompting new flags to be packed into existing bytes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Pack NodeInfo cache booleans into 1-bit fields Fold hasDecodedBitfield and keySignerProven into adjacent uint8_t:1 bitfields so they share a single byte, reserving 6 spare bits for future flags without growing the 2000-entry PSRAM array. sizeof is unchanged (still one packed tail word, sizeof(meshtastic_User) + 20); access is by name exactly as before, so no call sites change. Reorders decodedBitfield ahead of the flags so the two 1-bit members stay adjacent and the compiler packs them together. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Gate NodeInfo replay on signer-proven provenance (compile-time, default on) Add TMM_NODEINFO_REPLAY_REQUIRE_SIGNED (default 1): the direct-response path now only spoofs a reply for a node whose key is signer-proven, in addition to the staleness gate - so replay is based on flagging AND staleness. Replay is a courtesy feature, and vouching for an unverified (trust-on-first-use) identity to other nodes is the risk this closes, so signed-only is the safer default. Define the macro to 0 at build time to also serve fresh TOFU-only nodes. Both paths are gated: the PSRAM path checks the cached keySignerProven flag, the NodeDB fallback checks nodeInfoLiteHasXeddsaSigned(node). The effective gate (TMM_NODEINFO_REPLAY_SIGNED_GATE) is bypassed when PKI is excluded from the build, since nothing can be signed there and the courtesy feature would otherwise be disabled outright. Tests: existing reply-expecting tests establish signer-proven state (a new markKeySignerProvenForTest hook for the PSRAM cache, the NodeDB signed bit for the fallback path); add fallback and PSRAM tests asserting an unsigned/TOFU node is withheld under the default gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Rehydrate re-admitted node names from the TMM NodeInfo cache The warm tier keeps an evicted node's key but not its name (the 40-byte record has no room), so a re-admitted long-tail node is nameless until its next NodeInfo broadcast. The TrafficManagement NodeInfo cache is much larger (2000 PSRAM entries) and commonly still holds the full User, so use it as a name reservoir the warm tier structurally cannot be. On re-admission (getOrCreateMeshNode), after the warm-tier restore, copy the cached identity from the TMM cache via a new copyUser() getter - but only when its cached key matches the key just restored from warm, so a name never attaches to a different identity than the one we encrypt to (key-matched trust). Guarded by HAS_TRAFFIC_MANAGEMENT and a null check; a no-op without the PSRAM cache or when no key is present. CopyUserToNodeInfoLite sets only user-related bits, so the warm-restored signer bit is preserved. Limitations (by design): PSRAM-only, and the TMM cache is RAM-only, so this helps within an uptime session, not across reboot. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Drop non-portable NodeInfoPayloadEntry size assert (fixes pico build) The static_assert pinned sizeof(NodeInfoPayloadEntry) to sizeof(meshtastic_User) + 20, but nanopb packs the generated User struct differently per platform: on pico its size is not a multiple of 4, so alignment padding before the uint32 timestamps makes the overhead 22, not 20, and the assert failed the build (native happened to be +20 and passed, hiding it). The bitfield packing it was guarding still stands; replace the fixed-count assert with a comment, since no portable byte count exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Run the NodeInfo cache paths in native tests (TMM_HAS_NODEINFO_CACHE) The NodeInfo direct-response cache and everything layered on it - key pinning, signer provenance, staleness, throttle - was compiled only for ARCH_ESP32 + BOARD_HAS_PSRAM, so its entire test set was skipped by the native CI suite and ran nowhere but real hardware. Fold the scattered platform guards into one TMM_HAS_NODEINFO_CACHE macro that also enables the cache (plain heap, delete[] path already existed) for ARCH_PORTDUINO unit-test builds. Production portduino and embedded test builds are unchanged. Tests that exercise the NodeDB fallback path drop the cache explicitly via a new dropNodeInfoCacheForTest() hook, so both response paths are covered in one binary. Native suite: 60/60 (was 50 with the 10 cache tests compiled out). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit b2dafbbbd6501b42fc27175e599fb38be292b0e3) * Pin NodeInfo cache keys against the warm tier, not just the hot store cacheNodeInfoPacket() mirrored NodeDB::updateUser()'s key pin but checked only getMeshNode() - the hot store. updateUser's own pin effectively covers the warm tier too (getOrCreateMeshNode rehydrates the warm key before the check), so the mirror had a gap: for a node evicted to the warm tier whose cache slot was also gone, an attacker's NodeInfo with a bogus key passed the pin, and the cache's TOFU pin then locked the genuine node's frames out until the poisoned entry aged away. Split the authoritative lookup out of NodeDB::copyPublicKey() as copyPublicKeyAuthoritative() (hot store, then warm tier - never the opportunistic TMM tier, which would compare the cache against itself) and pin against that. Test: warm-tier-only key rejects a mismatching NodeInfo and accepts the matching one. Native suite: 61/61. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit 28e880875ef1f8bf16bdf301a4e6036537a84b4b) * Convert NodeInfo cache clocks to uint8 tick counters Replace the entry's uint32 millis stamps (lastObservedMs, lastResponseMs) with three uint8 free-running tick clocks - the same modular scheme the UnifiedCache counters already use: obsTick 3 min/tick (12.8 h period) - replay staleness gate respTick 5 s/tick (21.3 min) - per-target response throttle retTick 1 h/tick (10.67 d) - retention TTL + LRU age Validity is an explicit flag bit (hasObserved / hasResponded), not a 0 sentinel, and the maintenance sweep saturates a stamp (clears its flag) once its window passes, so no stamp can age toward its ~256-tick aliasing horizon. That retires the wrap/sentinel special cases (T4, T9) - nowStampMs() survives only for the fallback path's module-global millis stamp. obsTick is separated from retention by design: only a genuinely heard NODEINFO frame stamps it, so later membership-based retention refresh can never make a silent node look servable. Also drops lastObservedRxTime, which was only echoed in a debug log. Entry shrinks 136 -> 128 B; the 2000-entry array 272 kB -> 256 kB, 8 kB below the original develop footprint while keeping the throttle and retention features. Tick granularity costs at most one tick per window (+-5 s on 30 s, +-3 min on 6 h, +-1 h on 7 d). Native suite: 61/61. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit e48ed9cfacfed0131b77ed2f1992e8720720d387) * Keep the NodeInfo cache a superset of NodeDB (seed + keep-alive) The cache learned identities only from overheard NODEINFO frames, so its membership was opportunistic: a NodeDB-tier node whose frame was never heard this boot had no entry (no key pin to protect it, no name to rehydrate), and the retention TTL evicted entries for nodes that still lived in the hot store or warm tier. Add an anti-entropy pass to the maintenance sweep (reconcileNodeInfoMembershipLocked): every hot-store / warm-tier node gets an entry - full identity from the hot store (hasFullUser), key-only from the warm tier - with signer provenance inherited key-matched, and NodeDB's key adopted wholesale on conflict. Member entries (isMember) are re-stamped each sweep, so they never age toward the retention TTL; when a node leaves both tiers the keep-alive stops and its entry ages out from its final member re-stamp. Membership also outranks key trust in LRU victim selection, and the reconcile pass skips seeding rather than churn one member out for another when hot+warm exceeds the cache. Two properties are load-bearing: - seeding/keep-alive never touch obsTick/hasObserved, so a seeded or retained entry is never served as a spoofed reply - only genuinely heard frames make a node servable; - copyUser() now requires hasFullUser, so a key-only warm seed can never stamp HAS_USER onto a nameless node via name-rehydration. Native suite: 64/64. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit 4f1b30d838b2f7b11173373b8f1ba97cf91bef6b) * Write-through NodeDB identity commits into the NodeInfo cache updateUser() is the single chokepoint through which every remote identity/key write enters NodeDB (key-verification keys stay in CryptoEngine's pending buffer until a NodeInfo carries them here), so one hook at its tail lets the NodeInfo cache reflect a commit immediately instead of waiting up to a minute for the reconcile sweep - which stays in place as the anti-entropy backstop. onNodeIdentityCommitted() upserts the full User: NodeDB's key is authoritative (a conflicting cached key is stale residue - replaced, provenance dropped), a keyless commit keeps an already-cached TOFU key, and signer provenance transfers only for the committed key. The observation stamp is never touched: knowledge is not observation, so a hook write can never make a never-heard node servable - covered by the new test alongside immediate copyUser/copyPublicKey visibility. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit e160b899728be8d951d4e7bdf5106c777832b917) * Purge TrafficManagement caches on explicit node removal NodeDB::removeNodeByNum() already forgets the node everywhere NodeDB owns - hot store, satellite stores, warm tier - but the TrafficManagement caches kept the deleted identity: the NodeInfo entry went on feeding the key pool (NodeDB::copyPublicKey) and name rehydration, and the unified slot kept its role/next-hop/dedup state, resurrecting the node on next contact. Add purgeNode(): clears both the unified cache slot and the NodeInfo cache entry, called from removeNodeByNum() alongside the warm-tier removal. Removal means full removal; passive eviction never calls this, and the reconcile sweep will not re-seed a node absent from both NodeDB tiers. Test: an observed identity plus a next-hop hint both vanish after removeNodeByNum(). Left for CI to execute per request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit b5564c173337de99e15ef998a5c698550238fd3c) * Document the tmm-fix-2 / tmm-fix-superset reconciliation decisions Two parallel implementations of the same superset design are being combined on this branch. This decision matrix records every divergence and which side each reconciled feature takes, ahead of the code commits that apply them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Reconcile retention: no timed eviction, obsTick LRU, hourly seeding Adopt tmm-fix-superset's retention model (reconciliation decisions #3, #4, #9 in .notes/tmm-super-superset-reconciliation.md): - Entries are never evicted on a timer. The 7-day retention TTL and its third tick clock (retTick) are gone; wrap-safety rests entirely on the sweep's presence-bit saturation, and a quiet entry keeps its value (pubkey pool, name rehydration). Slots are reclaimed only by trust/membership-tiered LRU on insert - age scored by obsTick, with never-observed entries counting oldest - or by an explicit purge. - Membership refresh (entry -> NodeDB contains checks) runs every sweep; the heavy seeding pass runs at boot and then hourly, with the write-through hooks carrying immediacy in between. - Tick constants and helpers move into the header beside the existing UnifiedCache tick idiom, with static_asserts tying the tick windows to the fallback path's seconds/ms forms. Kept from tmm-fix-2 (decision #4): the warm tier is still seeded (key-only records via WarmNodeStore::entryAt) so the invariant remains cache superset-of hot AND warm, and the spareMembers guard still stops seeding from churning one member out for another when hot+warm exceeds the cache. Entry shrinks to 128 B (2000 entries = 256 kB). The TTL-based retention test is superseded by a no-timed-eviction test: a quiet keyed entry survives 9 days of sweeps while the serve gate saturates as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Reconcile hooks: key-commit write-through, purgeAll, key-matched signer Adopt tmm-fix-superset's hook coverage (reconciliation decisions #5-#8): - onNodeKeyCommitted(): write-through for the two key-write sites that bypass NodeDB::updateUser - the Router's admin-key learn (TOFU-grade) and KeyVerificationModule's manual-verification commit (proven=true, the strongest provenance the cache can carry). Both were coverage gaps in the tmm-fix-2 write-through design. - updateUser's hook call now transfers signer status key-matched (isVerifiedSignerForKey) instead of the node's bare signed flag, and runs on acceptance rather than only on change. - purgeAll(): factoryReset() and resetNodes() clear both TMM tables - removal-is-full-removal applies to bulk resets too, without relying on the usual post-reset reboot. - purgeNode() reuses the existing finders instead of open-coded scans and logs the (user-initiated) purge. - peekNodeInfoFlagsForTest(): flag introspection so saturation and membership tests can assert sweep effects directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Merge the two branches' test suites for the reconciled design Port tmm-fix-superset's unique tests, adapted to run natively under TMM_HAS_NODEINFO_CACHE (reconciliation decision #11): - keyHook_upsertsAndGovernsProvenance: TOFU learn -> manual-verification upgrade -> NodeDB-senior rotation resets provenance. - tickSaturation_sweepClearsObserved: the sweep saturates hasObserved past the serve window, the entry persists (no TTL), and a full 256-tick clock wrap cannot alias a saturated stamp back to fresh. - sweepMembershipMarking: reconciliation seeds a hot identity as member+fullUser+unobserved; the next sweep clears membership once the node leaves NodeDB, while the entry itself persists. tmm-fix-2's tests (warm pin, hot/warm seeding, updateUser write-through, removal purge - now also asserting the entry is gone via the new peek hook - and the no-timed-eviction rewrite) were already on this branch. Suite now counts 70 test functions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Adapt cherry-picked and seeding tests to the reconciled semantics Two tests needed updating for interactions the validation run surfaced: - The #11035 test (ignoresUnsignedSignerIdentity) was written for a world without the native NodeInfo cache: it needs the NodeDB fallback path (dropNodeInfoCacheForTest) and the signed replay gate satisfied for its target, like the other fallback tests on this branch. - The hot-seed test's "observed frame makes it servable" step now sends a signature-verified frame: its node is a known signer, and per #11035 an unsigned frame from a signer must not (and does not) drive cache writes - the gate working as designed. Full native suite: 70/70 under ASan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * trunk fml * Doc tidyup * TrafficManagement: key NodeInfo-cache maintenance to its own guard runOnce() nested the entire NodeInfo maintenance block (tick-stamp saturation, membership refresh, boot/hourly reconcile) inside #if TRAFFIC_MANAGEMENT_CACHE_SIZE > 0, so a variant overriding the unified cache to 0 on a PSRAM board would compile the NodeInfo cache without its maintenance: hasObserved would never saturate, obsTick would alias past its 12.8 h wrap, and the 6 h staleness gate - whose wrap-safety explicitly depends on the sweep - would serve spoofed replies for long-gone nodes. Extract maintainNodeInfoCacheLocked(), guarded by TMM_HAS_NODEINFO_CACHE alone, and give runOnce() the same independent-guard structure purgeAll() already has. Also close the runtime cousin of the same mismatch: the write-through hooks (onNodeIdentityCommitted / onNodeKeyCommitted) now no-op while moduleConfig.has_traffic_management is off. Previously they kept filling the cache from NodeDB commits while runOnce() returned INT32_MAX, accumulating entries that were never swept or reconciled. Purges and reads stay ungated: removal must always work, and reads just miss an empty cache. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: forward throttled NodeInfo requests instead of consuming them The direct-response throttle returned true, which made handleReceived() STOP the request: within the 30 s window a repeat request was neither answered nor forwarded toward the genuine target, and the call site still logged "respond" and bumped nodeinfo_cache_hits for a reply that was never sent. A requester whose first reply was lost on a noisy link got silence for the whole window. Return false instead: the request flows through normal relay handling, so the genuine node (or another cache-holder) can answer, while our own spoofed TX stays bounded. Repeats of the same packet id are already absorbed by the router's duplicate detection, and the stats counter now only counts replies actually sent. Also log purgeNode() only when a slot was actually cleared - it runs for every NodeDB removal, including nodes the caches never tracked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: consolidate NodeInfo-cache #if sprawl into one region Fourteen scattered #if TMM_HAS_NODEINFO_CACHE guards (one per function, each with its own #else stub of (void) casts) collapse into a single guarded region holding every NodeInfo-cache-only function, terminated by one #else block of no-op stub definitions. Because the stubs now exist on every build, the call sites in runOnce(), purgeNode() and purgeAll() drop their guards too; inner guards remain only for orthogonal features (PKI, warm tier) and for real conditional work (constructor allocation, PSRAM paths in shouldRespondToNodeInfo). No behavior change. Compile-checked on both sides of the macro: the native app build (stubs) and the native unit-test build (region). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Router: resolve sender key only for PKI-decrypt candidates perhapsDecode() resolved the sender's public key unconditionally for every encrypted packet, before even checking whether the packet could be PKI-decrypted (channel 0, unicast to us). Since copyPublicKey() grew its TrafficManagement fallback tier, a full hot+warm miss - the common case for channel traffic from senders outside both NodeDB tiers - additionally walked the 2000-entry NodeInfo cache under its lock, per packet, for a key that was then discarded. Move the resolution inside the PKI-candidate branch; remotePublic and haveRemoteKey had no consumers outside it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: refresh NodeInfo membership hourly, not per-sweep The 60 s sweep re-derived isMember with a per-entry NodeDB lookup: O(entries x members) - about 700k node-number comparisons per minute on a full PSRAM cache (2000 entries x 250 hot + 100 warm), with strided PSRAM reads, all while holding cacheLock against the packet path. Move the refresh into the hourly reconcile pass, which is already O(members x entries): after the seeding loops (so the upsert pass still sees last pass's bits for spareMembers protection), clear every isMember bit and re-mark from both NodeDB tiers - including keyless warm records, which seed nothing but are still members. Accepted tradeoff, now documented on the field: membership lags a passive NodeDB eviction by up to an hour (the entry just stays LRU-sticky slightly longer). Additions stay immediate via the write-through hooks, explicit removals via purgeNode(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * NodeDB: centralize bare-key commits in commitRemoteKey() The two key-write sites that bypass updateUser() (admin-channel learn in Router::perhapsDecode, manual verification in KeyVerificationModule) each carried their own #if HAS_TRAFFIC_MANAGEMENT write-through boilerplate - the pattern this PR itself had to retrofit twice, and which any future direct key write would silently miss, leaving the TrafficManagement cache divergent until the next hourly reconcile. Add NodeDB::commitRemoteKey(n, key32, KeyCommitTrust): writes the key to the hot store and routes the TrafficManagement write-through in one place, with provenance explicit at the call site (AdminChannelProven maps to TOFU-grade, ManuallyVerified to proven). The bypass sites keep their reason for existing - bare-key commits with provenance that updateUser's User-payload/TOFU-pin path cannot express - but no longer know about TrafficManagement at all; both stale includes are dropped (Router's was already unused on develop). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: tick wrap-safety analysis, updated cadences, CodeRabbit doc fixes node_info_stores.md gains a "Tick clocks and wrap safety" section recording which mechanism keeps each modular tick clock honest: the unified-cache clocks pair the 60 s sweep with read-time window resets, the NodeInfo obs/resp clocks are sweep-only (hence the compile-time invariant that maintainNodeInfoCacheLocked() is guarded by TMM_HAS_NODEINFO_CACHE alone), and the warm tier is immune by design - absolute unix-seconds, no wrap until 2106. Also brought current with this series: membership refresh moved to the hourly reconcile, throttled direct-response requests forward instead of being consumed, the commitRemoteKey() bare-key funnel, and the module-disabled gate on the write-through hooks. CodeRabbit doc review (PR #11050): present the NodeInfo payload cache as the third *identity* tier with the unified cache beside the chain, and describe NodeInfoLite's flattened fields / satellite copy-out accessors instead of the removed nested members. Two stale docs/tmm_node_stores.md references in the header now point at the real file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: adapt tests to forwarded throttles and hourly membership Throttle tests (PSRAM and NodeDB-fallback paths): a request inside the 30 s window now CONTINUEs into normal relay handling instead of being consumed - assert no spoofed TX, no NAK suppression, and (PSRAM path) that nodeinfo_cache_hits counts only replies actually sent. Membership test (renamed reconcileMembershipMarking): the per-minute sweep no longer refreshes isMember, so the test now pins down both halves of the new contract - the very next sweep after a passive NodeDB eviction still shows the stale member bit (the documented up-to-an-hour lag), and a reconcile interval's worth of sweeps clears it. Disabled-module test additionally proves the write-through hooks share the has_traffic_management gate: a key commit while disabled must not land in the NodeInfo cache. Not covered here: a build permutation with TRAFFIC_MANAGEMENT_CACHE_SIZE overridden to 0 (the configuration the maintenance-guard fix protects) would need its own PlatformIO env plus guards on every unified-cache test - left as a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * expand test coverage * nitpicks and docs update * more comment fixes * nitpicks * test: make TMM fixture cleanup abort-safe in tearDown() Several tests reset per-test global state (trafficManagementModule pointing at a stack `module`, owner.public_key, the RTC fake clock) only after their assertions. A TEST_ASSERT_* failure longjmps out and skips that trailing cleanup, leaving the state to dangle into later cases - concretely, the next setUp()'s resetNodes() would call trafficManagementModule->purgeAll() on a destroyed object. Move the resets into tearDown(), which runs unconditionally between tests, and drop the now-redundant per-test cleanups. Addresses a CodeRabbit review comment on PR #11050. clod helped too * docs tidy * Throttle NodeInfo direct responses A direct response is addressed to the requesting packet's from field, which is unauthenticated, and is sent by every neighbour that holds the target in cache. One request therefore makes several nodes transmit at an address the requester chose, and nothing limited how often that could be repeated. Replies are now spaced per requester, which bounds how much any single node can be made to receive, plus a global floor on how much airtime the feature can consume. The check sits where a reply is about to be sent, so requests declined for other reasons do not consume the budget. * test: reconcile TMM direct-response tests with #11104 throttle Makes the suite green for now. #11104's per-requester + global-floor throttle (60 s) is layered over the branch's per-target throttle, so the three existing "served again" checks (psram, fallback, sweep) now use a fresh requester to avoid the per-requester window masking the mechanism each one actually exercises. Adds a dedicated test for the new per-requester + 1 s global-floor behaviour (the reflector-flood gap the per-target throttle leaves open, and the bound that survives on non-PSRAM builds). Deferred: the branch's now-redundant fallback global stamp (nodeInfoFallbackLastResponseMs) is left in place as a follow-up cleanup. Clod wants credit. * TMM: unify direct-response throttles into per-sender + per-target RAM tables Replace the two path-specific NodeInfo-spoof throttles (per-entry respTick on the PSRAM cache path, single global stamp on the fallback path) with three symmetric bounds that behave identically with and without PSRAM: - per requester (60 s): how much any one node can be made to receive; - per target (60 s): how often we vouch for the same identity; - global floor (1 s): total airtime, the backstop once an attacker cycles requester/target past the 8-slot tables. Both axes are fixed 8-slot LRU tables in internal RAM (not the PSRAM NodeInfo cache), compared by wrap-safe uint32 ms subtraction, so there is no tick clock and no sweep to maintain. directResponseAllowed() resolves both slots before stamping either, so a reply one axis throttles never consumes the other's budget, and records the send itself. Retires: nodeInfoFallbackLastResponseMs, kNodeInfoResponseThrottleMs, nowStampMs, the respTick byte + hasResponded bit on every cache entry, currentRespTick/kNodeInfoRespTickMs/kNodeInfoThrottleTicks, and the sweep's respTick clear. Renumbers peekNodeInfoFlagsForTest (drops the responded bit). Tests: the psram/fallback throttle tests now assert the per-target axis at 60 s, isolated via a different requester, proving it holds with and without PSRAM; perRequesterAndGlobalFloor isolates the other two axes; the obsolete respTick wrap-safety test is removed. 83/83 native cases pass. clod helped too * whats up, doc? * trimming the comments * test: bump native-suite-count to 38 after upstream rebase Upstream develop carries 38 test_* suite directories but its native-suite-count file still reads 37 (a suite was added without bumping the count). Rebasing onto develop inherits that stale file, so the runner flags AMBER. Correct the registered total to 38. clod helped too --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> |
||
|
Thomas GöttgensandGitHub
|
40e583c102 | Limit node database churn from unauthenticated packets (#11102) | ||
|
Benjamin Faershtein
|
08cfb3d683 | Merge upstream develop into packet authentication policy | ||
|
Thomas GöttgensandGitHub
|
5ded0ec8c5 | Gate identity learning on signature in NodeDB::updateUser (#11084) | ||
|
Benjamin Faershtein
|
59ab12675a | Merge develop into packet authentication policy | ||
 HarukiToredaandGitHub
|
8c6cc5dbde |
Don't show messages from Ignored nodes (#11068)
* Honor Node Ignore messages * Screenless node fix |
||
 oscgonferandGitHub
|
73180141b3 |
Add userprefs for telemetry screen settings (#11051)
* Add telemetry update interval to userprefs * Add telemetry screen configs to userprefs * Removed duped code from cherry-pick |
||
|
b71c1adb26 |
stm32wl: add hardware RTC support (rak3172) (#10961)
* stm32wl: add hardware RTC support infrastructure Wires the STM32WL chip's internal RTC (running off the LSE 32.768kHz crystal) into meshtastic's existing time-of-day framework (perhapsSetRTC()/readFromRTC()), following the same pattern already used for I2C RTC chips (RV3028, PCF8563/85063, RX8130CE). LSE is started and polled manually before ever calling into the STM32RTC library, with our own bounded timeout - the library's own internal LSE startup path has no bounded fallback and hangs forever via Error_Handler() if the crystal never locks, so this is required for a board with a missing/faulty crystal to boot normally rather than hang. Gated behind a new HAS_LSE variant flag (currently unset everywhere, so this is inert until a variant opts in - see follow-up commit). Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * gps: qualify RTC.h includes to avoid case-insensitive filesystem collision with STM32RTC The stm32duino STM32RTC library (added to lib_deps in a follow-up commit) ships its own src/rtc.h. On case-insensitive filesystems (the macOS default), an unqualified #include "RTC.h"/<RTC.h> from any file outside src/gps/ resolves to the library's rtc.h instead of src/gps/RTC.h, since PlatformIO's LDF puts lib_deps include paths ahead of the project's own -Isrc/gps. Qualify every include as gps/RTC.h so it can't collide with any same-named header a future dependency might ship, regardless of filesystem case sensitivity. Purely mechanical, no behavior change. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * stm32wl(rak3172): enable hardware RTC support Opts rak3172 into the HAS_LSE infrastructure added previously: sets STM32WL_LSE_DRIVE to a conservative default and pulls in the STM32RTC library. rak3172 has ~63KB flash headroom going in; build-verified at 76.7% flash usage after this change (up from a 73.8% baseline), well within budget. wio-e5 is not opted in here despite sharing the same STM32WLE5 chip - it's already at 96.8% flash usage today (GPS + I2C sensor support compiled in, unlike rak3172), leaving too little headroom to safely add STM32RTC without first trimming something else. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * stm32wl: add docstrings for LSE/RTC setup functions Addresses CodeRabbit's docstring coverage check on PR #10961. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * stm32wl: address CodeRabbit nitpicks on PR #10961 - Brace the single-statement HAS_LSE branch in perhapsSetRTC() to match the sibling readFromRTC() branch's style. - Quote the RTC.h include in PhoneAPI.cpp for consistency with every other qualified include site. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> --------- Signed-off-by: Andrew Yong <me@ndoo.sg> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
oscgonferandGitHub
|
f18a8b0552 | Add AQ Telemetry to userPrefs (#11029) | ||
|
Thomas GöttgensandGitHub
|
381f9c196d | NodeDB: clear only the slots removeNodeByNum() vacates (#11022) | ||
|
Thomas GöttgensandGitHub
|
e219e24b09 | WarmNodeStore: carry the XEdDSA signer flag through the warm tier (#11020) | ||
|
36bfb86937 |
Remove UA_868 as obsolete (#10994)
Decree of the Cabinet of Ministers of Ukraine dated February 25, 2026,* No. 262, on amendments to Section 2 of the Plan for the Allocation and Use of the Radio Frequency Spectrum in Ukraine, has harmonised Ukrainian regulations on the 868 MHz spectrum with the EU's, thereby making UA_868 legally obsolete. * https://zakon.rada.gov.ua/laws/show/262-2026-п Co-authored-by: Tom <116762865+NomDeTom@users.noreply.github.com> |
||
|
|
269b974e96 |
feat: populate MyNodeInfo.device_id on all platforms (#10995)
* feat: populate MyNodeInfo.device_id on all platforms RP2040/RP2350 use the 64-bit pico unique board id, STM32WL the 96-bit silicon UID, ESP32-S2 joins the existing OPTIONAL_UNIQUE_ID efuse branch, and everything else (classic ESP32 in particular) falls back to a deterministic factory-MAC-derived id, resolving the long-standing FIXME. Portduino keeps the config-supplied id preferred and now uses the MAC fallback when the config omits one. No proto or persistence changes; the id is re-read from silicon each boot and PhoneAPI still zeroes it for unauthenticated clients. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: declare zero_mac const to satisfy cppcheck Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address review feedback on device_id derivation Clear any disk-loaded device_id before the silicon derivation so a failed read leaves it unset rather than stale (Copilot), and size the portduino config copy with sizeof instead of a literal 16 (CodeRabbit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: extract device_id generation into per-arch getDeviceId() Per review feedback on #10995: move the platform-specific MyNodeInfo.device_id derivation out of the #if/#elif ladder in NodeDB.cpp into a getDeviceId() interface (target_specific.h) implemented per-architecture alongside each platform's getMacAddr(): - esp32: efuse OPTIONAL_UNIQUE_ID (C3/S2/S3/C6); classic ESP32 -> MAC - nrf52: FICR DEVICEID + DEVICEADDR - nrf54l15: FICR->INFO.DEVICEID + DEVICEADDR (NRF_FICR-guarded, MAC fallback) - rp2xx0: pico_get_unique_board_id() - stm32wl: HAL_GetUIDw0/1/2() - portduino: config-supplied id preferred, else MAC The shared MAC-derived fallback moves to meshUtils as getMacAddrDeviceId(). NodeDB.cpp now zero-inits the field and makes a single getDeviceId() call, dropping ~65 lines of platform boilerplate plus the esp_efuse/pico/stm32 includes that came with it. No behavior change: device_id is still re-read from silicon each boot and never persisted. Builds green: native-macos, tbeam, heltec-v3, rak4631, rak11310, rak3172. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: address review of device_id refactor - getMacAddrDeviceId(): zero-init the mac[6] buffer. getMacAddr() can return without writing (e.g. Portduino with no MAC source), so the old uninitialized buffer let stack garbage pass the all-zeros guard and become device_id. The pre-refactor code relied on the zero-initialized static ourMacAddr; restore that guarantee. - nrf54l15 getDeviceId(): drop the `#if defined(NRF_FICR)` guard and read FICR unconditionally (as the pre-refactor NodeDB code did). The guarded #else fell back to getMacAddr()'s hard-coded placeholder MAC, which would give every unit an identical device_id; a missing NRF_FICR should be a loud compile error. - NodeDB.cpp: `#include "target_specific.h"` instead of hand-copied externs for getMacAddr/getDeviceId; retire the stale FIXME. Same for meshUtils.cpp (whose extern comment wrongly claimed the TU was Arduino-free). - Delete the orphaned commented-out device_id hex-dump block in NodeDB.cpp. - Trim/de-duplicate the getDeviceId contract comments (single-sourced in target_specific.h). Builds green: native-macos, tbeam. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: trim device_id comments to the repo's 1-2 line guideline Addresses CodeRabbit review nitpick on #10995: shorten the getDeviceId() (target_specific.h), getMacAddrDeviceId() (meshUtils.h), device-id refresh (NodeDB.cpp), and nrf54l15 getDeviceId comments to two lines each, per the "one or two lines maximum" coding guideline. Comment-only; no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Thomas GöttgensandGitHub
|
7a25ffd0a2 | fix: evaluate pre-hop hop_start bitfield guard post-decode instead of before decryption (#10758) | ||
|
Benjamin Faershtein
|
d6b12ea3f1 | feat(security): enforce packet authenticity policies | ||
|
Andrew YongandGitHub
|
0ae44d7017 |
fix(router): null-check packetPool/clientNotificationPool allocations (#10948)
On platforms where these pools are heap-backed (MemoryDynamic - used whenever there isn't enough static RAM for a fixed pool, e.g. ARCH_STM32WL or BOARD_HAS_PSRAM), allocCopy()/allocZeroed() return nullptr on allocation failure and already log a warning, but most callers dereferenced the result unconditionally. Under real heap pressure this reliably produced a HardFault - reproduced on STM32WL hardware under sustained mesh traffic, including the RX entry point (RadioLibInterface::handleReceiveInterrupt) where every received packet is allocated. Adds null checks at all call sites that were missing one, mirroring the guard pattern already used correctly elsewhere in the same files (e.g. RadioInterface.cpp's sendErrorNotification). On allocation failure, callers now skip the send/retransmission/notification and log nothing further (the allocator already did) rather than crash. Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Andrew Yong <me@ndoo.sg> |
||
|
Ben MeadorsandGitHub
|
8d20606203 |
feat(config): make position & telemetry broadcast opt-in (#10929)
Position sharing was opt-out (the default primary channel shipped position_precision=13) while device telemetry was already opt-in, and a normal firmware upgrade preserves saved config, so existing nodes stayed position-on. This makes both broadcasts opt-in, both on a fresh flash and via a one-time migration for upgrading nodes. - Fresh default: initDefaultChannel now sets position_precision=0. - One-time migration in loadFromDisk, gated on a dedicated POSITION_TELEMETRY_OPTIN_VER (26) watermark kept separate from DEVICESTATE_CUR_VER (bumping that would re-run the NodeDatabase v24 legacy decoder on already-migrated v25 DBs): disables position broadcast on PUBLIC/default-PSK channels and forces every device-telemetry mesh-broadcast flag plus the MQTT map-report location off. - Private-PSK channels are left untouched: a channel with a custom key is a deliberate trusted-group setup, so its configured precision is preserved. - Idempotent: ordinary saves never re-stamp .version, so a user who re-enables sharing is not re-disabled on the next boot. - Added pure channelFileUsesPublicKey() (operates on the raw ChannelFile, since the channels singleton isn't initialized during loadFromDisk) and refactored Channels::usesPublicKey() to delegate to it. - New test/test_optin_migration native suite (14 cases). |
||
|
Ben MeadorsandGitHub
|
6c7ee8afc7 |
Add MemAudit: per-subsystem heap accounting in the boot log (#10900)
* Add MemAudit: per-subsystem heap accounting in the boot log The 2.8.0 nRF52840 heap-exhaustion field reports had to be diagnosed by hand, reconstructing each subsystem's heap footprint from source and build flags one report at a time. This makes every future report self-diagnosing from the serial log: a tiny fixed-size registry (src/memory/MemAudit.*) that big long-lived allocations report into, printed as one line at the end of setup() and alongside the periodic "Heap free:" log: MemAudit[boot]: tmm=2500 warm=4000 pkthist=5824 nodedb=13440 msgstore=2200 pktpool(live)=3270 total=31234 Instrumented: NodeDB hot vector (nodedb) + satellite maps (satmaps, rb-tree overhead estimated), WarmNodeStore (warm), PacketHistory records and hash index (pkthist), TrafficManagement caches (tmm/tmm_ni), MessageStore text pool (msgstore), TFT line/repaint buffers (display), and live in-flight packets (pktpool(live)) via an optional audit tag on the packet pool allocator - the one hot path, counted with a relaxed 32-bit atomic add (single instructions on Cortex-M, no locks). Cost: 128 B RAM for the 16-tag table, well under 1 KB flash on rak4631. MESHTASTIC_MEM_AUDIT=0 compiles it out to inline no-op stubs (call sites need no ifdefs); STM32WL, the tightest flash target, defaults off. New native suite test_mem_audit covers add/set/snapshot arithmetic, tag reuse (pointer and cross-TU strcmp fallback), null/unknown tags, and table-full behavior; test/native-suite-count bumped to 28. * native-wasm: add src/memory/ to the curated source filter The wasm env denies all sources and adds an explicit file list; MemAudit callers (main, MeshService, NodeDB, PacketHistory) are in that list but src/memory/MemAudit.cpp was not, so wasm-ld failed on undefined memaudit:: symbols. |
||
|
Jonathan BennettandGitHub
|
fe6bf91343 | Add tx_power to userprefs (#10889) | ||
|
Ben MeadorsandGitHub
|
b4dd76a4db |
Harden against crafted-packet crashes + adversarial fuzzing (#10862)
Audit and fuzzing of the RF-packet decode -> dispatch -> display/phone paths for the "crash a node or phone with a crafted packet" surface, beyond the XEdDSA authenticity work. Crash fixes (reproduced under AddressSanitizer / UBSan): - GeoCoord::latLongToUTM/latLongToMGRS read fixed letter tables out of bounds on extreme latitude_i/longitude_i from a received Position, and narrowed out-of-range easting/northing doubles to unsigned (float-cast-overflow UB). Clamp the UTM zone, the easting/northing narrowing, and the band/col/row indices. Regression: test_geocoord_extreme_coords_no_oob. - EnvironmentTelemetry/AirQualityTelemetry render attacker floats via String(float), which on nRF52/RP2040/STM32/portduino formats into a fixed char[33] (dtostrf) and overflows near FLT_MAX. Clamp the rendered metrics via UnitConversions::displaySafeFloat (finite + magnitude <= 1e9), unit-tested in test_type_conversions. Defense-in-depth + robustness: - TraceRouteModule::printRoute: fix an snr_back[-1] OOB read (wrong count in the guard) and stop formatting the INT8_MIN "unknown SNR" sentinel as a dB value. - WaypointModule/NodeDB: sanitize untrusted strings before the OLED renderer and the phone-facing ClientNotification (belt-and-suspenders vs PB_VALIDATE_UTF8). - MeshService::sendToPhone: withhold NODEINFO/WAYPOINT packets whose nested string won't cleanly decode, protecting strict phone protobuf decoders without affecting mesh relay. Tests: new test_fuzz_decode (protobuf decode + UTF-8 sanitizer fuzz) and test_fuzz_packets (perhapsDecode / module-handler / traceroute / phone-gate fuzz), all under AddressSanitizer; native-suite-count 25 -> 27. Full suite 515/515 green. |
||
 Ethac.chenandGitHub
|
4e36f4271e |
fix(tap_v2): disable Bluetooth by default on TFT device (#10857)
TAP V2 has a TFT screen and belongs to the same device category as T-Deck / T-Watch S3 / HELTEC_V4_TFT -- Bluetooth should default to OFF to avoid unwanted BLE connections and reduce power consumption. Users can re-enable BT via the TFT settings menu when needed. |
||
|
TomandGitHub
|
3becaf2d95 | emdashes begone (#10847) | ||
|
TomandGitHub
|
fd62322876 |
Log tidy (#10798)
* clarify channel number formatting and fix the fixed-width padded nodeIDs * use the explicit 0x%08x for packets and nodeIDs throughout * LLM instructions |
||
|
|
0cf6fe72fd |
fix self-nodeID change (#10822)
* fix se0fl-nodeID change * address review: avoid redundant satellite erase + add braces in createNewIdentity removeNodeByNum() already drops the satellite stores (and warm-tier copy) when it removes the node, so only erase satellites directly in the rare case the lite entry was already absent. Brace the removal block. --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
Jonathan BennettandGitHub
|
fd8ba4fa4e |
Chop down tft-related loooong ifdefs chains (#10803)
* Chop down tft-related loooong ifdefs chains * Fix accidental sweep-up of portduino into screen #ifdef refactor |
||
|
Ben MeadorsandGitHub
|
0488a46a3c |
fix: stop unexpected NodeNum regeneration from PKI key loss (#10808)
* fix: stop unexpected NodeNum regeneration from PKI key loss A node's NodeNum is derived from its PKI public key (my_node_num == crc32(public_key)), so it changes only when the keypair is regenerated -- which happens only when a keygen path runs while config.security.private_key.size != 32. Two paths could trigger that without the user intending a new identity: 1. Boot: loadFromDisk() collapsed every non-success loadProto() result for the config file into installDefaultConfig() (preserveKey=false), which memset()s config and wipes the private key. loadProto() distinguishes DECODE_FAILED (file present but undecodable/undecryptable) from OTHER_FAILURE (absent), so a transient/corrupt read silently minted a new identity. A new configDecodeFailed flag now gates the boot keygen and the boot config-save directly (not via region, so it survives the userprefs/region overrides that run later in loadFromDisk): identity is frozen, the unreadable file is left intact for a later clean boot to recover, and the node boots radio-silent (region UNSET, TX off). A genuinely-absent config still gets defaults + keygen. 2. AdminModule security SET: config.security = c.payload_variant.security wholesale-clobbered the keypair, then regenerated whenever the incoming private_key.size != 32. A client editing an unrelated security field without round-tripping the private key would re-mint identity. The existing keypair is now preserved when the incoming config omits it; first provisioning and key import are unchanged. Intentional reset goes through factory_reset. Native PlatformIO unit suite (Docker): 499/499 test cases pass. * test: cover security keypair preservation; clarify load-failure comment Addresses PR review feedback: - Add native unit tests (test_admin_radio) asserting handleSetConfig preserves the existing identity keypair when a security SET omits the private key, and applies a full supplied keypair (key import). Guards against reintroduced NodeNum/keypair regeneration. AdminModuleTestShim (now a friend) defers saves so the lightweight harness doesn't saveToDisk an uninitialized node database. - Clarify the non-DECODE_FAILED config-load comment: OTHER_FAILURE / NO_FILESYSTEM cover an absent or unopenable file, not just first boot. |
||
   TomGitHubSteve GilberdDarafei Praliaskouskigithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Claude Opus 4.8 |
ec5d230305 |
Feat/mesh beacon (#10618)
* Tips robot virtual node / relayer to different LoRa modes & channels
Note that this commit has details hardcoded for the Wellington (NZ)
mesh, and also requires the following patch to the protobufs:
-----
diff --git a/meshtastic/mesh.proto b/meshtastic/mesh.proto
index 03162d8..ec54c99 100644
--- a/meshtastic/mesh.proto
+++ b/meshtastic/mesh.proto
@@ -1393,6 +1393,21 @@ message MeshPacket {
* Set by the firmware internally, clients are not supposed to set this.
*/
uint32 tx_after = 20;
+
+ /*
+ * The modem preset to use fo rthis packet
+ */
+ uint32 modem_preset = 21;
+
+ /*
+ * The frequency slot to use for this packet
+ */
+ uint32 frequency_slot = 22;
+
+ /*
+ * Whether the packet has a nonstandard radio config
+ */
+ bool nonstandard_radio_config = 23;
}
/*
-----
* fix: repair mesh tips CI build
* feat: add MeshBeacon module (Phase 1 — proto + generated code + initial stub)
* feat(beacon): implement broadcaster + listener (phases 2-5)
* feat(beacon): wire RadioLibInterface hooks + admin validation (phases 6-7)
* fix(beacon): fix LocalModuleConfig flat access (no payload_variant), add localonly proto field
* feat(beacon): fix broadcaster inheritance, add preset/region validation + proto cache
- MeshBeaconBroadcastModule now inherits ProtobufModule<meshtastic_MeshBeacon>
(alongside private MeshBeaconModule + OSThread), giving it allocDataPacket()
and setStartDelay() without extra includes.
- Payload cache: rebuildCache() encodes the MeshBeacon protobuf once and stores
it in payloadCache[]/payloadCacheSize; sendBeacon() only calls rebuildCache()
when payloadCacheDirty==true. AdminModule calls invalidateCache() after saving
new config so the next broadcast picks up changes.
- Region/preset validation in handleSetModuleConfig (mesh_beacon_tag):
broadcast_on_preset is validated against the device's current region via
RadioInterface::validateConfigLora(); broadcast_offer_region is validated via
RadioInterface::validateConfigRegion(). Invalid values are zeroed with a
LOG_WARN before saving.
* feat(beacon): add unit tests for MeshBeaconModule and AdminModule configuration validation
* remove old meshtips
* more validation in NodeDB and AdminModule, and userprefs for baked in goodness
* copilot is my gravity
* mmmmm... beacon
* oops
* Enhance unit tests for MeshBeaconModule with detailed validation checks and output formatting
* new lines. Why not?
* finally
* legacy mode activate!
* Update protobufs (#17)
Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com>
* better logic, fixed a test
* updated for packet signing
fixed a test
added guards for licensed/ham mode
* channel numbers
* beacon: encrypt on the beacon channel PSK; fix split note
When broadcast_on_channel overrides the primary channel's name/PSK, the
beacon was encrypted with the PRIMARY PSK: perhapsEncode keys encryption
off the primary slot, but the radio-thread channel switch happens only
after encryption. sendBeaconPacket() now installs the beacon channel into
the primary slot for the synchronous duration of send() (cooperative
threading => no interleaving) so encryption/hash use the beacon channel,
then restores it. A shared beaconChannelSettings() helper builds the
channel for both the encrypt-time swap and the RF-time swap so the
key+hash cannot drift.
Also: correct the legacy-split comments (both packets go out on the same
beacon radio settings, not the normal config) and merge the two
consecutive `if (hasText)` blocks in the listener (cppcheck
duplicateCondition).
Tests: add channelPskOverride_swapsBeaconChannelAndRestores and
noChannelOverride_doesNotSwapPrimary; MockRouter snapshots the primary
channel at send() time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test/beacon: drain toPhoneQueue in tearDown to fix LSan leak abort
The listener delivers received text via MeshService::sendToPhone(), which
enqueues the packet into toPhoneQueue and takes ownership. Nothing dequeues
it in tests, so the three listener tests carrying message text stranded a
MeshPacket each — 1272 bytes / 3 allocations that LeakSanitizer flagged at
process exit, aborting the coverage run (surfaced by pio as [ERRORED] /
SIGHUP even though all 40 assertions passed).
Drain the phone queue in tearDown (getForPhone()/releaseToPool) so the
packets return to packetPool. Suite is now GREEN with no sanitizer abort.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* legacy hop override for zero-hoppers
* ever more beacons
* beacon: comment out broadcast_send_as_node pending further review
Functionality preserved in comments with full signing/has_bitfield notes
for when it is re-enabled. Proto tag 3 retained on the wire.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test/beacon: fix fromIsCustomNodeWhenSet now that send-as-node is disabled
broadcast_send_as_node is commented out; from is always the local node.
Update the test assertion and doc comment to match current behaviour.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Update protobufs (#21)
Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com>
* flags for beacons
* beacon: do more with less — slot-index targets + validation
Multi-target beacons embedded a full ChannelSettings in every BroadcastTarget,
blowing ModuleConfig past the 512-byte BLE FromRadio budget so the firmware would
not compile. Targets now reference an existing channel-table slot by channel_index
and the broadcaster resolves it via channels.getByIndex() at TX time. Net effect:
the same multi-target capability for a fraction of the bytes —
FromRadio 609 -> 510 B, MeshBeaconConfig 596 -> 324 B, AdminMessage 615 -> 511 B.
- proto: BroadcastTarget.channel (embedded) -> channel_index (uint32 ref); regen all
generated headers (size constants propagate to admin/localonly/deviceonly/mesh).
- broadcaster: resolve channel_index from the channel table; an out-of-range or blank
slot falls back to the default channel for the target preset rather than borrowing
the primary's name/PSK.
- AdminModule: validate broadcast_targets entries on write (region/preset sanitised
like the single-target fields; channel_index range-checked).
- userPrefs: TARGET_<n>_CHANNEL_{NAME,NUM,PSK} collapse to a single CHANNEL_INDEX.
- docs: two-step (set_channel -> set_module_config) multi-target setup, inline-vs-
reference distinction, and single-/multi-target are equal (not "legacy") options.
- tests: target validation + channel-index resolution incl. blank-slot fallback
(47/47 green on `./bin/run-tests.sh -e native -f test_mesh_beacon`).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NRAF5csgsMn6p1zEcFL8Qz
* throttling after reboot
* address copilot review
* simplify
* fix(beacon): use 0x%08x for node/packet IDs in logs; register test suite
The %#08lx log specifiers passed uint32_t (NodeNum/PacketId) to a %lx
length modifier — undefined behaviour on 64-bit (native test) targets and
non-standard width. Switch to the project-standard 0x%08x. Also bump
test/native-suite-count to 25 for the added test_mesh_beacon suite.
clod helped too
* copilot & clarity
clod helped too
* refactor(beacon): use auto for the sanitized config copy
clod helped too
* fix(beacon): guard empty-payload sends; gate has_mesh_beacon on build flag; document ISR_TX pre-switch
clod helped too
---------
Co-authored-by: Steve Gilberd <steve@erayd.net>
Co-authored-by: Darafei Praliaskouski <me@komzpa.net>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Thomas GöttgensandGitHub
|
c4fdf374e1 |
fix: right-size warm tier for constrained platforms (#10746, #10705) (#10759)
* fix: right-size warm tier for constrained platforms and feed RP2040 watchdog during NodeDB save * fix: size warm tier and traffic cache per-MCU RAM, lowering no-PSRAM ESP32 (classic/S2/C3) tiers * docs: document nRF52/RP2040 #else fall-through in warm tier and TM cache cascades |
||
|
3501f620a3 |
fix: restore fixed position into localPosition on boot (#10670)
* NodeDB: restore fixed position into localPosition at boot Fixed-position nodes without a GPS stop broadcasting their position (and publishing MQTT map reports) after a reboot. On boot localPosition is empty, and NodeDB::hasValidPosition() for the local node only inspects localPosition, not the persisted node position. PositionModule therefore never sends a position, so the lazy localPosition backfill in getPositionPacket() never runs and localPosition stays at (0,0) indefinitely. Restore the configured fixed position into localPosition during NodeDB construction so position broadcasts and map reports resume automatically after a reboot. * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
|
c51c01607d |
Traffic Management module: dedup, rate limiting, role-aware policing (#10706)
Adds the Traffic Management module (TMM) plus the NodeDB/warm-store and next-hop foundations it builds on: - Unified per-node cache (flat array, 8-bit relative ticks) shared by all features; role-aware throttles for tracker / lost-and-found. - Position deduplication: drop unchanged position rebroadcasts within a configurable interval; precision driven off the channel ceiling (clamped to the public-key max on well-known channels). Enabled by default at 11h. - Per-node rate limiting and unknown-packet filtering (config-driven; a non-zero companion field enables each feature -- no bool toggles). - NodeInfo direct response from cache with role-based hop clamps. - Persistent next-hop overflow store: confirmed hops have no TTL, are seeded from NodeInfoLite at boot, and survive hot-store eviction. - Three-tier sender-role resolution (hot NodeInfoLite -> warm store -> TMM cache). Role is cached write-time (seeded on first track, refreshed from NodeInfo), pins its cache entry like a next-hop hint, and is evicted last. - Warm store caches device role + protected category across reboot/eviction. - PositionModule stationary floor for tracker / lost-and-found. - PSRAM gating for warm/satellite/TMM cache sizes; STM32WL excluded. Protobufs: TrafficManagementConfig trimmed to the five uint32 fields actually used; submodule repointed to protobufs develop. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
d8d92b7b71 |
Fix/zerohop and hopscale mqtt (#10753)
* fix for prehopdrop on zerohop packets * hopscaling: exclude MQTT-origin packets from the node histogram The hop-scaling histogram gate only checked transport_mechanism == TRANSPORT_LORA, which excludes packets received from the broker but NOT MQTT-origin packets that a gateway rebroadcasts onto LoRa (those arrive as TRANSPORT_LORA with via_mqtt set). Counting them inflates the local mesh-size/density estimate with nodes that aren't real RF participants — and since the bridged copy usually carries hop_start==0 they land in the hop-0 bucket, the one that pulls getLastRequiredHop() lowest, over- shrinking NodeInfo/position/telemetry hop limits. Add `!mp.via_mqtt` to the gate. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Ben Meadors
|
161cd26519 | Add conditional compilation for warm node count checks | ||
|
|
22072c5f4b |
Pr1.5 tmm nexthop (#10745)
* TrafficManagement: flat unified cache + persistent next-hop overflow store Reworks the TrafficManagementModule cache layer (policing behaviour unchanged from upstream) and adds a routing-hint overflow store: - Flatten the ring: replace the cuckoo-hashed unified cache and the bucketed PSRAM NodeInfo index with plain flat arrays + linear scan (same idiom as WarmNodeStore). At LoRa packet rates an O(n) scan of the cache is negligible, and it removes a large amount of hashing/displacement complexity. The cache entry is 11 B; timestamps use a uniform +1 presence-offset so a 0 byte always means "empty" across every sub-store. Adds rebaseEpoch() so cached state survives the ~19 h relative-timestamp horizon instead of being flushed. - Next-hop overflow cache: setNextHop/getNextHopHint store a confirmed last-byte relay for a destination, written only from NextHopRouter's ACK-confirmed decision (and mirrored from TraceRoute). NextHopRouter::getNextHop falls back to this cache when the hot NodeDB has no hint, so DMs/relays to long-tail nodes keep routing after the node ages out of NodeInfoLite. - Persistence: preloadNextHopsFromNodeDB warm-starts the cache from persisted NodeInfoLite hints on first maintenance pass; next_hop entries are kept alive across the maintenance sweep (no TTL) and never clobbered by a stale preload. All packet-policing logic (rate limit, position dedup, unknown-packet drop, NodeInfo direct response, hop exhaustion) is the existing upstream behaviour, untouched. HAS_TRAFFIC_MANAGEMENT defaults on so the module is compiled in. (see note). Tests: upstream policing suite now actually runs (adds the MeshTypes.h include that gates HAS_TRAFFIC_MANAGEMENT) plus 4 next-hop tests. Role-aware throttles, politeness, precision clamp, port-interval and mesh-radius gating — and the rate-limit >255 saturation fix — are deferred to the advanced-TMM branch. Note: default dedup movement grid moves to ~91m, which also means 1.5km required to end up with the same signature position - coarser and therefore further than before. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * TrafficManagement: fix cppcheck constVariablePointer warning `node` in preloadNextHopsFromNodeDB() is never written through — mark it const to satisfy cppcheck's constVariablePointer check in CI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add multi-hop NextHop recovery tests and unit tests for routing reliability - Introduced a new test suite for multi-hop NextHop directed-message delivery and relay recovery in `test_nexthop_multihop_recovery.py`. This includes tests for end-to-end delivery and recovery after relay drop. - Implemented unit tests in `test_main.cpp` for NextHop routing reliability mitigations, covering: - M1: Ambiguity-aware last-byte resolution. - M2: NextHopRouter's strict-neighbor gate and hop limit checks. - M3: Route-health freshness and failure decay. - Enhanced mock classes to facilitate controlled testing of node behaviors and routing logic. * grafting fixed * Address Copilot review for PR #10735 (NextHop improvements) - docs/nexthop-routing-reliability.md: update status from "no code changes yet" to reflect that mitigations and tests are implemented RAM pressure and MIGRATION_VERBOSE concerns addressed upstream in PR2.5 (per-platform TRAFFIC_MANAGEMENT_CACHE_SIZE) and PR2 (verbose default=0) respectively; (0,0) sentinel fixed in PR2.5. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * CI: fix cppcheck constVariablePointer and test include path - NextHopRouter.cpp: qualify two RouteHealth *h locals as const — only read for stale-route checks, never mutated through the pointer - Router.cpp: qualify meshtastic_NodeInfoLite *node as const in shouldDecrementHopLimit — only read for favorite/role predicate - test_position_module/test_main.cpp: change bare PositionModule.h to modules/PositionModule.h — build_flags sets -Isrc, not -Isrc/modules, so the bare form fails to resolve in the native PlatformIO test env Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * WarmStore: cache device role + protected category in last_heard low bits Steal the low 6 bits of WarmNodeEntry.last_heard to carry an evicted node's device role (4 bits) and a protected category (2 bits) for the hop-trim path, at zero record-size cost (entry stays 40 B; no RAM/flash growth). The high bits remain a real unix-seconds timestamp, quantised to 64 s — ample for warm LRU ordering of long-tail nodes. - absorb() packs role/protectedCat; place()/ring replay store the raw word so metadata round-trips through flash. LRU compares masked time (warmTimeOf). - take() rehydration masks the metadata bits and restores the cached role so a re-admitted node isn't stuck at CLIENT until its next NodeInfo. - NodeDB classifies the category (favorite/ignored/verified -> Flag; tracker/sensor/tak_tracker -> Role) at each eviction site. - WarmNodeStore::lookupMeta() exposes role/category to consumers. - Bump WARM_RING_MAGIC (WRNG->WRN2): old rings read as erased and rebuild; warm data is a non-critical evictee cache, so discard-on-upgrade is safe. Tests: test_warm_store 11/11 (new meta round-trip + quantisation-aware ordering); NodeDB compiles (test_nodedb_blocked 4/4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: migrate v1 rings/files by discarding last_heard, not the data Previously the WRNG->WRN2 magic bump treated old rings as erased, discarding all warm entries — including the PKI public keys that let evicted nodes keep decrypting DMs. Instead, read v1 (WRNG / WRM1) records and keep each node's identity + public key, discarding only last_heard (its low bits would otherwise be misread as the new role/protected metadata). Records re-rank and re-learn their role on next contact. - Ring backend (nRF52840): ringReadHeader accepts both magics and reports v1 via an out-param; replay zeroes last_heard for v1 records. If the active head page is v1, force a rotation so new v2 records never land in a v1-headered page (which would discard their freshly-set role on the next load). Legacy pages convert to v2 as the ring rotates. - File backend (warm.dat): bump WARM_STORE_MAGIC WRM1->WRM2; accept WRM1, verify CRC against the stored bytes, then discard last_heard and mark dirty so the next save rewrites as v2. Tests: test_warm_store 12/12 (adds test_ws_v1_migration_discardsLastHeard: key survives, role/protected reset). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: guard role bit-width + test eviction carries role/protected - static_assert that the device role enum still fits the 4-bit warm metadata field (WARM_ROLE_MASK); fails the build loudly if a new role is added past 15 rather than silently truncating role on eviction. (Max role today = 12.) - Add test_migration_carriesRoleAndProtectedIntoWarm: a demoted TRACKER lands in the warm tier with its key, role=TRACKER and protected category=Role; a demoted CLIENT carries role=CLIENT/None. Exercises the NodeDB eviction path + warmProtectedCategory classification (the warm-store unit tests only cover absorb() directly). Tests: test_nodedb_blocked 5/5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix copilot comments * fix(test): restore #if HAS_TRAFFIC_MANAGEMENT guard in TMM test The rebase onto PR1.5 lost the top-level HAS_TRAFFIC_MANAGEMENT guard that PR1.5 introduced, leaving the #else/#endif tail orphaned and causing compile errors on non-TMM builds. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
TomGitHubClaude Opus 4.8github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Ben Meadors |
79a7dcc46c |
Pr1 nodedb warmstore (#10705)
* NodeDB: 3-tier node store with persistent warm tier (long-tail identity retention)
Introduces a tiered NodeDB so the device retains identity (public key,
last_heard) for far more nodes than fit in the full-record hot store,
without growing heap or the persisted nodes.proto unboundedly.
- Hot store: full NodeInfoLite, MAX_NUM_NODES (120 on nRF52).
- Satellite maps: position/telemetry/environment/status capped at
MAX_SATELLITE_NODES (40 freshest); eviction via enforceSatelliteCaps /
evictSatelliteOverCap.
- Warm tier (WarmNodeStore): 40 B {num,last_heard,public_key} records for
evicted nodes so DMs to/from long-tail nodes keep encrypting/decrypting.
Persisted to /prefs/warm.dat, or on nRF52840 a dedicated 12 KB raw-flash
record-ring below LittleFS (3x4 KB pages; see linker scripts + the
nrf52_warm_region.py post-link guard).
NodeDB::getOrCreateMeshNode now demotes evicted nodes into the warm tier and
re-admits them (restoring key/last_heard). Router PKI decrypt/encode resolve
the peer key via NodeDB::copyPublicKey (hot store, then warm tier).
NodeInfoLite gains snr_q4 (sint32, Q4-encoded dB); the float snr is zeroed on
disk. NodeInfoLite grows 105 -> 112 B; backup 2432 -> 2468 B.
Note: the snr_q4 .proto change still needs to land in the protobufs submodule
(generated header is updated here; submodule pointer left at upstream).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* NodeDB: robust receive + retention for blocked (ignored) nodes
Hardens how ignored/favourite nodes are received over admin and retained,
closing paths where a block could be lost or accidentally cleared.
- Blocking keeps the node's public key (admin set_ignored_node and
addFromContact no longer zero it / drop the warm-tier key), so a blocked
peer stays a verifiable identity.
- set_ignored_node creates the node if absent, so a block by node ID sticks
even for a node we've never heard from (e.g. pushed by a remote admin) with
no NodeInfo or key.
- Eviction protection (favourite/ignored/manually-verified) now also applies to
the load-time hot-store migration and is never undone by cleanupMeshDB, which
previously purged ignored nodes that lacked user info.
- The hot-store migration leaves our own node (index 0) in place and prefers to
demote non-protected nodes, like the runtime eviction scan.
Caps the protected set (favourite + ignored + verified) at MAX_NUM_NODES-2 via
NodeDB::setProtectedFlag(), so at least two evictable slots always remain and
getOrCreateMeshNode can always make room — replacing the previous unconditional
append that could run off the end of the node vector when every node was
protected. A locally-set favourite/ignore that hits the cap reports back to the
phone via a ClientNotification.
Adds test_nodedb_blocked covering the migration, favourite/ignored eviction
protection, ignored-survives-cleanup, and the protected-node cap. The
maintenance methods stay private in production; the test reaches them through a
PIO_UNIT_TESTING-guarded friend shim.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts:
# src/mesh/NodeDB.h
* fix copilot comments
* once again
* WarmNodeStore: fix cppcheck warnings (uninitvar, constVariablePointer)
Zero-initialise `stranded[]` and `seqs[]/order[]` VLAs so cppcheck can
verify there are no unguarded reads of uninitialised memory (the guards
exist but are not visible to static analysis). Mark two local pointers
`const` where the pointed-to entry is never mutated after assignment.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* self-care added to assist 2.7 and 2.8 nodedb migration
* Tidy warm-store/self-care: comments, guards, log + flash cleanup
Style/cleanup pass over the branch (no behavior change except the noted
preprocessor simplifications, which are semantically identical):
- Comments: move function descriptions to the headers, cap in-function
comments at ~3-4 lines, drop leading-number step markers, label stacked
#endif blocks, de-decorate banner comments.
- dumpToLog: fully gate decl + definition + AdminModule call site behind
MESHTASTIC_NODEDB_MIGRATION_VERBOSE so it compiles out when disabled
(~1.2 KB when off).
- mesh-pb-constants: drop the dead nRF52832 WARM_NODE_COUNT branch and trim
the macro docs.
- WarmNodeStore: simplify the redundant `ARCH_NRF52 && NRF52840_XXAA` guards
to `NRF52840_XXAA`, add a kNoPage sentinel for the ring page state.
- Shorten the always-on LOG_WARN strings (~120 B flash).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* more tidying up, aligning with docs and undoing other-arch regressions
* Update protobufs (#19)
Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com>
* made the migration pathway cleareer
* address copilot review
* fixed a copilot review on a downstream PR.
* Address Copilot review comments for PR #10705 (warmstore/nodedb)
- WarmNodeStore.h: default MIGRATION_VERBOSE to 0 (suppress info-level
chatter on production builds; opt in with =1)
- WarmNodeStore.cpp load(): move memset to top of function so all
failure paths (header-read fail, invalid header) leave entries clear
- WarmNodeStore.cpp save(): replace manual spiLock lock/unlock around
mkdir with LockGuard covering the full SafeFile sequence, matching
the lock discipline in load()
- Router.cpp: memcpy(&p->public_key.bytes, ...) -> memcpy(p->public_key.bytes,
...) — pass decayed uint8_t* rather than pointer-to-array
- AdminModule.cpp: check setProtectedFlag return for PKC auto-favorite;
log cap-refusal warning instead of unconditional "auto-favoriting"
- nrf52_warm_region.py: error message references both v6.ld and v7.ld
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* NodeDB: formatting cleanup (blank lines after preprocessor blocks)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Lukewarm store
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
|
||
|
|
5ffd30c4a8 |
Fix PKC on portduino sim by working around region blocks keygen and packet length (#10730)
* Fix PKC on portduino sim by working around region blocks keygen and packet length * Reserve Compressed framing overhead in sim loopback capacity check The sim loopback re-encodes the Compressed wrapper back into decoded.payload.bytes (the same 233-byte field its data is copied from), so the carried bytes must leave room for protobuf framing. Checking against sizeof(c.data.bytes) (233) let near-max payloads overflow pb_encode_to_bytes(), which returns 0 and silently sends an empty loopback payload. Cap at sizeof(decoded.payload.bytes) minus the worst-case Compressed framing (meshtastic_Compressed_size - data size) for both the ciphertext and plaintext paths. Addresses Copilot review feedback on #10730. * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Run trunk fmt --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
Jonathan Bennett
|
7424631a27 |
Beta fixes (#10728)
* Wipe message Store on factory reset * Check for destination 0 in a new message, and convert to broadcast * Make sure CHARGE_LED_state gets turned off, to avoid stuck LEDs * Take the spiLock in MessageStore when clearing messages * Trunk * Add thinknode M5 voltage curve * Fix the oops |
||
  Jonathan BennettGitHubBen MeadorsCopilotWesselcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>thebentern |
8267bb22bd |
Packet Signing via XEdDSA (#10478)
* Test commit for XEdDSA support * Update to Crypto lib in Meshtatic org * Generate a new node identity on key generation (#7628) * Generate a new node identity on key generation * Fixes * Fixes * Fixes * Messed up * Fixes * Update src/modules/AdminModule.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/NodeDB.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Figured it out! * Cleanup * Update src/mesh/NodeDB.h Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/NodeDB.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/modules/AdminModule.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update crypto commit hash * Some fixes for xeddsa pr (#9610) * fix: add null check for getMeshNode() in NodeInfoModule getMeshNode() can return nullptr for unknown nodes. Dereferencing without a check crashes the firmware when receiving NodeInfo from a node not yet in the database. * fix: enforce XEdDSA signature verification and prevent stripping Previously, failed signature verification still allowed the packet through, making signatures purely cosmetic. Now: - Failed verification drops the packet (DECODE_FAILURE) - Successfully verified nodes get HAS_XEDDSA_SIGNED bitfield set - Unsigned packets from previously-signing nodes are rejected - Log levels reduced from WARN/ERROR to DEBUG/WARN as appropriate * fix: include packet metadata in XEdDSA signature The signature now covers [fromNode | packetId | portnum | payload] instead of just the payload bytes. This prevents: - Replay attacks (different packetId fails verification) - Reattribution (different fromNode fails verification) - Portnum redirection (different portnum fails verification) Also adds a key initialization check to xeddsa_sign (returns false if XEdDSA keys are all zeros) and checks the return value in the encode path. * fix: handle existing key pair in AdminModule security config When a user provides both a valid private key and public key via admin config, the crypto engine's DH private key and owner public key were never loaded. DMs and XEdDSA signing would silently break. Add an else branch to load both keys into the crypto engine. * perf: cache Ed25519 public key conversion in xeddsa_verify curve_to_ed_pub() performs field element parsing, inversion, and multiplication on every call. Since packets from the same node tend to arrive in bursts, a single-entry cache avoids repeating this expensive conversion for consecutive packets from one sender. * fix: skip identity cleanup when node number is unchanged createNewIdentity() was called on every generateCryptoKeyPair(), including normal boots where the same key is regenerated. This caused unnecessary NodeDB writes and old-node cleanup logic to run when the node number hadn't actually changed. Also fixes only zeroing byte[0] of the old node's public key instead of clearing the entire array. * fix: replace hardcoded 120 with derived XEDDSA_SIGNATURE_SIZE constant The payload size check for XEdDSA signing used a magic number (120). Replace with a derivation from DATA_PAYLOAD_LEN and XEDDSA_SIGNATURE_SIZE so the limit adjusts automatically if constants change. This also increases the max signable payload from 120 to 169 bytes, which is still safe since the actual encoded size is checked after pb_encode. * fix: add const qualifiers to XEdDSA verify and curve_to_ed_pub inputs pubKey, payload, and signature parameters in xeddsa_verify are input-only and should not be modified. Same for curve_pubkey in curve_to_ed_pub. * chore: remove commented-out old Crypto dependency in portduino.ini * Leave out the admin module change for now --------- Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz> * trunk * protobuf re-update * Protobufs * Merge resolution fix * Put XEDDSA on the right bit * NodeDB update to new nodeInfoLite accessors, etc * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Refine unsigned packet rejection logic in Router (#10534) * use hardware random to fill the first 32 signature bytes with entropy prior to signing. * Add XEdDSA packet-signing policy tests and update dependencies for macos * Minor fixes * integrate XEdDSA support and update dependencies across multiple modules --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Wessel <github@weebl.me> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> |
||
|
07a87a8254 |
security: runtime-toggleable MESHTASTIC_LOCKDOWN hardening for nRF52 (#10349)
* security: add MESHTASTIC_LOCKDOWN hardened build option
Meshtastic nodes ship with secrets on flash (channel PSKs, the device
private key, admin keys, wifi PSK) and over-the-wire access to admin
APIs that can re-key the mesh. Lose the device, at a border crossing,
in a raid, off a backpack, and an attacker reads everything in 30s
with a USB cable. There's no at-rest encryption, no client auth, the
screen leaks contents, and SWD is wide open. This adds an opt-in
hardened build for users who care.
-DMESHTASTIC_LOCKDOWN=1 on nRF52 (CC310) turns on:
DEBUG_MUTE silence USB/serial logs
MESHTASTIC_ENCRYPTED_STORAGE AES-128-CTR + HMAC-SHA256 on
LocalConfig / channels / NodeDB.
Passphrase-gated DEK, TTL/boot
unlock token, failed-attempt
backoff (within-boot, wall-clock,
persisted bootsSinceFail).
MESHTASTIC_PHONEAPI_ACCESS_CONTROL per-connection auth gate. Secrets
emitted as empty proto structs
to unauthenticated clients.
MESHTASTIC_ENABLE_APPROTECT one-way UICR APPROTECT, reset
applied same boot. Recoverable
only via \`nrfjprog --recover\`,
which also wipes the DEK.
LockdownDisplay screen shows "LOCKED" when locked
or idle 30s. OLED only; InkHUD /
niche / device-ui not yet wired.
Wire format is the LockdownAuth / LockdownStatus pair from
meshtastic/protobufs#911 (AdminMessage tag 104, FromRadio tag 18).
Access-control state is a file-scope 6-slot table in PhoneAPI.cpp
keyed by \`this\`, not class members. Adding *any* per-instance field
to PhoneAPI breaks USB-CDC enumeration on the current nRF52 Adafruit
framework, one volatile bool was enough. Out-of-line side-steps it.
lockdown_auth is handled synchronously in PhoneAPI::handleToRadioPacket
rather than routed through the mesh Router into AdminModule. Two
reasons: the passphrase never travels through a routed MeshPacket
queue, and per-connection authorization runs while \`this\` is still on
the call stack. The previous async-via-router design lost connection
identity (g_currentContext was null by the time AdminModule processed
the auth), so per-connection unlock never actually took effect on the
originating client.
Non-nRF52: #warning, only DEBUG_MUTE activates. tools/lockdown_provision.py
drives provision / unlock / lock-now / watch over USB.
Display privacy is a screen-lock latch separate from storage-lock
state: shouldRedactDisplay() is true when storage is locked OR the
latch is set. Screen::setOn(false) sets the latch when the stock idle
timeout powers the display off (reusing config.display.screen_on_secs,
no second timer); it is cleared only when a client authenticates with
the passphrase. A device idling on the mesh keeps routing but hides
its screen until re-auth; button input wakes the backlight to the
LOCKED frame, not content. The earlier lockdown-specific 30s idle
timer is removed — it duplicated PowerFSM idle detection and showed a
misleading LOCKED screen on a merely-idle device.
Unlock-token TTL fix: a token carrying both a boot-count and a
wall-clock TTL is no longer destroyed when the RTC is invalid at cold
boot. The boot count is independently verifiable without a clock, so
the token falls back to boot-count enforcement instead of being
deleted. A token is only hard-rejected when its wall-clock TTL can be
evaluated and is found expired.
NodeDB::reloadFromDisk() after unlock is deferred to the main loop via
lockdownReloadPending rather than run inline on the transport callback
stack — the reload is too heavy for the BLE/serial task stack and was
resetting the device immediately after a successful unlock.
The screen-lock latch also swallows local input events in
InputBroker::handleInputEvent while it (or storage-locked) is set.
Without that, a blind operator could drive on-device menus, fire
canned messages, or change settings through the joystick/buttons even
though the screen content was hidden. PowerFSM is still triggered
first so the backlight wakes to the LOCKED frame; the event is dropped
before reaching the UI observers.
The screen-lock latch is initialised to true at boot, so even a
token-auto-unlocked cold boot comes up redacted. Otherwise an attacker
holding a screen-locked device could power-cycle it (the RAM latch
resets) and recover a content screen. After any boot, the operator
must authenticate from a client to reveal screen content.
MyNodeInfo.device_id is also redacted for unauthenticated clients —
it is a stable hardware identifier useful to an attacker for
fingerprinting / correlating the device across observations. The
public mesh fields (my_node_num, owner short/long name, public key,
hw model) are left as-is because they are already broadcast on-mesh.
ModuleConfig.mqtt is also redacted for unauthenticated clients —
MQTTConfig carries broker username, password, server address, and
root_topic. The empty MQTTConfig is emitted via the same zero-init
pattern as the other gated sections.
Uptime-based session limit (MESHTASTIC_LOCKDOWN_SESSION_DEFAULT_SECONDS)
caps how long a single auto-unlocked session can hold storage open,
measured in firmware millis() since unlock. 0 = unlimited (existing
token-only behavior, suitable for tower/infra nodes); non-zero arms a
timer on every passphrase unlock and on every token-auto-unlock that
inherits the value, since the cap is persisted in the token (token
format bumped to v2: adds sessionMaxSeconds, body 56→60 bytes).
On expiry the device revokes per-connection auth, re-engages the
screen-lock latch, and reboots WITHOUT deleting the token. Next boot
auto-unlocks via the boot count (decrementing it) and arms a fresh
session window. Hard exposure ceiling: bootsRemaining * sessionMaxSeconds.
Explicit user Lock Now still deletes the token (passphrase required to
recover); only session expiry preserves it.
Why uptime, not wall-clock: getValidTime() is fed by GPS/RTC/client
time pushes — all manipulable by an attacker with the device (GPS
spoof to roll the clock back, pull the RTC backup cell, Faraday-cage
the whole thing). millis() comes off the Cortex-M's internal cycle
counter, sealed inside the chip; the only way to reset it is a reboot,
which costs a boot from the on-flash token counter. APPROTECT remains
the load-bearing defense against forging higher boot counts via SWD.
A future LockdownAuth.max_session_seconds proto field will let the
client set this per-token; until that lands the build-time
MESHTASTIC_LOCKDOWN_SESSION_DEFAULT_SECONDS macro is the only source.
Session expiry now decrements the on-flash boot count in place and
re-arms the uptime timer WITHOUT rebooting, while budget remains.
Mesh routing keeps running across session boundaries; the device only
reboots when bootsRemaining reaches zero (rollback budget exhausted),
at which point it hard-locks and forces passphrase re-entry.
Each session boundary still: revokes per-connection admin auth so
clients must re-authenticate to see content, re-engages the screen
lock latch, and emits LockdownStatus{LOCKED, needs_auth, boots=N}
so connected clients see the decremented count and know to re-auth.
Storage stays unlocked (DEK in RAM) for continuity.
The boot count's role as the rollback ledger is unchanged — it
decrements monotonically once per session boundary, whether the
session ends in a reboot or an in-place roll. Attacker who power-
cycles to dodge the session timer still pays a boot via the existing
readAndConsumeToken decrement-at-load path. APPROTECT remains the
only defense against forging higher counts.
Net effect for an unattended/tower node with bootsRemaining=50,
sessionSeconds=3600: 50 hours of continuous mesh service, one
reboot at the end, vs. the previous design's 50 reboots over the
same period. Same exposure ceiling, far better uptime.
LockdownAuth.max_session_seconds (proto tag 5) is now consumed: when
non-zero the client value wins; 0 falls back to the firmware-side
MESHTASTIC_LOCKDOWN_SESSION_DEFAULT_SECONDS, matching the boots_remaining
sentinel convention. Protobufs submodule pin bumped to develop tip
which contains meshtastic/protobufs#916 (merged).
* security: drop dead is_managed allowlist for set_config(security).private_key
The 'isLockdownSecurityCmd' allowlist in handleReceivedProtobuf dates
from the pre-LockdownAuth design when the passphrase was smuggled
through SecurityConfig.private_key. With lockdown_auth handled
synchronously in PhoneAPI::handleToRadioPacket before any admin message
reaches the Router, this allowlist now serves no legitimate purpose
and lets an unauthenticated local client mutate security settings on
a managed device by setting private_key.size>=1 — including
potentially disabling is_managed itself.
Remove the allowlist. Managed-mode local admin now requires a
PhoneAPI connection that has already authenticated via lockdown_auth
(or, on the pki_encrypted branch below, a valid PKC admin key).
Resolves Copilot review feedback on src/modules/AdminModule.cpp:105.
* security: protect lockdown-status drain slot from concurrent writers
g_pendingLockdownStatus / g_hasPendingLockdownStatus are written from
multiple call sites (PhoneAPI::handleLockdownAuthInline on the BLE/USB
transport callback, AdminModule on the Router thread, main loop session
expiry) and read in getFromRadio() on whichever transport is draining
FromRadio. The struct read/write was unprotected, so a writer could
corrupt the slot mid-encode. Same pattern as nodeInfoMutex — wrap
both the queue path and the drain in a small lock. Drain re-checks
the bool under the lock to handle the case where another reader
grabbed the slot first.
Resolves Copilot review feedback on src/mesh/PhoneAPI.cpp:1560.
* security: derive readAndDecrypt size cap from caller buffer, not a hardcoded 64 KB
The MAX_PROTO_FILE_SIZE = 65536 + OVERHEAD ceiling was an absolute
constant chosen against a since-outdated assumption that 'meshtastic
proto files are well under 64 KB'. On variants where MAX_NUM_NODES
pushes the serialised NodeDatabase past 64 KB the legitimate file gets
rejected at load and the device treats its own real config as corrupt.
The caller already knows the maximum plaintext it expects (outBufSize).
Cap the ciphertext at outBufSize + OVERHEAD instead — this is the tightest
sound bound (anything larger could not possibly decode into the caller's
buffer), still defends against OOM / integer overflow, and scales with
the platform's actual NodeDB size rather than an arbitrary constant.
Resolves Copilot review feedback on src/security/EncryptedStorage.cpp:1327.
* docs: fix stale 'passphrase delivery via AdminModule' references in configuration.h
The lockdown overview comment block was written when passphrase delivery
ran through AdminModule's handleReceivedProtobuf. With the synchronous
refactor that path now lives in PhoneAPI::handleLockdownAuthInline,
called before the admin message reaches the Router. Update both the
nRF52 feature list and the non-nRF52 degraded-mode rationale to point
at the current code path.
Resolves Copilot review feedback on src/configuration.h:578 (and :604).
* docs: refresh unlock-token format doc to match v2 layout
The header comment for the UTOK file still described v1 (version 0x01,
no session_max_seconds, 71 bytes) even after the in-flight bump to
TOKEN_VERSION=0x02 and TOKEN_TOTAL_SIZE=75. The inline body-size
breakdown comment was also wrong (claimed 39 bytes and mismatched the
real NONCE_SIZE/AES_KEY_SIZE constants). Rewrite both to match the
actual on-flash layout and note how v1 tokens are handled on upgrade
(rejected via the version byte; passphrase re-entry mints a v2).
Resolves Copilot review feedback on src/security/EncryptedStorage.h:50.
* docs: correct session-limit comment re: token-auto-unlock behavior
The s_sessionMaxMs comment block claimed 'token-auto-unlocked
sessions have no session timer (the session feature is a
passphrase-unlock-only knob)'. Stale: readAndConsumeToken() now
persists sessionMaxSeconds in the token file and re-calls
setSession() from the token-load path, so token-auto-unlocked
sessions DO inherit the same cap (and consumeSessionBoot() re-arms
in place between sessions on a single boot). Update the comment to
match.
Resolves Copilot review feedback on src/security/EncryptedStorage.cpp:72.
* docs: clarify input-swallow gate re: screen-lock latch vs storage state
The previous comment said input is swallowed 'until a client authenticates
and unlockScreen() clears the latch (or storage is unlocked)'. The
parenthetical was misleading: storage being unlocked is not in itself
enough to clear the latch — the latch persists across the
storage-unlocked-but-screen-locked steady state, and only an explicit
unlockScreen() (called from a successful passphrase auth path) clears
it. Reword so the only-passphrase-clears-the-latch invariant is
explicit and local input is named as something that does NOT clear it.
Resolves Copilot review feedback on src/input/InputBroker.cpp:134.
* docs: fix reloadFromDisk() trigger comment in NodeDB.h
The header still claimed reloadFromDisk() is called by AdminModule
after a successful passphrase op. With the synchronous PhoneAPI
refactor the actual trigger is PhoneAPI::handleLockdownAuthInline
setting lockdownReloadPending, with main.cpp's loop() dispatching
the heavy reload on the main thread (the transport callback stack
isn't large enough). Update the comment to point at the real path
and explain why the deferral exists.
Resolves Copilot review feedback on src/mesh/NodeDB.h:393.
* style: clang-format lockdown sources
Apply trunk clang-format (16.0.3) to satisfy the format check.
* style: black-format lockdown_provision.py
Satisfy the trunk black formatter check.
* security: drop unused v1 EncryptedStorage formats and migration
This storage layer has never shipped, so there are no v1 DEK files,
v1 unlock tokens, or v1 backoff records anywhere to stay compatible
with. Remove the dead compatibility machinery:
- legacy init() (FICR-only KEK, no passphrase) — had no callers
- deriveKEKv1() / loadDEKv1() and the v1->v2 DEK migration paths in
provisionPassphrase() and unlockWithPassphrase()
- the 5-byte v1 backoff file format
Also drop the now-pointless version byte from the on-disk MENC, MDEK,
and UTOK formats. Each is identified by its 4-byte magic (and, for the
keyed formats, its HMAC); with only one version that will ever exist,
the version field added nothing. Sizes shrink by one byte each
(overhead 54->53, DEK 66->65, token 75->74).
Rename the surviving helpers to drop the _v2 suffix (deriveKEK,
loadDEK, saveDEK, KEK_DOMAIN). No behavioral change for provisioning,
unlock, token consumption, or session handling.
Verified with an nRF52 lockdown build (rak4631).
* fix(lockdown): harden auth-table and lockdown_auth handler (audit)
Audit findings addressed:
C3 — `~PhoneAPI()` now clears its auth slot unconditionally. The previous
slot-clear in `close()` was gated on `state != STATE_SEND_NOTHING`, so a
PhoneAPI that never reached config (or that already closed) left
`slot.who` pointing at freed memory; a future PhoneAPI heap-allocated at
the same address would inherit the prior session's authorization through
`findOrAllocSlot`.
C4 — All access to `g_authSlots`, `g_authEpoch`, and `g_currentContext` is
now serialised through `g_authSlotsMutex`. Previously these were touched
without locking from BLE/USB/TCP/Router tasks, so two parallel slot scans
could hand out the same slot and mid-update reads could observe
authorized=true alongside a stale epoch. Granularity is fine — every
critical section is a short linear scan over six entries, and getFromRadio
(which calls `getAdminAuthorized()` per redaction check) tolerates the
brief blocking.
A4 / H1 — `lock_now` now requires the originating connection to be
already authorized. Previously any unauthenticated client (BLE/USB/TCP)
could submit `lockdown_auth { lock_now=true }` and force a reboot,
which was a trivial local-presence DoS — an attacker near the radio
could brick-loop it indefinitely. The original "panic button without
auth" property is dropped; panic now requires the operator to have
passphrase-unlocked the connection.
H2 — Empty-passphrase `lockdown_auth` (with `lock_now=false`) used to
silently return success. The client received no feedback distinguishing
that case from a real success, and an attacker could probe lockdown
state for free. Now emits UNLOCK_FAILED with no backoff increment
(empty-passphrase is more likely a client bug than an attack, but the
honest signal still lets the client correct itself).
H14 — `la.boots_remaining > 255` previously truncated silently
(256 → 0 → mapped to TOKEN_DEFAULT_BOOTS=50; 257 → 1). Honest clients
could not detect the misbehavior. Now rejected explicitly with
UNLOCK_FAILED.
L1 — The `to == nodeDB->getNodeNum()` allowance in the unauth ToRadio
gate now also requires `getNodeNum() != 0`. During the locked-default
boot path `getNodeNum()` returns 0, so a packet with `to=0` could
otherwise satisfy the equality and bypass the gate.
L2 — Comment added on `g_authEpoch` wrap. Practically unreachable
(2^32 lockNow events on one boot), but worth recording the behavior.
M17 — `findOrAllocSlot_LH` now evicts the first unauthorized stale slot
when the table is full of non-nullptr entries, rather than failing
closed. Authorized slots are never evicted — they represent live
operator sessions. Fail-closed (with LOG_WARN) only when every slot
holds a different live authorized PhoneAPI, which would require seven
simultaneous authed connections.
M18 — `s_screenLocked` is now `std::atomic<bool>` with relaxed ordering.
Plain bool happened to work on single-core Cortex-M4 today but breaks
silently if lockdown ports to ESP32 / RP2040, or under LTO whole-
program elision.
Verified with an nRF52 lockdown build (rak4631).
* fix(lockdown): gate every admin op on per-connection auth + storage unlock
Audit findings addressed:
H6 — Unauthenticated local clients could previously set_config / set_module_config /
set_channel etc. on a lockdown device whenever is_managed was unset.
The previous gate inside AdminModule's is_managed branch consulted
PhoneAPI::isLocalAdminAuthorized(), which reads a global g_currentContext
set during synchronous PhoneAPI dispatch — but AdminModule runs on the
Router task, by which time the dispatch task has exited and the global is
unrelated to the originating connection. The check was both broken (always
false on Router, so even authed clients were rejected) and unsafe (when it
did fire, the wrong connection could be authorized).
The fix relocates the gate to PhoneAPI::handleToRadioPacket, where dispatch
is synchronous and getAdminAuthorized() can be trusted. The admin payload
is already decoded there to extract lockdown_auth; extend the same branch
so that any non-lockdown_auth admin variant from an unauthorized connection
is dropped before ever reaching the Router queue.
H7 — Same root cause: get_config_request / get_module_config_request /
get_channel_request handlers returned full security/network/mqtt content
to unauthorized local clients. With the H6 gate in PhoneAPI, these
requests never reach AdminModule, so handleGetConfig / handleGetModuleConfig
/ handleGetChannel are only callable from authorized connections.
H9 — Remote admin (PKC-authorized peers, mesh-relayed admin) bypassed
lockdown entirely. If admin_keys were baked in via USERPREFS or set on a
prior unlocked boot, a remote attacker could drive factory_reset /
set_config against a locked device before the operator ever unlocked it.
Added an EncryptedStorage::isUnlocked() early-return at the top of
AdminModule::handleReceivedProtobuf. The local lockdown_auth path is
unaffected because PhoneAPI handles it synchronously before AdminModule
runs.
H10 — Removed g_currentContext, the ContextGuard, authorizeLocalAdmin(),
and isLocalAdminAuthorized() entirely. The audit's race (Router-thread
reads a pointer set by an unrelated parallel dispatch and authorizes the
wrong PhoneAPI) and the always-false-on-Router behavior both disappear
with the code that produced them. The PKC-admin auto-authorize path is
gone — PKC admin and the per-connection lockdown auth are now
independent: clients using PKC admin from a local app must also send
lockdown_auth to unlock the redacted FromRadio stream.
Cleaned up AdminModule's is_managed branch: under lockdown the
PhoneAPI-layer gate has already done its job, so no additional check
is needed; without lockdown the legacy is_managed-blocks-plain-admin
semantics are preserved.
Verified with an nRF52 lockdown build (rak4631).
* fix(lockdown): hold radio silent until storage is unlocked
Audit finding H8: while locked, the device beaconed nodeinfo and
telemetry on the public LongFast default PSK and routed incoming default-
channel packets through the locked router. The locked-default boot path
in NodeDB::loadFromDisk installs config via installDefaultConfig, which
honours USERPREFS_CONFIG_LORA_REGION (the common shape for managed
deployments) and synthesises the default LongFast channel. So a locked
device on managed firmware came up TX-enabled on a well-known PSK
before any operator interaction.
Force config.lora.region = UNSET in the locked-boot block.
RadioLibInterface gates both TX (startSend) and RX (readData) on
region != UNSET — locked devices no longer initialise the SX12xx for
either direction. Also set tx_enabled = false for any code path that
checks the flag directly without consulting region.
reloadFromDisk() restores the persisted lora config once the operator
unlocks. Note: until the audit's M8 (radio re-init after reload, the
upcoming commit 5 in this remediation series) lands, an unlocked
device may need to reboot before its radio fully comes up under the
real config; this is no worse than the pre-fix state, where the radio
was already running on the wrong (default) config and any real config
change required an explicit reconfigure or reboot anyway.
Verified with an nRF52 lockdown build (rak4631).
* fix(lockdown): per-connection status queue, redaction expansion, log/banner mute (audit)
M14 — Replaces the single file-scope LockdownStatus slot with a per-
PhoneAPI table keyed by PhoneAPI*, parallel to the auth-slot table and
sharing g_authSlotsMutex. Previously a status produced for connection
A (UNLOCKED with the active TTL, or UNLOCK_FAILED with a backoff)
could be drained by connection B before A read it, leaking A's auth
state to B. queueLockdownStatus is now a per-instance method writing
to this->slot. A new static broadcastLockdownStatus exists for the
main-loop session-expiry callers that have no PhoneAPI* in hand —
those want every connected client to learn about the session roll,
which is the only legitimate broadcast use case. hasPendingLockdownStatus
is a const helper for the FromRadio available()/drain check.
M13 — buildStatus_LH (the single point where lock_reason crosses into
the on-wire LockdownStatus) collapses any token_* reason to a generic
"locked" before emission. The specific reasons (token_hmac_fail,
token_wrong_size, token_bad_magic, token_boots_zero, token_expired,
token_dek_fail, token_missing) still go to local logs, but no longer
tell an unauthenticated client that the firmware noticed their
tampering / rollback / corrupt-file attempt.
M15 — Extended the STATE_SEND_MY_INFO redaction (previously device_id
only) to also wipe pio_env and min_app_version for unauth clients —
both are pure build-fingerprint vectors that tell an attacker which
known issues to probe. Kept my_node_num (broadcast on the mesh anyway)
and nodedb_count (clients need it post-unlock to decide whether to
pull the node DB). Added equivalent redaction for STATE_SEND_METADATA:
the whole DeviceMetadata struct is wiped for unauth clients
(firmware_version, device_state_version, hw_model, hw_model_string,
has_bluetooth/has_wifi/has_ethernet, role, position_flags,
excluded_modules). Clients re-fetch after authenticating.
M16 — LoRa config is now whitelisted for unauth clients to the set
that is intrinsically observable on the air anyway: region,
modem_preset, use_preset, channel_num, hop_limit. Operator-private
knobs (ignore_incoming, override_duty_cycle, override_frequency,
sx126x_rx_boosted_gain, tx_power, ignore_mqtt, fem_lna_mode,
config_ok_to_mqtt) are zeroed. The whitelist is built as a fresh
LoRaConfig stack copy rather than masked in place to avoid touching
the persisted struct.
M12 — Skip the DEBUG_MUTE "we are muted, FYI" banner under
MESHTASTIC_LOCKDOWN. The banner spilled APP_VERSION / APP_ENV /
APP_REPO over USB CDC even with all other logging suppressed, which
defeats the muting in lockdown builds and gives a USB-attached
attacker a free firmware-fingerprint primitive.
L9 — Removed the numeric backoff value from the LOG_WARN unlock-
failed message. The client receives backoff_seconds via the
UNLOCK_FAILED status; printing it again to USB serial under
non-DEBUG_MUTE builds (i.e. MESHTASTIC_LOCKDOWN_DEBUG dev builds)
was the only place it appeared in logs.
Verified with an nRF52 lockdown build (rak4631).
* fix(lockdown): atomic post-unlock reload with corruption surface (audit)
Closes M6, M7, M8, M9 from the lockdown security audit.
M6 — handleLockdownAuthInline no longer flips the connection to
authorized or emits UNLOCKED on the cold-unlock path (the first
successful passphrase verify after a locked boot). The client keeps
seeing LOCKED until reloadFromDisk has actually populated config /
channelFile / nodeDatabase with the operator's real values. Without
this, the window between the auth call and the main-loop reload
exposed two race-friendly bugs: (a) the client could read the
locked-default placeholders as if they were the real config, and (b)
a set_config in the window would silently overwrite a corrupted
baseline once the reload swapped values in.
A new per-status-slot bool pendingUnlockAfterReload records that the
connection is mid-unlock. The re-verify path (storage already
unlocked) is unchanged and authorizes immediately — there is nothing
to reload.
M7 — reloadFromDisk now holds a new file-scope mutex
(g_reloadFromDiskMutex) against itself, parks the radio in sleep
mode before swapping config / channelFile, and reconfigures the
radio with the now-real settings after. Other readers of config.lora
/ channelFile / nodeDatabase do not take this lock today; closing
those races is a wider locking-discipline change outside the audit's
M7 scope. The radio standby+reconfigure prevents the SX12xx from
sitting in a half-old/half-new register set across the swap, which
otherwise required a reboot to recover from.
M8 — RadioInterface::reconfigure() is now called at the end of a
successful reload, so the SX12xx register set actually reflects
the unlocked operator settings (region, modem preset, channels)
rather than staying on the locked-default placeholder. Routed through
a new Router::getRadioIface() accessor — the radio interface is
owned by Router as a unique_ptr and was not exposed.
M9 — NodeDB::loadProto now sets a NodeDB::storageCorruptThisLoad
flag whenever an encrypted file fails to decrypt or proto-decode.
reloadFromDisk consumes the flag and returns false on any failure
instead of silently falling back to defaults. main.cpp's reload
service then calls EncryptedStorage::lockNow() and
PhoneAPI::revokeAllAuth(), and the new
PhoneAPI::completePendingUnlocks(false) emits LOCKED(storage_corrupt)
to every pending connection — they stay unauthorized so any
set_config they send is dropped at the existing unauth gates.
The lock_reason string passes through buildStatus_LH's M13
redaction unchanged because it does not start with token_.
The success path goes through PhoneAPI::completePendingUnlocks(true)
which authorizes each pending connection, emits UNLOCKED with the
current TTL, and clears the screen-lock latch once. Snapshots the
target PhoneAPI* list outside the auth-table lock to avoid re-entry
when setAdminAuthorized takes the same lock.
Verified with an nRF52 lockdown build (rak4631).
* fix(lockdown): UI/pairing fixes for first-pair + content-flash + e-ink (audit)
Closes H13, M19, M20, L4 from the lockdown audit. (L3 dropped per
explicit decision — battery level is not a meaningful security side
channel.)
H13 — BLE pairing PIN was suppressed by the lockdown lock screen on
locked devices. Screen.cpp updateUiFrame's lockdown short-circuit
intercepts before ui->update() runs, so the pairing-PIN overlay
banner that NRF52Bluetooth::onPairingPasskey queued never painted.
Net effect: a freshly-locked device on first BLE pair could not be
unlocked over BLE because the operator could never see the PIN —
chicken and egg.
Adds a new notificationTypeEnum::pairing_pin value and special-cases
it in the short-circuit: paint the LOCKED frame first (so the
underlying background remains the redacted view, never dashboard
content) then let ui->update() composite the PIN banner overlay on
top. The PIN itself is an ephemeral pair-handshake artifact
(regenerated per attempt, dies on banner timeout) and is not
operator content, so this does not regress the redaction guarantee.
NRF52Bluetooth::onPairingPasskey switches from showSimpleBanner to
showOverlayBanner with notificationType = pairing_pin so the
short-circuit's lookup matches.
M19 — Brief content-visible window on Screen::handleSetOn(true)
wake. OLED GDDRAM physically retains the last-rendered frame while
the panel is powered off; the next ui->update() after displayOn() is
async, so an observer (or shoulder-surfer) could see the previous
frame's content for 16-50 ms on every wake. Under MESHTASTIC_LOCKDOWN
we now paint the LOCKED frame into GDDRAM in handleSetOn(false)
before calling displayOff(). On wake the only thing the panel can
flash is the redacted view. Gated on lockdown only — non-lockdown
builds keep the previous frame as a UX cue.
M20 — E-ink panels physically retain the last-rendered image
without power. A power-cycled lockdown handheld kept showing
operator-identifying content (position, messages, nodeinfo) until
the firmware's first natural refresh — which on e-ink can be
seconds into boot. Now, under MESHTASTIC_LOCKDOWN && USE_EINK, the
panel init path in Screen::setup() paints the LOCKED frame and
forces a full refresh (forceDisplay) immediately after ui->init()
and before any other rendering. Persistent pixels are wiped to the
redacted view before an observer can see them. Build-tested on
seeed_wio_tracker_L1_eink; hardware-verified visual confirmation
is pending a T-Echo session.
L4 — Screen::blink() bypasses the normal ui->update() path that
the lockdown short-circuit gates. It draws arbitrary geometry, not
node data, so it does not actually leak today; but any future
change that puts content into blink would silently leak past
redaction. Added an early-return on shouldRedactDisplay() to make
the function honor the redaction contract.
Verified with nRF52 lockdown builds on both rak4631 (OLED) and
seeed_wio_tracker_L1_eink (e-ink).
* fix(lockdown): refuse APPROTECT on vulnerable silicon, gate on provision (audit)
Closes M22 and M23 from the lockdown audit.
M22 — APPROTECT lockout on nRF52840 is publicly known to be bypassable
on every silicon revision shipping in current Meshtastic hardware
(AAB0..AAF0) via SWD glitching, per LimitedResults' published research
on the nRF52 series. Engaging APPROTECT on these revisions has two
bad properties: (1) the lockout is irreversible without a destructive
nrfjprog --recover, and (2) it gives the operator a false sense of
security because the lockout itself can be defeated by anyone with
ten minutes and a glitcher.
enableAPProtect() now reads FICR.INFO.VARIANT (encoded as a 4-byte
ASCII word) and refuses to engage on any known-vulnerable revision,
logging the variant so the operator knows their device's specific
build code. To override (e.g. for end-to-end testing of the engage
path on hardware that's known affected), rebuild with
-DMESHTASTIC_APPROTECT_OVERRIDE_VULNERABLE_SILICON=1.
The vulnerable list is explicit and easy to update: any future
revision shown to be fixed can be removed from the list and APPROTECT
will engage on it as before.
M23 — APPROTECT engagement moved from very early in setup() to
after fsInit() + EncryptedStorage::initLocked(), and gated on
EncryptedStorage::isProvisioned(). A misconfigured CI build of a
lockdown variant flashed to a dev board would otherwise burn SWD on
first boot before the operator had set any passphrase, taking the
board out of the development/recovery workflow with zero real
security benefit (there is no DEK to protect on an unprovisioned
device). Engagement now follows operator intent: SWD locks only
once they've committed to lockdown via passphrase provisioning.
The SWD-attachable window between boot and APPROTECT engagement
widens slightly from this reorder (now ~hundreds of ms while fsInit
runs) but APPROTECT remains effective on the only payload it could
protect (the in-RAM DEK loaded by initLocked which now runs *after*
APPROTECT for already-provisioned devices).
Verified with an nRF52 lockdown build (rak4631).
* tools: harden lockdown_provision.py (audit)
Closes M26-M30 and addresses L7.
M26 — passphrase input. --passphrase on argv now requires
--insecure-passphrase-on-cmdline as an explicit acknowledgement;
without it the tool refuses and points at --passphrase-file or the
interactive prompt. --passphrase-file refuses to read anything that
isn't mode 0600 (so a passphrase another user can read off the
filesystem doesn't silently succeed). With neither, the tool reads
the passphrase via getpass.getpass — and on 'provision' double-prompts
with a confirm.
M27 — provision now requires an explicit 'yes' confirmation unless
--yes is passed, after printing the warning that the passphrase
cannot be recovered. The double-passphrase prompt is built into
gather_passphrase(confirm=True). Reduces the chance of a typo
binding a device to an unrecoverable passphrase.
M28 — 'lock' subcommand gains a 'lock-now' alias, matching how the
audit and wire docs refer to it everywhere. Both forms now require
'yes' confirmation unless --yes is set, so an accidental command
doesn't immediately reboot the device into a locked state.
M29 — the 4-second sleep is gone. Replaced with a StatusFuture
single-shot that the FromRadio interceptor signals when the next
LockdownStatus arrives. provision/unlock/lock wait up to --wait
seconds (default 8) for the actual reply and exit non-zero with the
device's reason on UNLOCK_FAILED, surfacing backoff_seconds in the
error line. Exit codes are now meaningful:
0 = UNLOCKED
1 = no status / unexpected
2 = NEEDS_PROVISION (or a precondition fault: missing pkg, bad args)
3 = LOCKED (ambiguous: device reported locked rather than the
expected unlocked result)
4 = UNLOCK_FAILED
This lets ops scripts decide what to do without parsing stdout.
M30 — top-of-file docstring gained an explicit SECURITY MODEL block
that names the threat model (USB-only, passphrase cleartext on the
cable) and forbids extension to TCP/BLE/UDP without a redesign. A
runtime banner reprints the headline on every invocation. --port
values starting with tcp:/tcp://, ble:/ble://, udp:/udp://, ws:/wss:
are rejected at argument parse before any connection attempt; a
copy-paste of an example into a context with a different --port
cannot silently leak credentials to the wire.
L7 — private meshtastic APIs (_handleFromRadio, _sendToRadio,
_generatePacketId) are still in use because the lib does not yet
dispatch LockdownStatus on a public pubsub topic and there is no
public seam for raw ToRadio. Their use is now wrapped in
getattr-with-clear-error so a future lib version that removes them
produces an actionable error instead of an obscure traceback. The
top-of-file note explains why we're on the private surface.
Verified end-to-end on hardware (R1-Neo + Seeed Wio Tracker L1)
during the audit-remediation hardware test pass:
- provision (interactive, with confirm and double-prompt)
- unlock (success returns UNLOCKED + boots TTL)
- watch (passive listener emits LockdownStatus events)
- lock-now (with --yes)
* fix(lockdown): H13 — render pairing PIN steady over LOCKED frame
Two bugs in the H13 fix from commit 614b7f001:
1. NotificationRenderer::drawBannercallback's switch had no case for
the new notificationTypeEnum::pairing_pin. The function fell through
to no-op so the banner never rendered. Added pairing_pin alongside
text_banner so it dispatches to drawAlertBannerOverlay (same
rendering, distinct type so the lockdown short-circuit in Screen.cpp
can recognise it).
2. updateUiFrame's lockdown short-circuit called ui->update() to
composite the banner. That redraws the current carousel frame
(the dashboard) into the host framebuffer BEFORE the overlay
paints, so the panel flashed dashboard content under the banner
on every cycle. Replaced with a direct call to drawBannercallback
so only the banner box is painted on top of the LOCKED pixels.
Also: drawLockdownLockScreen used to commit to the panel
(display->display()) at its end. With the banner overlay then
painting and committing a second time, the panel visibly flickered
between 'just LOCKED' and 'LOCKED + banner' on every render cycle.
Split into drawLockdownLockScreenIntoBuffer (no commit) for the
lockdown short-circuit, and a thin drawLockdownLockScreen wrapper
that calls Buffer + display() for the other call sites that don't
composite anything on top. The short-circuit now commits exactly
once per frame after both LOCKED + any overlay are in the buffer.
Verified end-to-end on hardware (Seeed Wio Tracker L1, OLED):
fresh BLE pair against a locked device now shows the pairing PIN
steadily on top of the LOCKED frame, no flicker, no dashboard
leak, and pair completes normally.
* fix(lockdown): backoff MAC + atomic writes + fault wipe + size cap (audit)
Closes H3, H4, H12, M10, M11, M25 from the lockdown audit. Non-format-
breaking: existing devices keep their .dek and .unlock_token but their
old plaintext .backoff file (6 bytes, no MAC) is silently rejected as
tampered on first read and reseeded with the MAC'd 38-byte format on
the next failed-attempt OR successful unlock.
H3 — Pre-increment the failed-attempt counter BEFORE running the HMAC
verify in unlockWithPassphrase. The previous order wrote the counter
only after a failed verify, so an attacker glitching the chip between
verify and write could skip the increment and bypass backoff. The
slot is now reserved atomically up front; the success path writes
attempts=0 to clear the reservation. Worst case for a legitimate user
who power-cycles mid-success is one phantom attempt — backoff
recovers next try.
H4 — .backoff file is now MAC'd with HMAC-SHA256(ephemeralKEK,
"backoff-auth" || body) (32-byte tag), and written atomically via
SafeFile (tmp + readback verify + rename). readBackoff treats
missing / wrong-size / MAC-fail uniformly as max-attempts (255) so an
attacker who deletes or rewrites the file can only INCREASE the wait,
never decrease it. clearBackoff() now writes an attempts=0 sentinel
instead of removing the file, so 'missing == tamper' is unambiguous
post-provision. bumpBootsSinceFailOnBoot() skips on un-provisioned
devices to avoid false 'tamper' detection during the legitimate fresh
window between fsInit and provisionPassphrase.
H12 — saveDEK and writeUnlockToken now write via SafeFile in
fullAtomic mode (tmp file + readback verify + atomic rename) instead
of remove-then-open-then-write. Power loss during a DEK or token
write previously left the device unable to unlock — the encrypted
prefs files are unreadable without a valid DEK. The atomic path
rolls back to the previous file on partial write.
M10 — readAndConsumeToken's 74-byte stack buffer (entire wrapped
DEK + HMAC, explicitly called out by the audit as never wiped before
return) is now a meshtastic_security::ZeroizingBuffer that the
destructor scrubs on every return path. Same treatment for the
computedHmac stack array next to it, and for the new backoff state
buffers in readBackoff / writeBackoff / computeBackoffHmac. Removes
the manual secure_zero calls those buffers had on success paths and
fixes the missing wipes on the failure-return paths.
M11 — Added EncryptedStorage::secureWipeKeys() public API that
zeros dek/kek/ephemeralKek in BSS without touching flash, no
logging, no locks (safe from interrupt context). HardFault_Impl now
calls it as the very first thing on entry, before the diagnostic
print / coredump path runs, so a hard-fault crash dump won't capture
the DEK / KEK material that the rest of the module leaves in RAM.
M25 — migrateFile now refuses to allocate a buffer for any file
larger than 64 KiB. The legitimate ceiling is well under that on
every supported variant; anything larger is either corrupt or a
DFU-injected OOM attempt.
Verified with an nRF52 lockdown build (rak4631).
* fix(lockdown): MENC header MAC + token rollback counter (audit)
Closes M2 and M4 from the lockdown audit. **FORMAT-BREAKING** — devices
provisioned with prior lockdown firmware must factory-erase /prefs and
reprovision; the previous tokens and encrypted prefs files will not
decrypt under the new HMAC/body layouts.
M2 — The HMAC on MENC encrypted proto files now covers the full on-disk
header (4-byte magic + 13-byte nonce + 4-byte plaintext_len + ciphertext)
instead of just (nonce + ciphertext). Without this, magic and
plaintext_len were integrity-protected only by the equality check
`plaintextLen == ciphertextLen` — which holds today (no padding /
compression / AAD) but would silently produce length-oracle and
downgrade vulnerabilities the instant any of those got added. Putting
the header inside the MAC closes that pre-condition cleanly. The
verify side in readAndDecrypt and the compose side in encryptAndWrite
update in lockstep.
M4 — UTOK gains a 4-byte monotonic counter field inside its MAC'd
body. The highest counter ever issued is persisted to a new
/prefs/.tokmono file MAC'd with HMAC-SHA256(ephemeralKEK,
"tokmono-auth" || counter). On every readAndConsumeToken, any
token whose counter is less than the persisted value is rejected as
a rollback attempt and deleted. Defeats the audit's threat: an
attacker who once captured a token (e.g. bootsRemaining=255 from
before the operator lowered the policy) tries to write it back to
disk later. Counter is incremented monotonically across the device's
lifetime so any captured snapshot loses to the persisted max-seen.
Self-heal: a token whose counter exceeds the persisted value (e.g.
the .tokmono write itself failed after the token committed, or the
.tokmono got wiped via factory-erase) is accepted AND the counter
file is promoted to match. This avoids spuriously rejecting valid
tokens after partial-update recovery.
Threat model caveat (consistent with C2 acceptance): an attacker who
has both flash extraction AND FICR can recompute the .tokmono MAC
and restore a matching pair (.unlock_token + .tokmono) from an
earlier capture. M4 raises the bar to that combined capability;
the flash-write-only attacker is now blocked.
Verified with an nRF52 lockdown build (rak4631).
MIGRATION: devices already provisioned with the prior lockdown
firmware will fail to auto-unlock at boot (token format mismatch),
fall back to LOCKED(needs_auth), and every passphrase attempt will
fail because the encrypted /prefs files are HMAC'd against the old
input. Recovery is: factory-erase via the bootloader UF2 then
re-provision via lockdown_provision.py or the Android app.
* feat(lockdown): make lockdown a runtime client-toggleable setting
Converts MESHTASTIC_LOCKDOWN from a per-variant compile-time flag that
forced lockdown ON into an internal capability that is ALWAYS compiled
in for nRF52 and gated purely at runtime by whether a passphrase has
been provisioned. A device that has never been provisioned (or that the
operator disabled) behaves exactly like stock firmware.
Build/config:
- configuration.h auto-defines MESHTASTIC_LOCKDOWN (+ ACCESS_CONTROL,
ENCRYPTED_STORAGE, APPROTECT-capable) for ARCH_NRF52 unconditionally.
No variant sets -DMESHTASTIC_LOCKDOWN anymore. Flash-constrained
variants can opt out with -DMESHTASTIC_EXCLUDE_LOCKDOWN=1. DEBUG_MUTE
is no longer coupled to lockdown (a capable-but-off device must log
normally). rak4631 lands at 96.2% flash with lockdown always-in.
Runtime predicate:
- EncryptedStorage::isLockdownActive() == isProvisioned() (.dek exists)
is the single source of truth for active/inactive.
- PhoneAPI::getAdminAuthorized() returns true when lockdown is inactive,
so every existing redaction gate no-ops on a capable-but-off device
with no per-site changes. The locked-boot defaults path (NodeDB), the
AdminModule storage-locked gate, the screen-redaction predicate, and
the plaintext->encrypted migrate block are all additionally gated on
isLockdownActive() so an un-provisioned device loads/serves plaintext
normally.
- sendConfigComplete emits LockdownStatus{DISABLED} when capable-but-off
so the client renders its toggle OFF.
Enable (off->on): client provisions a passphrase. provisionPassphrase
generates the DEK; the existing reload path encrypts the plaintext
config in place (migration runs live with the DEK in RAM) and authorizes
the connection -> UNLOCKED. No reboot.
Disable (on->off): LockdownAuth{passphrase, disable=true}. PhoneAPI
verifies the passphrase (loads DEK), sets lockdownDisablePending; the
main loop runs NodeDB::disableLockdownToPlaintext() which decrypts every
pref via EncryptedStorage::migrateFileToPlaintext() then
removeLockdownArtifacts() deletes the DEK/token/counter/backoff (the
.dek delete is the atomic commit), then reboots into normal mode.
Power-loss safe and re-runnable without a persistent marker — and the
crypto runs live with the operator's passphrase in RAM rather than via
a boot-time marker an attacker could plant to trigger an unprompted
decrypt. APPROTECT is NOT reversed (sticky; permanent on silicon where
it engaged).
Generated bindings (admin.pb.h / mesh.pb.h) regenerated against
protobufs#927 (LockdownAuth.disable, LockdownStatus.State.DISABLED).
Submodule pointer stays at the pinned develop commit; the bindings are
ahead until #927 merges and the submodule is bumped, same flow as the
max_session_seconds work.
Builds clean: rak4631 with no flags now auto-includes lockdown.
NOTE: this changes the LockdownStatus the firmware emits and adds the
disable path; pairs with protobufs#927 and the upcoming Android client
toggle work.
* fix(lockdown): re-lock per-connection auth on BLE reconnect
A provisioned device reused a single BLE PhoneAPI instance, and the
per-connection auth slot (keyed by that instance) was only cleared on the
!isConnected() disconnect transition. A fast disconnect/reconnect could
begin a new config burst while state was still STATE_SEND_PACKETS, so the
reconnected client inherited the prior session's authorization: it received
SecurityConfig in the clear and no LockdownStatus, and never re-authenticated.
Reset the auth slot in NRF52Bluetooth onConnect(), which fires once per
physical link, so every new connection starts locked regardless of whether
the previous link's close() raced the new handshake. handleStartConfig keeps
its !isConnected() reset (do NOT reset on a same-connection want_config: the
post-unlock re-fetch is the client pulling now-unredacted config and must keep
the auth it just earned, otherwise config comes back redacted and set_config
writes get dropped).
* fix(lockdown): persist config on a lockdown-capable but disabled device
saveProto always called encryptAndWrite when encrypted storage was compiled,
and saveToDiskNoRetry skipped every save when !isUnlocked(). On a disabled
(never provisioned) device there is no DEK and isUnlocked() is always false,
so both paths fired and NO config ever persisted: a LoRa region set before
enabling lockdown lived only in RAM, then provisioning migrated the UNSET
default from disk and the region was lost.
Gate both on isLockdownActive(): when lockdown is inactive the device writes
plaintext exactly like stock firmware; the reloadFromDisk migrate pass then
re-saves those plaintext files encrypted once the device is provisioned.
Verified on hardware: region set while disabled now survives enable, reboot,
and unlock.
* fix(lockdown): suppress LoRa region picker under the lock screen
A locked-boot lockdown device installs region=UNSET as a deliberate RAM
placeholder (the real region is in encrypted storage, restored on unlock).
Screen.cpp popped the region picker / onboard message whenever region==UNSET,
so it rendered over the lock screen and trapped input with no way out. Skip it
while the display is being redacted for lockdown.
* fix(lockdown): silence cppcheck void* false positive + ruff docstring lints
The nRF52 `check` (cppcheck --fail-on-defect=low) flagged
arithOperationsOnVoidPointer on EncryptedStorage.cpp buffers. These are
false positives: make_zeroizing_array() returns unique_ptr<uint8_t[], ...>
so .get() is uint8_t*, not void* — cppcheck just can't resolve the
custom-deleter alias. File-scoped suppression, matching the existing
crypto-code convention in suppressions.txt.
Trunk flagged 5 ruff docstring issues in lockdown_provision.py: D301
(backslashes need a raw docstring) and D405/D407/D411/D413 (the EXAMPLES
heading was being parsed as a numpydoc section). Made the docstring raw
and renamed the heading to USAGE to dodge section detection while keeping
the ASCII-box formatting.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(lockdown): resolve cppcheck const/null-deref defects
The nRF52 `check` job (pio check --fail-on-defect=low) flagged seven
real cppcheck defects in the lockdown code:
- EncryptedStorage.cpp: nonce/encDek are read-only views into the
token buffer -> const uint8_t *.
- NodeDB.cpp: segments[] lookup table is never mutated -> const.
- PhoneAPI.cpp: clearStatusSlot_LH's p is only compared; the auth-check
slot and the hasPendingLockdownStatus loop var are read-only -> const.
- Screen.cpp: the MESHTASTIC_LOCKDOWN drawLockdownLockScreen() guard
introduced a redundant null check (nullPointerRedundantCheck) since
dispdev->displayOff() right below derefs it unguarded, as does the
rest of the file. Dropped the guard.
Verified with cppcheck 2.21 locally against the project suppressions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(lockdown): const-qualify clearAuthSlot_LH param (cppcheck cascade)
Making clearStatusSlot_LH take const PhoneAPI* let cppcheck propagate the
same to clearAuthSlot_LH, whose p is only compared and forwarded. The
remaining PhoneAPI* params (findOrAlloc*Slot_LH) store p into the slot
table, so they correctly stay non-const.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(lockdown): wire runtime-toggle disable flow into provision tool
Addresses Copilot review on tools/lockdown_provision.py — the reference
tool advertised the runtime-toggle disable lifecycle but couldn't exercise
it:
- _STATE_NAMES: map LockdownStatus.DISABLED so a capable-but-off boot
prints DISABLED instead of an opaque state=<num>.
- build_lockdown_auth(): add a disable param that actually sets
la.disable, failing loudly on pre-runtime-toggle bindings instead of
silently sending a plain unlock.
- cmd_disable() + 'disable' subcommand: send LockdownAuth{disable=true,
passphrase=...} and wait for the resulting LockdownStatus. Mirrors the
firmware: non-empty passphrase required, DISABLED broadcast precedes
the reboot, TTL/session fields ignored.
- _exit_code_for_status(): treat DISABLED as a success (exit 0) like
UNLOCKED.
All DISABLED/disable references are hasattr-guarded so the tool still
imports and runs the lock/unlock/provision paths against the currently
released meshtastic package (verified: it has LockdownAuth but not yet
disable/DISABLED). Verified with ruff 0.15.13 and black.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Ben Meadors
|
c2bcec93d0 | Fix long name clamping and adjust related structures for compatibility | ||
|
Thomas GöttgensandGitHub
|
8c4900a52f |
Prevent ghost nodes during onboarding (#10647)
* Prevent ghost nodes during onboarding * Coplilot is exceptionally nit-picky today |
||
|
|
de345939af |
Automatic variable hop limits based on mesh activity and size estimation (#10176)
* asdf * Implement SphereOfInfluenceModule for traffic management and eviction tracking * Implement Sphere of Influence module for dynamic hop limit adjustment and role-based floor * Update SAMPLING_DENOMINATOR to improve mesh size estimation accuracy * Add debug logging for scale factor estimation and per-hop node counts in SphereOfInfluenceModule * Enable variable hop limits and role-based hop floors in Sphere of Influence module * Respond to copilot review * Disable variable hop limits and role-based hop floors in Sphere of Influence module * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Implement adaptive sampling for unique node ID tracking in Sphere of Influence module * Add state persistence for Sphere of Influence module * Enhance Sphere of Influence module with state management and adaptive sampling adjustments * Refactor hop scaling functionality: remove SphereOfInfluenceModule and introduce HopScalingModule - Deleted SphereOfInfluenceModule.h, consolidating its responsibilities into the new HopScalingModule. - Added HopScalingModule.h and HopScalingModule.cpp to manage hop scaling logic, including eviction tracking and sampling-based mesh size estimation. - Implemented methods for recording evictions and packet senders, estimating scale factors, and computing required hops based on node activity. - Introduced state persistence for hop scaling parameters to maintain continuity across reboots. - Enhanced thread safety and modularity by utilizing concurrency features. * Guard out STM32. Sowwy. * Refactor HopScalingModule: enhance sampling logic and improve state management * Add unit tests for HopScalingModule: implement mock database and various test scenarios * Refactor test output in HopScalingModule tests: replace printf with TEST_MESSAGE for better integration with Unity * Refactor HopScalingModule logging: replace lastStatusMode with descriptive mode names for improved readability * Refactor test_main.cpp: change NodeNum variable to static and improve comments for clarity * Remove unnecessary delay in setup function for improved test performance * Add missing include for MeshTypes in test_main.cpp * Refactor HopScalingModule tests: enhance mesh topology scenarios and improve test clarity * Update HopScalingModule tests: flesh out node scenarios and improve clarity for dense and sparse mesh cases * Fix politeness factor calculations in HopScalingModule and update related test scenarios for clarity. Remove outdated design doc. * Enhance HopScalingModule: add sampled estimate for scaling decisions and refactor initial run state management * Add sample traffic injection for HopScaling tests to enhance sampledEst visibility * Enhance HopScalingModule: adjust windowFraction calculation for early triggers and improve test output formatting * Enhance HopScalingModule: add jitter functionality to sampling denominator and update tests for consistent behavior * Enhance HopScalingModule: implement adaptive sampling denominator adjustment and add reset functionality for tests * Enhance HopScalingTestShim: add test-only clock and window helpers, update injectSampleTraffic for adaptive sampling, and improve scenario summary output * Enhance HopScalingModule: add detailed documentation for functions, improve clarity of jitter and sampling logic, and reset functionality in tests * Enhance HopScalingModule: add evictionEstimate parameter to estimateScaleFactor and update related logging for improved mesh size estimation * Enhance HopScalingModule: adjust effective rolls calculation for improved accuracy, add eviction estimate logic, responding to all copilot review points * Implement CompactHistogram for parallel hop scaling sampling - Added CompactHistogram class to track node hop distances with bitwise sampling. - Integrated CompactHistogram into HopScalingModule for independent packet sampling. - Updated NodeDB to feed both the hop scaling module and the new histogram sampler. - Enhanced HopScalingModule with methods to sample packets for the histogram and retrieve hop distribution statistics. - Implemented tests for CompactHistogram functionality, including sampling, window rolling, and adaptive denominator scaling. - Updated existing tests to validate the integration of the new histogram sampling mechanism. * Enhance CompactHistogram and HopScalingModule: add per-hop distribution functionality, improve time handling for unit tests, and refine test setup for deterministic behavior * CompactHistogram: add mesh size estimation, improve entry replacement logic, and update logging for per-hop distribution * Refactor HopScalingModule and CompactHistogram integration - Removed the suggestedHopFromCompactHistogram function to streamline hop suggestion logic. - Updated HopScalingModule to directly utilize CompactHistogram's internal methods for hop suggestions and sampling. - Enhanced logging in HopScalingModule to provide detailed histogram statistics. - Modified test cases to ensure comprehensive coverage of new histogram behaviors and sampling logic. - Improved node ID distribution in tests to better exercise sampling mechanisms. - Ensured that filtering denominators are held for 12 hours before dropping, enhancing stability in sampling. * Refactor CompactHistogram to support 13-hour activity tracking and introduce politeness regimes - Updated the bitfield structure to accommodate 13-hour seen tracking. - Changed the logic in rollHour() to analyze activity over the last 0-2 hours vs. 1-3 hours for politeness factor calculation. - Introduced three politeness levels: GENEROUS, DEFAULT, and STRICT based on recent activity ratios. - Adjusted filtering and sampling logic to reflect the new 13-hour tracking period. - Updated unit tests to validate new behavior and ensure proper functionality of politeness regimes. * Enhance CompactHistogram and HopScalingModule for improved sampling and decision-making - Introduced a session-specific hash seed in CompactHistogram to reduce bias in node ID sampling. - Updated sampling logic to use hashed node IDs instead of raw IDs for filtering and entry management. - Added histogram rollover tracking in HopScalingModule to ensure proper decision-making after initial data collection. - Adjusted logging to reflect the active state of the histogram and its comparison with NodeDB advisory hops. - Enhanced unit tests to validate new sampling logic and memory layout changes. * Expose hashNodeId for testing in CompactHistogram * Add mesh trend statistics to CompactHistogram for enhanced activity tracking * Implement histogram state persistence in CompactHistogram with save and load functions * Refactor CompactHistogram to improve entry management and enhance rollHour logging * feat: add HopScalingModule for adaptive hop limit recommendations Introduces HopScalingModule, a sampled hop-distance histogram that recommends the minimum hop limit needed to reach ~40 nodes, and automatically reducing the hops as the mesh grows. Key design: - 512-byte packed histogram (128 × 4-byte Record entries) embedded in a new HopScalingModule. - Each Record: 16-bit node hash, 3-bit hop distance, 13-bit seen bitmap - Sampling filter: only nodes where (hash & (denom-1)) == 0 are kept; denominator doubles on overflow and halves when utilisation is low - Hourly rollHour(): tallies per-hop counts, walks scaled buckets to find the minimum hop satisfying TARGET_AFFECTED_NODES (40), applies a politeness extension based on recent/older activity ratio, shifts all seen bitmaps, and persists state to /prefs/hopScalingState.bin - Hop recommendation gated by bootstrap (requires >=1 rollHour before overriding HOP_MAX) - NodeDB calls samplePacketForHistogram() on every non-MQTT rx packet - Module also estimates total mesh size and logs useful information about mesh characteristics. Changes: - src/modules/HopScalingModule.h/.cpp: new module - src/mesh/NodeDB.cpp: wire up samplePacketForHistogram - src/mesh/Router.cpp: consume getLastRequiredHop() - test/test_hop_scaling/: 12-test suite covering all mesh topologies and anticipated operational requirements * test: increase run iterations in sparse to dense transition test * feat: refactor HopScalingModule to use RUNS_PER_HOUR constant and improve logging * feat: enhance HopScalingModule with filtering denominator management and add tests for state transitions * refactor: remove CompactHistogram module and related files * address copilot review comments * Tweak: packet sampling only lora * ove role-based hop floor logic and related definitions into the module - keep it in one place. * Refactor MockNodeDB to use nodeInfoLiteSetBit for MQTT flag setting * Refactor hop scaling parameters and logic to integer maths and put default values in defaults.h. Small flash size reduction, no functional impact. * Update unit test preprocessor directives to PIO_UNIT_TESTING for consistency * refactor: improve test output organization and clarity in hop scaling tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Thomas GöttgensandGitHub
|
1747e2d8e5 | T-Echo-Card support (#10267) | ||
     AustinGitHubmverch67ManuelCatalin Patuleacopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>vidplace7CopilotthebenternBen Meadors |
a541957480 |
ESP32: Migrate to Arduino 3.x (pioarduino) (#9122)
* Migrate esp32 families to pioarduino platform * ESP32c6 align text.handler_execute same as C3 * Use pioarduino `develop` The latest fixes and the latest bugs! * preliminary esp32p4.ini * pioarduino: Update LovyanGFX Includes Manuel's recent commit * pioarduino 3.3.6 * pioarduino 3.3.6 *release* chasing the release * pioarduino: Fix OG ESP32 duplicate libs * pioarduino: T-Beam 1W CDC mode * pioarduino: disable network provisioning (wifiprov) * pioarduino: use legacy esptoolpy naming (forward-compatible) * Update lovyangfx from `develop` commit to 1.2.19 * fix esp32p4.ini * check for esp32 w/ wifi * esp32-p4 specific adaptations * Switch to meshtastic/esp32_https_server fork (idf5 branch) * don't ignore esp_lcd * config for MUI * fix/workaround SDMMC * revert a6f6175, update to 3.3.8 * enable esp_hosted for esp32-p4 (experimental) * Pioarduino 55.03.38-1 * NimBLE-Arduino -> Arduino "BLE" (3.3.x) migration (#10164) * NimBLE-Arduino -> Arduino "BLE" (3.3.x) migration * More NimBLE * Fix Device Name in ATT Read Request (0x2A00). Device Name is exposed in two places: - Advertisement data: this is set properly in startAdvertising. - GATT attribute Device Name (0x2A00). This one is handled internally in NimBLE and comes from ble_svc_gap_device_name_set. This is set initially, but then BLEDevice::createServer calls ble_svc_gap_init which resets the device name. This causes the device to apparently "change name after pairing": < ACL Data TX:... flags 0x00 dlen 7 #113 [hci0] 14.241149 ATT: Read Request (0x0a) len 2 Handle: 0x0003 Type: Device Name (0x2a00) > ACL Data RX: Handle 2048 flags 0x02 dlen 11 #115 [hci0] 14.269050 ATT: Read Response (0x0b) len 6 Value[6]: 6e696d626c65 # "nimble" Workaround this by setting the device name once again after BLEDevice::createServer. * Temporarily lower CORE_DEBUG_LEVEL to INFO to avoid triggering an apparent ESP-IDF Bluetooth bug when re-connecting to Pixel 8 Android devices. Initial pairing works, but after ESP32 is rebooted, phone fails to reconnect. Meshtastic app shows it as disconnecting immediately. LightBlue shows a more detailed error "Peripheral Connection - Warning: onConnectionStatusChange: status 61" (0x3D - MIC Failure). Bug report to Espresssif: https://github.com/espressif/esp-idf/issues/18126#issuecomment-4286197744 * Temporarily disable ble_gap_set_data_len, causes crash with Pixel 8 Android reconnect. Crash looks like this: [ 11966][E][BLEAdvertising.cpp:341] setScanResponseData(): ble_gap_adv_rsp_set_data: 22 [ 11975][E][BLEAdvertising.cpp:1554] start(): Host reset, wait for sync. ERROR | ??:??:?? 11 BLE failed to start advertising Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled. Core 0 register dump: PC : 0x420e6190 PS : 0x00060730 A0 : 0x820e158b A1 : 0x3fce50c0 A2 : 0x00000000 A3 : 0x3fcb8600 A4 : 0x3fcb85cc A5 : 0x00000000 A6 : 0x00000000 A7 : 0x00000c03 A8 : 0x00000000 A9 : 0x3fce50b0 A10 : 0x0000000e A11 : 0x00000000 A12 : 0x00000010 A13 : 0x3fce50e0 A14 : 0x00000c03 A15 : 0x00000001 SAR : 0x0000001e EXCCAUSE: 0x0000001c EXCVADDR: 0x00000000 LBEG : 0x400570e8 LEND : 0x400570f3 LCOUNT : 0x00000000 Backtrace: 0x420e618d:0x3fce50c0 0x420e1588:0x3fce5110 0x420dfe87:0x3fce5200 0x420dfefb:0x3fce5220 0x420dff3f:0x3fce5240 0x4219602b:0x3fce5260 0x4037b0e5:0x3fce5280 0x4201edf3:0x3fce52a0 Connection seems fast enough even without this. We'll investigate the reason for the crash and re-enable once it's safe. --------- Co-authored-by: Catalin Patulea <cronos586@gmail.com> * Add extension from pioarduino nag "Jason2866.esp-decoder" * Cleanup after merge * ESP32: Disable classic bluetooth * Cleanup: Fix ADC channels on new variants * InkHUD: Fix type casting for message size in saveToFlash method inkhud compiles again! * update p4 esp_hosted for BT * I thought I fixed this * fix linker error using response file (p4 only) * fix infinite loop * Fix Power.cpp check warning Local variable 'config' shadows outer variable [shadowVariable] * Build ESP32 original with NimBLE ('custom_sdkconfig' approach). (#10235) * Re-enable littlefs json manifest This works locally again :) Not sure what changed * Re-add tool-mklittlefs * sensecap indicator fixes after upgrade arduino-esp & lovyanGFX libs * hackaday fix * robot tbeam cache error fix Co-authored-by: Copilot <copilot@github.com> * trunk fmt * ignore trunk * BLEDevice::deinit() added Co-authored-by: Copilot <copilot@github.com> * platformio-custom: Modify mtjson target dependency to prevent fake-success. (#10291) Co-authored-by: Copilot <copilot@github.com> * Fix ESP32-C6 linker errors. Align .text.handler_execute section to 4 bytes and update watchdog timer core mask configuration Co-authored-by: Copilot <copilot@github.com> * tlora-c6: Disable Screen MESHTASTIC_EXCLUDE_SCREEN=1 on tlora-c6. It doesn't have a screen, and this gets it compiling again (saving flash). * Use mverch's iram_memset hack for all OG-ESP32 * Refactor watchdog timer initialization and handling * use adc_channel_t in variant.h * Fix variant headers * More idiomatic default ethernet that doesn't break the build * Elecrows: Delete problematic variant.cpp Not needed after USE_ETHERNET_DEFAULT --------- Co-authored-by: mverch67 <manuel.verch@gmx.de> Co-authored-by: Manuel <71137295+mverch67@users.noreply.github.com> Co-authored-by: Catalin Patulea <cronos586@gmail.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: vidplace7 <1779290+vidplace7@users.noreply.github.com> Co-authored-by: Copilot <copilot@github.com> Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |