b2bc33cb2d0af4686f7737257e7899aa50141dca
292
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Thomas GöttgensandGitHub
|
a5940b4c6c |
Guard the deferred local queue and depth counter (#11284)
* Guard the deferred local queue and depth counter * Close the drain and enqueue race on the deferred queue * Route the raced loopback through handleReceived * Make the last-frame check and depth decrement atomic * Correct the handleReceived doc comment |
||
|
Thomas GöttgensandGitHub
|
9c260ad792 |
Signature key resolution and a send-path leak (#11283)
* Verify signatures against authoritative keys only * Release the packet on the unset-variant send error |
||
|
Thomas GöttgensandGitHub
|
df6e67f70b |
Signing and ingress hardening (#11282)
* Include warm-tier signers in the identity update gate * Fail the send when PKI encryption fails * Require signatures on licensed unicasts * Include warm-tier signers in the NodeInfo downgrade drop * Clamp hop fields on UDP multicast ingress * Address review comments on signing hardening Condense the updateUser rationale to two lines and stop calling the Balanced-mode drop a broadcast now that licensed unicasts reach it. |
||
 TomandGitHub
|
2024bb8384 |
Arrival time fix perhaps (#11274)
* Add explicit presence for MeshPacket.rx_time (arrival time) rx_time is now proto3 optional with a has_rx_time presence bit, matching the rx_rssi treatment. A node with no GPS and no phone connected yet has no time source at all, so a bare 0 was indistinguishable from a genuine 1970-01-01 reading; downstream consumers (replay packets, JSON serialization) now check has_rx_time instead of the value. * Dedupe rx_time stamping into a shared helper; trim a debug log string Extract the repeated haveTime/rx_time/has_rx_time stamp logic (5 call sites across Router.cpp, MeshBeaconModule.cpp, MeshService.cpp) into Router::computeRxTimeStamp()/stampRxTime(). Also shorten the new RTC.cpp LOG_DEBUG string. Saves 48 bytes of flash on rak4631 (measured), no behavior change. * Fix has_rx_rssi presence carried unconditionally through StoreForward replay preparePayload() set has_rx_rssi = true unconditionally on replay, regardless of whether the packet's rx_rssi at store time was a genuine measurement (e.g. MQTT-relayed packets carry no real RSSI). Store the presence bit alongside rx_rssi in PacketHistoryStruct and restore it on replay instead. Flagged by Copilot on #11271 (same root cause the has_rx_time explicit presence work fixes) but never addressed before that PR merged. * Trim comment blocks to the repo's 1-2 line guideline .github/copilot-instructions.md:338 caps code comments at 1-2 lines; several blocks added across the rx_time explicit-presence work ran well past that. Also consolidates Time.cpp's file-level doc comment into Time.h, where the rest of the Time:: API contract already lives. No behavior change. * Add rx_time explicit-presence test coverage - test_meshpacket_serializer: has_rx_time=false fixture plus tests asserting JsonSerialize/JsonSerializeEncrypted emit 0 rather than leaking the millis() placeholder, alongside the has_rx_time=true baseline. - test_stream_api: two tests driving a real PhoneAPI handshake (want_config_id through STATE_SEND_PACKETS) that simulate a phone time-giving transaction arriving before vs. after a queued packet is drained - covering both the reconciled and the ships-with-placeholder-absent paths of MeshService::reconcilePendingRxTimes(). * Fix three correctness issues flagged in review - Time.h: drop the reserved-identifier include guard (_MT_TIME_H); pragma once already covers it, matching convention elsewhere (e.g. RTC.h). - Time.cpp: rebase getMillis64()'s wrap accumulator when the test seam swaps clock sources, so a real<->injected clock jump isn't miscounted as a genuine 32-bit wrap. - NodeInfoModule: the 12h reply-suppression window is a local dedup duration, not a wall-clock reading - switch it to Time::getMillis64() so RTC-quality jumps and replayed packets' stale rx_time can't perturb it. - StoreForwardModule: has_rx_time was derived from *current* RTC quality at replay time rather than stored at capture time, so a history entry saved while time-blind could be misreported as a valid epoch once the clock later improved. Persist the presence bit in PacketHistoryStruct instead. * tryfix CI * post review fixes * more test fixes |
||
  
|
1e982fa78c |
Sign plaintext packets in licensed mode (#10969)
* feat: sign licensed plaintext packets Preserve and publish identity keys in licensed mode so existing XEdDSA signatures can authenticate plaintext traffic. Sign licensed broadcasts and direct messages when they fit, while keeping PKI encryption disabled and normal routing behavior unchanged. * fix(security): persist licensed channel sanitation * fix(security): close licensed migration lifecycle gaps * fix(security): address licensed signing review feedback * test: restore signing globals from Unity teardown * fix(baseui): allow confirmed ham region selection * fix(baseui): guard region picker validation * style: trunk fmt --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Austin <vidplace7@gmail.com> |
||
|
Thomas GöttgensandGitHub
|
b46c8c9f80 |
Router: defer nested local sends to avoid handleReceived re-entrancy (#11191)
Prevents an nRF52 task-stack overflow on config save. Replaces draft #11155. |
||
 nomdetom
|
583248c19d | claw back some space from signing policies | ||
|
Thomas GöttgensandGitHub
|
f09f2b1366 | Gate the PKI-decrypt key on signer-proven, not the opportunistic cache (#11116) | ||
|
Ben MeadorsandGitHub
|
e11635baa2 | Merge branch 'develop' into codex/packet-auth-policy | ||
|
Thomas GöttgensandGitHub
|
d587f08481 |
Pin admin responses to the stored peer key and request id (#11092)
* Pin admin responses to the stored peer key and request id noteOutgoingAdminRequest derived its PKC pin from p.public_key, which nothing populates on the outgoing path, so keyValid was false for every client request and the pin never engaged. The accepted-response predicate reduced to an unauthenticated from plus variant and subtype. Resolve the destination key from NodeDB the way perhapsEncode does, and pin it only when the request would actually be PKC-encrypted. Extract that condition from perhapsEncode as wouldEncryptWithPKC so both use one predicate. Also record the request's packet id and require the response to echo it. * Treat a zero request id as no pairing token |
||
|
Ben Meadors
|
7d954e668d |
Merge branch 'develop' into codex/packet-auth-policy
Resolve conflicts against the NodeDB signer/key primitives (#11050) and the admin-key PKI decrypt budget (#11100). - NodeDB: drop this branch's hasSeenXeddsaSigner in favour of develop's isKnownXeddsaSigner. They answer the same question, but develop's reads the dedicated warm signer bit (warmSignerOf) rather than the WarmProtected category, and TrafficManagementModule already depends on it. Keep develop's copyPublicKey/copyPublicKeyAuthoritative, isVerifiedSignerForKey and commitRemoteKey/KeyCommitTrust. - checkXeddsaReceivePolicy: keep this branch's Strict/Balanced/Compatible policy, which is a superset of develop's balanced-only downgrade gate, and call isKnownXeddsaSigner from it. develop's !pki_encrypted term is dropped because the policy returns early for PKI packets before that check. - perhapsDecode: keep develop's key resolution (NodeDB then pending-key, only for real PKI candidates) plus its admin-key token bucket, and re-apply this branch's pkiAttempted flag feeding the DECODE_OPAQUE verdict. Keep both passesRoutingAuthGate and adminKeyFallbackAllowed/Refund. - test_A17: model eviction the way NodeDB actually does it, passing the warm signer bit as well as the XeddsaSigner category, since isKnownXeddsaSigner reads the former. Native suite: 38 suites, 743/743 cases, no sanitizer findings. |
||
|
1872e5b306 |
TMM extend to ephemeral 3rd tier store (#11050)
* Harden TMM NodeInfo direct-response: staleness, key hygiene, throttle The NodeInfo direct-response path answered queries on behalf of other nodes from an unauthenticated, never-expiring cache while suppressing the genuine request (STOP). That let a long-gone or forged identity be served as a fresh-looking, authoritative reply indefinitely. Three fixes: - Staleness gate: refuse to spoof a reply for a node not actually heard within ~6h (PSRAM cache via lastObservedMs; NodeDB fallback via sinceLastSeen, which tolerates last_heard==0 when the clock is unset). A stale entry now falls through so the real request propagates instead of being answered for a dead node. - Key hygiene: mirror NodeDB::updateUser() PKI checks when caching overheard NodeInfo - reject a packet advertising our own public key, and pin keys (drop a NodeInfo whose key mismatches an already-known key from NodeDB or from our own cache). Prevents cache poisoning / key substitution via the spoofed-reply path. - Response throttle: at most one spoofed reply per target node per 30s. Direct responses bypass the per-sender rate limiter (they STOP the request first) and the reply target is attacker-controlled, so this bounds airtime exhaustion and reflected floods. Recorded in a new NodeInfoPayloadEntry.lastResponseMs (PSRAM; self-expiring timestamp compare, no sweep needed). Tests: native NodeDB-fallback staleness (stale drops, fresh serves) plus PSRAM-guarded staleness, throttle, and key-mismatch cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Add TODO(T1-T9) markers for code-review findings on NodeInfo hardening Comment-only. Tags each site flagged in the review of the direct-response throttle/staleness/key-hygiene change so the follow-ups are visible in-context: T1 throttle black-holes distinct requestors; T2 per-target key doesn't bound aggregate TX; T3 non-PSRAM fallback unthrottled; T4 millis-wrap defeats the staleness gate; T5 redundant second O(n) scan for the throttle stamp; T6 duplicated 32-byte key compares; T7 sinceLastSeen() called twice; T8 two 6h constants can desync; T9 clockMs()==0 collides with the 0 sentinel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Address code-review cleanups T6, T7, T8 in NodeInfo direct-response T8: derive kNodeInfoMaxServeAgeSecs from kNodeInfoMaxServeAgeMs so the PSRAM and NodeDB-fallback staleness windows cannot desync. T6: collapse the three duplicated 32-byte key compares in cacheNodeInfoPacket (owner-impersonation + NodeDB pin + cache TOFU pin) into one pubKeysEqual() helper, so the compare lives in one place. T7: compute sinceLastSeen(node) once on the fallback staleness path so the tested age and the logged age cannot diverge (it calls getTime() each time). Remaining review items still marked in-code: T1/T2/T3 (throttle design), T4/T9 (millis-wrap + 0-sentinel), T5 (redundant throttle-stamp scan). Native test_traffic_management suite: 48/48 passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Throttle the NodeDB fallback direct-response path (T3) The per-target NodeInfo response throttle lived only in the PSRAM NodeInfoPayloadEntry, so non-PSRAM boards - which answer from the NodeDB fallback path - emitted spoofed direct replies with no rate limit at all, leaving the airtime/reflection surface the throttle exists to close. Add a single module-global fallback stamp (nodeInfoFallbackLastResponseMs, 4 bytes, guarded by cacheLock) that throttles the fallback path to one spoofed reply per window across all targets. Coarser than the PSRAM per-target throttle, but the fallback path has no per-node slot to stamp, and a global cap still bounds spoofed transmissions - which is the point. Add a native test exercising the fallback throttle window. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Make NodeInfo staleness/throttle wrap- and sentinel-safe (T4, T9) T9: the millis-based fields that use 0 as a "never" sentinel (lastObservedMs, lastResponseMs, nodeInfoFallbackLastResponseMs) could be stamped with a literal 0 during the one-millisecond clockMs()==0 instant at each ~49.7-day wrap, momentarily colliding with the sentinel and disabling the staleness/throttle gate for that entry. Route all such stamps through a new nowStampMs() that maps 0 -> 1; the 1 ms skew is irrelevant to every window these fields feed. T4: the staleness gate's modular age compare is only unambiguous while true age < 2^32 ms (~49.7 days); an entry that lingered unrefreshed that long could wrap to a small age and read as fresh, defeating the gate. Add a NodeInfo eviction pass to the maintenance sweep that drops entries past the serve window, so an entry is removed long before its age can approach the wrap boundary. This also frees slots holding stale identities. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Stamp PSRAM throttle entry by captured index, not a rescan (T5) The post-send throttle stamp did a second full O(n) scan of the PSRAM NodeInfo array under a fresh lock, even though findNodeInfoEntry() had already located the slot at the top of shouldRespondToNodeInfo(). Capture the slot index during that initial lookup and address the entry directly on the stamp path. Because the cache lock is released between the two accesses, the slot could have been evicted or reused, so re-validate node == p->to under the lock before stamping. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Document accepted per-target throttle tradeoffs (T1, T2) Convert the T1/T2 TODO markers into a NOTE: the per-target PSRAM throttle keying and its lack of an aggregate bound are deliberate design choices, not pending work. A distinct requestor being throttled for one window is harmless on a redundant mesh, and keeping the PSRAM path per-target preserves throughput for legitimate multi-target responders (the fallback path already bounds aggregate via its global stamp). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Track signed-provenance of cached NodeInfo public keys Add keySignerProven to NodeInfoPayloadEntry, set when an observed NODEINFO frame's XEdDSA signature was verified (mp.xeddsa_signed). This distinguishes a trust-on-first-use key from one proven to belong to a signer. The flag is monotonic - once proven it stays proven, and the existing key-pin checks forbid the underlying key from changing - so a later unsigned frame cannot downgrade it. A signature can only be verified against a key we already held, so a first-contact key is always TOFU until a later signed frame upgrades it. Groundwork for using the TMM cache as a last-resort public-key source, where this flag serves as a trust tier. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Draw public keys from the TMM NodeInfo cache as a last resort Add TrafficManagementModule::copyPublicKey() and consult it from NodeDB::copyPublicKey() after the hot (NodeInfoLite) and warm (WarmNodeStore) tiers miss. This extends the pool of peers the node can encrypt to: a key the NodeInfo direct-response cache overheard for a node that has since aged out of both NodeDB tiers can still be used. The getter reports whether the key is signer-proven or trust-on-first-use. NodeDB serves TOFU keys here too - the same first-contact trust NodeDB already applies in updateUser() - so the pool actually expands to new long-tail nodes rather than only re-confirming known keys. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Retain keyed NodeInfo entries as a pubkey pool; trust-tier eviction Now that NodeDB::copyPublicKey() draws keys from the NodeInfo cache, the flat 6 h serve-window eviction would discard useful keys the moment a node stops being servable. Split retention: an entry carrying a 32-byte public key is kept for kNodeInfoKeyRetentionMs (7 d), while a keyless entry still expires at the serve window. Both windows stay well under the ~49.7-day millis wrap, preserving the T4 wrap-safety guarantee. Make LRU victim selection trust-tiered to match: a keyless slot is sacrificed before any keyed one, and a trust-on-first-use key before a signer-proven key; within a tier the oldest loses. Mirrors WarmNodeStore's keyed-first admission so the most valuable keys are the stickiest under memory pressure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Test signed-provenance flag and copyPublicKey key-pool source PSRAM-gated tests (run on the ESP32 build, alongside the existing NodeInfo PSRAM tests): - copyPublicKey serves a TOFU key learned from an unsigned NodeInfo and reports signerProven=false - a later signature-verified NodeInfo upgrades provenance to signer-proven while the pinned key bytes stay unchanged - copyPublicKey reports a miss for an uncached node Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Inherit signer provenance from NodeDB when caching a re-found node When the TrafficManagement NodeInfo cache re-caches a node, mark its key signer-proven if NodeDB already knows the node as a verified signer for that same key - even if this particular (unicast/unsigned) frame carried no signature. Previously the flag only upgraded on a frame we verified ourselves, so a node we had already proven elsewhere looked TOFU here. Add NodeDB::isVerifiedSignerForKey(), which checks both tiers - the hot store's signed bitfield and the warm tier's cached signer bit - and requires the key to match so a rotated/mismatched key never inherits a stale verdict. Add WarmNodeStore::isVerifiedSigner() to expose the warm signer bit (the rebase onto develop added the bit itself; this surfaces it for lookups). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Lock NodeInfoPayloadEntry packing with a size assert keySignerProven cost zero bytes: sourceChannel, the two bools, and decodedBitfield are four 1-byte fields that fill a single 4-byte tail word (struct alignment is 4), so the flag consumed former padding rather than growing the 2000-entry PSRAM array. Add a static_assert pinning the entry to sizeof(meshtastic_User) + 20 so a future 5th trailing byte - which would open a fresh word (~8 KB PSRAM across the array) - fails the build instead of silently costing memory, prompting new flags to be packed into existing bytes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Pack NodeInfo cache booleans into 1-bit fields Fold hasDecodedBitfield and keySignerProven into adjacent uint8_t:1 bitfields so they share a single byte, reserving 6 spare bits for future flags without growing the 2000-entry PSRAM array. sizeof is unchanged (still one packed tail word, sizeof(meshtastic_User) + 20); access is by name exactly as before, so no call sites change. Reorders decodedBitfield ahead of the flags so the two 1-bit members stay adjacent and the compiler packs them together. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Gate NodeInfo replay on signer-proven provenance (compile-time, default on) Add TMM_NODEINFO_REPLAY_REQUIRE_SIGNED (default 1): the direct-response path now only spoofs a reply for a node whose key is signer-proven, in addition to the staleness gate - so replay is based on flagging AND staleness. Replay is a courtesy feature, and vouching for an unverified (trust-on-first-use) identity to other nodes is the risk this closes, so signed-only is the safer default. Define the macro to 0 at build time to also serve fresh TOFU-only nodes. Both paths are gated: the PSRAM path checks the cached keySignerProven flag, the NodeDB fallback checks nodeInfoLiteHasXeddsaSigned(node). The effective gate (TMM_NODEINFO_REPLAY_SIGNED_GATE) is bypassed when PKI is excluded from the build, since nothing can be signed there and the courtesy feature would otherwise be disabled outright. Tests: existing reply-expecting tests establish signer-proven state (a new markKeySignerProvenForTest hook for the PSRAM cache, the NodeDB signed bit for the fallback path); add fallback and PSRAM tests asserting an unsigned/TOFU node is withheld under the default gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Rehydrate re-admitted node names from the TMM NodeInfo cache The warm tier keeps an evicted node's key but not its name (the 40-byte record has no room), so a re-admitted long-tail node is nameless until its next NodeInfo broadcast. The TrafficManagement NodeInfo cache is much larger (2000 PSRAM entries) and commonly still holds the full User, so use it as a name reservoir the warm tier structurally cannot be. On re-admission (getOrCreateMeshNode), after the warm-tier restore, copy the cached identity from the TMM cache via a new copyUser() getter - but only when its cached key matches the key just restored from warm, so a name never attaches to a different identity than the one we encrypt to (key-matched trust). Guarded by HAS_TRAFFIC_MANAGEMENT and a null check; a no-op without the PSRAM cache or when no key is present. CopyUserToNodeInfoLite sets only user-related bits, so the warm-restored signer bit is preserved. Limitations (by design): PSRAM-only, and the TMM cache is RAM-only, so this helps within an uptime session, not across reboot. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Drop non-portable NodeInfoPayloadEntry size assert (fixes pico build) The static_assert pinned sizeof(NodeInfoPayloadEntry) to sizeof(meshtastic_User) + 20, but nanopb packs the generated User struct differently per platform: on pico its size is not a multiple of 4, so alignment padding before the uint32 timestamps makes the overhead 22, not 20, and the assert failed the build (native happened to be +20 and passed, hiding it). The bitfield packing it was guarding still stands; replace the fixed-count assert with a comment, since no portable byte count exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G7QggdwMM1CJkguuCmMfZs * Run the NodeInfo cache paths in native tests (TMM_HAS_NODEINFO_CACHE) The NodeInfo direct-response cache and everything layered on it - key pinning, signer provenance, staleness, throttle - was compiled only for ARCH_ESP32 + BOARD_HAS_PSRAM, so its entire test set was skipped by the native CI suite and ran nowhere but real hardware. Fold the scattered platform guards into one TMM_HAS_NODEINFO_CACHE macro that also enables the cache (plain heap, delete[] path already existed) for ARCH_PORTDUINO unit-test builds. Production portduino and embedded test builds are unchanged. Tests that exercise the NodeDB fallback path drop the cache explicitly via a new dropNodeInfoCacheForTest() hook, so both response paths are covered in one binary. Native suite: 60/60 (was 50 with the 10 cache tests compiled out). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit b2dafbbbd6501b42fc27175e599fb38be292b0e3) * Pin NodeInfo cache keys against the warm tier, not just the hot store cacheNodeInfoPacket() mirrored NodeDB::updateUser()'s key pin but checked only getMeshNode() - the hot store. updateUser's own pin effectively covers the warm tier too (getOrCreateMeshNode rehydrates the warm key before the check), so the mirror had a gap: for a node evicted to the warm tier whose cache slot was also gone, an attacker's NodeInfo with a bogus key passed the pin, and the cache's TOFU pin then locked the genuine node's frames out until the poisoned entry aged away. Split the authoritative lookup out of NodeDB::copyPublicKey() as copyPublicKeyAuthoritative() (hot store, then warm tier - never the opportunistic TMM tier, which would compare the cache against itself) and pin against that. Test: warm-tier-only key rejects a mismatching NodeInfo and accepts the matching one. Native suite: 61/61. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit 28e880875ef1f8bf16bdf301a4e6036537a84b4b) * Convert NodeInfo cache clocks to uint8 tick counters Replace the entry's uint32 millis stamps (lastObservedMs, lastResponseMs) with three uint8 free-running tick clocks - the same modular scheme the UnifiedCache counters already use: obsTick 3 min/tick (12.8 h period) - replay staleness gate respTick 5 s/tick (21.3 min) - per-target response throttle retTick 1 h/tick (10.67 d) - retention TTL + LRU age Validity is an explicit flag bit (hasObserved / hasResponded), not a 0 sentinel, and the maintenance sweep saturates a stamp (clears its flag) once its window passes, so no stamp can age toward its ~256-tick aliasing horizon. That retires the wrap/sentinel special cases (T4, T9) - nowStampMs() survives only for the fallback path's module-global millis stamp. obsTick is separated from retention by design: only a genuinely heard NODEINFO frame stamps it, so later membership-based retention refresh can never make a silent node look servable. Also drops lastObservedRxTime, which was only echoed in a debug log. Entry shrinks 136 -> 128 B; the 2000-entry array 272 kB -> 256 kB, 8 kB below the original develop footprint while keeping the throttle and retention features. Tick granularity costs at most one tick per window (+-5 s on 30 s, +-3 min on 6 h, +-1 h on 7 d). Native suite: 61/61. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit e48ed9cfacfed0131b77ed2f1992e8720720d387) * Keep the NodeInfo cache a superset of NodeDB (seed + keep-alive) The cache learned identities only from overheard NODEINFO frames, so its membership was opportunistic: a NodeDB-tier node whose frame was never heard this boot had no entry (no key pin to protect it, no name to rehydrate), and the retention TTL evicted entries for nodes that still lived in the hot store or warm tier. Add an anti-entropy pass to the maintenance sweep (reconcileNodeInfoMembershipLocked): every hot-store / warm-tier node gets an entry - full identity from the hot store (hasFullUser), key-only from the warm tier - with signer provenance inherited key-matched, and NodeDB's key adopted wholesale on conflict. Member entries (isMember) are re-stamped each sweep, so they never age toward the retention TTL; when a node leaves both tiers the keep-alive stops and its entry ages out from its final member re-stamp. Membership also outranks key trust in LRU victim selection, and the reconcile pass skips seeding rather than churn one member out for another when hot+warm exceeds the cache. Two properties are load-bearing: - seeding/keep-alive never touch obsTick/hasObserved, so a seeded or retained entry is never served as a spoofed reply - only genuinely heard frames make a node servable; - copyUser() now requires hasFullUser, so a key-only warm seed can never stamp HAS_USER onto a nameless node via name-rehydration. Native suite: 64/64. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit 4f1b30d838b2f7b11173373b8f1ba97cf91bef6b) * Write-through NodeDB identity commits into the NodeInfo cache updateUser() is the single chokepoint through which every remote identity/key write enters NodeDB (key-verification keys stay in CryptoEngine's pending buffer until a NodeInfo carries them here), so one hook at its tail lets the NodeInfo cache reflect a commit immediately instead of waiting up to a minute for the reconcile sweep - which stays in place as the anti-entropy backstop. onNodeIdentityCommitted() upserts the full User: NodeDB's key is authoritative (a conflicting cached key is stale residue - replaced, provenance dropped), a keyless commit keeps an already-cached TOFU key, and signer provenance transfers only for the committed key. The observation stamp is never touched: knowledge is not observation, so a hook write can never make a never-heard node servable - covered by the new test alongside immediate copyUser/copyPublicKey visibility. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit e160b899728be8d951d4e7bdf5106c777832b917) * Purge TrafficManagement caches on explicit node removal NodeDB::removeNodeByNum() already forgets the node everywhere NodeDB owns - hot store, satellite stores, warm tier - but the TrafficManagement caches kept the deleted identity: the NodeInfo entry went on feeding the key pool (NodeDB::copyPublicKey) and name rehydration, and the unified slot kept its role/next-hop/dedup state, resurrecting the node on next contact. Add purgeNode(): clears both the unified cache slot and the NodeInfo cache entry, called from removeNodeByNum() alongside the warm-tier removal. Removal means full removal; passive eviction never calls this, and the reconcile sweep will not re-seed a node absent from both NodeDB tiers. Test: an observed identity plus a next-hop hint both vanish after removeNodeByNum(). Left for CI to execute per request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT (cherry picked from commit b5564c173337de99e15ef998a5c698550238fd3c) * Document the tmm-fix-2 / tmm-fix-superset reconciliation decisions Two parallel implementations of the same superset design are being combined on this branch. This decision matrix records every divergence and which side each reconciled feature takes, ahead of the code commits that apply them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Reconcile retention: no timed eviction, obsTick LRU, hourly seeding Adopt tmm-fix-superset's retention model (reconciliation decisions #3, #4, #9 in .notes/tmm-super-superset-reconciliation.md): - Entries are never evicted on a timer. The 7-day retention TTL and its third tick clock (retTick) are gone; wrap-safety rests entirely on the sweep's presence-bit saturation, and a quiet entry keeps its value (pubkey pool, name rehydration). Slots are reclaimed only by trust/membership-tiered LRU on insert - age scored by obsTick, with never-observed entries counting oldest - or by an explicit purge. - Membership refresh (entry -> NodeDB contains checks) runs every sweep; the heavy seeding pass runs at boot and then hourly, with the write-through hooks carrying immediacy in between. - Tick constants and helpers move into the header beside the existing UnifiedCache tick idiom, with static_asserts tying the tick windows to the fallback path's seconds/ms forms. Kept from tmm-fix-2 (decision #4): the warm tier is still seeded (key-only records via WarmNodeStore::entryAt) so the invariant remains cache superset-of hot AND warm, and the spareMembers guard still stops seeding from churning one member out for another when hot+warm exceeds the cache. Entry shrinks to 128 B (2000 entries = 256 kB). The TTL-based retention test is superseded by a no-timed-eviction test: a quiet keyed entry survives 9 days of sweeps while the serve gate saturates as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Reconcile hooks: key-commit write-through, purgeAll, key-matched signer Adopt tmm-fix-superset's hook coverage (reconciliation decisions #5-#8): - onNodeKeyCommitted(): write-through for the two key-write sites that bypass NodeDB::updateUser - the Router's admin-key learn (TOFU-grade) and KeyVerificationModule's manual-verification commit (proven=true, the strongest provenance the cache can carry). Both were coverage gaps in the tmm-fix-2 write-through design. - updateUser's hook call now transfers signer status key-matched (isVerifiedSignerForKey) instead of the node's bare signed flag, and runs on acceptance rather than only on change. - purgeAll(): factoryReset() and resetNodes() clear both TMM tables - removal-is-full-removal applies to bulk resets too, without relying on the usual post-reset reboot. - purgeNode() reuses the existing finders instead of open-coded scans and logs the (user-initiated) purge. - peekNodeInfoFlagsForTest(): flag introspection so saturation and membership tests can assert sweep effects directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Merge the two branches' test suites for the reconciled design Port tmm-fix-superset's unique tests, adapted to run natively under TMM_HAS_NODEINFO_CACHE (reconciliation decision #11): - keyHook_upsertsAndGovernsProvenance: TOFU learn -> manual-verification upgrade -> NodeDB-senior rotation resets provenance. - tickSaturation_sweepClearsObserved: the sweep saturates hasObserved past the serve window, the entry persists (no TTL), and a full 256-tick clock wrap cannot alias a saturated stamp back to fresh. - sweepMembershipMarking: reconciliation seeds a hot identity as member+fullUser+unobserved; the next sweep clears membership once the node leaves NodeDB, while the entry itself persists. tmm-fix-2's tests (warm pin, hot/warm seeding, updateUser write-through, removal purge - now also asserting the entry is gone via the new peek hook - and the no-timed-eviction rewrite) were already on this branch. Suite now counts 70 test functions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * Adapt cherry-picked and seeding tests to the reconciled semantics Two tests needed updating for interactions the validation run surfaced: - The #11035 test (ignoresUnsignedSignerIdentity) was written for a world without the native NodeInfo cache: it needs the NodeDB fallback path (dropNodeInfoCacheForTest) and the signed replay gate satisfied for its target, like the other fallback tests on this branch. - The hot-seed test's "observed frame makes it servable" step now sends a signature-verified frame: its node is a known signer, and per #11035 an unsigned frame from a signer must not (and does not) drive cache writes - the gate working as designed. Full native suite: 70/70 under ASan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KWHDDxY4U2XLYW2WJJ2oLT * trunk fml * Doc tidyup * TrafficManagement: key NodeInfo-cache maintenance to its own guard runOnce() nested the entire NodeInfo maintenance block (tick-stamp saturation, membership refresh, boot/hourly reconcile) inside #if TRAFFIC_MANAGEMENT_CACHE_SIZE > 0, so a variant overriding the unified cache to 0 on a PSRAM board would compile the NodeInfo cache without its maintenance: hasObserved would never saturate, obsTick would alias past its 12.8 h wrap, and the 6 h staleness gate - whose wrap-safety explicitly depends on the sweep - would serve spoofed replies for long-gone nodes. Extract maintainNodeInfoCacheLocked(), guarded by TMM_HAS_NODEINFO_CACHE alone, and give runOnce() the same independent-guard structure purgeAll() already has. Also close the runtime cousin of the same mismatch: the write-through hooks (onNodeIdentityCommitted / onNodeKeyCommitted) now no-op while moduleConfig.has_traffic_management is off. Previously they kept filling the cache from NodeDB commits while runOnce() returned INT32_MAX, accumulating entries that were never swept or reconciled. Purges and reads stay ungated: removal must always work, and reads just miss an empty cache. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: forward throttled NodeInfo requests instead of consuming them The direct-response throttle returned true, which made handleReceived() STOP the request: within the 30 s window a repeat request was neither answered nor forwarded toward the genuine target, and the call site still logged "respond" and bumped nodeinfo_cache_hits for a reply that was never sent. A requester whose first reply was lost on a noisy link got silence for the whole window. Return false instead: the request flows through normal relay handling, so the genuine node (or another cache-holder) can answer, while our own spoofed TX stays bounded. Repeats of the same packet id are already absorbed by the router's duplicate detection, and the stats counter now only counts replies actually sent. Also log purgeNode() only when a slot was actually cleared - it runs for every NodeDB removal, including nodes the caches never tracked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: consolidate NodeInfo-cache #if sprawl into one region Fourteen scattered #if TMM_HAS_NODEINFO_CACHE guards (one per function, each with its own #else stub of (void) casts) collapse into a single guarded region holding every NodeInfo-cache-only function, terminated by one #else block of no-op stub definitions. Because the stubs now exist on every build, the call sites in runOnce(), purgeNode() and purgeAll() drop their guards too; inner guards remain only for orthogonal features (PKI, warm tier) and for real conditional work (constructor allocation, PSRAM paths in shouldRespondToNodeInfo). No behavior change. Compile-checked on both sides of the macro: the native app build (stubs) and the native unit-test build (region). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Router: resolve sender key only for PKI-decrypt candidates perhapsDecode() resolved the sender's public key unconditionally for every encrypted packet, before even checking whether the packet could be PKI-decrypted (channel 0, unicast to us). Since copyPublicKey() grew its TrafficManagement fallback tier, a full hot+warm miss - the common case for channel traffic from senders outside both NodeDB tiers - additionally walked the 2000-entry NodeInfo cache under its lock, per packet, for a key that was then discarded. Move the resolution inside the PKI-candidate branch; remotePublic and haveRemoteKey had no consumers outside it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: refresh NodeInfo membership hourly, not per-sweep The 60 s sweep re-derived isMember with a per-entry NodeDB lookup: O(entries x members) - about 700k node-number comparisons per minute on a full PSRAM cache (2000 entries x 250 hot + 100 warm), with strided PSRAM reads, all while holding cacheLock against the packet path. Move the refresh into the hourly reconcile pass, which is already O(members x entries): after the seeding loops (so the upsert pass still sees last pass's bits for spareMembers protection), clear every isMember bit and re-mark from both NodeDB tiers - including keyless warm records, which seed nothing but are still members. Accepted tradeoff, now documented on the field: membership lags a passive NodeDB eviction by up to an hour (the entry just stays LRU-sticky slightly longer). Additions stay immediate via the write-through hooks, explicit removals via purgeNode(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * NodeDB: centralize bare-key commits in commitRemoteKey() The two key-write sites that bypass updateUser() (admin-channel learn in Router::perhapsDecode, manual verification in KeyVerificationModule) each carried their own #if HAS_TRAFFIC_MANAGEMENT write-through boilerplate - the pattern this PR itself had to retrofit twice, and which any future direct key write would silently miss, leaving the TrafficManagement cache divergent until the next hourly reconcile. Add NodeDB::commitRemoteKey(n, key32, KeyCommitTrust): writes the key to the hot store and routes the TrafficManagement write-through in one place, with provenance explicit at the call site (AdminChannelProven maps to TOFU-grade, ManuallyVerified to proven). The bypass sites keep their reason for existing - bare-key commits with provenance that updateUser's User-payload/TOFU-pin path cannot express - but no longer know about TrafficManagement at all; both stale includes are dropped (Router's was already unused on develop). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: tick wrap-safety analysis, updated cadences, CodeRabbit doc fixes node_info_stores.md gains a "Tick clocks and wrap safety" section recording which mechanism keeps each modular tick clock honest: the unified-cache clocks pair the 60 s sweep with read-time window resets, the NodeInfo obs/resp clocks are sweep-only (hence the compile-time invariant that maintainNodeInfoCacheLocked() is guarded by TMM_HAS_NODEINFO_CACHE alone), and the warm tier is immune by design - absolute unix-seconds, no wrap until 2106. Also brought current with this series: membership refresh moved to the hourly reconcile, throttled direct-response requests forward instead of being consumed, the commitRemoteKey() bare-key funnel, and the module-disabled gate on the write-through hooks. CodeRabbit doc review (PR #11050): present the NodeInfo payload cache as the third *identity* tier with the unified cache beside the chain, and describe NodeInfoLite's flattened fields / satellite copy-out accessors instead of the removed nested members. Two stale docs/tmm_node_stores.md references in the header now point at the real file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * TrafficManagement: adapt tests to forwarded throttles and hourly membership Throttle tests (PSRAM and NodeDB-fallback paths): a request inside the 30 s window now CONTINUEs into normal relay handling instead of being consumed - assert no spoofed TX, no NAK suppression, and (PSRAM path) that nodeinfo_cache_hits counts only replies actually sent. Membership test (renamed reconcileMembershipMarking): the per-minute sweep no longer refreshes isMember, so the test now pins down both halves of the new contract - the very next sweep after a passive NodeDB eviction still shows the stale member bit (the documented up-to-an-hour lag), and a reconcile interval's worth of sweeps clears it. Disabled-module test additionally proves the write-through hooks share the has_traffic_management gate: a key commit while disabled must not land in the NodeInfo cache. Not covered here: a build permutation with TRAFFIC_MANAGEMENT_CACHE_SIZE overridden to 0 (the configuration the maintenance-guard fix protects) would need its own PlatformIO env plus guards on every unified-cache test - left as a follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * expand test coverage * nitpicks and docs update * more comment fixes * nitpicks * test: make TMM fixture cleanup abort-safe in tearDown() Several tests reset per-test global state (trafficManagementModule pointing at a stack `module`, owner.public_key, the RTC fake clock) only after their assertions. A TEST_ASSERT_* failure longjmps out and skips that trailing cleanup, leaving the state to dangle into later cases - concretely, the next setUp()'s resetNodes() would call trafficManagementModule->purgeAll() on a destroyed object. Move the resets into tearDown(), which runs unconditionally between tests, and drop the now-redundant per-test cleanups. Addresses a CodeRabbit review comment on PR #11050. clod helped too * docs tidy * Throttle NodeInfo direct responses A direct response is addressed to the requesting packet's from field, which is unauthenticated, and is sent by every neighbour that holds the target in cache. One request therefore makes several nodes transmit at an address the requester chose, and nothing limited how often that could be repeated. Replies are now spaced per requester, which bounds how much any single node can be made to receive, plus a global floor on how much airtime the feature can consume. The check sits where a reply is about to be sent, so requests declined for other reasons do not consume the budget. * test: reconcile TMM direct-response tests with #11104 throttle Makes the suite green for now. #11104's per-requester + global-floor throttle (60 s) is layered over the branch's per-target throttle, so the three existing "served again" checks (psram, fallback, sweep) now use a fresh requester to avoid the per-requester window masking the mechanism each one actually exercises. Adds a dedicated test for the new per-requester + 1 s global-floor behaviour (the reflector-flood gap the per-target throttle leaves open, and the bound that survives on non-PSRAM builds). Deferred: the branch's now-redundant fallback global stamp (nodeInfoFallbackLastResponseMs) is left in place as a follow-up cleanup. Clod wants credit. * TMM: unify direct-response throttles into per-sender + per-target RAM tables Replace the two path-specific NodeInfo-spoof throttles (per-entry respTick on the PSRAM cache path, single global stamp on the fallback path) with three symmetric bounds that behave identically with and without PSRAM: - per requester (60 s): how much any one node can be made to receive; - per target (60 s): how often we vouch for the same identity; - global floor (1 s): total airtime, the backstop once an attacker cycles requester/target past the 8-slot tables. Both axes are fixed 8-slot LRU tables in internal RAM (not the PSRAM NodeInfo cache), compared by wrap-safe uint32 ms subtraction, so there is no tick clock and no sweep to maintain. directResponseAllowed() resolves both slots before stamping either, so a reply one axis throttles never consumes the other's budget, and records the send itself. Retires: nodeInfoFallbackLastResponseMs, kNodeInfoResponseThrottleMs, nowStampMs, the respTick byte + hasResponded bit on every cache entry, currentRespTick/kNodeInfoRespTickMs/kNodeInfoThrottleTicks, and the sweep's respTick clear. Renumbers peekNodeInfoFlagsForTest (drops the responded bit). Tests: the psram/fallback throttle tests now assert the per-target axis at 60 s, isolated via a different requester, proving it holds with and without PSRAM; perRequesterAndGlobalFloor isolates the other two axes; the obsolete respTick wrap-safety test is removed. 83/83 native cases pass. clod helped too * whats up, doc? * trimming the comments * test: bump native-suite-count to 38 after upstream rebase Upstream develop carries 38 test_* suite directories but its native-suite-count file still reads 37 (a suite was added without bumping the count). Rebasing onto develop inherits that stale file, so the runner flags AMBER. Correct the registered total to 38. clod helped too --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com> |
||
|
Thomas GöttgensandGitHub
|
7bdc2569e8 | Budget the admin-key PKI decrypt fallback (#11100) | ||
|
Thomas GöttgensandGitHub
|
d9f8839241 |
Accept a pending key only for the key-verification exchange (#11107)
perhapsDecode fell back to the not-yet-verified key held during a key verification handshake for any incoming PKI unicast. That key is supplied by whoever opened the handshake and proves only that they hold it, not that they are the node they claim to be, so until the session ended they could send DMs on any port that decrypted and were marked pki_encrypted. perhapsEncode already restricts the pending key to KEY_VERIFICATION_APP. The receive path now applies the same rule. |
||
|
Thomas GöttgensandGitHub
|
fb75410e53 |
Null-check packet allocations in allocForSending and its callers (#11086)
* Null-check packet allocations in allocForSending and its callers
Sibling of
|
||
|
Benjamin Faershtein
|
08cfb3d683 | Merge upstream develop into packet authentication policy | ||
|
Thomas GöttgensandGitHub
|
405a6bfd8a | Strip inner-message padding when sizing unsigned broadcasts (#11083) | ||
|
Thomas GöttgensandGitHub
|
023351a979 |
Remove dead code (#11082)
Removed: - MessageStore::addFromPacket and addFromString, superseded by tryAddFromPacket - GeoCoord rangeRadiansToMeters, distanceTo, bearingTo - Router::rawSend, declared virtual with no override and no caller - ContentHandler handleHotspot, handleFs, handleAdminSettings, handleAdminSettingsApply, handleDeleteFsContent and their commented route registrations, plus the now unreachable htmlDeleteDir and the handleUpdateFs declaration that had no definition - ContentHelper replaceAll - OnScreenKeyboardModule popup chain: showPopup, clearPopup, drawPopup, drawPopupOverlay and their state, unreachable since the frame based UI was replaced by baseUI - DebugRenderer drawDebugInfoTrampoline, drawDebugInfoSettingsTrampoline and the orphaned drawFrameSettings - NodeListRenderer calculateMaxScroll, drawColumns and a stale extern haveGlyphs declaration with no definition - UIRenderer::haveGlyphs, Screen::blink, NotificationRenderer::showKeyboardMessagePopupWithTitle, VirtualKeyboard::getInputText - InkHUD touchNavLeft, touchNavRight, Applet::getActiveNodeCount, ThreadedMessageApplet::saveMessagesToFlash - TwoButton::setHandlerUp, TwoButtonExtended setHandlerUp, setJoystickDownHandlers, setJoystickUpHandlers - CannedMessageModule LaunchRepeatDestination, isCharInputAllowed, hasMessages - TrafficManagementModule resetStats, recordRouterHopPreserved, saturatingIncrement - UnitConversions::MetersPerSecondToMilesPerHour - EncryptedStorage getSessionRemainingSeconds - BMI270Sensor::writeRegisters, GPS::hasFlow, FSCommon copyFile, SerialConsole consolePrintf, buzz playLongPressLeadUp, memGet displayPercentHeapFree |
||
|
Benjamin Faershtein
|
59ab12675a | Merge develop into packet authentication policy | ||
|
Thomas GöttgensandGitHub
|
8ebdae9921 |
Bound payload.bytes prints by payload.size (#11032)
decoded.payload.bytes is a 233-byte protobuf field that is not NUL-terminated. Printing it with a plain "%s" reads until a NUL, which for a full-length payload with no NUL runs past the field. Use "%.*s" with payload.size, matching the write form already used a few lines up in SerialModule. Live sites: RangeTestModule appendFile, SerialModule text output. The same fix is applied to the commented-out debug lines in RangeTestModule and Router so the pattern is consistent if they are re-enabled. |
||
|
b71c1adb26 |
stm32wl: add hardware RTC support (rak3172) (#10961)
* stm32wl: add hardware RTC support infrastructure Wires the STM32WL chip's internal RTC (running off the LSE 32.768kHz crystal) into meshtastic's existing time-of-day framework (perhapsSetRTC()/readFromRTC()), following the same pattern already used for I2C RTC chips (RV3028, PCF8563/85063, RX8130CE). LSE is started and polled manually before ever calling into the STM32RTC library, with our own bounded timeout - the library's own internal LSE startup path has no bounded fallback and hangs forever via Error_Handler() if the crystal never locks, so this is required for a board with a missing/faulty crystal to boot normally rather than hang. Gated behind a new HAS_LSE variant flag (currently unset everywhere, so this is inert until a variant opts in - see follow-up commit). Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * gps: qualify RTC.h includes to avoid case-insensitive filesystem collision with STM32RTC The stm32duino STM32RTC library (added to lib_deps in a follow-up commit) ships its own src/rtc.h. On case-insensitive filesystems (the macOS default), an unqualified #include "RTC.h"/<RTC.h> from any file outside src/gps/ resolves to the library's rtc.h instead of src/gps/RTC.h, since PlatformIO's LDF puts lib_deps include paths ahead of the project's own -Isrc/gps. Qualify every include as gps/RTC.h so it can't collide with any same-named header a future dependency might ship, regardless of filesystem case sensitivity. Purely mechanical, no behavior change. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * stm32wl(rak3172): enable hardware RTC support Opts rak3172 into the HAS_LSE infrastructure added previously: sets STM32WL_LSE_DRIVE to a conservative default and pulls in the STM32RTC library. rak3172 has ~63KB flash headroom going in; build-verified at 76.7% flash usage after this change (up from a 73.8% baseline), well within budget. wio-e5 is not opted in here despite sharing the same STM32WLE5 chip - it's already at 96.8% flash usage today (GPS + I2C sensor support compiled in, unlike rak3172), leaving too little headroom to safely add STM32RTC without first trimming something else. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * stm32wl: add docstrings for LSE/RTC setup functions Addresses CodeRabbit's docstring coverage check on PR #10961. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> * stm32wl: address CodeRabbit nitpicks on PR #10961 - Brace the single-statement HAS_LSE branch in perhapsSetRTC() to match the sibling readFromRTC() branch's style. - Quote the RTC.h include in PhoneAPI.cpp for consistency with every other qualified include site. Signed-off-by: Andrew Yong <me@ndoo.sg> Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> --------- Signed-off-by: Andrew Yong <me@ndoo.sg> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
Thomas GöttgensandGitHub
|
f52d7753ad | Router: size the fit check from the decoded Data (#11018) | ||
|
952c825167 |
Allow key verification to work for unknown nodes. (#10669)
* Allow key verification to work for unknown nodes. * trunk * More reliable admin key decryption * Add admin key fallback tests * Actually check haveRemoteKey * Logging cleanup * Address review feedback - Persist the committed key + manually-verified flag in commitVerifiedRemoteNode via saveToDisk(SEGMENT_NODEDATABASE), replacing the "todo: initiate save" - Guard the CryptoEngine pending-key slot with a dedicated internal lock; the Router reads it while already holding the non-recursive cryptLock, so the accessors cannot reuse that lock - Draw the security number from the hardware RNG (CryptRNG fallback) under cryptLock instead of random(); on nRF52 the entropy fill toggles the same CC310 the BLE task's packet crypto uses - Return true after fully handling the hash2 response (restores develop behavior; consistent with the hash1 branch) - Take uint32_t in the number-picker callbacks so 8-digit hex nodenums can't truncate through int - Trim over-long comments flagged by review * feat(tests): add deterministic tests for admin session-key behavior --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
Thomas GöttgensandGitHub
|
7a25ffd0a2 | fix: evaluate pre-hop hop_start bitfield guard post-decode instead of before decryption (#10758) | ||
|
Benjamin Faershtein
|
d6b12ea3f1 | feat(security): enforce packet authenticity policies | ||
|
Andrew YongandGitHub
|
0ae44d7017 |
fix(router): null-check packetPool/clientNotificationPool allocations (#10948)
On platforms where these pools are heap-backed (MemoryDynamic - used whenever there isn't enough static RAM for a fixed pool, e.g. ARCH_STM32WL or BOARD_HAS_PSRAM), allocCopy()/allocZeroed() return nullptr on allocation failure and already log a warning, but most callers dereferenced the result unconditionally. Under real heap pressure this reliably produced a HardFault - reproduced on STM32WL hardware under sustained mesh traffic, including the RX entry point (RadioLibInterface::handleReceiveInterrupt) where every received packet is allocated. Adds null checks at all call sites that were missing one, mirroring the guard pattern already used correctly elsewhere in the same files (e.g. RadioInterface.cpp's sendErrorNotification). On allocation failure, callers now skip the send/retransmission/notification and log nothing further (the allocator already did) rather than crash. Assisted-by: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Andrew Yong <me@ndoo.sg> |
||
|
Ben MeadorsandGitHub
|
6c7ee8afc7 |
Add MemAudit: per-subsystem heap accounting in the boot log (#10900)
* Add MemAudit: per-subsystem heap accounting in the boot log The 2.8.0 nRF52840 heap-exhaustion field reports had to be diagnosed by hand, reconstructing each subsystem's heap footprint from source and build flags one report at a time. This makes every future report self-diagnosing from the serial log: a tiny fixed-size registry (src/memory/MemAudit.*) that big long-lived allocations report into, printed as one line at the end of setup() and alongside the periodic "Heap free:" log: MemAudit[boot]: tmm=2500 warm=4000 pkthist=5824 nodedb=13440 msgstore=2200 pktpool(live)=3270 total=31234 Instrumented: NodeDB hot vector (nodedb) + satellite maps (satmaps, rb-tree overhead estimated), WarmNodeStore (warm), PacketHistory records and hash index (pkthist), TrafficManagement caches (tmm/tmm_ni), MessageStore text pool (msgstore), TFT line/repaint buffers (display), and live in-flight packets (pktpool(live)) via an optional audit tag on the packet pool allocator - the one hot path, counted with a relaxed 32-bit atomic add (single instructions on Cortex-M, no locks). Cost: 128 B RAM for the 16-tag table, well under 1 KB flash on rak4631. MESHTASTIC_MEM_AUDIT=0 compiles it out to inline no-op stubs (call sites need no ifdefs); STM32WL, the tightest flash target, defaults off. New native suite test_mem_audit covers add/set/snapshot arithmetic, tag reuse (pointer and cross-TU strcmp fallback), null/unknown tags, and table-full behavior; test/native-suite-count bumped to 28. * native-wasm: add src/memory/ to the curated source filter The wasm env denies all sources and adds an explicit file list; MemAudit callers (main, MeshService, NodeDB, PacketHistory) are in that list but src/memory/MemAudit.cpp was not, so wasm-ld failed on undefined memaudit:: symbols. |
||
|
Ben MeadorsandGitHub
|
0e84c1a827 |
Harden XEdDSA unsigned-packet policy and add coverage (#10858)
Audit of the XEdDSA packet-signing implementation (#10478) surfaced several issues in when unsigned packets are accepted on receive or emitted on send. This fixes them and adds regression coverage. - Unicast NodeInfo exchange no longer breaks against signer nodes: the NodeInfoModule downgrade drop is gated to broadcasts, since senders never sign unicast (want_response replies, directed exchanges). - Replace the payload-size sign heuristic with an exact encoded-size gate (signedDataFits) and mirror it on the receive side, removing a dead band where 167-168 B broadcasts were signed then failed TOO_LARGE. - Extract the receive policy into checkXeddsaReceivePolicy() and apply it to plaintext-MQTT decoded downlink, which previously skipped signature verification and downgrade protection entirely. - Reject signatures whose length is neither 0 nor 64 as malformed, so a crafted partial signature can't inflate the size estimate and dodge the unsigned-downgrade drop. - Hold cryptLock on the MQTT verify path (shared Ed25519 key cache). - Clear any client-preset signature on packets we originate, on all builds. - Randomized (hedged) signing per the Signal XEdDSA spec: bump the meshtastic/Crypto pin to the build where XEdDSA::sign mixes 32 bytes of caller randomness into the nonce as Z (meshtastic/Crypto#3), and seed those bytes in xeddsa_sign from HardwareRNG (checked, with a seeded-CSPRNG fallback). test_crypto pins that repeated signs differ and both verify. Adds test coverage: test_packet_signing groups A-E (receive matrix, send policy, NodeInfo backstop, encoding invariants, decoded-ingress policy), test_mqtt end-to-end downlink cases, and a test_crypto randomization check. |
||
|
TomandGitHub
|
3becaf2d95 | emdashes begone (#10847) | ||
|
TomandGitHub
|
fd62322876 |
Log tidy (#10798)
* clarify channel number formatting and fix the fixed-width padded nodeIDs * use the explicit 0x%08x for packets and nodeIDs throughout * LLM instructions |
||
   TomGitHubSteve GilberdDarafei Praliaskouskigithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Claude Opus 4.8 |
ec5d230305 |
Feat/mesh beacon (#10618)
* Tips robot virtual node / relayer to different LoRa modes & channels
Note that this commit has details hardcoded for the Wellington (NZ)
mesh, and also requires the following patch to the protobufs:
-----
diff --git a/meshtastic/mesh.proto b/meshtastic/mesh.proto
index 03162d8..ec54c99 100644
--- a/meshtastic/mesh.proto
+++ b/meshtastic/mesh.proto
@@ -1393,6 +1393,21 @@ message MeshPacket {
* Set by the firmware internally, clients are not supposed to set this.
*/
uint32 tx_after = 20;
+
+ /*
+ * The modem preset to use fo rthis packet
+ */
+ uint32 modem_preset = 21;
+
+ /*
+ * The frequency slot to use for this packet
+ */
+ uint32 frequency_slot = 22;
+
+ /*
+ * Whether the packet has a nonstandard radio config
+ */
+ bool nonstandard_radio_config = 23;
}
/*
-----
* fix: repair mesh tips CI build
* feat: add MeshBeacon module (Phase 1 — proto + generated code + initial stub)
* feat(beacon): implement broadcaster + listener (phases 2-5)
* feat(beacon): wire RadioLibInterface hooks + admin validation (phases 6-7)
* fix(beacon): fix LocalModuleConfig flat access (no payload_variant), add localonly proto field
* feat(beacon): fix broadcaster inheritance, add preset/region validation + proto cache
- MeshBeaconBroadcastModule now inherits ProtobufModule<meshtastic_MeshBeacon>
(alongside private MeshBeaconModule + OSThread), giving it allocDataPacket()
and setStartDelay() without extra includes.
- Payload cache: rebuildCache() encodes the MeshBeacon protobuf once and stores
it in payloadCache[]/payloadCacheSize; sendBeacon() only calls rebuildCache()
when payloadCacheDirty==true. AdminModule calls invalidateCache() after saving
new config so the next broadcast picks up changes.
- Region/preset validation in handleSetModuleConfig (mesh_beacon_tag):
broadcast_on_preset is validated against the device's current region via
RadioInterface::validateConfigLora(); broadcast_offer_region is validated via
RadioInterface::validateConfigRegion(). Invalid values are zeroed with a
LOG_WARN before saving.
* feat(beacon): add unit tests for MeshBeaconModule and AdminModule configuration validation
* remove old meshtips
* more validation in NodeDB and AdminModule, and userprefs for baked in goodness
* copilot is my gravity
* mmmmm... beacon
* oops
* Enhance unit tests for MeshBeaconModule with detailed validation checks and output formatting
* new lines. Why not?
* finally
* legacy mode activate!
* Update protobufs (#17)
Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com>
* better logic, fixed a test
* updated for packet signing
fixed a test
added guards for licensed/ham mode
* channel numbers
* beacon: encrypt on the beacon channel PSK; fix split note
When broadcast_on_channel overrides the primary channel's name/PSK, the
beacon was encrypted with the PRIMARY PSK: perhapsEncode keys encryption
off the primary slot, but the radio-thread channel switch happens only
after encryption. sendBeaconPacket() now installs the beacon channel into
the primary slot for the synchronous duration of send() (cooperative
threading => no interleaving) so encryption/hash use the beacon channel,
then restores it. A shared beaconChannelSettings() helper builds the
channel for both the encrypt-time swap and the RF-time swap so the
key+hash cannot drift.
Also: correct the legacy-split comments (both packets go out on the same
beacon radio settings, not the normal config) and merge the two
consecutive `if (hasText)` blocks in the listener (cppcheck
duplicateCondition).
Tests: add channelPskOverride_swapsBeaconChannelAndRestores and
noChannelOverride_doesNotSwapPrimary; MockRouter snapshots the primary
channel at send() time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test/beacon: drain toPhoneQueue in tearDown to fix LSan leak abort
The listener delivers received text via MeshService::sendToPhone(), which
enqueues the packet into toPhoneQueue and takes ownership. Nothing dequeues
it in tests, so the three listener tests carrying message text stranded a
MeshPacket each — 1272 bytes / 3 allocations that LeakSanitizer flagged at
process exit, aborting the coverage run (surfaced by pio as [ERRORED] /
SIGHUP even though all 40 assertions passed).
Drain the phone queue in tearDown (getForPhone()/releaseToPool) so the
packets return to packetPool. Suite is now GREEN with no sanitizer abort.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* legacy hop override for zero-hoppers
* ever more beacons
* beacon: comment out broadcast_send_as_node pending further review
Functionality preserved in comments with full signing/has_bitfield notes
for when it is re-enabled. Proto tag 3 retained on the wire.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test/beacon: fix fromIsCustomNodeWhenSet now that send-as-node is disabled
broadcast_send_as_node is commented out; from is always the local node.
Update the test assertion and doc comment to match current behaviour.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Update protobufs (#21)
Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com>
* flags for beacons
* beacon: do more with less — slot-index targets + validation
Multi-target beacons embedded a full ChannelSettings in every BroadcastTarget,
blowing ModuleConfig past the 512-byte BLE FromRadio budget so the firmware would
not compile. Targets now reference an existing channel-table slot by channel_index
and the broadcaster resolves it via channels.getByIndex() at TX time. Net effect:
the same multi-target capability for a fraction of the bytes —
FromRadio 609 -> 510 B, MeshBeaconConfig 596 -> 324 B, AdminMessage 615 -> 511 B.
- proto: BroadcastTarget.channel (embedded) -> channel_index (uint32 ref); regen all
generated headers (size constants propagate to admin/localonly/deviceonly/mesh).
- broadcaster: resolve channel_index from the channel table; an out-of-range or blank
slot falls back to the default channel for the target preset rather than borrowing
the primary's name/PSK.
- AdminModule: validate broadcast_targets entries on write (region/preset sanitised
like the single-target fields; channel_index range-checked).
- userPrefs: TARGET_<n>_CHANNEL_{NAME,NUM,PSK} collapse to a single CHANNEL_INDEX.
- docs: two-step (set_channel -> set_module_config) multi-target setup, inline-vs-
reference distinction, and single-/multi-target are equal (not "legacy") options.
- tests: target validation + channel-index resolution incl. blank-slot fallback
(47/47 green on `./bin/run-tests.sh -e native -f test_mesh_beacon`).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NRAF5csgsMn6p1zEcFL8Qz
* throttling after reboot
* address copilot review
* simplify
* fix(beacon): use 0x%08x for node/packet IDs in logs; register test suite
The %#08lx log specifiers passed uint32_t (NodeNum/PacketId) to a %lx
length modifier — undefined behaviour on 64-bit (native test) targets and
non-standard width. Switch to the project-standard 0x%08x. Also bump
test/native-suite-count to 25 for the added test_mesh_beacon suite.
clod helped too
* copilot & clarity
clod helped too
* refactor(beacon): use auto for the sanitized config copy
clod helped too
* fix(beacon): guard empty-payload sends; gate has_mesh_beacon on build flag; document ISR_TX pre-switch
clod helped too
---------
Co-authored-by: Steve Gilberd <steve@erayd.net>
Co-authored-by: Darafei Praliaskouski <me@komzpa.net>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
|
c51c01607d |
Traffic Management module: dedup, rate limiting, role-aware policing (#10706)
Adds the Traffic Management module (TMM) plus the NodeDB/warm-store and next-hop foundations it builds on: - Unified per-node cache (flat array, 8-bit relative ticks) shared by all features; role-aware throttles for tracker / lost-and-found. - Position deduplication: drop unchanged position rebroadcasts within a configurable interval; precision driven off the channel ceiling (clamped to the public-key max on well-known channels). Enabled by default at 11h. - Per-node rate limiting and unknown-packet filtering (config-driven; a non-zero companion field enables each feature -- no bool toggles). - NodeInfo direct response from cache with role-based hop clamps. - Persistent next-hop overflow store: confirmed hops have no TTL, are seeded from NodeInfoLite at boot, and survive hot-store eviction. - Three-tier sender-role resolution (hot NodeInfoLite -> warm store -> TMM cache). Role is cached write-time (seeded on first track, refreshed from NodeInfo), pins its cache entry like a next-hop hint, and is evicted last. - Warm store caches device role + protected category across reboot/eviction. - PositionModule stationary floor for tracker / lost-and-found. - PSRAM gating for warm/satellite/TMM cache sizes; STM32WL excluded. Protobufs: TrafficManagementConfig trimmed to the five uint32 fields actually used; submodule repointed to protobufs develop. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
22072c5f4b |
Pr1.5 tmm nexthop (#10745)
* TrafficManagement: flat unified cache + persistent next-hop overflow store Reworks the TrafficManagementModule cache layer (policing behaviour unchanged from upstream) and adds a routing-hint overflow store: - Flatten the ring: replace the cuckoo-hashed unified cache and the bucketed PSRAM NodeInfo index with plain flat arrays + linear scan (same idiom as WarmNodeStore). At LoRa packet rates an O(n) scan of the cache is negligible, and it removes a large amount of hashing/displacement complexity. The cache entry is 11 B; timestamps use a uniform +1 presence-offset so a 0 byte always means "empty" across every sub-store. Adds rebaseEpoch() so cached state survives the ~19 h relative-timestamp horizon instead of being flushed. - Next-hop overflow cache: setNextHop/getNextHopHint store a confirmed last-byte relay for a destination, written only from NextHopRouter's ACK-confirmed decision (and mirrored from TraceRoute). NextHopRouter::getNextHop falls back to this cache when the hot NodeDB has no hint, so DMs/relays to long-tail nodes keep routing after the node ages out of NodeInfoLite. - Persistence: preloadNextHopsFromNodeDB warm-starts the cache from persisted NodeInfoLite hints on first maintenance pass; next_hop entries are kept alive across the maintenance sweep (no TTL) and never clobbered by a stale preload. All packet-policing logic (rate limit, position dedup, unknown-packet drop, NodeInfo direct response, hop exhaustion) is the existing upstream behaviour, untouched. HAS_TRAFFIC_MANAGEMENT defaults on so the module is compiled in. (see note). Tests: upstream policing suite now actually runs (adds the MeshTypes.h include that gates HAS_TRAFFIC_MANAGEMENT) plus 4 next-hop tests. Role-aware throttles, politeness, precision clamp, port-interval and mesh-radius gating — and the rate-limit >255 saturation fix — are deferred to the advanced-TMM branch. Note: default dedup movement grid moves to ~91m, which also means 1.5km required to end up with the same signature position - coarser and therefore further than before. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * TrafficManagement: fix cppcheck constVariablePointer warning `node` in preloadNextHopsFromNodeDB() is never written through — mark it const to satisfy cppcheck's constVariablePointer check in CI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add multi-hop NextHop recovery tests and unit tests for routing reliability - Introduced a new test suite for multi-hop NextHop directed-message delivery and relay recovery in `test_nexthop_multihop_recovery.py`. This includes tests for end-to-end delivery and recovery after relay drop. - Implemented unit tests in `test_main.cpp` for NextHop routing reliability mitigations, covering: - M1: Ambiguity-aware last-byte resolution. - M2: NextHopRouter's strict-neighbor gate and hop limit checks. - M3: Route-health freshness and failure decay. - Enhanced mock classes to facilitate controlled testing of node behaviors and routing logic. * grafting fixed * Address Copilot review for PR #10735 (NextHop improvements) - docs/nexthop-routing-reliability.md: update status from "no code changes yet" to reflect that mitigations and tests are implemented RAM pressure and MIGRATION_VERBOSE concerns addressed upstream in PR2.5 (per-platform TRAFFIC_MANAGEMENT_CACHE_SIZE) and PR2 (verbose default=0) respectively; (0,0) sentinel fixed in PR2.5. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * CI: fix cppcheck constVariablePointer and test include path - NextHopRouter.cpp: qualify two RouteHealth *h locals as const — only read for stale-route checks, never mutated through the pointer - Router.cpp: qualify meshtastic_NodeInfoLite *node as const in shouldDecrementHopLimit — only read for favorite/role predicate - test_position_module/test_main.cpp: change bare PositionModule.h to modules/PositionModule.h — build_flags sets -Isrc, not -Isrc/modules, so the bare form fails to resolve in the native PlatformIO test env Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * WarmStore: cache device role + protected category in last_heard low bits Steal the low 6 bits of WarmNodeEntry.last_heard to carry an evicted node's device role (4 bits) and a protected category (2 bits) for the hop-trim path, at zero record-size cost (entry stays 40 B; no RAM/flash growth). The high bits remain a real unix-seconds timestamp, quantised to 64 s — ample for warm LRU ordering of long-tail nodes. - absorb() packs role/protectedCat; place()/ring replay store the raw word so metadata round-trips through flash. LRU compares masked time (warmTimeOf). - take() rehydration masks the metadata bits and restores the cached role so a re-admitted node isn't stuck at CLIENT until its next NodeInfo. - NodeDB classifies the category (favorite/ignored/verified -> Flag; tracker/sensor/tak_tracker -> Role) at each eviction site. - WarmNodeStore::lookupMeta() exposes role/category to consumers. - Bump WARM_RING_MAGIC (WRNG->WRN2): old rings read as erased and rebuild; warm data is a non-critical evictee cache, so discard-on-upgrade is safe. Tests: test_warm_store 11/11 (new meta round-trip + quantisation-aware ordering); NodeDB compiles (test_nodedb_blocked 4/4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: migrate v1 rings/files by discarding last_heard, not the data Previously the WRNG->WRN2 magic bump treated old rings as erased, discarding all warm entries — including the PKI public keys that let evicted nodes keep decrypting DMs. Instead, read v1 (WRNG / WRM1) records and keep each node's identity + public key, discarding only last_heard (its low bits would otherwise be misread as the new role/protected metadata). Records re-rank and re-learn their role on next contact. - Ring backend (nRF52840): ringReadHeader accepts both magics and reports v1 via an out-param; replay zeroes last_heard for v1 records. If the active head page is v1, force a rotation so new v2 records never land in a v1-headered page (which would discard their freshly-set role on the next load). Legacy pages convert to v2 as the ring rotates. - File backend (warm.dat): bump WARM_STORE_MAGIC WRM1->WRM2; accept WRM1, verify CRC against the stored bytes, then discard last_heard and mark dirty so the next save rewrites as v2. Tests: test_warm_store 12/12 (adds test_ws_v1_migration_discardsLastHeard: key survives, role/protected reset). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: guard role bit-width + test eviction carries role/protected - static_assert that the device role enum still fits the 4-bit warm metadata field (WARM_ROLE_MASK); fails the build loudly if a new role is added past 15 rather than silently truncating role on eviction. (Max role today = 12.) - Add test_migration_carriesRoleAndProtectedIntoWarm: a demoted TRACKER lands in the warm tier with its key, role=TRACKER and protected category=Role; a demoted CLIENT carries role=CLIENT/None. Exercises the NodeDB eviction path + warmProtectedCategory classification (the warm-store unit tests only cover absorb() directly). Tests: test_nodedb_blocked 5/5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix copilot comments * fix(test): restore #if HAS_TRAFFIC_MANAGEMENT guard in TMM test The rebase onto PR1.5 lost the top-level HAS_TRAFFIC_MANAGEMENT guard that PR1.5 introduced, leaving the #else/#endif tail orphaned and causing compile errors on non-TMM builds. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
TomGitHubClaude Opus 4.8github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Ben Meadors |
79a7dcc46c |
Pr1 nodedb warmstore (#10705)
* NodeDB: 3-tier node store with persistent warm tier (long-tail identity retention)
Introduces a tiered NodeDB so the device retains identity (public key,
last_heard) for far more nodes than fit in the full-record hot store,
without growing heap or the persisted nodes.proto unboundedly.
- Hot store: full NodeInfoLite, MAX_NUM_NODES (120 on nRF52).
- Satellite maps: position/telemetry/environment/status capped at
MAX_SATELLITE_NODES (40 freshest); eviction via enforceSatelliteCaps /
evictSatelliteOverCap.
- Warm tier (WarmNodeStore): 40 B {num,last_heard,public_key} records for
evicted nodes so DMs to/from long-tail nodes keep encrypting/decrypting.
Persisted to /prefs/warm.dat, or on nRF52840 a dedicated 12 KB raw-flash
record-ring below LittleFS (3x4 KB pages; see linker scripts + the
nrf52_warm_region.py post-link guard).
NodeDB::getOrCreateMeshNode now demotes evicted nodes into the warm tier and
re-admits them (restoring key/last_heard). Router PKI decrypt/encode resolve
the peer key via NodeDB::copyPublicKey (hot store, then warm tier).
NodeInfoLite gains snr_q4 (sint32, Q4-encoded dB); the float snr is zeroed on
disk. NodeInfoLite grows 105 -> 112 B; backup 2432 -> 2468 B.
Note: the snr_q4 .proto change still needs to land in the protobufs submodule
(generated header is updated here; submodule pointer left at upstream).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* NodeDB: robust receive + retention for blocked (ignored) nodes
Hardens how ignored/favourite nodes are received over admin and retained,
closing paths where a block could be lost or accidentally cleared.
- Blocking keeps the node's public key (admin set_ignored_node and
addFromContact no longer zero it / drop the warm-tier key), so a blocked
peer stays a verifiable identity.
- set_ignored_node creates the node if absent, so a block by node ID sticks
even for a node we've never heard from (e.g. pushed by a remote admin) with
no NodeInfo or key.
- Eviction protection (favourite/ignored/manually-verified) now also applies to
the load-time hot-store migration and is never undone by cleanupMeshDB, which
previously purged ignored nodes that lacked user info.
- The hot-store migration leaves our own node (index 0) in place and prefers to
demote non-protected nodes, like the runtime eviction scan.
Caps the protected set (favourite + ignored + verified) at MAX_NUM_NODES-2 via
NodeDB::setProtectedFlag(), so at least two evictable slots always remain and
getOrCreateMeshNode can always make room — replacing the previous unconditional
append that could run off the end of the node vector when every node was
protected. A locally-set favourite/ignore that hits the cap reports back to the
phone via a ClientNotification.
Adds test_nodedb_blocked covering the migration, favourite/ignored eviction
protection, ignored-survives-cleanup, and the protected-node cap. The
maintenance methods stay private in production; the test reaches them through a
PIO_UNIT_TESTING-guarded friend shim.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts:
# src/mesh/NodeDB.h
* fix copilot comments
* once again
* WarmNodeStore: fix cppcheck warnings (uninitvar, constVariablePointer)
Zero-initialise `stranded[]` and `seqs[]/order[]` VLAs so cppcheck can
verify there are no unguarded reads of uninitialised memory (the guards
exist but are not visible to static analysis). Mark two local pointers
`const` where the pointed-to entry is never mutated after assignment.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* self-care added to assist 2.7 and 2.8 nodedb migration
* Tidy warm-store/self-care: comments, guards, log + flash cleanup
Style/cleanup pass over the branch (no behavior change except the noted
preprocessor simplifications, which are semantically identical):
- Comments: move function descriptions to the headers, cap in-function
comments at ~3-4 lines, drop leading-number step markers, label stacked
#endif blocks, de-decorate banner comments.
- dumpToLog: fully gate decl + definition + AdminModule call site behind
MESHTASTIC_NODEDB_MIGRATION_VERBOSE so it compiles out when disabled
(~1.2 KB when off).
- mesh-pb-constants: drop the dead nRF52832 WARM_NODE_COUNT branch and trim
the macro docs.
- WarmNodeStore: simplify the redundant `ARCH_NRF52 && NRF52840_XXAA` guards
to `NRF52840_XXAA`, add a kNoPage sentinel for the ring page state.
- Shorten the always-on LOG_WARN strings (~120 B flash).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* more tidying up, aligning with docs and undoing other-arch regressions
* Update protobufs (#19)
Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com>
* made the migration pathway cleareer
* address copilot review
* fixed a copilot review on a downstream PR.
* Address Copilot review comments for PR #10705 (warmstore/nodedb)
- WarmNodeStore.h: default MIGRATION_VERBOSE to 0 (suppress info-level
chatter on production builds; opt in with =1)
- WarmNodeStore.cpp load(): move memset to top of function so all
failure paths (header-read fail, invalid header) leave entries clear
- WarmNodeStore.cpp save(): replace manual spiLock lock/unlock around
mkdir with LockGuard covering the full SafeFile sequence, matching
the lock discipline in load()
- Router.cpp: memcpy(&p->public_key.bytes, ...) -> memcpy(p->public_key.bytes,
...) — pass decayed uint8_t* rather than pointer-to-array
- AdminModule.cpp: check setProtectedFlag return for PKC auto-favorite;
log cap-refusal warning instead of unconditional "auto-favoriting"
- nrf52_warm_region.py: error message references both v6.ld and v7.ld
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* NodeDB: formatting cleanup (blank lines after preprocessor blocks)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Lukewarm store
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
|
||
   Jonathan BennettGitHubBen MeadorsCopilotWesselcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>thebentern |
8267bb22bd |
Packet Signing via XEdDSA (#10478)
* Test commit for XEdDSA support * Update to Crypto lib in Meshtatic org * Generate a new node identity on key generation (#7628) * Generate a new node identity on key generation * Fixes * Fixes * Fixes * Messed up * Fixes * Update src/modules/AdminModule.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/NodeDB.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Figured it out! * Cleanup * Update src/mesh/NodeDB.h Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/NodeDB.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/modules/AdminModule.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update crypto commit hash * Some fixes for xeddsa pr (#9610) * fix: add null check for getMeshNode() in NodeInfoModule getMeshNode() can return nullptr for unknown nodes. Dereferencing without a check crashes the firmware when receiving NodeInfo from a node not yet in the database. * fix: enforce XEdDSA signature verification and prevent stripping Previously, failed signature verification still allowed the packet through, making signatures purely cosmetic. Now: - Failed verification drops the packet (DECODE_FAILURE) - Successfully verified nodes get HAS_XEDDSA_SIGNED bitfield set - Unsigned packets from previously-signing nodes are rejected - Log levels reduced from WARN/ERROR to DEBUG/WARN as appropriate * fix: include packet metadata in XEdDSA signature The signature now covers [fromNode | packetId | portnum | payload] instead of just the payload bytes. This prevents: - Replay attacks (different packetId fails verification) - Reattribution (different fromNode fails verification) - Portnum redirection (different portnum fails verification) Also adds a key initialization check to xeddsa_sign (returns false if XEdDSA keys are all zeros) and checks the return value in the encode path. * fix: handle existing key pair in AdminModule security config When a user provides both a valid private key and public key via admin config, the crypto engine's DH private key and owner public key were never loaded. DMs and XEdDSA signing would silently break. Add an else branch to load both keys into the crypto engine. * perf: cache Ed25519 public key conversion in xeddsa_verify curve_to_ed_pub() performs field element parsing, inversion, and multiplication on every call. Since packets from the same node tend to arrive in bursts, a single-entry cache avoids repeating this expensive conversion for consecutive packets from one sender. * fix: skip identity cleanup when node number is unchanged createNewIdentity() was called on every generateCryptoKeyPair(), including normal boots where the same key is regenerated. This caused unnecessary NodeDB writes and old-node cleanup logic to run when the node number hadn't actually changed. Also fixes only zeroing byte[0] of the old node's public key instead of clearing the entire array. * fix: replace hardcoded 120 with derived XEDDSA_SIGNATURE_SIZE constant The payload size check for XEdDSA signing used a magic number (120). Replace with a derivation from DATA_PAYLOAD_LEN and XEDDSA_SIGNATURE_SIZE so the limit adjusts automatically if constants change. This also increases the max signable payload from 120 to 169 bytes, which is still safe since the actual encoded size is checked after pb_encode. * fix: add const qualifiers to XEdDSA verify and curve_to_ed_pub inputs pubKey, payload, and signature parameters in xeddsa_verify are input-only and should not be modified. Same for curve_pubkey in curve_to_ed_pub. * chore: remove commented-out old Crypto dependency in portduino.ini * Leave out the admin module change for now --------- Co-authored-by: Jonathan Bennett <jbennett@incomsystems.biz> * trunk * protobuf re-update * Protobufs * Merge resolution fix * Put XEDDSA on the right bit * NodeDB update to new nodeInfoLite accessors, etc * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Refine unsigned packet rejection logic in Router (#10534) * use hardware random to fill the first 32 signature bytes with entropy prior to signing. * Add XEdDSA packet-signing policy tests and update dependencies for macos * Minor fixes * integrate XEdDSA support and update dependencies across multiple modules --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Wessel <github@weebl.me> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> |
||
|
|
de345939af |
Automatic variable hop limits based on mesh activity and size estimation (#10176)
* asdf * Implement SphereOfInfluenceModule for traffic management and eviction tracking * Implement Sphere of Influence module for dynamic hop limit adjustment and role-based floor * Update SAMPLING_DENOMINATOR to improve mesh size estimation accuracy * Add debug logging for scale factor estimation and per-hop node counts in SphereOfInfluenceModule * Enable variable hop limits and role-based hop floors in Sphere of Influence module * Respond to copilot review * Disable variable hop limits and role-based hop floors in Sphere of Influence module * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Implement adaptive sampling for unique node ID tracking in Sphere of Influence module * Add state persistence for Sphere of Influence module * Enhance Sphere of Influence module with state management and adaptive sampling adjustments * Refactor hop scaling functionality: remove SphereOfInfluenceModule and introduce HopScalingModule - Deleted SphereOfInfluenceModule.h, consolidating its responsibilities into the new HopScalingModule. - Added HopScalingModule.h and HopScalingModule.cpp to manage hop scaling logic, including eviction tracking and sampling-based mesh size estimation. - Implemented methods for recording evictions and packet senders, estimating scale factors, and computing required hops based on node activity. - Introduced state persistence for hop scaling parameters to maintain continuity across reboots. - Enhanced thread safety and modularity by utilizing concurrency features. * Guard out STM32. Sowwy. * Refactor HopScalingModule: enhance sampling logic and improve state management * Add unit tests for HopScalingModule: implement mock database and various test scenarios * Refactor test output in HopScalingModule tests: replace printf with TEST_MESSAGE for better integration with Unity * Refactor HopScalingModule logging: replace lastStatusMode with descriptive mode names for improved readability * Refactor test_main.cpp: change NodeNum variable to static and improve comments for clarity * Remove unnecessary delay in setup function for improved test performance * Add missing include for MeshTypes in test_main.cpp * Refactor HopScalingModule tests: enhance mesh topology scenarios and improve test clarity * Update HopScalingModule tests: flesh out node scenarios and improve clarity for dense and sparse mesh cases * Fix politeness factor calculations in HopScalingModule and update related test scenarios for clarity. Remove outdated design doc. * Enhance HopScalingModule: add sampled estimate for scaling decisions and refactor initial run state management * Add sample traffic injection for HopScaling tests to enhance sampledEst visibility * Enhance HopScalingModule: adjust windowFraction calculation for early triggers and improve test output formatting * Enhance HopScalingModule: add jitter functionality to sampling denominator and update tests for consistent behavior * Enhance HopScalingModule: implement adaptive sampling denominator adjustment and add reset functionality for tests * Enhance HopScalingTestShim: add test-only clock and window helpers, update injectSampleTraffic for adaptive sampling, and improve scenario summary output * Enhance HopScalingModule: add detailed documentation for functions, improve clarity of jitter and sampling logic, and reset functionality in tests * Enhance HopScalingModule: add evictionEstimate parameter to estimateScaleFactor and update related logging for improved mesh size estimation * Enhance HopScalingModule: adjust effective rolls calculation for improved accuracy, add eviction estimate logic, responding to all copilot review points * Implement CompactHistogram for parallel hop scaling sampling - Added CompactHistogram class to track node hop distances with bitwise sampling. - Integrated CompactHistogram into HopScalingModule for independent packet sampling. - Updated NodeDB to feed both the hop scaling module and the new histogram sampler. - Enhanced HopScalingModule with methods to sample packets for the histogram and retrieve hop distribution statistics. - Implemented tests for CompactHistogram functionality, including sampling, window rolling, and adaptive denominator scaling. - Updated existing tests to validate the integration of the new histogram sampling mechanism. * Enhance CompactHistogram and HopScalingModule: add per-hop distribution functionality, improve time handling for unit tests, and refine test setup for deterministic behavior * CompactHistogram: add mesh size estimation, improve entry replacement logic, and update logging for per-hop distribution * Refactor HopScalingModule and CompactHistogram integration - Removed the suggestedHopFromCompactHistogram function to streamline hop suggestion logic. - Updated HopScalingModule to directly utilize CompactHistogram's internal methods for hop suggestions and sampling. - Enhanced logging in HopScalingModule to provide detailed histogram statistics. - Modified test cases to ensure comprehensive coverage of new histogram behaviors and sampling logic. - Improved node ID distribution in tests to better exercise sampling mechanisms. - Ensured that filtering denominators are held for 12 hours before dropping, enhancing stability in sampling. * Refactor CompactHistogram to support 13-hour activity tracking and introduce politeness regimes - Updated the bitfield structure to accommodate 13-hour seen tracking. - Changed the logic in rollHour() to analyze activity over the last 0-2 hours vs. 1-3 hours for politeness factor calculation. - Introduced three politeness levels: GENEROUS, DEFAULT, and STRICT based on recent activity ratios. - Adjusted filtering and sampling logic to reflect the new 13-hour tracking period. - Updated unit tests to validate new behavior and ensure proper functionality of politeness regimes. * Enhance CompactHistogram and HopScalingModule for improved sampling and decision-making - Introduced a session-specific hash seed in CompactHistogram to reduce bias in node ID sampling. - Updated sampling logic to use hashed node IDs instead of raw IDs for filtering and entry management. - Added histogram rollover tracking in HopScalingModule to ensure proper decision-making after initial data collection. - Adjusted logging to reflect the active state of the histogram and its comparison with NodeDB advisory hops. - Enhanced unit tests to validate new sampling logic and memory layout changes. * Expose hashNodeId for testing in CompactHistogram * Add mesh trend statistics to CompactHistogram for enhanced activity tracking * Implement histogram state persistence in CompactHistogram with save and load functions * Refactor CompactHistogram to improve entry management and enhance rollHour logging * feat: add HopScalingModule for adaptive hop limit recommendations Introduces HopScalingModule, a sampled hop-distance histogram that recommends the minimum hop limit needed to reach ~40 nodes, and automatically reducing the hops as the mesh grows. Key design: - 512-byte packed histogram (128 × 4-byte Record entries) embedded in a new HopScalingModule. - Each Record: 16-bit node hash, 3-bit hop distance, 13-bit seen bitmap - Sampling filter: only nodes where (hash & (denom-1)) == 0 are kept; denominator doubles on overflow and halves when utilisation is low - Hourly rollHour(): tallies per-hop counts, walks scaled buckets to find the minimum hop satisfying TARGET_AFFECTED_NODES (40), applies a politeness extension based on recent/older activity ratio, shifts all seen bitmaps, and persists state to /prefs/hopScalingState.bin - Hop recommendation gated by bootstrap (requires >=1 rollHour before overriding HOP_MAX) - NodeDB calls samplePacketForHistogram() on every non-MQTT rx packet - Module also estimates total mesh size and logs useful information about mesh characteristics. Changes: - src/modules/HopScalingModule.h/.cpp: new module - src/mesh/NodeDB.cpp: wire up samplePacketForHistogram - src/mesh/Router.cpp: consume getLastRequiredHop() - test/test_hop_scaling/: 12-test suite covering all mesh topologies and anticipated operational requirements * test: increase run iterations in sparse to dense transition test * feat: refactor HopScalingModule to use RUNS_PER_HOUR constant and improve logging * feat: enhance HopScalingModule with filtering denominator management and add tests for state transitions * refactor: remove CompactHistogram module and related files * address copilot review comments * Tweak: packet sampling only lora * ove role-based hop floor logic and related definitions into the module - keep it in one place. * Refactor MockNodeDB to use nodeInfoLiteSetBit for MQTT flag setting * Refactor hop scaling parameters and logic to integer maths and put default values in defaults.h. Small flash size reduction, no functional impact. * Update unit test preprocessor directives to PIO_UNIT_TESTING for consistency * refactor: improve test output organization and clarity in hop scaling tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Thomas GöttgensandGitHub
|
f86cb7781e | Remove fragile JSON libraries from the firmware while retaining Meshtasticd JSON support (#10152) | ||
 Clive BlackledgeandGitHub
|
32dcd90abf | Preserve forwarded position payload precision (#10554) | ||
|
5e69bc6c3f |
Enable Narrow and Lite regions for EU (#10120)
* Enable Lite and Narrow regions and introduce getEffectiveDutyCycle for Lite profiles * Add TrafficType enum and extend getConfiguredOrDefaultMsScaled to manage based on regionProfile settings * Refactor telemetry modules to include TrafficType in getConfiguredOrDefaultMsScaled calls * Update submodule protobufs to latest commit * Add support for new region presets and modem presets in menu options * Add new LoRa region codes and modem presets for EU bands * boof * Add modem presets for LITE and NARROW configurations * Update subproject commit reference in protobufs * Update protobufs * Refactor modem preset definitions to use macro for consistency and clarity * Refactor modem preset cases to use PRESET macro for consistency * fix: update LoRa region code for EU 868 narrowband configuration Co-authored-by: Copilot <copilot@github.com> * Fix test suite failure Co-authored-by: Copilot <copilot@github.com> * Add override slot override - for when one override isn't enough. Co-authored-by: Copilot <copilot@github.com> * address copilot comments --------- Co-authored-by: Copilot <copilot@github.com> |
||
|
4827498188 |
Clamp direct position packets to channel precision (fixes #8640) (#10383)
* Fix position precision for direct sends * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Clarify zero position precision logging * Use const channel reference for position precision * Use C linkage for position precision test entrypoints --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
Ben MeadorsGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>Copilot Autofix powered by AI |
94bb21ecc7 |
2.8: NodeDB shrink, decoupling, and restructuring (#10413)
* 2.8: NodeDB refactor to decouple satellite entries and decrease size * Regen * Refactor node mute handling to use dedicated functions for clarity and consistency * Develop ref * Fix NodeDB review follow-ups Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/6b1d6cf6-ed6b-43b6-95cb-8e141757664e Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Address review validation nits Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/6b1d6cf6-ed6b-43b6-95cb-8e141757664e Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Trunk * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Extract legacy NodeDatabase migration * Fix remaining NodeDB review issues Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/c76b9a5a-7244-4fbc-9ef0-98091d8caaea Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Fixes * Trunk * Fix latest review compile follow-ups Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/5198da01-ec4c-4c16-8a09-68b8e6d5d410 Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Fix cppcheck style warnings Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/e60287ba-4ece-46e0-83d8-a6d89664c0bb Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Change pointer type for mesh node in set_favorite function * Change pointer types for mesh node references to const in multiple applets * Add NodeDB layout v25 documentation and migration guidelines * Remove tests for uninitialized PacketHistory state due to undefined behavior * Fix code block formatting in copilot instructions --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
1bcabb893b |
mesh: bound the user-facing notification sprintf calls (#10437)
Two sites built ClientNotification messages with sprintf into a fixed-size proto buffer with no length cap. The current format strings fit comfortably, but a future caller editing either format string without rechecking the buffer size would get a silent stack/heap overrun. Switch to snprintf with sizeof so the bound is enforced at the call site. Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
bfadf0c36a |
fix(Router): localize p_encrypted to prevent recursive-overwrite leak (#10311)
Router::handleReceived stores its allocCopy of the encrypted packet in the class member p_encrypted. callModules() invokes module replies that re-enter the router via MeshService::sendToMesh -> Router::sendLocal, which on a broadcast reply recursively calls handleReceived. The inner call overwrites the outer's p_encrypted without releasing it; on the way out it nulls the member, the outer release(p_encrypted) now releases nullptr, and the original allocation is permanently leaked. ~381 B per recursion. Promote p_encrypted to a function-local so each invocation owns its own copy for its full lifetime. The MQTT-publish null check at the call site (added by PR #9136 as a workaround for this bug) stays in place because allocCopy can still legitimately return nullptr on packetPool exhaustion. Copilot's review of PR #8999 (the original introduction) flagged this exact pattern at merge time: "Storing p_encrypted as a class member can cause issues with recursive or concurrent calls to handleReceived() since each call would overwrite the previous packet pointer." The historical reason for the member (S&F needing to retain the encrypted copy across calls) was satisfied differently by PR #9799 (ServerAPI converted to std::unique_ptr + cleanup on connection close), so the member is no longer load-bearing. Reproduces issues #9632 / #10101 / #8729 (heap leak when MeshMonitor connected; TCP drops on Station G2 / LILYGO ServerAPI dump abort). Hardware A/B on Station G2 under sustained TCP-API retry storm (open :4403, request config, disconnect mid-stream, repeat at ~0.6/s) - 9 min run: | Build | heapFree drift | rebootCount delta | | this patch | -1.5 KB (noise)| 0 | | stock 2.7.13 | -73 KB (8.1KB/min) | +1 (OOM crash) | Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
nightjoker7andGitHub
|
399dde0f4b |
Router: demote cross-channel decrypt failures from ERROR to DEBUG (#10259)
The "Invalid protobufs (bad psk?)" and "Invalid portnum (bad psk?)" messages fire every time a neighbor transmits on a channel whose 8-bit hash matches one of ours but the PSK differs. In RF environments with multiple mesh groups nearby this is routine — a single device can see dozens of these per minute from SAR/MeshCA/private networks sharing a hash collision. LOG_ERROR for a benign "not our PSK" event: - spams the log when you have any neighboring mesh group - makes a genuine PSK misconfiguration on YOUR own channel indistinguishable from the constant cross-channel noise - hides actual errors in the stream LOG_DEBUG matches how similar decryption-failure paths are handled elsewhere (eg. the PKC "decrypt attempted but failed" uses LOG_WARN). Dropping the trailing "!" as well — these are expected events, not exceptional ones. |
||
  
|
ac7a58cd45 |
Fix: Traceroute through MQTT misses uplink node if MQTT is encrypted (#9798)
* Attempt to fix issue 9713 * Code formatting issue. * Remade the fix to follow Copilot observations on PR * Rebuild after AI and GUVWAF * Update src/mesh/Router.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/Router.cpp Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * trunk fmt --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: GUVWAF <thijs@havinga.eu> Co-authored-by: GUVWAF <78759985+GUVWAF@users.noreply.github.com> |
||
|
53c21eb30d |
Deprecate/block packets with a missing/invalid hop_start value (pre-hop firmware) (related to issue #7369) (#9476)
* Deprecate forwarding for invalid hop_start * Add pre-hop packet drop policy * Log ignored rebroadcasts for pre-hop packets * Respect pre-hop policy ALLOW in routing gates * Exempt local packets from pre-hop drop policy * Format pre-hop log line * Add MODERN_ONLY rebroadcast mode for pre-hop packets * Simplify implementation for drop packet only behaviour * Revert formatting-only changes * Match ReliableRouter EOF formatting * Make pre-hop drop a build-time flag * Rework to compile/build flag MESHTASTIC_PREHOP_DROP * Set MESHTASTIC_PREHOP_DROP off by default * Inline pre-hop hop_start validity check --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> Co-authored-by: Jord <650645+DivineOmega@users.noreply.github.com> |
||
|
016e68ec53 |
Traffic Management Module for packet forwarding logic (#9358)
* Add ESP32 Power Management lessons learned document Documents our experimentation with ESP-IDF DFS and why it doesn't work well for Meshtastic (RTOS locks, BLE locks, USB issues). Proposes simpler alternative: manual setCpuFrequencyMhz() control with explicit triggers for when to go fast vs slow. * Addition of traffic management module * Fixing compile issues, but may still need to update protobufs. * Fixing log2Floor in cuckoo hash function * Adding support for traffic management in PhoneAPI. * Making router_preserve_hops work without checking if the previous hop was a router. Also works for CLIENT_BASE. * Adding station-g2 and portduino varients to be able to use this module. * Spoofing from address for nodeinfo cache * Changing name and behavior for zero_hop_telemetry / zero_hop_position * Name change for exhausting telemetry packets and setting hop_limit to 1 so it will be 0 when sent. * Updated hop logic, including exhaustRequested flag to bypass some checks later in the code. * Reducing memory on nrf52 nodes further to 12 bytes per entry, 12KB total using 8 bit hashes with 0.4% collision. Probably ok. Adding portduino to the platforms that don't need to worry about memory as much. * Fixing hopsAway for nodeinfo responses. * traffic_management.nodeinfo_direct_response_min_hops -> traffic_management.nodeinfo_direct_response_max_hops * Removing dry run mode * Updates to UnifiedCacheEntry to use a common cache, created defaults for some values, reduced a couple bytes per entry by using a resolution-scale time selection based on configuration value. * Enhance traffic management logging and configuration. Updated log messages in NextHopRouter and Router to include more context. Adjusted traffic management configuration checks in AdminModule and improved cache handling in TrafficManagementModule. Ensured consistent enabling of traffic management across various variants. * Implement destructor for TrafficManagementModule and improve cache allocation handling. The destructor ensures proper deallocation of cache memory based on its allocation source (PSRAM or heap). Additionally, updated cache allocation logic to log warnings only when PSRAM allocation fails. * Update TrafficManagementModule with enhanced comments for clarity and improve cache handling logic. Update protobuf submodule to latest commit. * Creating consistent log messages * Remove docs/ESP32_Power_Management.md from traffic_module * Add unit tests for Traffic Management Module functionality * Fixing compile issues, but may still need to update protobufs. * Adding support for traffic management in PhoneAPI. * Making router_preserve_hops work without checking if the previous hop was a router. Also works for CLIENT_BASE. * Enhance traffic management logging and configuration. Updated log messages in NextHopRouter and Router to include more context. Adjusted traffic management configuration checks in AdminModule and improved cache handling in TrafficManagementModule. Ensured consistent enabling of traffic management across various variants. * Implement destructor for TrafficManagementModule and improve cache allocation handling. The destructor ensures proper deallocation of cache memory based on its allocation source (PSRAM or heap). Additionally, updated cache allocation logic to log warnings only when PSRAM allocation fails. * Update TrafficManagementModule with enhanced comments for clarity and improve cache handling logic. Update protobuf submodule to latest commit. * Add mock classes and unit tests for Traffic Management Module functionality. * Refactor setup and loop functions in test_main.cpp to include extern "C" linkage * Update comment to include reduced memory requirements Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Re-arranging comments for programmers with the attention span of less than 5 lines of code. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update comments in TrafficManagementModule to reflect changes in timestamp epoch handling and memory optimization details. * bug: Use node-wide config_ok_to_mqtt setting for cached NodeInfo replies. * Better way to handle clearing the ok_to_mqtt bit * Add bucketing to cuckoo hashing, allowing for 95% occupied rate before major eviction problems. * Extend nodeinfo cache for psram devices. * Refactor traffic management to make hop exhaustion packet-scoped. Nice catch. * Implement better position precision sanitization in TrafficManagementModule. * Added logic in TrafficManagementModule to invalidate stale traffic state. Also, added some tests to avoid future me from creating a regression here. * Fixing tests for native * Enhance TrafficManagementModule to improve NodeInfo response handling and position deduplication logic. Added tests to ensure local packets bypass transit filters and that NodeInfo requests correctly update the requester information in the cache. Updated deduplication checks to prevent dropping valid position packets under certain conditions. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
|
0ed537a336 |
Add json file rotation option (#9783)
Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
|
7b03980e0a |
Refuse to send legacy DMs simply because the remote public key is unknown (#9485)
* Refuse to send legacy DMs simply because the remote public key is unknown * Update src/mesh/Router.cpp - use more specific error Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/mesh/Router.cpp - more detail in warning message Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
f710cd6ecb |
Support fully direct request/responses (#9455)
Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |