b93239bb0a7a2b642fab90de2d5e62c7195a8ec5
41
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Thomas GöttgensandGitHub
|
c4fdf374e1 |
fix: right-size warm tier for constrained platforms (#10746, #10705) (#10759)
* fix: right-size warm tier for constrained platforms and feed RP2040 watchdog during NodeDB save * fix: size warm tier and traffic cache per-MCU RAM, lowering no-PSRAM ESP32 (classic/S2/C3) tiers * docs: document nRF52/RP2040 #else fall-through in warm tier and TM cache cascades |
||
 
|
c51c01607d |
Traffic Management module: dedup, rate limiting, role-aware policing (#10706)
Adds the Traffic Management module (TMM) plus the NodeDB/warm-store and next-hop foundations it builds on: - Unified per-node cache (flat array, 8-bit relative ticks) shared by all features; role-aware throttles for tracker / lost-and-found. - Position deduplication: drop unchanged position rebroadcasts within a configurable interval; precision driven off the channel ceiling (clamped to the public-key max on well-known channels). Enabled by default at 11h. - Per-node rate limiting and unknown-packet filtering (config-driven; a non-zero companion field enables each feature -- no bool toggles). - NodeInfo direct response from cache with role-based hop clamps. - Persistent next-hop overflow store: confirmed hops have no TTL, are seeded from NodeInfoLite at boot, and survive hot-store eviction. - Three-tier sender-role resolution (hot NodeInfoLite -> warm store -> TMM cache). Role is cached write-time (seeded on first track, refreshed from NodeInfo), pins its cache entry like a next-hop hint, and is evicted last. - Warm store caches device role + protected category across reboot/eviction. - PositionModule stationary floor for tracker / lost-and-found. - PSRAM gating for warm/satellite/TMM cache sizes; STM32WL excluded. Protobufs: TrafficManagementConfig trimmed to the five uint32 fields actually used; submodule repointed to protobufs develop. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
22072c5f4b |
Pr1.5 tmm nexthop (#10745)
* TrafficManagement: flat unified cache + persistent next-hop overflow store Reworks the TrafficManagementModule cache layer (policing behaviour unchanged from upstream) and adds a routing-hint overflow store: - Flatten the ring: replace the cuckoo-hashed unified cache and the bucketed PSRAM NodeInfo index with plain flat arrays + linear scan (same idiom as WarmNodeStore). At LoRa packet rates an O(n) scan of the cache is negligible, and it removes a large amount of hashing/displacement complexity. The cache entry is 11 B; timestamps use a uniform +1 presence-offset so a 0 byte always means "empty" across every sub-store. Adds rebaseEpoch() so cached state survives the ~19 h relative-timestamp horizon instead of being flushed. - Next-hop overflow cache: setNextHop/getNextHopHint store a confirmed last-byte relay for a destination, written only from NextHopRouter's ACK-confirmed decision (and mirrored from TraceRoute). NextHopRouter::getNextHop falls back to this cache when the hot NodeDB has no hint, so DMs/relays to long-tail nodes keep routing after the node ages out of NodeInfoLite. - Persistence: preloadNextHopsFromNodeDB warm-starts the cache from persisted NodeInfoLite hints on first maintenance pass; next_hop entries are kept alive across the maintenance sweep (no TTL) and never clobbered by a stale preload. All packet-policing logic (rate limit, position dedup, unknown-packet drop, NodeInfo direct response, hop exhaustion) is the existing upstream behaviour, untouched. HAS_TRAFFIC_MANAGEMENT defaults on so the module is compiled in. (see note). Tests: upstream policing suite now actually runs (adds the MeshTypes.h include that gates HAS_TRAFFIC_MANAGEMENT) plus 4 next-hop tests. Role-aware throttles, politeness, precision clamp, port-interval and mesh-radius gating — and the rate-limit >255 saturation fix — are deferred to the advanced-TMM branch. Note: default dedup movement grid moves to ~91m, which also means 1.5km required to end up with the same signature position - coarser and therefore further than before. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * TrafficManagement: fix cppcheck constVariablePointer warning `node` in preloadNextHopsFromNodeDB() is never written through — mark it const to satisfy cppcheck's constVariablePointer check in CI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add multi-hop NextHop recovery tests and unit tests for routing reliability - Introduced a new test suite for multi-hop NextHop directed-message delivery and relay recovery in `test_nexthop_multihop_recovery.py`. This includes tests for end-to-end delivery and recovery after relay drop. - Implemented unit tests in `test_main.cpp` for NextHop routing reliability mitigations, covering: - M1: Ambiguity-aware last-byte resolution. - M2: NextHopRouter's strict-neighbor gate and hop limit checks. - M3: Route-health freshness and failure decay. - Enhanced mock classes to facilitate controlled testing of node behaviors and routing logic. * grafting fixed * Address Copilot review for PR #10735 (NextHop improvements) - docs/nexthop-routing-reliability.md: update status from "no code changes yet" to reflect that mitigations and tests are implemented RAM pressure and MIGRATION_VERBOSE concerns addressed upstream in PR2.5 (per-platform TRAFFIC_MANAGEMENT_CACHE_SIZE) and PR2 (verbose default=0) respectively; (0,0) sentinel fixed in PR2.5. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * CI: fix cppcheck constVariablePointer and test include path - NextHopRouter.cpp: qualify two RouteHealth *h locals as const — only read for stale-route checks, never mutated through the pointer - Router.cpp: qualify meshtastic_NodeInfoLite *node as const in shouldDecrementHopLimit — only read for favorite/role predicate - test_position_module/test_main.cpp: change bare PositionModule.h to modules/PositionModule.h — build_flags sets -Isrc, not -Isrc/modules, so the bare form fails to resolve in the native PlatformIO test env Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * WarmStore: cache device role + protected category in last_heard low bits Steal the low 6 bits of WarmNodeEntry.last_heard to carry an evicted node's device role (4 bits) and a protected category (2 bits) for the hop-trim path, at zero record-size cost (entry stays 40 B; no RAM/flash growth). The high bits remain a real unix-seconds timestamp, quantised to 64 s — ample for warm LRU ordering of long-tail nodes. - absorb() packs role/protectedCat; place()/ring replay store the raw word so metadata round-trips through flash. LRU compares masked time (warmTimeOf). - take() rehydration masks the metadata bits and restores the cached role so a re-admitted node isn't stuck at CLIENT until its next NodeInfo. - NodeDB classifies the category (favorite/ignored/verified -> Flag; tracker/sensor/tak_tracker -> Role) at each eviction site. - WarmNodeStore::lookupMeta() exposes role/category to consumers. - Bump WARM_RING_MAGIC (WRNG->WRN2): old rings read as erased and rebuild; warm data is a non-critical evictee cache, so discard-on-upgrade is safe. Tests: test_warm_store 11/11 (new meta round-trip + quantisation-aware ordering); NodeDB compiles (test_nodedb_blocked 4/4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: migrate v1 rings/files by discarding last_heard, not the data Previously the WRNG->WRN2 magic bump treated old rings as erased, discarding all warm entries — including the PKI public keys that let evicted nodes keep decrypting DMs. Instead, read v1 (WRNG / WRM1) records and keep each node's identity + public key, discarding only last_heard (its low bits would otherwise be misread as the new role/protected metadata). Records re-rank and re-learn their role on next contact. - Ring backend (nRF52840): ringReadHeader accepts both magics and reports v1 via an out-param; replay zeroes last_heard for v1 records. If the active head page is v1, force a rotation so new v2 records never land in a v1-headered page (which would discard their freshly-set role on the next load). Legacy pages convert to v2 as the ring rotates. - File backend (warm.dat): bump WARM_STORE_MAGIC WRM1->WRM2; accept WRM1, verify CRC against the stored bytes, then discard last_heard and mark dirty so the next save rewrites as v2. Tests: test_warm_store 12/12 (adds test_ws_v1_migration_discardsLastHeard: key survives, role/protected reset). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: guard role bit-width + test eviction carries role/protected - static_assert that the device role enum still fits the 4-bit warm metadata field (WARM_ROLE_MASK); fails the build loudly if a new role is added past 15 rather than silently truncating role on eviction. (Max role today = 12.) - Add test_migration_carriesRoleAndProtectedIntoWarm: a demoted TRACKER lands in the warm tier with its key, role=TRACKER and protected category=Role; a demoted CLIENT carries role=CLIENT/None. Exercises the NodeDB eviction path + warmProtectedCategory classification (the warm-store unit tests only cover absorb() directly). Tests: test_nodedb_blocked 5/5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix copilot comments * fix(test): restore #if HAS_TRAFFIC_MANAGEMENT guard in TMM test The rebase onto PR1.5 lost the top-level HAS_TRAFFIC_MANAGEMENT guard that PR1.5 introduced, leaving the #else/#endif tail orphaned and causing compile errors on non-TMM builds. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
f2f23f8978 |
NimBLE params overhaul and try-fix for incompatible bond cleanup (#10741)
* NimBLE params overhaul and try-fix for incompatible bond cleanup * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Address PR review: remove dead clearNVS(), defer bond-purge log below init - Delete unused clearNVS() (no callers; should have been removed in #10264). - Move purgeIncompatibleBleBonds() after the "Init the NimBLE" log so bond-cleanup output doesn't appear to precede module init; it still runs before BLEDevice::init() reads the store. * Update MAX_SATELLITE_NODES and WARM_NODE_COUNT definitions for ESP32-S3 PSRAM support --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
 TomGitHubClaude Opus 4.8github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Ben Meadors |
79a7dcc46c |
Pr1 nodedb warmstore (#10705)
* NodeDB: 3-tier node store with persistent warm tier (long-tail identity retention)
Introduces a tiered NodeDB so the device retains identity (public key,
last_heard) for far more nodes than fit in the full-record hot store,
without growing heap or the persisted nodes.proto unboundedly.
- Hot store: full NodeInfoLite, MAX_NUM_NODES (120 on nRF52).
- Satellite maps: position/telemetry/environment/status capped at
MAX_SATELLITE_NODES (40 freshest); eviction via enforceSatelliteCaps /
evictSatelliteOverCap.
- Warm tier (WarmNodeStore): 40 B {num,last_heard,public_key} records for
evicted nodes so DMs to/from long-tail nodes keep encrypting/decrypting.
Persisted to /prefs/warm.dat, or on nRF52840 a dedicated 12 KB raw-flash
record-ring below LittleFS (3x4 KB pages; see linker scripts + the
nrf52_warm_region.py post-link guard).
NodeDB::getOrCreateMeshNode now demotes evicted nodes into the warm tier and
re-admits them (restoring key/last_heard). Router PKI decrypt/encode resolve
the peer key via NodeDB::copyPublicKey (hot store, then warm tier).
NodeInfoLite gains snr_q4 (sint32, Q4-encoded dB); the float snr is zeroed on
disk. NodeInfoLite grows 105 -> 112 B; backup 2432 -> 2468 B.
Note: the snr_q4 .proto change still needs to land in the protobufs submodule
(generated header is updated here; submodule pointer left at upstream).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* NodeDB: robust receive + retention for blocked (ignored) nodes
Hardens how ignored/favourite nodes are received over admin and retained,
closing paths where a block could be lost or accidentally cleared.
- Blocking keeps the node's public key (admin set_ignored_node and
addFromContact no longer zero it / drop the warm-tier key), so a blocked
peer stays a verifiable identity.
- set_ignored_node creates the node if absent, so a block by node ID sticks
even for a node we've never heard from (e.g. pushed by a remote admin) with
no NodeInfo or key.
- Eviction protection (favourite/ignored/manually-verified) now also applies to
the load-time hot-store migration and is never undone by cleanupMeshDB, which
previously purged ignored nodes that lacked user info.
- The hot-store migration leaves our own node (index 0) in place and prefers to
demote non-protected nodes, like the runtime eviction scan.
Caps the protected set (favourite + ignored + verified) at MAX_NUM_NODES-2 via
NodeDB::setProtectedFlag(), so at least two evictable slots always remain and
getOrCreateMeshNode can always make room — replacing the previous unconditional
append that could run off the end of the node vector when every node was
protected. A locally-set favourite/ignore that hits the cap reports back to the
phone via a ClientNotification.
Adds test_nodedb_blocked covering the migration, favourite/ignored eviction
protection, ignored-survives-cleanup, and the protected-node cap. The
maintenance methods stay private in production; the test reaches them through a
PIO_UNIT_TESTING-guarded friend shim.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# Conflicts:
# src/mesh/NodeDB.h
* fix copilot comments
* once again
* WarmNodeStore: fix cppcheck warnings (uninitvar, constVariablePointer)
Zero-initialise `stranded[]` and `seqs[]/order[]` VLAs so cppcheck can
verify there are no unguarded reads of uninitialised memory (the guards
exist but are not visible to static analysis). Mark two local pointers
`const` where the pointed-to entry is never mutated after assignment.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* self-care added to assist 2.7 and 2.8 nodedb migration
* Tidy warm-store/self-care: comments, guards, log + flash cleanup
Style/cleanup pass over the branch (no behavior change except the noted
preprocessor simplifications, which are semantically identical):
- Comments: move function descriptions to the headers, cap in-function
comments at ~3-4 lines, drop leading-number step markers, label stacked
#endif blocks, de-decorate banner comments.
- dumpToLog: fully gate decl + definition + AdminModule call site behind
MESHTASTIC_NODEDB_MIGRATION_VERBOSE so it compiles out when disabled
(~1.2 KB when off).
- mesh-pb-constants: drop the dead nRF52832 WARM_NODE_COUNT branch and trim
the macro docs.
- WarmNodeStore: simplify the redundant `ARCH_NRF52 && NRF52840_XXAA` guards
to `NRF52840_XXAA`, add a kNoPage sentinel for the ring page state.
- Shorten the always-on LOG_WARN strings (~120 B flash).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* more tidying up, aligning with docs and undoing other-arch regressions
* Update protobufs (#19)
Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com>
* made the migration pathway cleareer
* address copilot review
* fixed a copilot review on a downstream PR.
* Address Copilot review comments for PR #10705 (warmstore/nodedb)
- WarmNodeStore.h: default MIGRATION_VERBOSE to 0 (suppress info-level
chatter on production builds; opt in with =1)
- WarmNodeStore.cpp load(): move memset to top of function so all
failure paths (header-read fail, invalid header) leave entries clear
- WarmNodeStore.cpp save(): replace manual spiLock lock/unlock around
mkdir with LockGuard covering the full SafeFile sequence, matching
the lock discipline in load()
- Router.cpp: memcpy(&p->public_key.bytes, ...) -> memcpy(p->public_key.bytes,
...) — pass decayed uint8_t* rather than pointer-to-array
- AdminModule.cpp: check setProtectedFlag return for PKC auto-favorite;
log cap-refusal warning instead of unconditional "auto-favoriting"
- nrf52_warm_region.py: error message references both v6.ld and v7.ld
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* NodeDB: formatting cleanup (blank lines after preprocessor blocks)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Lukewarm store
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
|
||
|
|
de345939af |
Automatic variable hop limits based on mesh activity and size estimation (#10176)
* asdf * Implement SphereOfInfluenceModule for traffic management and eviction tracking * Implement Sphere of Influence module for dynamic hop limit adjustment and role-based floor * Update SAMPLING_DENOMINATOR to improve mesh size estimation accuracy * Add debug logging for scale factor estimation and per-hop node counts in SphereOfInfluenceModule * Enable variable hop limits and role-based hop floors in Sphere of Influence module * Respond to copilot review * Disable variable hop limits and role-based hop floors in Sphere of Influence module * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Implement adaptive sampling for unique node ID tracking in Sphere of Influence module * Add state persistence for Sphere of Influence module * Enhance Sphere of Influence module with state management and adaptive sampling adjustments * Refactor hop scaling functionality: remove SphereOfInfluenceModule and introduce HopScalingModule - Deleted SphereOfInfluenceModule.h, consolidating its responsibilities into the new HopScalingModule. - Added HopScalingModule.h and HopScalingModule.cpp to manage hop scaling logic, including eviction tracking and sampling-based mesh size estimation. - Implemented methods for recording evictions and packet senders, estimating scale factors, and computing required hops based on node activity. - Introduced state persistence for hop scaling parameters to maintain continuity across reboots. - Enhanced thread safety and modularity by utilizing concurrency features. * Guard out STM32. Sowwy. * Refactor HopScalingModule: enhance sampling logic and improve state management * Add unit tests for HopScalingModule: implement mock database and various test scenarios * Refactor test output in HopScalingModule tests: replace printf with TEST_MESSAGE for better integration with Unity * Refactor HopScalingModule logging: replace lastStatusMode with descriptive mode names for improved readability * Refactor test_main.cpp: change NodeNum variable to static and improve comments for clarity * Remove unnecessary delay in setup function for improved test performance * Add missing include for MeshTypes in test_main.cpp * Refactor HopScalingModule tests: enhance mesh topology scenarios and improve test clarity * Update HopScalingModule tests: flesh out node scenarios and improve clarity for dense and sparse mesh cases * Fix politeness factor calculations in HopScalingModule and update related test scenarios for clarity. Remove outdated design doc. * Enhance HopScalingModule: add sampled estimate for scaling decisions and refactor initial run state management * Add sample traffic injection for HopScaling tests to enhance sampledEst visibility * Enhance HopScalingModule: adjust windowFraction calculation for early triggers and improve test output formatting * Enhance HopScalingModule: add jitter functionality to sampling denominator and update tests for consistent behavior * Enhance HopScalingModule: implement adaptive sampling denominator adjustment and add reset functionality for tests * Enhance HopScalingTestShim: add test-only clock and window helpers, update injectSampleTraffic for adaptive sampling, and improve scenario summary output * Enhance HopScalingModule: add detailed documentation for functions, improve clarity of jitter and sampling logic, and reset functionality in tests * Enhance HopScalingModule: add evictionEstimate parameter to estimateScaleFactor and update related logging for improved mesh size estimation * Enhance HopScalingModule: adjust effective rolls calculation for improved accuracy, add eviction estimate logic, responding to all copilot review points * Implement CompactHistogram for parallel hop scaling sampling - Added CompactHistogram class to track node hop distances with bitwise sampling. - Integrated CompactHistogram into HopScalingModule for independent packet sampling. - Updated NodeDB to feed both the hop scaling module and the new histogram sampler. - Enhanced HopScalingModule with methods to sample packets for the histogram and retrieve hop distribution statistics. - Implemented tests for CompactHistogram functionality, including sampling, window rolling, and adaptive denominator scaling. - Updated existing tests to validate the integration of the new histogram sampling mechanism. * Enhance CompactHistogram and HopScalingModule: add per-hop distribution functionality, improve time handling for unit tests, and refine test setup for deterministic behavior * CompactHistogram: add mesh size estimation, improve entry replacement logic, and update logging for per-hop distribution * Refactor HopScalingModule and CompactHistogram integration - Removed the suggestedHopFromCompactHistogram function to streamline hop suggestion logic. - Updated HopScalingModule to directly utilize CompactHistogram's internal methods for hop suggestions and sampling. - Enhanced logging in HopScalingModule to provide detailed histogram statistics. - Modified test cases to ensure comprehensive coverage of new histogram behaviors and sampling logic. - Improved node ID distribution in tests to better exercise sampling mechanisms. - Ensured that filtering denominators are held for 12 hours before dropping, enhancing stability in sampling. * Refactor CompactHistogram to support 13-hour activity tracking and introduce politeness regimes - Updated the bitfield structure to accommodate 13-hour seen tracking. - Changed the logic in rollHour() to analyze activity over the last 0-2 hours vs. 1-3 hours for politeness factor calculation. - Introduced three politeness levels: GENEROUS, DEFAULT, and STRICT based on recent activity ratios. - Adjusted filtering and sampling logic to reflect the new 13-hour tracking period. - Updated unit tests to validate new behavior and ensure proper functionality of politeness regimes. * Enhance CompactHistogram and HopScalingModule for improved sampling and decision-making - Introduced a session-specific hash seed in CompactHistogram to reduce bias in node ID sampling. - Updated sampling logic to use hashed node IDs instead of raw IDs for filtering and entry management. - Added histogram rollover tracking in HopScalingModule to ensure proper decision-making after initial data collection. - Adjusted logging to reflect the active state of the histogram and its comparison with NodeDB advisory hops. - Enhanced unit tests to validate new sampling logic and memory layout changes. * Expose hashNodeId for testing in CompactHistogram * Add mesh trend statistics to CompactHistogram for enhanced activity tracking * Implement histogram state persistence in CompactHistogram with save and load functions * Refactor CompactHistogram to improve entry management and enhance rollHour logging * feat: add HopScalingModule for adaptive hop limit recommendations Introduces HopScalingModule, a sampled hop-distance histogram that recommends the minimum hop limit needed to reach ~40 nodes, and automatically reducing the hops as the mesh grows. Key design: - 512-byte packed histogram (128 × 4-byte Record entries) embedded in a new HopScalingModule. - Each Record: 16-bit node hash, 3-bit hop distance, 13-bit seen bitmap - Sampling filter: only nodes where (hash & (denom-1)) == 0 are kept; denominator doubles on overflow and halves when utilisation is low - Hourly rollHour(): tallies per-hop counts, walks scaled buckets to find the minimum hop satisfying TARGET_AFFECTED_NODES (40), applies a politeness extension based on recent/older activity ratio, shifts all seen bitmaps, and persists state to /prefs/hopScalingState.bin - Hop recommendation gated by bootstrap (requires >=1 rollHour before overriding HOP_MAX) - NodeDB calls samplePacketForHistogram() on every non-MQTT rx packet - Module also estimates total mesh size and logs useful information about mesh characteristics. Changes: - src/modules/HopScalingModule.h/.cpp: new module - src/mesh/NodeDB.cpp: wire up samplePacketForHistogram - src/mesh/Router.cpp: consume getLastRequiredHop() - test/test_hop_scaling/: 12-test suite covering all mesh topologies and anticipated operational requirements * test: increase run iterations in sparse to dense transition test * feat: refactor HopScalingModule to use RUNS_PER_HOUR constant and improve logging * feat: enhance HopScalingModule with filtering denominator management and add tests for state transitions * refactor: remove CompactHistogram module and related files * address copilot review comments * Tweak: packet sampling only lora * ove role-based hop floor logic and related definitions into the module - keep it in one place. * Refactor MockNodeDB to use nodeInfoLiteSetBit for MQTT flag setting * Refactor hop scaling parameters and logic to integer maths and put default values in defaults.h. Small flash size reduction, no functional impact. * Update unit test preprocessor directives to PIO_UNIT_TESTING for consistency * refactor: improve test output organization and clarity in hop scaling tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
Ben MeadorsandGitHub
|
eead467ce6 |
Added NodeDB fixtures and refactored to use std maps for better memory efficiency (#10464)
* Added NodeDB fixtures and refactored to use std maps for better efficiency * Defer NodeDB save during xmodem transfer to prevent mid-transfer fsFormat |
||
|
Ben MeadorsGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>Copilot Autofix powered by AI |
94bb21ecc7 |
2.8: NodeDB shrink, decoupling, and restructuring (#10413)
* 2.8: NodeDB refactor to decouple satellite entries and decrease size * Regen * Refactor node mute handling to use dedicated functions for clarity and consistency * Develop ref * Fix NodeDB review follow-ups Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/6b1d6cf6-ed6b-43b6-95cb-8e141757664e Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Address review validation nits Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/6b1d6cf6-ed6b-43b6-95cb-8e141757664e Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Trunk * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Extract legacy NodeDatabase migration * Fix remaining NodeDB review issues Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/c76b9a5a-7244-4fbc-9ef0-98091d8caaea Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Fixes * Trunk * Fix latest review compile follow-ups Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/5198da01-ec4c-4c16-8a09-68b8e6d5d410 Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Fix cppcheck style warnings Agent-Logs-Url: https://github.com/meshtastic/firmware/sessions/e60287ba-4ece-46e0-83d8-a6d89664c0bb Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com> * Change pointer type for mesh node in set_favorite function * Change pointer types for mesh node references to const in multiple applets * Add NodeDB layout v25 documentation and migration guidelines * Remove tests for uninitialized PacketHistory state due to undefined behavior * Fix code block formatting in copilot instructions --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
016e68ec53 |
Traffic Management Module for packet forwarding logic (#9358)
* Add ESP32 Power Management lessons learned document Documents our experimentation with ESP-IDF DFS and why it doesn't work well for Meshtastic (RTOS locks, BLE locks, USB issues). Proposes simpler alternative: manual setCpuFrequencyMhz() control with explicit triggers for when to go fast vs slow. * Addition of traffic management module * Fixing compile issues, but may still need to update protobufs. * Fixing log2Floor in cuckoo hash function * Adding support for traffic management in PhoneAPI. * Making router_preserve_hops work without checking if the previous hop was a router. Also works for CLIENT_BASE. * Adding station-g2 and portduino varients to be able to use this module. * Spoofing from address for nodeinfo cache * Changing name and behavior for zero_hop_telemetry / zero_hop_position * Name change for exhausting telemetry packets and setting hop_limit to 1 so it will be 0 when sent. * Updated hop logic, including exhaustRequested flag to bypass some checks later in the code. * Reducing memory on nrf52 nodes further to 12 bytes per entry, 12KB total using 8 bit hashes with 0.4% collision. Probably ok. Adding portduino to the platforms that don't need to worry about memory as much. * Fixing hopsAway for nodeinfo responses. * traffic_management.nodeinfo_direct_response_min_hops -> traffic_management.nodeinfo_direct_response_max_hops * Removing dry run mode * Updates to UnifiedCacheEntry to use a common cache, created defaults for some values, reduced a couple bytes per entry by using a resolution-scale time selection based on configuration value. * Enhance traffic management logging and configuration. Updated log messages in NextHopRouter and Router to include more context. Adjusted traffic management configuration checks in AdminModule and improved cache handling in TrafficManagementModule. Ensured consistent enabling of traffic management across various variants. * Implement destructor for TrafficManagementModule and improve cache allocation handling. The destructor ensures proper deallocation of cache memory based on its allocation source (PSRAM or heap). Additionally, updated cache allocation logic to log warnings only when PSRAM allocation fails. * Update TrafficManagementModule with enhanced comments for clarity and improve cache handling logic. Update protobuf submodule to latest commit. * Creating consistent log messages * Remove docs/ESP32_Power_Management.md from traffic_module * Add unit tests for Traffic Management Module functionality * Fixing compile issues, but may still need to update protobufs. * Adding support for traffic management in PhoneAPI. * Making router_preserve_hops work without checking if the previous hop was a router. Also works for CLIENT_BASE. * Enhance traffic management logging and configuration. Updated log messages in NextHopRouter and Router to include more context. Adjusted traffic management configuration checks in AdminModule and improved cache handling in TrafficManagementModule. Ensured consistent enabling of traffic management across various variants. * Implement destructor for TrafficManagementModule and improve cache allocation handling. The destructor ensures proper deallocation of cache memory based on its allocation source (PSRAM or heap). Additionally, updated cache allocation logic to log warnings only when PSRAM allocation fails. * Update TrafficManagementModule with enhanced comments for clarity and improve cache handling logic. Update protobuf submodule to latest commit. * Add mock classes and unit tests for Traffic Management Module functionality. * Refactor setup and loop functions in test_main.cpp to include extern "C" linkage * Update comment to include reduced memory requirements Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Re-arranging comments for programmers with the attention span of less than 5 lines of code. Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update comments in TrafficManagementModule to reflect changes in timestamp epoch handling and memory optimization details. * bug: Use node-wide config_ok_to_mqtt setting for cached NodeInfo replies. * Better way to handle clearing the ok_to_mqtt bit * Add bucketing to cuckoo hashing, allowing for 95% occupied rate before major eviction problems. * Extend nodeinfo cache for psram devices. * Refactor traffic management to make hop exhaustion packet-scoped. Nice catch. * Implement better position precision sanitization in TrafficManagementModule. * Added logic in TrafficManagementModule to invalidate stale traffic state. Also, added some tests to avoid future me from creating a regression here. * Fixing tests for native * Enhance TrafficManagementModule to improve NodeInfo response handling and position deduplication logic. Added tests to ensure local packets bypass transit filters and that NodeInfo requests correctly update the requester information in the cache. Updated deduplication checks to prevent dropping valid position packets under certain conditions. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
 JorropoandGitHub
|
beb268ff25 |
Revert "add a .clang-format file (#9154)" (#9172)
I thought git would be smart enough to understand all the whitespace changes but even with all the flags I know to make it ignore theses it still blows up if there are identical changes on both sides.
I have a solution but it require creating a new commit at the merge base for each conflicting PR and merging it into develop.
I don't think blowing up all PRs is worth for now, maybe if we can coordinate this for V3 let's say.
This reverts commit
|
||
|
JorropoandGitHub
|
0d11331d18 | add a .clang-format file (#9154) | ||
|
Ben Meadors
|
9977035499 | Fix DRAM overflow on old esp32 targets | ||
|
Ben Meadors
|
096afa07f8 | Tweak maximums | ||
|
Ben Meadors
|
8989de118c | Only queue 2 client notification | ||
|
Ben MeadorsandGitHub
|
d1d16fc25f |
Make phone queues use a static pointer queue (#7919)
* Make phone queues use a static pointer queue * Static init * Compile time constants now * Instead, lets just use the normal pointerqueue for linux native builds and static for IoT platforms * Add missing method * Missing methods * Update variant.h |
||
|
Ben Meadors
|
6041357cbb | Increase the debt ceiling | ||
|
850d21dcb9 |
Add a static_assert to verify assumption about NodeInfoLite size (#6428)
Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
 AustinandGitHub
|
6673cb9292 | Increase MAX_NUM_NODES on high-flash ESP32_S3 (#6311) | ||
+9      Ben MeadorsGitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>GUVWAFGUVWAFThomas GöttgensTom Fifieldmverch67Manueltodd-herbert Jason Murray Jason Murraydependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Jonathan BennettAustin virgil Mark Trevor Birss Kalle Lilja rcarteraz |
99d3e5eb70 |
2.6 changes (#5806)
* 2.6 protos
* [create-pull-request] automated change (#5789)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
* Hello world support for UDP broadcasts over the LAN on ESP32 (#5779)
* UDP local area network meshing on ESP32
* Logs
* Comment
* Update UdpMulticastThread.h
* Changes
* Only use router->send
* Make NodeDatabase (and file) independent of DeviceState (#5813)
* Make NodeDatabase (and file) independent of DeviceState
* 70
* Remove logging statement no longer needed
* Explicitly set CAD symbols, improve slot time calculation and adjust CW size accordingly (#5772)
* File system persistence fixes
* [create-pull-request] automated change (#6000)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
* Update ref
* Back to 80
* [create-pull-request] automated change (#6002)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
* 2.6 <- Next hop router (#6005)
* Initial version of NextHopRouter
* Set original hop limit in header flags
* Short-circuit to FloodingRouter for broadcasts
* If packet traveled 1 hop, set `relay_node` as `next_hop` for the original transmitter
* Set last byte to 0xFF if it ended at 0x00
As per an idea of @S5NC
* Also update next-hop based on received DM for us
* temp
* Add 1 retransmission for intermediate hops when using NextHopRouter
* Add next_hop and relayed_by in PacketHistory for setting next-hop and handle flooding fallback
* Update protos, store multiple relayers
* Remove next-hop update logic from NeighborInfoModule
* Fix retransmissions
* Improve ACKs for repeated packets and responses
* Stop retransmission even if there's not relay node
* Revert perhapsRebroadcast()
* Remove relayer if we cancel a transmission
* Better checking for fallback to flooding
* Fix newlines in traceroute print logs
* Stop retransmission for original packet
* Use relayID
* Also when want_ack is set, we should try to retransmit
* Fix cppcheck error
* Fix 'router' not in scope error
* Fix another cppcheck error
* Check for hop_limit and also update next hop when `hop_start == hop_limit` on ACK
Also check for broadcast in `getNextHop()`
* Formatting and correct NUM_RETRANSMISSIONS
* Update protos
* Start retransmissions in NextHopRouter if ReliableRouter didn't do it
* Handle repeated/fallback to flooding packets properly
First check if it's not still in the TxQueue
* Guard against clients setting `next_hop`/`relay_node`
* Don't cancel relay if we were the assigned next-hop
* Replies (e.g. tapback emoji) are also a valid confirmation of receipt
---------
Co-authored-by: GUVWAF <thijs@havinga.eu>
Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com>
Co-authored-by: Tom Fifield <tom@tomfifield.net>
Co-authored-by: GUVWAF <78759985+GUVWAF@users.noreply.github.com>
* fix "native" compiler errors/warnings NodeDB.h
* fancy T-Deck / SenseCAP Indicator / unPhone / PICOmputer-S3 TFT screen (#3259)
* lib update: light theme
* fix merge issue
* lib update: home buttons + button try-fix
* lib update: icon color fix
* lib update: fix instability/crash on notification
* update lib: timezone
* timezone label
* lib update: fix set owner
* fix spiLock in RadioLibInterface
* add picomputer tft build
* picomputer build
* fix compiler error std::find()
* fix merge
* lib update: theme runtime config
* lib update: packet logger + T-Deck Plus
* lib update: mesh detector
* lib update: fix brightness & trackball crash
* try-fix less paranoia
* sensecap indicator updates
* lib update: indicator fix
* lib update: statistic & some fixes
* lib-update: other T-Deck touch driver
* use custom touch driver for Indicator
* lower tft task prio
* prepare LVGL ST7789 driver
* lib update: try-fix audio
* Drop received packets from self
* Additional decoded packet ignores
* Honor flip & color for Heltec T114 and T190 (#4786)
* Honor TFT_MESH color if defined for Heltec T114 or T190
* Temporary: point lib_deps at fork of Heltec's ST7789 library
For demo only, until ST7789 is merged
* Update lib_deps; tidy preprocessor logic
* Download debian files after firmware zip
* set title for protobufs bump PR (#4792)
* set title for version bump PR (#4791)
* Enable Dependabot
* chore: trunk fmt
* fix dependabot syntax (#4795)
* fix dependabot syntax
* Update dependabot.yml
* Update dependabot.yml
* Bump peter-evans/create-pull-request from 6 to 7 in /.github/workflows (#4797)
* Bump docker/build-push-action from 5 to 6 in /.github/workflows (#4800)
* Actions: Semgrep Images have moved from returntocorp to semgrep (#4774)
https://hub.docker.com/r/returntocorp/semgrep notes: "We've moved!
Official Docker images for Semgrep now available at semgrep/semgrep."
Patch updates our CI workflow for these images.
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
* Bump meshtestic from `31ee3d9` to `37245b3` (#4799)
Bumps [meshtestic](https://github.com/meshtastic/meshTestic) from `31ee3d9` to `37245b3`.
- [Commits](https://github.com/meshtastic/meshTestic/compare/31ee3d90c8bef61e835c3271be2c7cda8c4a5cc2...37245b3d612a9272f546bbb092837bafdad46bc2)
---
updated-dependencies:
- dependency-name: meshtestic
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* [create-pull-request] automated change (#4789)
Co-authored-by: thebentern <9000580+thebentern@users.noreply.github.com>
* Bump pnpm/action-setup from 2 to 4 in /.github/workflows (#4798)
Bumps [pnpm/action-setup](https://github.com/pnpm/action-setup) from 2 to 4.
- [Release notes](https://github.com/pnpm/action-setup/releases)
- [Commits](https://github.com/pnpm/action-setup/compare/v2...v4)
---
updated-dependencies:
- dependency-name: pnpm/action-setup
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Raspberry Pico2 - needs protos
* Re-order doDeepSleep (#4802)
Make sure PMU sleep takes place before I2C ends
* [create-pull-request] automated change
* heltec-wireless-bridge
requires Proto PR first
* feat: trigger class update when protobufs are changed
* meshtastic/ is a test suite; protobufs/ contains protobufs;
* Update platform-native to pick up portduino crash fix (#4807)
* Hopefully extract and commit to meshtastic.github.io
* CI fixes
* [Board] DIY "t-energy-s3_e22" (#4782)
* New variant "t-energy-s3_e22"
- Lilygo T-Energy-S3
- NanoVHF "Mesh-v1.06-TTGO-T18" board
- Ebyte E22 Series
* add board_level = extra
* Update variant.h
---------
Co-authored-by: Thomas Göttgens <tgoettgens@gmail.com>
Co-authored-by: Tom Fifield <tom@tomfifield.net>
* Consolidate variant build steps (#4806)
* poc: consolidate variant build steps
* use build-variant action
* only checkout once and clean up after run
* Revert "Consolidate variant build steps (#4806)" (#4816)
This reverts commit
|
||
|
Ben Meadors
|
fdc473e5fa | Trunk | ||
|
Ben Meadors
|
add70b5229 | Rework some things | ||
|
Ben MeadorsandGitHub
|
ec6949fdc0 | Migrate NRF52 devices max nodes down to 80 for now to prevent brownouts (#5346) | ||
|
Ben MeadorsandGitHub
|
3a66c738bd |
Revert "Decrease max nodes for NRF52 to 80 as workaround to prevent FS blowou…" (#5340)
This reverts commit
|
||
|
Ben MeadorsandGitHub
|
ea150c32f3 |
Decrease max nodes for NRF52 to 80 as workaround to prevent FS blowouts (#5338)
* Decrease max nodes for NRF52 to 80 as workaround to prevent FS blowouts * Don't redefine * Newline |
||
|
Jonathan BennettandGitHub
|
d2ea430a3e |
Make SPI frequency and TOPHONE queue size configurable on Native (#4369)
* Make SPI frequency configurable on Native * Make the tophone queue size configurable for Portduino * The modified SPISettings must be configured in setup(), after config.yaml is processed * make MeshService a pointer, so we can configure MAX_RX_TOPHONE at run time * Got a little over excited with refactoring * Silence a warning |
||
|
|
35754d661d |
Make MAX_NUM_NODES configurable in variant.h (#3453)
Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
|
dfcd0d14f6 |
Add MaxNodes to Native config (#3427)
* Add MaxNodes to Native * It compiles... * Convert nodedb to use new * Closer but still broken. * Finally working * Remove unintended lines * Don't include a pointer * Capitalization matters. * avoid rename in protocol regen * When trimming the nodeDB, start with a cleanup * Remove extra cleanupMeshDB() call for now --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |
||
|
Ben Meadors
|
08f1ac785a | Gut NodeDb and remove deprecated protos | ||
|
Thomas Göttgens
|
6fdb93cd16 |
re-add namespacing in protobufs. Let's see what i missed. Portduino likely ...
Checking in generated on purpose. |
||
|
Ben Meadors
|
9046dacec8 | I don't like this formatting but I need trunk to STFU | ||
|
Ben Meadors
|
4d4af7de59 | generated paths experiment | ||
|
Thomas Göttgens
|
f6982ca726 | fix building with new peotobuf structure. | ||
 Kevin Hester
|
ab9fe42f58 | wip - move channels | ||
|
Kevin Hester
|
5ae4edf8fd | move admin out of core proto | ||
|
Kevin Hester
|
42ae27973e | WIP 1.2 move routing into plugin | ||
|
Kevin Hester
|
45caf394f0 | WIP multichannel support | ||
|
Kevin Hester
|
6e31ba30c7 | move generated protobuf c code to own directory | ||
|
Kevin Hester
|
d6912cfd8e | mqtt design progress | ||
|
geeksville
|
f35b15b09c | Shrink devicestate for small mcus | ||
|
geeksville
|
16812c3ee4 | add ignore_incoming to user preferences, for automated testing of DSR topologies | ||
|
geeksville
|
dec4870649 | begin cleaning up mesh library layer so that it could be split someday |