kevin/meshtastic_firmware
1
0
Fork 0
Code Issues Pull Requests Actions 10 Packages Projects Releases Wiki Activity

Pr1.5 tmm nexthop (#10745)

Browse Source 
* TrafficManagement: flat unified cache + persistent next-hop overflow store

Reworks the TrafficManagementModule cache layer (policing behaviour unchanged
from upstream) and adds a routing-hint overflow store:

- Flatten the ring: replace the cuckoo-hashed unified cache and the bucketed
  PSRAM NodeInfo index with plain flat arrays + linear scan (same idiom as
  WarmNodeStore). At LoRa packet rates an O(n) scan of the cache is negligible,
  and it removes a large amount of hashing/displacement complexity. The cache
  entry is 11 B; timestamps use a uniform +1 presence-offset so a 0 byte always
  means "empty" across every sub-store. Adds rebaseEpoch() so cached state
  survives the ~19 h relative-timestamp horizon instead of being flushed.

- Next-hop overflow cache: setNextHop/getNextHopHint store a confirmed last-byte
  relay for a destination, written only from NextHopRouter's ACK-confirmed
  decision (and mirrored from TraceRoute). NextHopRouter::getNextHop falls back
  to this cache when the hot NodeDB has no hint, so DMs/relays to long-tail
  nodes keep routing after the node ages out of NodeInfoLite.

- Persistence: preloadNextHopsFromNodeDB warm-starts the cache from persisted
  NodeInfoLite hints on first maintenance pass; next_hop entries are kept alive
  across the maintenance sweep (no TTL) and never clobbered by a stale preload.

All packet-policing logic (rate limit, position dedup, unknown-packet drop,
NodeInfo direct response, hop exhaustion) is the existing upstream behaviour,
untouched. HAS_TRAFFIC_MANAGEMENT defaults on so the module is compiled in. (see note).

Tests: upstream policing suite now actually runs (adds the MeshTypes.h include
that gates HAS_TRAFFIC_MANAGEMENT) plus 4 next-hop tests. Role-aware throttles,
politeness, precision clamp, port-interval and mesh-radius gating — and the
rate-limit >255 saturation fix — are deferred to the advanced-TMM branch.

Note: default dedup movement grid moves to ~91m, which also means 1.5km required to end up with the same signature position - coarser and therefore further than before.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* TrafficManagement: fix cppcheck constVariablePointer warning

`node` in preloadNextHopsFromNodeDB() is never written through — mark
it const to satisfy cppcheck's constVariablePointer check in CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Add multi-hop NextHop recovery tests and unit tests for routing reliability

- Introduced a new test suite for multi-hop NextHop directed-message delivery and relay recovery in `test_nexthop_multihop_recovery.py`. This includes tests for end-to-end delivery and recovery after relay drop.
- Implemented unit tests in `test_main.cpp` for NextHop routing reliability mitigations, covering:
  - M1: Ambiguity-aware last-byte resolution.
  - M2: NextHopRouter's strict-neighbor gate and hop limit checks.
  - M3: Route-health freshness and failure decay.
- Enhanced mock classes to facilitate controlled testing of node behaviors and routing logic.

* grafting fixed

* Address Copilot review for PR #10735 (NextHop improvements)

- docs/nexthop-routing-reliability.md: update status from "no code
  changes yet" to reflect that mitigations and tests are implemented

RAM pressure and MIGRATION_VERBOSE concerns addressed upstream in
PR2.5 (per-platform TRAFFIC_MANAGEMENT_CACHE_SIZE) and PR2 (verbose
default=0) respectively; (0,0) sentinel fixed in PR2.5.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* CI: fix cppcheck constVariablePointer and test include path

- NextHopRouter.cpp: qualify two RouteHealth *h locals as const — only
  read for stale-route checks, never mutated through the pointer
- Router.cpp: qualify meshtastic_NodeInfoLite *node as const in
  shouldDecrementHopLimit — only read for favorite/role predicate
- test_position_module/test_main.cpp: change bare PositionModule.h to
  modules/PositionModule.h — build_flags sets -Isrc, not -Isrc/modules,
  so the bare form fails to resolve in the native PlatformIO test env

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* WarmStore: cache device role + protected category in last_heard low bits

Steal the low 6 bits of WarmNodeEntry.last_heard to carry an evicted node's
device role (4 bits) and a protected category (2 bits) for the hop-trim path,
at zero record-size cost (entry stays 40 B; no RAM/flash growth). The high bits
remain a real unix-seconds timestamp, quantised to 64 s — ample for warm LRU
ordering of long-tail nodes.

- absorb() packs role/protectedCat; place()/ring replay store the raw word so
  metadata round-trips through flash. LRU compares masked time (warmTimeOf).
- take() rehydration masks the metadata bits and restores the cached role so a
  re-admitted node isn't stuck at CLIENT until its next NodeInfo.
- NodeDB classifies the category (favorite/ignored/verified -> Flag;
  tracker/sensor/tak_tracker -> Role) at each eviction site.
- WarmNodeStore::lookupMeta() exposes role/category to consumers.
- Bump WARM_RING_MAGIC (WRNG->WRN2): old rings read as erased and rebuild;
  warm data is a non-critical evictee cache, so discard-on-upgrade is safe.

Tests: test_warm_store 11/11 (new meta round-trip + quantisation-aware ordering);
NodeDB compiles (test_nodedb_blocked 4/4).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* WarmStore: migrate v1 rings/files by discarding last_heard, not the data

Previously the WRNG->WRN2 magic bump treated old rings as erased, discarding all
warm entries — including the PKI public keys that let evicted nodes keep
decrypting DMs. Instead, read v1 (WRNG / WRM1) records and keep each node's
identity + public key, discarding only last_heard (its low bits would otherwise
be misread as the new role/protected metadata). Records re-rank and re-learn
their role on next contact.

- Ring backend (nRF52840): ringReadHeader accepts both magics and reports v1 via
  an out-param; replay zeroes last_heard for v1 records. If the active head page
  is v1, force a rotation so new v2 records never land in a v1-headered page
  (which would discard their freshly-set role on the next load). Legacy pages
  convert to v2 as the ring rotates.
- File backend (warm.dat): bump WARM_STORE_MAGIC WRM1->WRM2; accept WRM1, verify
  CRC against the stored bytes, then discard last_heard and mark dirty so the
  next save rewrites as v2.

Tests: test_warm_store 12/12 (adds test_ws_v1_migration_discardsLastHeard:
key survives, role/protected reset).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* WarmStore: guard role bit-width + test eviction carries role/protected

- static_assert that the device role enum still fits the 4-bit warm metadata
  field (WARM_ROLE_MASK); fails the build loudly if a new role is added past 15
  rather than silently truncating role on eviction. (Max role today = 12.)
- Add test_migration_carriesRoleAndProtectedIntoWarm: a demoted TRACKER lands in
  the warm tier with its key, role=TRACKER and protected category=Role; a demoted
  CLIENT carries role=CLIENT/None. Exercises the NodeDB eviction path +
  warmProtectedCategory classification (the warm-store unit tests only cover
  absorb() directly).

Tests: test_nodedb_blocked 5/5.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix copilot comments

* fix(test): restore #if HAS_TRAFFIC_MANAGEMENT guard in TMM test

The rebase onto PR1.5 lost the top-level HAS_TRAFFIC_MANAGEMENT guard
that PR1.5 introduced, leaving the #else/#endif tail orphaned and
causing compile errors on non-TMM builds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
This commit is contained in:
Tom
2026-06-19 19:52:58 -05:00
committed by GitHub
co-authored by GitHub Claude Opus 4.8 Ben Meadors
parent ca7d82629d
commit 22072c5f4b
21 changed files with 2529 additions and 737 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/test_nodedb_blocked/test_main.cpp
+35 -1
View File
@@ -26,17 +26,22 @@ class NodeDBTestShim : public NodeDB
void runDemote() { demoteOldestHotNodesToWarm(); }
void runCleanup() { cleanupMeshDB(); }
// Read back the role + protected category the warm tier cached for a node.
bool warmMeta(NodeNum n, uint8_t &role, uint8_t &prot) { return warmStore.lookupMeta(n, role, prot); }
void clearHot()
{
meshNodes->clear();
numMeshNodes = 0;
}
void push(NodeNum num, uint32_t lastHeard, bool favorite, bool ignored, bool withUser, bool withKey)
void push(NodeNum num, uint32_t lastHeard, bool favorite, bool ignored, bool withUser, bool withKey,
meshtastic_Config_DeviceConfig_Role role = meshtastic_Config_DeviceConfig_Role_CLIENT)
{
meshtastic_NodeInfoLite n = meshtastic_NodeInfoLite_init_zero;
n.num = num;
n.last_heard = lastHeard;
n.role = role;
if (favorite)
nodeInfoLiteSetBit(&n, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true);
if (ignored)
@@ -99,6 +104,34 @@ static void test_migration_demotesOldestKeepsKeepersAndSelf(void)
TEST_ASSERT_TRUE(warmHasKey(2000 + 3)); // ...but its key kept in the warm tier
}
// Eviction carries the device role + protected category into the warm tier. A TRACKER is
// hop-protected but NOT eviction-protected, so it gets demoted with its key; the warm
// record must report role=TRACKER / category=Role. A plain CLIENT carries role=CLIENT/None.
static void test_migration_carriesRoleAndProtectedIntoWarm(void)
{
db->seedSelf();
const int extra = MAX_NUM_NODES + 30; // overflow so the oldest non-protected are demoted
for (int i = 1; i <= extra; i++) {
const auto role = (i == 3) ? meshtastic_Config_DeviceConfig_Role_TRACKER : meshtastic_Config_DeviceConfig_Role_CLIENT;
db->push(2000 + i, /*last_heard=*/i, /*favorite=*/false, /*ignored=*/false, /*withUser=*/true,
/*withKey=*/true, role);
}
db->runDemote();
uint8_t role = 0xFF, prot = 0xFF;
// TRACKER (i=3): demoted out of hot, key kept, role + protected carried into warm.
TEST_ASSERT_NULL(db->getMeshNode(2000 + 3));
TEST_ASSERT_TRUE(warmHasKey(2000 + 3));
TEST_ASSERT_TRUE(db->warmMeta(2000 + 3, role, prot));
TEST_ASSERT_EQUAL(meshtastic_Config_DeviceConfig_Role_TRACKER, role);
TEST_ASSERT_EQUAL((uint8_t)WarmProtected::Role, prot);
// CLIENT (i=4): also demoted, carries role=CLIENT / category=None.
TEST_ASSERT_TRUE(db->warmMeta(2000 + 4, role, prot));
TEST_ASSERT_EQUAL(meshtastic_Config_DeviceConfig_Role_CLIENT, role);
TEST_ASSERT_EQUAL((uint8_t)WarmProtected::None, prot);
}
// Favourite handling: a favourite is never the eviction victim, even when it is
// the oldest node in a full hot store.
static void test_eviction_preservesFavorite(void)
@@ -172,6 +205,7 @@ NDB_TEST_ENTRY void setup()
UNITY_BEGIN();
RUN_TEST(test_migration_demotesOldestKeepsKeepersAndSelf);
RUN_TEST(test_migration_carriesRoleAndProtectedIntoWarm);
RUN_TEST(test_eviction_preservesFavorite);
RUN_TEST(test_ignored_survivesEvictionAndCleanup);
RUN_TEST(test_protectedCap_refusesBeyondLimit);