Pr1.5 tmm nexthop (#10745)
* TrafficManagement: flat unified cache + persistent next-hop overflow store Reworks the TrafficManagementModule cache layer (policing behaviour unchanged from upstream) and adds a routing-hint overflow store: - Flatten the ring: replace the cuckoo-hashed unified cache and the bucketed PSRAM NodeInfo index with plain flat arrays + linear scan (same idiom as WarmNodeStore). At LoRa packet rates an O(n) scan of the cache is negligible, and it removes a large amount of hashing/displacement complexity. The cache entry is 11 B; timestamps use a uniform +1 presence-offset so a 0 byte always means "empty" across every sub-store. Adds rebaseEpoch() so cached state survives the ~19 h relative-timestamp horizon instead of being flushed. - Next-hop overflow cache: setNextHop/getNextHopHint store a confirmed last-byte relay for a destination, written only from NextHopRouter's ACK-confirmed decision (and mirrored from TraceRoute). NextHopRouter::getNextHop falls back to this cache when the hot NodeDB has no hint, so DMs/relays to long-tail nodes keep routing after the node ages out of NodeInfoLite. - Persistence: preloadNextHopsFromNodeDB warm-starts the cache from persisted NodeInfoLite hints on first maintenance pass; next_hop entries are kept alive across the maintenance sweep (no TTL) and never clobbered by a stale preload. All packet-policing logic (rate limit, position dedup, unknown-packet drop, NodeInfo direct response, hop exhaustion) is the existing upstream behaviour, untouched. HAS_TRAFFIC_MANAGEMENT defaults on so the module is compiled in. (see note). Tests: upstream policing suite now actually runs (adds the MeshTypes.h include that gates HAS_TRAFFIC_MANAGEMENT) plus 4 next-hop tests. Role-aware throttles, politeness, precision clamp, port-interval and mesh-radius gating — and the rate-limit >255 saturation fix — are deferred to the advanced-TMM branch. Note: default dedup movement grid moves to ~91m, which also means 1.5km required to end up with the same signature position - coarser and therefore further than before. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * TrafficManagement: fix cppcheck constVariablePointer warning `node` in preloadNextHopsFromNodeDB() is never written through — mark it const to satisfy cppcheck's constVariablePointer check in CI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add multi-hop NextHop recovery tests and unit tests for routing reliability - Introduced a new test suite for multi-hop NextHop directed-message delivery and relay recovery in `test_nexthop_multihop_recovery.py`. This includes tests for end-to-end delivery and recovery after relay drop. - Implemented unit tests in `test_main.cpp` for NextHop routing reliability mitigations, covering: - M1: Ambiguity-aware last-byte resolution. - M2: NextHopRouter's strict-neighbor gate and hop limit checks. - M3: Route-health freshness and failure decay. - Enhanced mock classes to facilitate controlled testing of node behaviors and routing logic. * grafting fixed * Address Copilot review for PR #10735 (NextHop improvements) - docs/nexthop-routing-reliability.md: update status from "no code changes yet" to reflect that mitigations and tests are implemented RAM pressure and MIGRATION_VERBOSE concerns addressed upstream in PR2.5 (per-platform TRAFFIC_MANAGEMENT_CACHE_SIZE) and PR2 (verbose default=0) respectively; (0,0) sentinel fixed in PR2.5. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * CI: fix cppcheck constVariablePointer and test include path - NextHopRouter.cpp: qualify two RouteHealth *h locals as const — only read for stale-route checks, never mutated through the pointer - Router.cpp: qualify meshtastic_NodeInfoLite *node as const in shouldDecrementHopLimit — only read for favorite/role predicate - test_position_module/test_main.cpp: change bare PositionModule.h to modules/PositionModule.h — build_flags sets -Isrc, not -Isrc/modules, so the bare form fails to resolve in the native PlatformIO test env Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * WarmStore: cache device role + protected category in last_heard low bits Steal the low 6 bits of WarmNodeEntry.last_heard to carry an evicted node's device role (4 bits) and a protected category (2 bits) for the hop-trim path, at zero record-size cost (entry stays 40 B; no RAM/flash growth). The high bits remain a real unix-seconds timestamp, quantised to 64 s — ample for warm LRU ordering of long-tail nodes. - absorb() packs role/protectedCat; place()/ring replay store the raw word so metadata round-trips through flash. LRU compares masked time (warmTimeOf). - take() rehydration masks the metadata bits and restores the cached role so a re-admitted node isn't stuck at CLIENT until its next NodeInfo. - NodeDB classifies the category (favorite/ignored/verified -> Flag; tracker/sensor/tak_tracker -> Role) at each eviction site. - WarmNodeStore::lookupMeta() exposes role/category to consumers. - Bump WARM_RING_MAGIC (WRNG->WRN2): old rings read as erased and rebuild; warm data is a non-critical evictee cache, so discard-on-upgrade is safe. Tests: test_warm_store 11/11 (new meta round-trip + quantisation-aware ordering); NodeDB compiles (test_nodedb_blocked 4/4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: migrate v1 rings/files by discarding last_heard, not the data Previously the WRNG->WRN2 magic bump treated old rings as erased, discarding all warm entries — including the PKI public keys that let evicted nodes keep decrypting DMs. Instead, read v1 (WRNG / WRM1) records and keep each node's identity + public key, discarding only last_heard (its low bits would otherwise be misread as the new role/protected metadata). Records re-rank and re-learn their role on next contact. - Ring backend (nRF52840): ringReadHeader accepts both magics and reports v1 via an out-param; replay zeroes last_heard for v1 records. If the active head page is v1, force a rotation so new v2 records never land in a v1-headered page (which would discard their freshly-set role on the next load). Legacy pages convert to v2 as the ring rotates. - File backend (warm.dat): bump WARM_STORE_MAGIC WRM1->WRM2; accept WRM1, verify CRC against the stored bytes, then discard last_heard and mark dirty so the next save rewrites as v2. Tests: test_warm_store 12/12 (adds test_ws_v1_migration_discardsLastHeard: key survives, role/protected reset). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: guard role bit-width + test eviction carries role/protected - static_assert that the device role enum still fits the 4-bit warm metadata field (WARM_ROLE_MASK); fails the build loudly if a new role is added past 15 rather than silently truncating role on eviction. (Max role today = 12.) - Add test_migration_carriesRoleAndProtectedIntoWarm: a demoted TRACKER lands in the warm tier with its key, role=TRACKER and protected category=Role; a demoted CLIENT carries role=CLIENT/None. Exercises the NodeDB eviction path + warmProtectedCategory classification (the warm-store unit tests only cover absorb() directly). Tests: test_nodedb_blocked 5/5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix copilot comments * fix(test): restore #if HAS_TRAFFIC_MANAGEMENT guard in TMM test The rebase onto PR1.5 lost the top-level HAS_TRAFFIC_MANAGEMENT guard that PR1.5 introduced, leaving the #else/#endif tail orphaned and causing compile errors on non-TMM builds. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
This commit is contained in:
Tom
co-authored by
GitHub
Claude Opus 4.8
Ben Meadors
GitHub
Claude Opus 4.8
Ben Meadors
parent
ca7d82629d
commit
22072c5f4b
+203
-14
@@ -98,21 +98,38 @@ void NextHopRouter::sniffReceived(const meshtastic_MeshPacket *p, const meshtast
|
||||
// destination
|
||||
if (p->from != 0) {
|
||||
meshtastic_NodeInfoLite *origTx = nodeDB->getMeshNode(p->from);
|
||||
if (origTx) {
|
||||
// Either relayer of ACK was also a relayer of the packet, or we were the *only* relayer and the ACK came
|
||||
// directly from the destination
|
||||
// Single lookup for both relayer checks on the same (request_id, to) pair
|
||||
bool wasAlreadyRelayer = false;
|
||||
bool weWereSoleRelayer = false;
|
||||
bool weWereRelayer = false;
|
||||
checkRelayers(p->relay_node, ourRelayID, p->decoded.request_id, p->to, &wasAlreadyRelayer, &weWereRelayer,
|
||||
&weWereSoleRelayer);
|
||||
if ((weWereRelayer && wasAlreadyRelayer) || (getHopsAway(*p) == 0 && weWereSoleRelayer)) {
|
||||
if (origTx->next_hop != p->relay_node) { // Not already set
|
||||
// Either relayer of ACK was also a relayer of the packet, or we were the *only* relayer and the ACK came
|
||||
// directly from the destination. checkRelayers is read-only on PacketHistory and O(1), so we run it even
|
||||
// when origTx is absent — that lets us still capture the confirmed hop into the TMM overflow cache below.
|
||||
// Single lookup for both relayer checks on the same (request_id, to) pair
|
||||
bool wasAlreadyRelayer = false;
|
||||
bool weWereSoleRelayer = false;
|
||||
bool weWereRelayer = false;
|
||||
checkRelayers(p->relay_node, ourRelayID, p->decoded.request_id, p->to, &wasAlreadyRelayer, &weWereRelayer,
|
||||
&weWereSoleRelayer);
|
||||
if ((weWereRelayer && wasAlreadyRelayer) || (getHopsAway(*p) == 0 && weWereSoleRelayer)) {
|
||||
// M1/M2: only learn a next hop whose last byte maps to a single plausible relay. On a dense
|
||||
// mesh the byte may be ambiguous; storing it would aim future DMs at the wrong node. This gate
|
||||
// now protects BOTH the hot-store route (NodeInfoLite.next_hop) AND the TMM overflow cache —
|
||||
// the overflow cache deliberately holds many more next-hop bytes (long-tail nodes), so it is
|
||||
// even more collision-prone and must never store an ambiguous byte either. Ambiguous/unknown
|
||||
// -> store nothing and keep flooding (safe).
|
||||
if (nodeDB->resolveUniqueLastByte(p->relay_node, /*requireDirectNeighbor=*/false)) {
|
||||
if (origTx && origTx->next_hop != p->relay_node) { // Not already set
|
||||
LOG_INFO("Update next hop of 0x%x to 0x%x based on ACK/reply (was relayer %d we were sole %d)", p->from,
|
||||
p->relay_node, wasAlreadyRelayer, weWereSoleRelayer);
|
||||
origTx->next_hop = p->relay_node;
|
||||
}
|
||||
noteRouteLearned(p->from, p->relay_node, millis()); // M3: anchor freshness (hot or overflow route)
|
||||
#if HAS_TRAFFIC_MANAGEMENT
|
||||
// Mirror the confirmed (and now unique-resolved) hop into the TMM overflow cache so it
|
||||
// survives even when the source isn't (or is no longer) in the hot NodeDB.
|
||||
if (trafficManagementModule)
|
||||
trafficManagementModule->setNextHop(p->from, p->relay_node);
|
||||
#endif
|
||||
} else {
|
||||
LOG_DEBUG("Not learning next hop for 0x%x: relay byte 0x%x ambiguous/unknown; keep flooding", p->from,
|
||||
p->relay_node);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -144,6 +161,11 @@ bool NextHopRouter::perhapsRebroadcast(const meshtastic_MeshPacket *p)
|
||||
if (!isToUs(p) && !isFromUs(p) && (p->hop_limit > 0 || exhaustHops)) {
|
||||
if (p->id != 0) {
|
||||
if (isRebroadcaster()) {
|
||||
// NOTE: this is a self-identity match (is the addressed next_hop OUR last byte?), so it
|
||||
// cannot be hardened with resolveLastByte() — a remote node that legitimately shares our
|
||||
// last byte will also match here and rebroadcast. That residual collision needs a wider
|
||||
// on-wire field to fix. M1/M2 instead shrink the blast radius by reducing how often an
|
||||
// ambiguous next_hop byte is ever learned (sniffReceived) or originated (getNextHop).
|
||||
if (p->next_hop == NO_NEXT_HOP_PREFERENCE || p->next_hop == nodeDB->getLastByteOfNodeNum(getNodeNum())) {
|
||||
meshtastic_MeshPacket *tosend = packetPool.allocCopy(*p); // keep a copy because we will be sending it
|
||||
LOG_INFO("Rebroadcast received message coming from %x", p->relay_node);
|
||||
@@ -194,15 +216,63 @@ std::optional<uint8_t> NextHopRouter::getNextHop(NodeNum to, uint8_t relay_node)
|
||||
if (isBroadcast(to))
|
||||
return std::nullopt;
|
||||
|
||||
// Hot store first: a direct array hit on the live NodeDB entry.
|
||||
meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(to);
|
||||
if (node && node->next_hop) {
|
||||
// M3: proactively decay a stale or repeatedly-failing route back to flooding, so a dead hop
|
||||
// isn't trusted on the next DM's first (and on dense meshes, slowest) attempt. We only act on
|
||||
// a health record that still matches the stored byte; a next_hop set by another path (e.g.
|
||||
// TraceRouteModule) with no matching record is left authoritative.
|
||||
const RouteHealth *h = findRouteHealth(to);
|
||||
if (h && h->lastNextHop == node->next_hop && isRouteStale(*h, millis())) {
|
||||
LOG_INFO("Next hop 0x%x for 0x%x is stale (age/fails); flood and clear", node->next_hop, to);
|
||||
node->next_hop = NO_NEXT_HOP_PREFERENCE; // clear persisted route
|
||||
clearRouteHealth(to); // clear RAM health
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
// We are careful not to return the relay node as the next hop
|
||||
if (node->next_hop != relay_node) {
|
||||
// LOG_DEBUG("Next hop for 0x%x is 0x%x", to, node->next_hop);
|
||||
return node->next_hop;
|
||||
// M1/M2: only emit a stored next_hop if its last byte still maps to a UNIQUE, currently
|
||||
// reachable direct neighbor. On a dense mesh the last byte collides, so an ambiguous byte
|
||||
// would unicast a hint toward the wrong physical node; if the neighbor has gone away we'd
|
||||
// unicast into a void. In both cases flood instead (managed flooding still delivers).
|
||||
ResolvedNode r = nodeDB->resolveLastByte(node->next_hop, /*requireDirectNeighbor=*/true);
|
||||
if (r.status == LastByteResolution::Unique)
|
||||
return node->next_hop;
|
||||
LOG_WARN("Next hop 0x%x for 0x%x %s; set no pref", node->next_hop, to,
|
||||
r.status == LastByteResolution::Ambiguous ? "ambiguous among neighbors" : "not a known neighbor");
|
||||
} else
|
||||
LOG_WARN("Next hop for 0x%x is 0x%x, same as relayer; set no pref", to, node->next_hop);
|
||||
}
|
||||
|
||||
#if HAS_TRAFFIC_MANAGEMENT
|
||||
// Fallback: TMM overflow cache holds confirmed hops for nodes that have aged out of the hot store.
|
||||
// It is the same byte source/confidence as NodeInfoLite.next_hop, so it gets the same M1/M2/M3
|
||||
// protection: decay a stale/failing route, then only emit a byte that still resolves to a unique
|
||||
// reachable neighbor. Without this the overflow cache (which holds MORE bytes for MORE nodes) would
|
||||
// reintroduce exactly the silent-misroute that M1/M2 closes on the hot path.
|
||||
if (trafficManagementModule) {
|
||||
uint8_t hint = trafficManagementModule->getNextHopHint(to);
|
||||
if (hint && hint != relay_node) {
|
||||
const RouteHealth *h = findRouteHealth(to);
|
||||
if (h && h->lastNextHop == hint && isRouteStale(*h, millis())) {
|
||||
LOG_INFO("TMM next hop 0x%x for 0x%x is stale (age/fails); flood and clear", hint, to);
|
||||
trafficManagementModule->clearNextHop(to); // clear overflow route (setNextHop won't store 0)
|
||||
clearRouteHealth(to); // clear RAM health
|
||||
return std::nullopt;
|
||||
}
|
||||
ResolvedNode r = nodeDB->resolveLastByte(hint, /*requireDirectNeighbor=*/true);
|
||||
if (r.status == LastByteResolution::Unique) {
|
||||
LOG_DEBUG("Next hop for 0x%x is 0x%x (TMM cache)", to, hint);
|
||||
return hint;
|
||||
}
|
||||
LOG_WARN("TMM next hop 0x%x for 0x%x %s; set no pref", hint, to,
|
||||
r.status == LastByteResolution::Ambiguous ? "ambiguous among neighbors" : "not a known neighbor");
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
@@ -311,7 +381,10 @@ int32_t NextHopRouter::doRetransmissions()
|
||||
|
||||
if (!isBroadcast(p.packet->to)) {
|
||||
if (p.numRetransmissions == 1) {
|
||||
// Last retransmission, reset next_hop (fallback to FloodingRouter)
|
||||
// Last retransmission: this directed delivery went un-ACKed. Record the failure
|
||||
// (M3 — accumulates across DMs to age out a flapping/dead route) and reset
|
||||
// next_hop so the final try falls back to FloodingRouter.
|
||||
noteRouteFailure(p.packet->to);
|
||||
p.packet->next_hop = NO_NEXT_HOP_PREFERENCE;
|
||||
// Also reset it in the nodeDB
|
||||
meshtastic_NodeInfoLite *sentTo = nodeDB->getMeshNode(p.packet->to);
|
||||
@@ -319,9 +392,32 @@ int32_t NextHopRouter::doRetransmissions()
|
||||
LOG_INFO("Resetting next hop for packet with dest 0x%x\n", p.packet->to);
|
||||
sentTo->next_hop = NO_NEXT_HOP_PREFERENCE;
|
||||
}
|
||||
#if HAS_TRAFFIC_MANAGEMENT
|
||||
if (trafficManagementModule) {
|
||||
trafficManagementModule->clearNextHop(p.packet->to);
|
||||
}
|
||||
#endif
|
||||
FloodingRouter::send(packetPool.allocCopy(*p.packet));
|
||||
} else {
|
||||
#if NEXTHOP_EARLY_FLOOD_ON_UNVERIFIED
|
||||
// M4 (gated): if the route isn't proven healthy, don't spend a second directed
|
||||
// attempt — start flooding one retry sooner to cut recovery latency. A verified
|
||||
// route (fresh, zero recent failures) keeps the unchanged directed-retry path so
|
||||
// the sparse-mesh happy path is untouched.
|
||||
RouteHealth *h = findRouteHealth(p.packet->to);
|
||||
bool verified = h && h->consecutiveFailures == 0 && !isRouteStale(*h, now);
|
||||
if (!verified) {
|
||||
p.packet->next_hop = NO_NEXT_HOP_PREFERENCE;
|
||||
meshtastic_NodeInfoLite *sentTo = nodeDB->getMeshNode(p.packet->to);
|
||||
if (sentTo)
|
||||
sentTo->next_hop = NO_NEXT_HOP_PREFERENCE;
|
||||
FloodingRouter::send(packetPool.allocCopy(*p.packet));
|
||||
} else {
|
||||
NextHopRouter::send(packetPool.allocCopy(*p.packet));
|
||||
}
|
||||
#else
|
||||
NextHopRouter::send(packetPool.allocCopy(*p.packet));
|
||||
#endif
|
||||
}
|
||||
} else {
|
||||
// Note: we call the superclass version because we don't want to have our version of send() add a new
|
||||
@@ -355,3 +451,96 @@ void NextHopRouter::setNextTx(PendingPacket *pending)
|
||||
printPacket("", pending->packet);
|
||||
setReceivedMessage(); // Run ASAP, so we can figure out our correct sleep time
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// M3: RAM route-health table. Bounded array with reuse-oldest eviction (same discipline as
|
||||
// PacketHistory). All age comparisons use unsigned subtraction so they survive the 49.7-day millis()
|
||||
// rollover. dest == 0 marks an empty slot; learnedAtMsec is normalized to 1 on write so an occupied
|
||||
// slot is never read as infinitely old.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
RouteHealth *NextHopRouter::findRouteHealth(NodeNum dest)
|
||||
{
|
||||
if (dest == 0)
|
||||
return nullptr;
|
||||
for (auto &h : routeHealth)
|
||||
if (h.dest == dest)
|
||||
return &h;
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
RouteHealth *NextHopRouter::getOrAllocRouteHealth(NodeNum dest, uint32_t now)
|
||||
{
|
||||
if (dest == 0)
|
||||
return nullptr;
|
||||
|
||||
RouteHealth *oldest = &routeHealth[0];
|
||||
RouteHealth *freeSlot = nullptr;
|
||||
for (auto &h : routeHealth) {
|
||||
if (h.dest == dest)
|
||||
return &h; // existing record
|
||||
if (h.dest == 0) {
|
||||
if (!freeSlot)
|
||||
freeSlot = &h; // remember the first free slot; prefer it over evicting
|
||||
continue;
|
||||
}
|
||||
// Track the oldest occupied slot in case the table is full (rollover-safe).
|
||||
if ((uint32_t)(now - h.learnedAtMsec) > (uint32_t)(now - oldest->learnedAtMsec))
|
||||
oldest = &h;
|
||||
}
|
||||
// Claim the free slot if there is one, else reuse the oldest. Reset before use and stamp the dest
|
||||
// so the record is findable.
|
||||
RouteHealth *slot = freeSlot ? freeSlot : oldest;
|
||||
*slot = RouteHealth{};
|
||||
slot->dest = dest;
|
||||
return slot;
|
||||
}
|
||||
|
||||
void NextHopRouter::noteRouteLearned(NodeNum dest, uint8_t nextHop, uint32_t now)
|
||||
{
|
||||
if (dest == 0 || nextHop == NO_NEXT_HOP_PREFERENCE)
|
||||
return;
|
||||
RouteHealth *h = getOrAllocRouteHealth(dest, now);
|
||||
if (!h)
|
||||
return;
|
||||
// A genuinely new next hop earns a clean slate; re-learning the SAME hop keeps the accumulated
|
||||
// failure count so an asymmetric reverse path that keeps re-teaching a dead forward hop still ages
|
||||
// out instead of resetting the counter every time.
|
||||
if (h->lastNextHop != nextHop) {
|
||||
h->lastNextHop = nextHop;
|
||||
h->consecutiveFailures = 0;
|
||||
}
|
||||
h->learnedAtMsec = now ? now : 1;
|
||||
}
|
||||
|
||||
void NextHopRouter::noteRouteSuccess(NodeNum dest, uint32_t now)
|
||||
{
|
||||
RouteHealth *h = findRouteHealth(dest);
|
||||
if (!h)
|
||||
return; // only routes we actually learned have health to refresh
|
||||
h->consecutiveFailures = 0;
|
||||
h->learnedAtMsec = now ? now : 1;
|
||||
}
|
||||
|
||||
void NextHopRouter::noteRouteFailure(NodeNum dest)
|
||||
{
|
||||
RouteHealth *h = findRouteHealth(dest);
|
||||
if (!h)
|
||||
return; // nothing to penalize (we were flooding, or never learned a route here)
|
||||
if (h->consecutiveFailures < 255)
|
||||
h->consecutiveFailures++;
|
||||
}
|
||||
|
||||
bool NextHopRouter::isRouteStale(const RouteHealth &h, uint32_t now) const
|
||||
{
|
||||
if (h.consecutiveFailures >= ROUTE_FAILURE_THRESHOLD)
|
||||
return true;
|
||||
return (uint32_t)(now - h.learnedAtMsec) >= ROUTE_TTL_MSEC;
|
||||
}
|
||||
|
||||
void NextHopRouter::clearRouteHealth(NodeNum dest)
|
||||
{
|
||||
RouteHealth *h = findRouteHealth(dest);
|
||||
if (h)
|
||||
*h = RouteHealth{};
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user