* TrafficManagement: flat unified cache + persistent next-hop overflow store Reworks the TrafficManagementModule cache layer (policing behaviour unchanged from upstream) and adds a routing-hint overflow store: - Flatten the ring: replace the cuckoo-hashed unified cache and the bucketed PSRAM NodeInfo index with plain flat arrays + linear scan (same idiom as WarmNodeStore). At LoRa packet rates an O(n) scan of the cache is negligible, and it removes a large amount of hashing/displacement complexity. The cache entry is 11 B; timestamps use a uniform +1 presence-offset so a 0 byte always means "empty" across every sub-store. Adds rebaseEpoch() so cached state survives the ~19 h relative-timestamp horizon instead of being flushed. - Next-hop overflow cache: setNextHop/getNextHopHint store a confirmed last-byte relay for a destination, written only from NextHopRouter's ACK-confirmed decision (and mirrored from TraceRoute). NextHopRouter::getNextHop falls back to this cache when the hot NodeDB has no hint, so DMs/relays to long-tail nodes keep routing after the node ages out of NodeInfoLite. - Persistence: preloadNextHopsFromNodeDB warm-starts the cache from persisted NodeInfoLite hints on first maintenance pass; next_hop entries are kept alive across the maintenance sweep (no TTL) and never clobbered by a stale preload. All packet-policing logic (rate limit, position dedup, unknown-packet drop, NodeInfo direct response, hop exhaustion) is the existing upstream behaviour, untouched. HAS_TRAFFIC_MANAGEMENT defaults on so the module is compiled in. (see note). Tests: upstream policing suite now actually runs (adds the MeshTypes.h include that gates HAS_TRAFFIC_MANAGEMENT) plus 4 next-hop tests. Role-aware throttles, politeness, precision clamp, port-interval and mesh-radius gating — and the rate-limit >255 saturation fix — are deferred to the advanced-TMM branch. Note: default dedup movement grid moves to ~91m, which also means 1.5km required to end up with the same signature position - coarser and therefore further than before. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * TrafficManagement: fix cppcheck constVariablePointer warning `node` in preloadNextHopsFromNodeDB() is never written through — mark it const to satisfy cppcheck's constVariablePointer check in CI. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add multi-hop NextHop recovery tests and unit tests for routing reliability - Introduced a new test suite for multi-hop NextHop directed-message delivery and relay recovery in `test_nexthop_multihop_recovery.py`. This includes tests for end-to-end delivery and recovery after relay drop. - Implemented unit tests in `test_main.cpp` for NextHop routing reliability mitigations, covering: - M1: Ambiguity-aware last-byte resolution. - M2: NextHopRouter's strict-neighbor gate and hop limit checks. - M3: Route-health freshness and failure decay. - Enhanced mock classes to facilitate controlled testing of node behaviors and routing logic. * grafting fixed * Address Copilot review for PR #10735 (NextHop improvements) - docs/nexthop-routing-reliability.md: update status from "no code changes yet" to reflect that mitigations and tests are implemented RAM pressure and MIGRATION_VERBOSE concerns addressed upstream in PR2.5 (per-platform TRAFFIC_MANAGEMENT_CACHE_SIZE) and PR2 (verbose default=0) respectively; (0,0) sentinel fixed in PR2.5. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * CI: fix cppcheck constVariablePointer and test include path - NextHopRouter.cpp: qualify two RouteHealth *h locals as const — only read for stale-route checks, never mutated through the pointer - Router.cpp: qualify meshtastic_NodeInfoLite *node as const in shouldDecrementHopLimit — only read for favorite/role predicate - test_position_module/test_main.cpp: change bare PositionModule.h to modules/PositionModule.h — build_flags sets -Isrc, not -Isrc/modules, so the bare form fails to resolve in the native PlatformIO test env Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * WarmStore: cache device role + protected category in last_heard low bits Steal the low 6 bits of WarmNodeEntry.last_heard to carry an evicted node's device role (4 bits) and a protected category (2 bits) for the hop-trim path, at zero record-size cost (entry stays 40 B; no RAM/flash growth). The high bits remain a real unix-seconds timestamp, quantised to 64 s — ample for warm LRU ordering of long-tail nodes. - absorb() packs role/protectedCat; place()/ring replay store the raw word so metadata round-trips through flash. LRU compares masked time (warmTimeOf). - take() rehydration masks the metadata bits and restores the cached role so a re-admitted node isn't stuck at CLIENT until its next NodeInfo. - NodeDB classifies the category (favorite/ignored/verified -> Flag; tracker/sensor/tak_tracker -> Role) at each eviction site. - WarmNodeStore::lookupMeta() exposes role/category to consumers. - Bump WARM_RING_MAGIC (WRNG->WRN2): old rings read as erased and rebuild; warm data is a non-critical evictee cache, so discard-on-upgrade is safe. Tests: test_warm_store 11/11 (new meta round-trip + quantisation-aware ordering); NodeDB compiles (test_nodedb_blocked 4/4). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: migrate v1 rings/files by discarding last_heard, not the data Previously the WRNG->WRN2 magic bump treated old rings as erased, discarding all warm entries — including the PKI public keys that let evicted nodes keep decrypting DMs. Instead, read v1 (WRNG / WRM1) records and keep each node's identity + public key, discarding only last_heard (its low bits would otherwise be misread as the new role/protected metadata). Records re-rank and re-learn their role on next contact. - Ring backend (nRF52840): ringReadHeader accepts both magics and reports v1 via an out-param; replay zeroes last_heard for v1 records. If the active head page is v1, force a rotation so new v2 records never land in a v1-headered page (which would discard their freshly-set role on the next load). Legacy pages convert to v2 as the ring rotates. - File backend (warm.dat): bump WARM_STORE_MAGIC WRM1->WRM2; accept WRM1, verify CRC against the stored bytes, then discard last_heard and mark dirty so the next save rewrites as v2. Tests: test_warm_store 12/12 (adds test_ws_v1_migration_discardsLastHeard: key survives, role/protected reset). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WarmStore: guard role bit-width + test eviction carries role/protected - static_assert that the device role enum still fits the 4-bit warm metadata field (WARM_ROLE_MASK); fails the build loudly if a new role is added past 15 rather than silently truncating role on eviction. (Max role today = 12.) - Add test_migration_carriesRoleAndProtectedIntoWarm: a demoted TRACKER lands in the warm tier with its key, role=TRACKER and protected category=Role; a demoted CLIENT carries role=CLIENT/None. Exercises the NodeDB eviction path + warmProtectedCategory classification (the warm-store unit tests only cover absorb() directly). Tests: test_nodedb_blocked 5/5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix copilot comments * fix(test): restore #if HAS_TRAFFIC_MANAGEMENT guard in TMM test The rebase onto PR1.5 lost the top-level HAS_TRAFFIC_MANAGEMENT guard that PR1.5 introduced, leaving the #else/#endif tail orphaned and causing compile errors on non-TMM builds. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
212 lines
8.6 KiB
C++
212 lines
8.6 KiB
C++
#pragma once
|
||
|
||
#include "FloodingRouter.h"
|
||
#include <optional>
|
||
#include <unordered_map>
|
||
|
||
/**
|
||
* An identifier for a globally unique message - a pair of the sending nodenum and the packet id assigned
|
||
* to that message
|
||
*/
|
||
struct GlobalPacketId {
|
||
NodeNum node;
|
||
PacketId id;
|
||
|
||
bool operator==(const GlobalPacketId &p) const { return node == p.node && id == p.id; }
|
||
|
||
explicit GlobalPacketId(const meshtastic_MeshPacket *p)
|
||
{
|
||
node = getFrom(p);
|
||
id = p->id;
|
||
}
|
||
|
||
GlobalPacketId(NodeNum _from, PacketId _id)
|
||
{
|
||
node = _from;
|
||
id = _id;
|
||
}
|
||
};
|
||
|
||
/**
|
||
* A packet queued for retransmission
|
||
*/
|
||
struct PendingPacket {
|
||
meshtastic_MeshPacket *packet;
|
||
|
||
/** The next time we should try to retransmit this packet */
|
||
uint32_t nextTxMsec = 0;
|
||
|
||
/** Starts at NUM_RETRANSMISSIONS -1 and counts down. Once zero it will be removed from the list */
|
||
uint8_t numRetransmissions = 0;
|
||
|
||
PendingPacket() {}
|
||
explicit PendingPacket(meshtastic_MeshPacket *p, uint8_t numRetransmissions);
|
||
};
|
||
|
||
/**
|
||
* RAM-only per-destination route health. Tracks how fresh a learned next_hop is and how many
|
||
* consecutive directed deliveries to it have failed, so getNextHop() can proactively decay a stale or
|
||
* repeatedly-failing route back to flooding instead of trusting a dead hop on the next (and on dense
|
||
* meshes, slowest) attempt. Not persisted: the learned next_hop itself lives in NodeInfoLite; this is
|
||
* just freshness/failure metadata.
|
||
*/
|
||
struct RouteHealth {
|
||
NodeNum dest = 0; ///< destination this record describes; 0 == empty slot
|
||
uint32_t learnedAtMsec = 0; ///< millis() when next_hop was last (re)learned (rollover-aware)
|
||
uint8_t consecutiveFailures = 0; ///< directed deliveries to `dest` that went un-ACKed
|
||
uint8_t lastNextHop = NO_NEXT_HOP_PREFERENCE; ///< the relay byte this health refers to
|
||
};
|
||
|
||
// M4 (optional, off by default): when a route is not proven healthy, fall back to flooding one retry
|
||
// earlier instead of spending a second directed attempt. Trades airtime for recovery latency on dense
|
||
// meshes; leaves the sparse-mesh happy path (fresh, verified routes) unchanged. Measure on the
|
||
// simulator before enabling broadly.
|
||
#ifndef NEXTHOP_EARLY_FLOOD_ON_UNVERIFIED
|
||
#define NEXTHOP_EARLY_FLOOD_ON_UNVERIFIED 0
|
||
#endif
|
||
|
||
class GlobalPacketIdHashFunction
|
||
{
|
||
public:
|
||
size_t operator()(const GlobalPacketId &p) const { return (std::hash<NodeNum>()(p.node)) ^ (std::hash<PacketId>()(p.id)); }
|
||
};
|
||
|
||
/*
|
||
Router for direct messages, which only relays if it is the next hop for a packet. The next hop is set by the current
|
||
relayer of a packet, which bases this on information from a previous successful delivery to the destination via flooding.
|
||
Namely, in the PacketHistory, we keep track of (up to 3) relayers of a packet. When the ACK is delivered back to us via a node
|
||
that also relayed the original packet, we use that node as next hop for the destination from then on. This makes sure that only
|
||
when there’s a two-way connection, we assign a next hop. Both the ReliableRouter and NextHopRouter will do retransmissions (the
|
||
NextHopRouter only 1 time). For the final retry, if no one actually relayed the packet, it will reset the next hop in order to
|
||
fall back to the FloodingRouter again. Note that thus also intermediate hops will do a single retransmission if the intended
|
||
next-hop didn’t relay, in order to fix changes in the middle of the route.
|
||
*/
|
||
class NextHopRouter : public FloodingRouter
|
||
{
|
||
public:
|
||
/**
|
||
* Constructor
|
||
*
|
||
*/
|
||
NextHopRouter();
|
||
|
||
/**
|
||
* Send a packet
|
||
* @return an error code
|
||
*/
|
||
virtual ErrorCode send(meshtastic_MeshPacket *p) override;
|
||
|
||
/** Do our retransmission handling */
|
||
virtual int32_t runOnce() override
|
||
{
|
||
// Note: We must doRetransmissions FIRST, because it might queue up work for the base class runOnce implementation
|
||
doRetransmissions();
|
||
|
||
int32_t r = FloodingRouter::runOnce();
|
||
|
||
// Also after calling runOnce there might be new packets to retransmit
|
||
auto d = doRetransmissions();
|
||
return min(d, r);
|
||
}
|
||
|
||
// The number of retransmissions intermediate nodes will do (actually 1 less than this)
|
||
constexpr static uint8_t NUM_INTERMEDIATE_RETX = 2;
|
||
// The number of retransmissions the original sender will do
|
||
constexpr static uint8_t NUM_RELIABLE_RETX = 3;
|
||
|
||
// M3: bounded RAM route-health table (reuse-oldest eviction, like PacketHistory)
|
||
constexpr static uint8_t ROUTE_HEALTH_MAX = 32; // ~12B/slot -> ~384B
|
||
constexpr static uint32_t ROUTE_TTL_MSEC = 30UL * 60 * 1000; // re-discover a route unconfirmed for 30 min
|
||
constexpr static uint8_t ROUTE_FAILURE_THRESHOLD = 3; // consecutive un-ACKed directed deliveries -> dead
|
||
|
||
protected:
|
||
/**
|
||
* Pending retransmissions
|
||
*/
|
||
std::unordered_map<GlobalPacketId, PendingPacket, GlobalPacketIdHashFunction> pending;
|
||
|
||
/**
|
||
* Per-destination route health (M3). Bounded array, reuse-oldest eviction. RAM-only.
|
||
*/
|
||
RouteHealth routeHealth[ROUTE_HEALTH_MAX] = {};
|
||
|
||
/**
|
||
* Should this incoming filter be dropped?
|
||
*
|
||
* Called immediately on reception, before any further processing.
|
||
* @return true to abandon the packet
|
||
*/
|
||
virtual bool shouldFilterReceived(const meshtastic_MeshPacket *p) override;
|
||
|
||
/**
|
||
* Look for packets we need to relay
|
||
*/
|
||
virtual void sniffReceived(const meshtastic_MeshPacket *p, const meshtastic_Routing *c) override;
|
||
|
||
/**
|
||
* Try to find the pending packet record for this ID (or NULL if not found)
|
||
*/
|
||
PendingPacket *findPendingPacket(NodeNum from, PacketId id) { return findPendingPacket(GlobalPacketId(from, id)); }
|
||
PendingPacket *findPendingPacket(GlobalPacketId p);
|
||
|
||
/**
|
||
* Add p to the list of packets to retransmit occasionally. We will free it once we stop retransmitting.
|
||
*/
|
||
PendingPacket *startRetransmission(meshtastic_MeshPacket *p, uint8_t numReTx = NUM_INTERMEDIATE_RETX);
|
||
|
||
// Return true if we're allowed to cancel a packet in the txQueue (so we may never transmit it even once)
|
||
bool roleAllowsCancelingFromTxQueue(const meshtastic_MeshPacket *p);
|
||
|
||
/**
|
||
* Stop any retransmissions we are doing of the specified node/packet ID pair
|
||
*
|
||
* @return true if we found and removed a transmission with this ID
|
||
*/
|
||
bool stopRetransmission(NodeNum from, PacketId id);
|
||
bool stopRetransmission(GlobalPacketId p);
|
||
|
||
/**
|
||
* Do any retransmissions that are scheduled (FIXME - for the time being called from loop)
|
||
*
|
||
* @return the number of msecs until our next retransmission or MAXINT if none scheduled
|
||
*/
|
||
int32_t doRetransmissions();
|
||
|
||
void setNextTx(PendingPacket *pending);
|
||
|
||
// --- M3 route-health helpers (RAM-only). Protected so ReliableRouter (a subclass) can record
|
||
// delivery success, and so the unit-test shim can reach them via `using`. All take `now` where
|
||
// time matters so the decay logic is pure and testable without a clock mock. ---
|
||
|
||
/// @return the health record for `dest`, or nullptr if we hold none.
|
||
RouteHealth *findRouteHealth(NodeNum dest);
|
||
/// @return an existing record for `dest`, else a freshly claimed slot (reuse-oldest on overflow).
|
||
RouteHealth *getOrAllocRouteHealth(NodeNum dest, uint32_t now);
|
||
/// Record that we (re)learned `nextHop` for `dest`. Resets the failure count only when the hop
|
||
/// changed (so a flapping reverse-path re-learn of the same dead hop still ages out).
|
||
void noteRouteLearned(NodeNum dest, uint8_t nextHop, uint32_t now);
|
||
/// Record an end-to-end delivery success to `dest` (clears failures, refreshes freshness).
|
||
void noteRouteSuccess(NodeNum dest, uint32_t now);
|
||
/// Record that a directed delivery to `dest` went un-ACKed (no-op if we hold no record).
|
||
void noteRouteFailure(NodeNum dest);
|
||
/// @return true if the route is too old (TTL) or has failed too many times in a row.
|
||
bool isRouteStale(const RouteHealth &h, uint32_t now) const;
|
||
/// Forget any health record for `dest`.
|
||
void clearRouteHealth(NodeNum dest);
|
||
|
||
#ifdef PIO_UNIT_TESTING
|
||
public: // expose getNextHop to the test shim without widening production visibility
|
||
#else
|
||
private:
|
||
#endif
|
||
/**
|
||
* Get the next hop for a destination, given the relay node
|
||
* @return the node number of the next hop, 0 if no preference (fallback to FloodingRouter)
|
||
*/
|
||
std::optional<uint8_t> getNextHop(NodeNum to, uint8_t relay_node);
|
||
|
||
private:
|
||
/** Check if we should be rebroadcasting this packet if so, do so.
|
||
* @return true if we did rebroadcast */
|
||
bool perhapsRebroadcast(const meshtastic_MeshPacket *p) override;
|
||
}; |