Files
meshtastic_firmware/src/mesh/PacketHistory.h
T
Ben MeadorsandGitHub 12b2d973a4 Add central memory-class ladder (MemClass.h) with fail-safe-small defaults (#10901)
* Right-size nRF52 heap tiers after 2.8.0 heap-exhaustion field reports

Field reports on 2.8.0 show nRF52840 devices at 99% heap (114/115 KB)
within minutes of boot; operator new asserts on OOM, so these devices
are one allocation from a reboot. The 2.8.0 cache sizing ladders gave
nRF52 the largest non-PSRAM tiers on the assumption that a BLE-only
part has a roomy heap - the arena is actually ~125 KB shared with the
FreeRTOS task stacks.

Per-target retiers (nRF52840 unless noted):
- Traffic Management cache 1000 -> 250 entries (10 KB -> 2.5 KB); the
  unclassified fallthrough drops 1000 -> 400 to match the classic-ESP32
  tier (also affects RP2040/RP2350)
- Warm node store 200 -> 100 entries (8 KB -> 4 KB); the non-XXAA
  fallthrough drops 320 -> 100 so an unclassified RAM-constrained part
  can't boot-allocate 12.8 KB
- MESSAGE_HISTORY_LIMIT 20 -> 10 (text pool 4.4 KB -> 2.2 KB), the tier
  classic ESP32 already ships
- MAX_RX_TOPHONE 32 -> 16, shrinking the static packet pool 70 -> 54
  slots (~6.6 KB of .bss returned to the heap arena)
- PacketHistory hash index off arch-wide (1 KB); O(n) over 240 records
  is negligible at LoRa packet rates
- OLEDDISPLAY_REDUCE_MEMORY arch-wide (~1 KB OLED back buffer); the five
  TFT variants -U it because TFTDisplay.cpp needs buffer_back for
  dirty-window diffing
- Drop the stale "for testing" 1024-entry TMM override on T1000-E

Measured on rak4631: heap arena grows 124,572 -> 131,180 B and boot
allocations drop ~15.7 KB, roughly +22 KB free heap on the field-report
device class.

Migration: the nRF52840 warm flash ring replays through place() (LRU),
so the newest 100 identities survive the shrink; the file backend
rejects oversized snapshots cleanly (new test covers this). Native
suites pass (536/536 Docker, 13/13 native-macos warm store); rak4631,
heltec-mesh-node-t114 (TFT) and tracker-t1000-e build green.

* Add central memory-class ladder (MemClass.h) with fail-safe-small defaults

The 2.8.0 nRF52840 heap exhaustion happened because each RAM-sized cache
picked its per-platform tier from its own chip #ifdef ladder, and every
ladder's fallthrough default was its largest non-PSRAM tier - nRF52 was
never named, so it silently got 1000-entry caches on a ~115 KB arena.

This introduces src/memory/MemClass.h: a single MESHTASTIC_MEM_CLASS
(TINY / SMALL / MEDIUM / LARGE) ranked by usable app heap after platform
overheads, with the deliberate property that an unclassified chip lands
in SMALL - a new target boots with small caches until someone opts it up
in one visible place.

The TMM cache, warm store, MAX_RX_TOPHONE and MAX_SATELLITE_NODES
ladders in mesh-pb-constants.h now key off the class; branches pinned by
something other than RAM stay explicit and say why (nRF52840's SoftDevice
arena, RP2040's warm.dat watchdog bound). MAX_NUM_NODES intentionally
stays separate - it is flash-shaped (nodes.proto vs LittleFS), not
heap-shaped.

A per-class MESHTASTIC_BOOT_CACHE_BUDGET static_assert now covers the
three big boot-allocated caches, so the next cache-adding PR that would
blow a small platform's budget fails to compile instead of exhausting
heap in the field.

No values change for any existing target: rak4631, tbeam, rak11310 and
wio-e5 build byte-identical before/after; all ladders remain
#ifndef-guarded so variant overrides keep working.

* Address review: fix RP2350 class-table doc, add PacketRecord static_assert

- MemClass.h's class table claimed RP2350 was MEDIUM while the mapping
  ladder classifies it SMALL (with RP2040) - the table now matches the
  ladder, with a note that RP2350 is a MEDIUM candidate whenever someone
  wants to tune it up (kept SMALL here so this header stays a behavioral
  no-op).
- The boot-cache budget comment referenced a static_assert pinning
  PacketHistory::PacketRecord at 20 B that did not exist (only a layout
  comment). Add the real static_assert so the budget math in
  mesh-pb-constants.h fails to compile if the record layout changes.

Also merges develop (the base #10898 landed there as a squash, which is
what made this stacked branch conflict); develop's mesh-pb-constants.h
is byte-identical to this branch's base, so the resolution keeps the
MemClass ladder unchanged. rak4631 and wio-e5 build green;
test_packet_history 47/47.

* Address review: share PACKETHISTORY_MAX, trim policy comments

- Hoist PACKETHISTORY_MAX from PacketHistory.cpp into
  mesh-pb-constants.h (next to the MAX_NUM_NODES it derives from) so the
  constructor clamp and the boot-cache budget static_assert use one
  definition instead of hand-mirrored arithmetic that could drift. The
  expression stays valid where MAX_NUM_NODES resolves at runtime
  (ESP32-S3, portduino); the pointless 2.0 double math becomes integer.
- Trim the MemClass.h header (36 -> 16 comment lines) and the budget /
  sizing-policy comments per the repo comment-length guideline, keeping
  the class table, the fail-safe-small rule, the override mechanism, and
  the include-order constraint.

rak4631 (compile-time MAX_NUM_NODES) and heltec-v3 (runtime) build
green; test_packet_history 47/47.
2026-07-06 14:51:41 -05:00

105 lines
5.5 KiB
C++

#pragma once
#include "NodeDB.h"
#include <memory>
// Number of relayers we keep track of. Use 6 to be efficient with memory alignment of PacketRecord to 20 bytes
#define NUM_RELAYERS 6
#define HOP_LIMIT_HIGHEST_MASK 0x07 // Bits 0-2
#define HOP_LIMIT_OUR_TX_MASK 0x38 // Bits 3-5
#define HOP_LIMIT_OUR_TX_SHIFT 3 // Bits 3-5
/**
* This is a mixin that adds a record of past packets we have seen
*/
class PacketHistory
{
private:
struct PacketRecord { // A record of a recent message broadcast, no need to be visible outside this class.
NodeNum sender;
PacketId id;
uint32_t rxTimeMsec; // Unix time in msecs - the time we received it, 0 means empty
uint8_t next_hop; // The next hop asked for this packet
uint8_t hop_limit; // bit 0-2: Highest hop limit observed for this packet,
// bit 3-5: our hop limit when we first transmitted it
uint8_t relayed_by[NUM_RELAYERS]; // Array of nodes that relayed this packet
}; // 4B + 4B + 4B + 1B + 1B + 6B = 20B
static_assert(sizeof(PacketRecord) == 20,
"PacketRecord size feeds the boot-cache budget math in mesh-pb-constants.h - update both together");
uint32_t recentPacketsCapacity =
0; // Can be set in constructor, no need to recompile. Used to allocate memory for mx_recentPackets.
std::unique_ptr<PacketRecord[]> recentPackets; // Simple and fixed in size. Debloat.
#if !MESHTASTIC_EXCLUDE_PKT_HISTORY_HASH
// Open-addressing hash table for O(1) lookup in find(), replacing the O(N) linear scan.
// Maps (sender, id) -> index into recentPackets[]. Uses linear probing with a load factor <= 0.5.
// The load factor invariant holds permanently: hashCapacity = 2 * nextPowerOf2(recentPacketsCapacity),
// and at most recentPacketsCapacity entries can ever be live (one per recentPackets[] slot).
static constexpr uint16_t HASH_EMPTY = 0xFFFF;
std::unique_ptr<uint16_t[]> hashIndex;
uint32_t hashCapacity = 0; // Always a power of 2
uint32_t hashMask = 0; // hashCapacity - 1, for fast modular indexing
uint32_t hashSlot(NodeNum sender, PacketId id) const;
void hashInsert(NodeNum sender, PacketId id, uint16_t slotIdx);
void hashRemove(NodeNum sender, PacketId id);
void hashRebuild();
#endif
/** Find a packet record in history.
* @param sender NodeNum
* @param id PacketId
* @return pointer to PacketRecord if found, NULL if not found */
PacketRecord *find(NodeNum sender, PacketId id);
/** Insert/Replace oldest PacketRecord in mx_recentPackets.
* @param r PacketRecord to insert or replace */
void insert(const PacketRecord &r); // Insert or replace a packet record in the history
/* Check if a certain node was a relayer of a packet in the history given iterator
* If wasSole is not nullptr, it will be set to true if the relayer was the only relayer of that packet
* @return true if node was indeed a relayer, false if not */
bool wasRelayer(const uint8_t relayer, const PacketRecord &r, bool *wasSole = nullptr);
uint8_t getHighestHopLimit(const PacketRecord &r);
void setHighestHopLimit(PacketRecord &r, uint8_t hopLimit);
uint8_t getOurTxHopLimit(const PacketRecord &r);
void setOurTxHopLimit(PacketRecord &r, uint8_t hopLimit);
public:
explicit PacketHistory(uint32_t size = -1); // Constructor with size parameter, default is PACKETHISTORY_MAX
/**
* Update recentBroadcasts and return true if we have already seen this packet
*
* @param withUpdate if true and not found we add an entry to recentPackets
* @param wasFallback if not nullptr, packet will be checked for fallback to flooding and value will be set to true if so
* @param weWereNextHop if not nullptr, packet will be checked for us being the next hop and value will be set to true if so
* @param wasUpgraded if not nullptr, will be set to true if this packet has better hop_limit than previously seen
*/
bool wasSeenRecently(const meshtastic_MeshPacket *p, bool withUpdate = true, bool *wasFallback = nullptr,
bool *weWereNextHop = nullptr, bool *wasUpgraded = nullptr);
/* Check if a certain node was a relayer of a packet in the history given an ID and sender
* If wasSole is not nullptr, it will be set to true if the relayer was the only relayer of that packet
* @return true if node was indeed a relayer, false if not */
bool wasRelayer(const uint8_t relayer, const uint32_t id, const NodeNum sender, bool *wasSole = nullptr);
/**
* Check two relayers against the same packet record with a single lookup.
* Avoids redundant find() calls when checking multiple relayers for the same (id, sender) pair.
* @param r1Result set to true if relayer1 was a relayer
* @param r2Result set to true if relayer2 was a relayer
* @param r2WasSole if not nullptr, set to true if relayer2 was the sole relayer
*/
void checkRelayers(uint8_t relayer1, uint8_t relayer2, uint32_t id, NodeNum sender, bool *r1Result, bool *r2Result,
bool *r2WasSole = nullptr);
// Remove a relayer from the list of relayers of a packet in the history given an ID and sender
void removeRelayer(const uint8_t relayer, const uint32_t id, const NodeNum sender);
// To check if the PacketHistory was initialized correctly by constructor
bool initOk(void) { return recentPackets != nullptr && recentPacketsCapacity != 0; }
};