From 12b2d973a4c872fd39d1b3321a75ae44d23a333d Mon Sep 17 00:00:00 2001 From: Ben Meadors Date: Mon, 6 Jul 2026 14:51:41 -0500 Subject: [PATCH] Add central memory-class ladder (MemClass.h) with fail-safe-small defaults (#10901) * Right-size nRF52 heap tiers after 2.8.0 heap-exhaustion field reports Field reports on 2.8.0 show nRF52840 devices at 99% heap (114/115 KB) within minutes of boot; operator new asserts on OOM, so these devices are one allocation from a reboot. The 2.8.0 cache sizing ladders gave nRF52 the largest non-PSRAM tiers on the assumption that a BLE-only part has a roomy heap - the arena is actually ~125 KB shared with the FreeRTOS task stacks. Per-target retiers (nRF52840 unless noted): - Traffic Management cache 1000 -> 250 entries (10 KB -> 2.5 KB); the unclassified fallthrough drops 1000 -> 400 to match the classic-ESP32 tier (also affects RP2040/RP2350) - Warm node store 200 -> 100 entries (8 KB -> 4 KB); the non-XXAA fallthrough drops 320 -> 100 so an unclassified RAM-constrained part can't boot-allocate 12.8 KB - MESSAGE_HISTORY_LIMIT 20 -> 10 (text pool 4.4 KB -> 2.2 KB), the tier classic ESP32 already ships - MAX_RX_TOPHONE 32 -> 16, shrinking the static packet pool 70 -> 54 slots (~6.6 KB of .bss returned to the heap arena) - PacketHistory hash index off arch-wide (1 KB); O(n) over 240 records is negligible at LoRa packet rates - OLEDDISPLAY_REDUCE_MEMORY arch-wide (~1 KB OLED back buffer); the five TFT variants -U it because TFTDisplay.cpp needs buffer_back for dirty-window diffing - Drop the stale "for testing" 1024-entry TMM override on T1000-E Measured on rak4631: heap arena grows 124,572 -> 131,180 B and boot allocations drop ~15.7 KB, roughly +22 KB free heap on the field-report device class. Migration: the nRF52840 warm flash ring replays through place() (LRU), so the newest 100 identities survive the shrink; the file backend rejects oversized snapshots cleanly (new test covers this). Native suites pass (536/536 Docker, 13/13 native-macos warm store); rak4631, heltec-mesh-node-t114 (TFT) and tracker-t1000-e build green. * Add central memory-class ladder (MemClass.h) with fail-safe-small defaults The 2.8.0 nRF52840 heap exhaustion happened because each RAM-sized cache picked its per-platform tier from its own chip #ifdef ladder, and every ladder's fallthrough default was its largest non-PSRAM tier - nRF52 was never named, so it silently got 1000-entry caches on a ~115 KB arena. This introduces src/memory/MemClass.h: a single MESHTASTIC_MEM_CLASS (TINY / SMALL / MEDIUM / LARGE) ranked by usable app heap after platform overheads, with the deliberate property that an unclassified chip lands in SMALL - a new target boots with small caches until someone opts it up in one visible place. The TMM cache, warm store, MAX_RX_TOPHONE and MAX_SATELLITE_NODES ladders in mesh-pb-constants.h now key off the class; branches pinned by something other than RAM stay explicit and say why (nRF52840's SoftDevice arena, RP2040's warm.dat watchdog bound). MAX_NUM_NODES intentionally stays separate - it is flash-shaped (nodes.proto vs LittleFS), not heap-shaped. A per-class MESHTASTIC_BOOT_CACHE_BUDGET static_assert now covers the three big boot-allocated caches, so the next cache-adding PR that would blow a small platform's budget fails to compile instead of exhausting heap in the field. No values change for any existing target: rak4631, tbeam, rak11310 and wio-e5 build byte-identical before/after; all ladders remain #ifndef-guarded so variant overrides keep working. * Address review: fix RP2350 class-table doc, add PacketRecord static_assert - MemClass.h's class table claimed RP2350 was MEDIUM while the mapping ladder classifies it SMALL (with RP2040) - the table now matches the ladder, with a note that RP2350 is a MEDIUM candidate whenever someone wants to tune it up (kept SMALL here so this header stays a behavioral no-op). - The boot-cache budget comment referenced a static_assert pinning PacketHistory::PacketRecord at 20 B that did not exist (only a layout comment). Add the real static_assert so the budget math in mesh-pb-constants.h fails to compile if the record layout changes. Also merges develop (the base #10898 landed there as a squash, which is what made this stacked branch conflict); develop's mesh-pb-constants.h is byte-identical to this branch's base, so the resolution keeps the MemClass ladder unchanged. rak4631 and wio-e5 build green; test_packet_history 47/47. * Address review: share PACKETHISTORY_MAX, trim policy comments - Hoist PACKETHISTORY_MAX from PacketHistory.cpp into mesh-pb-constants.h (next to the MAX_NUM_NODES it derives from) so the constructor clamp and the boot-cache budget static_assert use one definition instead of hand-mirrored arithmetic that could drift. The expression stays valid where MAX_NUM_NODES resolves at runtime (ESP32-S3, portduino); the pointless 2.0 double math becomes integer. - Trim the MemClass.h header (36 -> 16 comment lines) and the budget / sizing-policy comments per the repo comment-length guideline, keeping the class table, the fail-safe-small rule, the override mechanism, and the include-order constraint. rak4631 (compile-time MAX_NUM_NODES) and heltec-v3 (runtime) build green; test_packet_history 47/47. --- src/memory/MemClass.h | 53 +++++++++++ src/mesh/PacketHistory.cpp | 4 - src/mesh/PacketHistory.h | 2 + src/mesh/mesh-pb-constants.h | 90 ++++++++++++------- .../heltec_mesh_node_t114/platformio.ini | 1 + .../nrf52840/heltec_mesh_solar/platformio.ini | 1 + 6 files changed, 116 insertions(+), 35 deletions(-) create mode 100644 src/memory/MemClass.h diff --git a/src/memory/MemClass.h b/src/memory/MemClass.h new file mode 100644 index 000000000..83e08757f --- /dev/null +++ b/src/memory/MemClass.h @@ -0,0 +1,53 @@ +#pragma once + +// Central memory-class ladder: RAM-sized caches key their per-platform tier off +// MESHTASTIC_MEM_CLASS. Classes rank *usable app heap after platform overheads* +// (SoftDevice, WiFi+BLE stacks) - not raw RAM - and an unclassified chip lands in +// SMALL on purpose: small caches are a recoverable default, an exhausted heap is not. +// +// MEM_CLASS_TINY <32 KB free heap STM32WL +// MEM_CLASS_SMALL ~100-250 KB nRF52840, classic ESP32/S2/C3, RP2040/RP2350 +// MEM_CLASS_MEDIUM ~250-500 KB, no PSRAM ESP32-S3/C6/P4 without PSRAM +// MEM_CLASS_LARGE PSRAM or host ESP32-S3+PSRAM, portduino +// +// Compare ordinally (>=). RP2350 rides with RP2040 so this header stays a behavioral +// no-op (MEDIUM candidate later). Variants may predefine MESHTASTIC_MEM_CLASS or any +// cache constant - consumer ladders stay #ifndef-guarded. MAX_NUM_NODES is deliberately +// unclassed (flash-shaped: nodes.proto vs filesystem). Included before configuration.h, +// so only toolchain/board -D macros and the ARCH_* macros the ladders already use are +// safe here. + +#define MEM_CLASS_TINY 1 +#define MEM_CLASS_SMALL 2 +#define MEM_CLASS_MEDIUM 3 +#define MEM_CLASS_LARGE 4 + +#ifndef MESHTASTIC_MEM_CLASS +#if defined(ARCH_STM32WL) +#define MESHTASTIC_MEM_CLASS MEM_CLASS_TINY +#elif (defined(CONFIG_IDF_TARGET_ESP32S3) && defined(BOARD_HAS_PSRAM)) || defined(ARCH_PORTDUINO) +#define MESHTASTIC_MEM_CLASS MEM_CLASS_LARGE +#elif defined(CONFIG_IDF_TARGET_ESP32S3) || defined(CONFIG_IDF_TARGET_ESP32C6) || defined(CONFIG_IDF_TARGET_ESP32P4) +#define MESHTASTIC_MEM_CLASS MEM_CLASS_MEDIUM +#else +// Classic ESP32 / S2 / C3, all nRF52, RP2040/RP2350, and - by design - any chip +// nobody has classified yet: small caches are a recoverable default, an +// exhausted heap is not. New RAM-rich targets opt up by adding a branch above. +#define MESHTASTIC_MEM_CLASS MEM_CLASS_SMALL +#endif +#endif // MESHTASTIC_MEM_CLASS + +// Ceiling for the boot-allocated mesh caches (TMM + warm store + packet history); +// mesh-pb-constants.h static_asserts their sum, so an oversized cache fails the build. +// Raising a budget is allowed - as a visible, reviewable one-line diff. +#ifndef MESHTASTIC_BOOT_CACHE_BUDGET +#if MESHTASTIC_MEM_CLASS <= MEM_CLASS_TINY +#define MESHTASTIC_BOOT_CACHE_BUDGET (8 * 1024) +#elif MESHTASTIC_MEM_CLASS == MEM_CLASS_SMALL +#define MESHTASTIC_BOOT_CACHE_BUDGET (16 * 1024) +#elif MESHTASTIC_MEM_CLASS == MEM_CLASS_MEDIUM +#define MESHTASTIC_BOOT_CACHE_BUDGET (32 * 1024) +#else +#define MESHTASTIC_BOOT_CACHE_BUDGET (256 * 1024) +#endif +#endif // MESHTASTIC_BOOT_CACHE_BUDGET diff --git a/src/mesh/PacketHistory.cpp b/src/mesh/PacketHistory.cpp index 9627d2425..2522c30ca 100644 --- a/src/mesh/PacketHistory.cpp +++ b/src/mesh/PacketHistory.cpp @@ -9,10 +9,6 @@ #endif #include "Throttle.h" -#define PACKETHISTORY_MAX \ - max((uint32_t)(MAX_NUM_NODES * 2.0), \ - (uint32_t)100) // x2..3 Should suffice. Empirical setup. 16B per record malloc'ed, but no less than 100 - #define RECENT_WARN_AGE (10 * 60 * 1000L) // Warn if the packet that gets removed was more recent than 10 min #define VERBOSE_PACKET_HISTORY 0 // Set to 1 for verbose logging, 2 for heavy debugging diff --git a/src/mesh/PacketHistory.h b/src/mesh/PacketHistory.h index e45df8474..84af59874 100644 --- a/src/mesh/PacketHistory.h +++ b/src/mesh/PacketHistory.h @@ -24,6 +24,8 @@ class PacketHistory // bit 3-5: our hop limit when we first transmitted it uint8_t relayed_by[NUM_RELAYERS]; // Array of nodes that relayed this packet }; // 4B + 4B + 4B + 1B + 1B + 6B = 20B + static_assert(sizeof(PacketRecord) == 20, + "PacketRecord size feeds the boot-cache budget math in mesh-pb-constants.h - update both together"); uint32_t recentPacketsCapacity = 0; // Can be set in constructor, no need to recompile. Used to allocate memory for mx_recentPackets. diff --git a/src/mesh/mesh-pb-constants.h b/src/mesh/mesh-pb-constants.h index 040bdf732..9a84e83a9 100644 --- a/src/mesh/mesh-pb-constants.h +++ b/src/mesh/mesh-pb-constants.h @@ -1,12 +1,16 @@ #pragma once #include +#include "memory/MemClass.h" #include "mesh/generated/meshtastic/admin.pb.h" #include "mesh/generated/meshtastic/deviceonly.pb.h" #include "mesh/generated/meshtastic/localonly.pb.h" #include "mesh/generated/meshtastic/mesh.pb.h" // this file defines constants which come from mesh.options +// +// RAM-shaped cache tiers key off MESHTASTIC_MEM_CLASS (memory/MemClass.h) so +// unclassified chips fail safe-small; class-deviant branches say why inline. // Tricky macro to let you find the sizeof a type member #define member_size(type, member) sizeof(((type *)0)->member) @@ -23,8 +27,13 @@ // the 8 classic ESP32 has shipped with for years; drops start when a stalled phone/serial client has // 16 packets queued. #define MAX_RX_TOPHONE 16 -#else +#elif MESHTASTIC_MEM_CLASS >= MEM_CLASS_MEDIUM || defined(ARCH_RP2040) || defined(CONFIG_IDF_TARGET_ESP32C3) || \ + defined(ARCH_STM32WL) +// RP2040/RP2350, ESP32-C3 and STM32WL keep their historical 32 (no field pressure to cut them; +// STM32WL's pool is dynamic, so the constant only bounds in-flight packets there). #define MAX_RX_TOPHONE 32 +#else +#define MAX_RX_TOPHONE 16 // unclassified small parts: fail safe-small #endif #endif @@ -109,13 +118,20 @@ static inline int get_max_num_nodes() #endif // platform #endif // MAX_NUM_NODES +/// Packet-history capacity: 2x the hot store so dedup/relayer state survives a +/// full mesh, floored at 100. Shared by PacketHistory's constructor clamp and +/// the boot-cache budget assert below so the two cannot drift. +#ifndef PACKETHISTORY_MAX +#define PACKETHISTORY_MAX (MAX_NUM_NODES * 2 > 100 ? (uint32_t)(MAX_NUM_NODES * 2) : (uint32_t)100) +#endif + /// Per-map cap (position/telemetry/environment/status): only the freshest /// MAX_SATELLITE_NODES nodes keep satellite payloads, the rest just the /// NodeInfoLite header. RAM-bound (the maps are internal-SRAM, not PSRAM), so /// flash-rich hosts get a cap >= their hot store (satellites for every node, as /// before the cap existed) while constrained parts stay at 40. #ifndef MAX_SATELLITE_NODES -#if (defined(CONFIG_IDF_TARGET_ESP32S3) && defined(BOARD_HAS_PSRAM)) || defined(ARCH_PORTDUINO) +#if MESHTASTIC_MEM_CLASS >= MEM_CLASS_LARGE #define MAX_SATELLITE_NODES 250 #else #define MAX_SATELLITE_NODES 40 // nRF52840, generic ESP32, and ESP32-S3 without PSRAM @@ -126,7 +142,7 @@ static inline int get_max_num_nodes() /// so DMs to/from them keep decrypting. 0 disables it; size is per-platform /// below, persisted to /prefs/warm.dat (or the nRF52840 raw-flash ring). #ifndef WARM_NODE_COUNT -#if defined(ARCH_STM32WL) +#if MESHTASTIC_MEM_CLASS <= MEM_CLASS_TINY #define WARM_NODE_COUNT 0 #elif defined(NRF52840_XXAA) // Keyed on the NRF52840_XXAA build flag, not ARCH_NRF52: the latter (from @@ -136,30 +152,28 @@ static inline int get_max_num_nodes() // arena, which 2.8.0 field reports showed at 99% use; 120 hot + 100 warm // identities still covers meshes well past the hot cap. #define WARM_NODE_COUNT 100 -#elif (defined(CONFIG_IDF_TARGET_ESP32S3) && defined(BOARD_HAS_PSRAM)) || defined(ARCH_PORTDUINO) -#define WARM_NODE_COUNT 2000 // PSRAM-equipped ESP32-S3 / native host; warm cache in PSRAM (~80 KB) -#elif defined(CONFIG_IDF_TARGET_ESP32S3) || defined(CONFIG_IDF_TARGET_ESP32C6) || defined(CONFIG_IDF_TARGET_ESP32P4) -#define WARM_NODE_COUNT 150 // 512 KB+ SRAM, no PSRAM (S3/C6/P4): ~6 KB heap (#10705) -#elif defined(ARCH_ESP32) -#define WARM_NODE_COUNT 100 // classic ESP32 (520 KB) / S2 (320 KB) / C3 (400 KB): tightest free heap w/ BLE+WiFi, ~4 KB (#10705) #elif defined(ARCH_RP2040) -#define WARM_NODE_COUNT 150 // RP2040 (264 KB) / RP2350 (520 KB): bounded so warm.dat write fits the 8s watchdog (#10746) +// Class-deviant on purpose: bounded so the warm.dat write fits the 8s watchdog (#10746), +// not by RAM (RP2040 264 KB / RP2350 520 KB could hold more). +#define WARM_NODE_COUNT 150 +#elif MESHTASTIC_MEM_CLASS >= MEM_CLASS_LARGE +#define WARM_NODE_COUNT 2000 // PSRAM-equipped ESP32-S3 / native host; warm cache in PSRAM (~80 KB) +#elif MESHTASTIC_MEM_CLASS == MEM_CLASS_MEDIUM +#define WARM_NODE_COUNT 150 // 512 KB+ SRAM, no PSRAM (S3/C6/P4): ~6 KB heap (#10705) #else -// nRF52840 is handled explicitly above (raw-flash ring). Any other nRF52 (non-XXAA) and any future -// non-ESP32/non-RP LittleFS part lands here. Reviewed after the 2.8.0 nRF52840 heap-exhaustion -// reports: fail small (match nRF52840) so an unclassified RAM-constrained part can't boot-allocate -// 12.8 KB; RAM-rich targets should opt up explicitly in their variant. -#define WARM_NODE_COUNT 100 // other LittleFS-backed parts (e.g. non-nRF52840 nRF52) -#endif // platform -#endif // WARM_NODE_COUNT +// MEM_CLASS_SMALL: classic ESP32/S2/C3 (~4 KB heap, #10705) and anything unclassified - +// fail small so a new RAM-constrained part can't boot-allocate 12.8 KB (2.8.0 lesson). +#define WARM_NODE_COUNT 100 +#endif // platform +#endif // WARM_NODE_COUNT /// Max number of channels allowed #define MAX_NUM_CHANNELS (member_size(meshtastic_ChannelFile, channels) / member_size(meshtastic_ChannelFile, channels[0])) // Traffic Management module configuration -// Enabled by default; STM32WL is excluded due to RAM constraints (MAX_NUM_NODES=10). +// Enabled by default; TINY parts (STM32WL) are excluded due to RAM constraints (MAX_NUM_NODES=10). // Disable per-variant by defining HAS_TRAFFIC_MANAGEMENT=0 in variant.h -#ifdef ARCH_STM32WL +#if MESHTASTIC_MEM_CLASS <= MEM_CLASS_TINY #define HAS_TRAFFIC_MANAGEMENT 0 #endif #ifndef HAS_TRAFFIC_MANAGEMENT @@ -181,25 +195,39 @@ static inline int get_max_num_nodes() #ifndef TRAFFIC_MANAGEMENT_CACHE_SIZE #if !HAS_TRAFFIC_MANAGEMENT #define TRAFFIC_MANAGEMENT_CACHE_SIZE 0 -#elif (defined(CONFIG_IDF_TARGET_ESP32S3) && defined(BOARD_HAS_PSRAM)) || defined(ARCH_PORTDUINO) -#define TRAFFIC_MANAGEMENT_CACHE_SIZE 2048 // PSRAM-equipped ESP32-S3 / native host -#elif defined(CONFIG_IDF_TARGET_ESP32S3) || defined(CONFIG_IDF_TARGET_ESP32C6) || defined(CONFIG_IDF_TARGET_ESP32P4) -#define TRAFFIC_MANAGEMENT_CACHE_SIZE 500 // 512 KB+ SRAM, no PSRAM (S3/C6/P4): ~5 KB heap (#10705) -#elif defined(ARCH_ESP32) -#define TRAFFIC_MANAGEMENT_CACHE_SIZE 400 // classic ESP32 / S2 / C3: tightest free heap, ~4 KB (#10705) #elif defined(NRF52840_XXAA) -// Keyed on NRF52840_XXAA (not ARCH_NRF52) for the same include-order reason as WARM_NODE_COUNT above. -// The 256 KB part shares its ~115 KB heap arena with SoftDevice and the FreeRTOS task stacks; 2.8.0 -// field reports hit 99% heap use with the old 1000-entry (~10 KB) cache. 250 entries (2.5 KB) still -// tracks >2x the 120-node hot store; LRU victim recycling absorbs busier meshes. +// Class-deviant on purpose (SMALL would be 400): the 256 KB part shares its ~115 KB heap arena with +// SoftDevice and the FreeRTOS task stacks; 2.8.0 field reports hit 99% heap use with the old +// 1000-entry (~10 KB) cache. 250 entries (2.5 KB) still tracks >2x the 120-node hot store; LRU +// victim recycling absorbs busier meshes. #define TRAFFIC_MANAGEMENT_CACHE_SIZE 250 +#elif MESHTASTIC_MEM_CLASS >= MEM_CLASS_LARGE +#define TRAFFIC_MANAGEMENT_CACHE_SIZE 2048 // PSRAM-equipped ESP32-S3 / native host +#elif MESHTASTIC_MEM_CLASS == MEM_CLASS_MEDIUM +#define TRAFFIC_MANAGEMENT_CACHE_SIZE 500 // 512 KB+ SRAM, no PSRAM (S3/C6/P4): ~5 KB heap (#10705) #else -// RP2040/RP2350 and anything unclassified land here: match the classic-ESP32 tier rather than -// defaulting large. RAM-rich targets should opt up explicitly in their variant. +// MEM_CLASS_SMALL: classic ESP32/S2/C3 (~4 KB heap, #10705), RP2040/RP2350, and anything +// unclassified - fail small rather than defaulting large (2.8.0 lesson). #define TRAFFIC_MANAGEMENT_CACHE_SIZE 400 #endif #endif // TRAFFIC_MANAGEMENT_CACHE_SIZE +// Enforce the per-class boot-cache budget (memory/MemClass.h) over the three big +// boot-allocated mesh caches. Entry sizes are pinned by static_asserts at their +// definitions: TMM UnifiedCacheEntry = 10 B (TrafficManagementModule.h), +// WarmNodeEntry = 40 B (WarmNodeStore.h), PacketHistory::PacketRecord = 20 B +// (PacketHistory.h; its optional hash index is excluded here). Skipped where +// MAX_NUM_NODES is not a compile-time constant: ESP32-S3 (runtime, from flash +// size) and portduino (runtime, from portduino_config.MaxNodes via variant.h). +// If this fires on a new feature: shrink the cache for this class, or raise the +// class budget in MemClass.h as a visible, reviewable decision. +#if !defined(CONFIG_IDF_TARGET_ESP32S3) && !defined(ARCH_PORTDUINO) +static_assert((uint32_t)TRAFFIC_MANAGEMENT_CACHE_SIZE * 10u + (uint32_t)WARM_NODE_COUNT * 40u + 20u * PACKETHISTORY_MAX <= + MESHTASTIC_BOOT_CACHE_BUDGET, + "Boot-allocated mesh caches exceed this memory class's budget - shrink a tier or consciously raise " + "MESHTASTIC_BOOT_CACHE_BUDGET in memory/MemClass.h"); +#endif + /// helper function for encoding a record as a protobuf, any failures to encode are fatal and we will panic /// returns the encoded packet size size_t pb_encode_to_bytes(uint8_t *destbuf, size_t destbufsize, const pb_msgdesc_t *fields, const void *src_struct); diff --git a/variants/nrf52840/heltec_mesh_node_t114/platformio.ini b/variants/nrf52840/heltec_mesh_node_t114/platformio.ini index c9f998240..0f8727453 100644 --- a/variants/nrf52840/heltec_mesh_node_t114/platformio.ini +++ b/variants/nrf52840/heltec_mesh_node_t114/platformio.ini @@ -16,6 +16,7 @@ debug_tool = jlink # add -DCFG_SYSVIEW if you want to use the Segger systemview tool for OS profiling. build_flags = ${nrf52840_base.build_flags} + -UOLEDDISPLAY_REDUCE_MEMORY ; TFTDisplay.cpp needs the lib's buffer_back for dirty-window diffing -Ivariants/nrf52840/heltec_mesh_node_t114 -DHELTEC_T114 diff --git a/variants/nrf52840/heltec_mesh_solar/platformio.ini b/variants/nrf52840/heltec_mesh_solar/platformio.ini index e4d3c5058..536045cfa 100644 --- a/variants/nrf52840/heltec_mesh_solar/platformio.ini +++ b/variants/nrf52840/heltec_mesh_solar/platformio.ini @@ -105,6 +105,7 @@ build_flags = ${heltec_mesh_solar_base.build_flags} [env:heltec-mesh-solar-tft] extends = heltec_mesh_solar_base build_flags = ${heltec_mesh_solar_base.build_flags} + -UOLEDDISPLAY_REDUCE_MEMORY ; TFTDisplay.cpp needs the lib's buffer_back for dirty-window diffing -DHELTEC_MESH_SOLAR_TFT -DSPI_INTERFACES_COUNT=2 -DHAS_SPI_TFT=1