Add central memory-class ladder (MemClass.h) with fail-safe-small defaults (#10901)

* Right-size nRF52 heap tiers after 2.8.0 heap-exhaustion field reports

Field reports on 2.8.0 show nRF52840 devices at 99% heap (114/115 KB)
within minutes of boot; operator new asserts on OOM, so these devices
are one allocation from a reboot. The 2.8.0 cache sizing ladders gave
nRF52 the largest non-PSRAM tiers on the assumption that a BLE-only
part has a roomy heap - the arena is actually ~125 KB shared with the
FreeRTOS task stacks.

Per-target retiers (nRF52840 unless noted):
- Traffic Management cache 1000 -> 250 entries (10 KB -> 2.5 KB); the
  unclassified fallthrough drops 1000 -> 400 to match the classic-ESP32
  tier (also affects RP2040/RP2350)
- Warm node store 200 -> 100 entries (8 KB -> 4 KB); the non-XXAA
  fallthrough drops 320 -> 100 so an unclassified RAM-constrained part
  can't boot-allocate 12.8 KB
- MESSAGE_HISTORY_LIMIT 20 -> 10 (text pool 4.4 KB -> 2.2 KB), the tier
  classic ESP32 already ships
- MAX_RX_TOPHONE 32 -> 16, shrinking the static packet pool 70 -> 54
  slots (~6.6 KB of .bss returned to the heap arena)
- PacketHistory hash index off arch-wide (1 KB); O(n) over 240 records
  is negligible at LoRa packet rates
- OLEDDISPLAY_REDUCE_MEMORY arch-wide (~1 KB OLED back buffer); the five
  TFT variants -U it because TFTDisplay.cpp needs buffer_back for
  dirty-window diffing
- Drop the stale "for testing" 1024-entry TMM override on T1000-E

Measured on rak4631: heap arena grows 124,572 -> 131,180 B and boot
allocations drop ~15.7 KB, roughly +22 KB free heap on the field-report
device class.

Migration: the nRF52840 warm flash ring replays through place() (LRU),
so the newest 100 identities survive the shrink; the file backend
rejects oversized snapshots cleanly (new test covers this). Native
suites pass (536/536 Docker, 13/13 native-macos warm store); rak4631,
heltec-mesh-node-t114 (TFT) and tracker-t1000-e build green.

* Add central memory-class ladder (MemClass.h) with fail-safe-small defaults

The 2.8.0 nRF52840 heap exhaustion happened because each RAM-sized cache
picked its per-platform tier from its own chip #ifdef ladder, and every
ladder's fallthrough default was its largest non-PSRAM tier - nRF52 was
never named, so it silently got 1000-entry caches on a ~115 KB arena.

This introduces src/memory/MemClass.h: a single MESHTASTIC_MEM_CLASS
(TINY / SMALL / MEDIUM / LARGE) ranked by usable app heap after platform
overheads, with the deliberate property that an unclassified chip lands
in SMALL - a new target boots with small caches until someone opts it up
in one visible place.

The TMM cache, warm store, MAX_RX_TOPHONE and MAX_SATELLITE_NODES
ladders in mesh-pb-constants.h now key off the class; branches pinned by
something other than RAM stay explicit and say why (nRF52840's SoftDevice
arena, RP2040's warm.dat watchdog bound). MAX_NUM_NODES intentionally
stays separate - it is flash-shaped (nodes.proto vs LittleFS), not
heap-shaped.

A per-class MESHTASTIC_BOOT_CACHE_BUDGET static_assert now covers the
three big boot-allocated caches, so the next cache-adding PR that would
blow a small platform's budget fails to compile instead of exhausting
heap in the field.

No values change for any existing target: rak4631, tbeam, rak11310 and
wio-e5 build byte-identical before/after; all ladders remain
#ifndef-guarded so variant overrides keep working.

* Address review: fix RP2350 class-table doc, add PacketRecord static_assert

- MemClass.h's class table claimed RP2350 was MEDIUM while the mapping
  ladder classifies it SMALL (with RP2040) - the table now matches the
  ladder, with a note that RP2350 is a MEDIUM candidate whenever someone
  wants to tune it up (kept SMALL here so this header stays a behavioral
  no-op).
- The boot-cache budget comment referenced a static_assert pinning
  PacketHistory::PacketRecord at 20 B that did not exist (only a layout
  comment). Add the real static_assert so the budget math in
  mesh-pb-constants.h fails to compile if the record layout changes.

Also merges develop (the base #10898 landed there as a squash, which is
what made this stacked branch conflict); develop's mesh-pb-constants.h
is byte-identical to this branch's base, so the resolution keeps the
MemClass ladder unchanged. rak4631 and wio-e5 build green;
test_packet_history 47/47.

* Address review: share PACKETHISTORY_MAX, trim policy comments

- Hoist PACKETHISTORY_MAX from PacketHistory.cpp into
  mesh-pb-constants.h (next to the MAX_NUM_NODES it derives from) so the
  constructor clamp and the boot-cache budget static_assert use one
  definition instead of hand-mirrored arithmetic that could drift. The
  expression stays valid where MAX_NUM_NODES resolves at runtime
  (ESP32-S3, portduino); the pointless 2.0 double math becomes integer.
- Trim the MemClass.h header (36 -> 16 comment lines) and the budget /
  sizing-policy comments per the repo comment-length guideline, keeping
  the class table, the fail-safe-small rule, the override mechanism, and
  the include-order constraint.

rak4631 (compile-time MAX_NUM_NODES) and heltec-v3 (runtime) build
green; test_packet_history 47/47.

src/memory/MemClass.h

@@ -0,0 +1,53 @@
+#pragma once
+
+// Central memory-class ladder: RAM-sized caches key their per-platform tier off
+// MESHTASTIC_MEM_CLASS. Classes rank *usable app heap after platform overheads*
+// (SoftDevice, WiFi+BLE stacks) - not raw RAM - and an unclassified chip lands in
+// SMALL on purpose: small caches are a recoverable default, an exhausted heap is not.
+//
+//   MEM_CLASS_TINY    <32 KB free heap      STM32WL
+//   MEM_CLASS_SMALL   ~100-250 KB           nRF52840, classic ESP32/S2/C3, RP2040/RP2350
+//   MEM_CLASS_MEDIUM  ~250-500 KB, no PSRAM ESP32-S3/C6/P4 without PSRAM
+//   MEM_CLASS_LARGE   PSRAM or host         ESP32-S3+PSRAM, portduino
+//
+// Compare ordinally (>=). RP2350 rides with RP2040 so this header stays a behavioral
+// no-op (MEDIUM candidate later). Variants may predefine MESHTASTIC_MEM_CLASS or any
+// cache constant - consumer ladders stay #ifndef-guarded. MAX_NUM_NODES is deliberately
+// unclassed (flash-shaped: nodes.proto vs filesystem). Included before configuration.h,
+// so only toolchain/board -D macros and the ARCH_* macros the ladders already use are
+// safe here.
+
+#define MEM_CLASS_TINY 1
+#define MEM_CLASS_SMALL 2
+#define MEM_CLASS_MEDIUM 3
+#define MEM_CLASS_LARGE 4
+
+#ifndef MESHTASTIC_MEM_CLASS
+#if defined(ARCH_STM32WL)
+#define MESHTASTIC_MEM_CLASS MEM_CLASS_TINY
+#elif (defined(CONFIG_IDF_TARGET_ESP32S3) && defined(BOARD_HAS_PSRAM)) || defined(ARCH_PORTDUINO)
+#define MESHTASTIC_MEM_CLASS MEM_CLASS_LARGE
+#elif defined(CONFIG_IDF_TARGET_ESP32S3) || defined(CONFIG_IDF_TARGET_ESP32C6) || defined(CONFIG_IDF_TARGET_ESP32P4)
+#define MESHTASTIC_MEM_CLASS MEM_CLASS_MEDIUM
+#else
+// Classic ESP32 / S2 / C3, all nRF52, RP2040/RP2350, and - by design - any chip
+// nobody has classified yet: small caches are a recoverable default, an
+// exhausted heap is not. New RAM-rich targets opt up by adding a branch above.
+#define MESHTASTIC_MEM_CLASS MEM_CLASS_SMALL
+#endif
+#endif // MESHTASTIC_MEM_CLASS
+
+// Ceiling for the boot-allocated mesh caches (TMM + warm store + packet history);
+// mesh-pb-constants.h static_asserts their sum, so an oversized cache fails the build.
+// Raising a budget is allowed - as a visible, reviewable one-line diff.
+#ifndef MESHTASTIC_BOOT_CACHE_BUDGET
+#if MESHTASTIC_MEM_CLASS <= MEM_CLASS_TINY
+#define MESHTASTIC_BOOT_CACHE_BUDGET (8 * 1024)
+#elif MESHTASTIC_MEM_CLASS == MEM_CLASS_SMALL
+#define MESHTASTIC_BOOT_CACHE_BUDGET (16 * 1024)
+#elif MESHTASTIC_MEM_CLASS == MEM_CLASS_MEDIUM
+#define MESHTASTIC_BOOT_CACHE_BUDGET (32 * 1024)
+#else
+#define MESHTASTIC_BOOT_CACHE_BUDGET (256 * 1024)
+#endif
+#endif // MESHTASTIC_BOOT_CACHE_BUDGET

src/mesh/PacketHistory.cpp

@@ -9,10 +9,6 @@
 #endif
 #include "Throttle.h"
 
-#define PACKETHISTORY_MAX                                                                                                        \
-    max((uint32_t)(MAX_NUM_NODES * 2.0),                                                                                         \
-        (uint32_t)100) // x2..3  Should suffice. Empirical setup. 16B per record malloc'ed, but no less than 100
-
 #define RECENT_WARN_AGE (10 * 60 * 1000L) // Warn if the packet that gets removed was more recent than 10 min
 
 #define VERBOSE_PACKET_HISTORY 0     // Set to 1 for verbose logging, 2 for heavy debugging

src/mesh/PacketHistory.h

@@ -24,6 +24,8 @@ class PacketHistory
                                           // bit 3-5: our hop limit when we first transmitted it
         uint8_t relayed_by[NUM_RELAYERS]; // Array of nodes that relayed this packet
     };                                    // 4B + 4B + 4B + 1B + 1B + 6B = 20B
+    static_assert(sizeof(PacketRecord) == 20,
+                  "PacketRecord size feeds the boot-cache budget math in mesh-pb-constants.h - update both together");
 
     uint32_t recentPacketsCapacity =
         0; // Can be set in constructor, no need to recompile. Used to allocate memory for mx_recentPackets.

src/mesh/mesh-pb-constants.h

@@ -1,12 +1,16 @@
 #pragma once
 #include <vector>
 
+#include "memory/MemClass.h"
 #include "mesh/generated/meshtastic/admin.pb.h"
 #include "mesh/generated/meshtastic/deviceonly.pb.h"
 #include "mesh/generated/meshtastic/localonly.pb.h"
 #include "mesh/generated/meshtastic/mesh.pb.h"
 
 // this file defines constants which come from mesh.options
+//
+// RAM-shaped cache tiers key off MESHTASTIC_MEM_CLASS (memory/MemClass.h) so
+// unclassified chips fail safe-small; class-deviant branches say why inline.
 
 // Tricky macro to let you find the sizeof a type member
 #define member_size(type, member) sizeof(((type *)0)->member)
@@ -23,8 +27,13 @@
 // the 8 classic ESP32 has shipped with for years; drops start when a stalled phone/serial client has
 // 16 packets queued.
 #define MAX_RX_TOPHONE 16
-#else
+#elif MESHTASTIC_MEM_CLASS >= MEM_CLASS_MEDIUM || defined(ARCH_RP2040) || defined(CONFIG_IDF_TARGET_ESP32C3) ||                  \
+    defined(ARCH_STM32WL)
+// RP2040/RP2350, ESP32-C3 and STM32WL keep their historical 32 (no field pressure to cut them;
+// STM32WL's pool is dynamic, so the constant only bounds in-flight packets there).
 #define MAX_RX_TOPHONE 32
+#else
+#define MAX_RX_TOPHONE 16 // unclassified small parts: fail safe-small
 #endif
 #endif
 
@@ -109,13 +118,20 @@ static inline int get_max_num_nodes()
 #endif                    // platform
 #endif                    // MAX_NUM_NODES
 
+/// Packet-history capacity: 2x the hot store so dedup/relayer state survives a
+/// full mesh, floored at 100. Shared by PacketHistory's constructor clamp and
+/// the boot-cache budget assert below so the two cannot drift.
+#ifndef PACKETHISTORY_MAX
+#define PACKETHISTORY_MAX (MAX_NUM_NODES * 2 > 100 ? (uint32_t)(MAX_NUM_NODES * 2) : (uint32_t)100)
+#endif
+
 /// Per-map cap (position/telemetry/environment/status): only the freshest
 /// MAX_SATELLITE_NODES nodes keep satellite payloads, the rest just the
 /// NodeInfoLite header. RAM-bound (the maps are internal-SRAM, not PSRAM), so
 /// flash-rich hosts get a cap >= their hot store (satellites for every node, as
 /// before the cap existed) while constrained parts stay at 40.
 #ifndef MAX_SATELLITE_NODES
-#if (defined(CONFIG_IDF_TARGET_ESP32S3) && defined(BOARD_HAS_PSRAM)) || defined(ARCH_PORTDUINO)
+#if MESHTASTIC_MEM_CLASS >= MEM_CLASS_LARGE
 #define MAX_SATELLITE_NODES 250
 #else
 #define MAX_SATELLITE_NODES 40 // nRF52840, generic ESP32, and ESP32-S3 without PSRAM
@@ -126,7 +142,7 @@ static inline int get_max_num_nodes()
 /// so DMs to/from them keep decrypting. 0 disables it; size is per-platform
 /// below, persisted to /prefs/warm.dat (or the nRF52840 raw-flash ring).
 #ifndef WARM_NODE_COUNT
-#if defined(ARCH_STM32WL)
+#if MESHTASTIC_MEM_CLASS <= MEM_CLASS_TINY
 #define WARM_NODE_COUNT 0
 #elif defined(NRF52840_XXAA)
 // Keyed on the NRF52840_XXAA build flag, not ARCH_NRF52: the latter (from
@@ -136,30 +152,28 @@ static inline int get_max_num_nodes()
 // arena, which 2.8.0 field reports showed at 99% use; 120 hot + 100 warm
 // identities still covers meshes well past the hot cap.
 #define WARM_NODE_COUNT 100
-#elif (defined(CONFIG_IDF_TARGET_ESP32S3) && defined(BOARD_HAS_PSRAM)) || defined(ARCH_PORTDUINO)
-#define WARM_NODE_COUNT 2000 // PSRAM-equipped ESP32-S3 / native host; warm cache in PSRAM (~80 KB)
-#elif defined(CONFIG_IDF_TARGET_ESP32S3) || defined(CONFIG_IDF_TARGET_ESP32C6) || defined(CONFIG_IDF_TARGET_ESP32P4)
-#define WARM_NODE_COUNT 150 // 512 KB+ SRAM, no PSRAM (S3/C6/P4): ~6 KB heap (#10705)
-#elif defined(ARCH_ESP32)
-#define WARM_NODE_COUNT 100 // classic ESP32 (520 KB) / S2 (320 KB) / C3 (400 KB): tightest free heap w/ BLE+WiFi, ~4 KB (#10705)
 #elif defined(ARCH_RP2040)
-#define WARM_NODE_COUNT 150 // RP2040 (264 KB) / RP2350 (520 KB): bounded so warm.dat write fits the 8s watchdog (#10746)
+// Class-deviant on purpose: bounded so the warm.dat write fits the 8s watchdog (#10746),
+// not by RAM (RP2040 264 KB / RP2350 520 KB could hold more).
+#define WARM_NODE_COUNT 150
+#elif MESHTASTIC_MEM_CLASS >= MEM_CLASS_LARGE
+#define WARM_NODE_COUNT 2000 // PSRAM-equipped ESP32-S3 / native host; warm cache in PSRAM (~80 KB)
+#elif MESHTASTIC_MEM_CLASS == MEM_CLASS_MEDIUM
+#define WARM_NODE_COUNT 150 // 512 KB+ SRAM, no PSRAM (S3/C6/P4): ~6 KB heap (#10705)
 #else
-// nRF52840 is handled explicitly above (raw-flash ring). Any other nRF52 (non-XXAA) and any future
+// MEM_CLASS_SMALL: classic ESP32/S2/C3 (~4 KB heap, #10705) and anything unclassified -
-// non-ESP32/non-RP LittleFS part lands here. Reviewed after the 2.8.0 nRF52840 heap-exhaustion
+// fail small so a new RAM-constrained part can't boot-allocate 12.8 KB (2.8.0 lesson).
-// reports: fail small (match nRF52840) so an unclassified RAM-constrained part can't boot-allocate
+#define WARM_NODE_COUNT 100
-// 12.8 KB; RAM-rich targets should opt up explicitly in their variant.
+#endif // platform
-#define WARM_NODE_COUNT 100 // other LittleFS-backed parts (e.g. non-nRF52840 nRF52)
+#endif // WARM_NODE_COUNT
-#endif                      // platform
-#endif                      // WARM_NODE_COUNT
 
 /// Max number of channels allowed
 #define MAX_NUM_CHANNELS (member_size(meshtastic_ChannelFile, channels) / member_size(meshtastic_ChannelFile, channels[0]))
 
 // Traffic Management module configuration
-// Enabled by default; STM32WL is excluded due to RAM constraints (MAX_NUM_NODES=10).
+// Enabled by default; TINY parts (STM32WL) are excluded due to RAM constraints (MAX_NUM_NODES=10).
 // Disable per-variant by defining HAS_TRAFFIC_MANAGEMENT=0 in variant.h
-#ifdef ARCH_STM32WL
+#if MESHTASTIC_MEM_CLASS <= MEM_CLASS_TINY
 #define HAS_TRAFFIC_MANAGEMENT 0
 #endif
 #ifndef HAS_TRAFFIC_MANAGEMENT
@@ -181,25 +195,39 @@ static inline int get_max_num_nodes()
 #ifndef TRAFFIC_MANAGEMENT_CACHE_SIZE
 #if !HAS_TRAFFIC_MANAGEMENT
 #define TRAFFIC_MANAGEMENT_CACHE_SIZE 0
-#elif (defined(CONFIG_IDF_TARGET_ESP32S3) && defined(BOARD_HAS_PSRAM)) || defined(ARCH_PORTDUINO)
-#define TRAFFIC_MANAGEMENT_CACHE_SIZE 2048 // PSRAM-equipped ESP32-S3 / native host
-#elif defined(CONFIG_IDF_TARGET_ESP32S3) || defined(CONFIG_IDF_TARGET_ESP32C6) || defined(CONFIG_IDF_TARGET_ESP32P4)
-#define TRAFFIC_MANAGEMENT_CACHE_SIZE 500 // 512 KB+ SRAM, no PSRAM (S3/C6/P4): ~5 KB heap (#10705)
-#elif defined(ARCH_ESP32)
-#define TRAFFIC_MANAGEMENT_CACHE_SIZE 400 // classic ESP32 / S2 / C3: tightest free heap, ~4 KB (#10705)
 #elif defined(NRF52840_XXAA)
-// Keyed on NRF52840_XXAA (not ARCH_NRF52) for the same include-order reason as WARM_NODE_COUNT above.
+// Class-deviant on purpose (SMALL would be 400): the 256 KB part shares its ~115 KB heap arena with
-// The 256 KB part shares its ~115 KB heap arena with SoftDevice and the FreeRTOS task stacks; 2.8.0
+// SoftDevice and the FreeRTOS task stacks; 2.8.0 field reports hit 99% heap use with the old
-// field reports hit 99% heap use with the old 1000-entry (~10 KB) cache. 250 entries (2.5 KB) still
+// 1000-entry (~10 KB) cache. 250 entries (2.5 KB) still tracks >2x the 120-node hot store; LRU
-// tracks >2x the 120-node hot store; LRU victim recycling absorbs busier meshes.
+// victim recycling absorbs busier meshes.
 #define TRAFFIC_MANAGEMENT_CACHE_SIZE 250
+#elif MESHTASTIC_MEM_CLASS >= MEM_CLASS_LARGE
+#define TRAFFIC_MANAGEMENT_CACHE_SIZE 2048 // PSRAM-equipped ESP32-S3 / native host
+#elif MESHTASTIC_MEM_CLASS == MEM_CLASS_MEDIUM
+#define TRAFFIC_MANAGEMENT_CACHE_SIZE 500 // 512 KB+ SRAM, no PSRAM (S3/C6/P4): ~5 KB heap (#10705)
 #else
-// RP2040/RP2350 and anything unclassified land here: match the classic-ESP32 tier rather than
+// MEM_CLASS_SMALL: classic ESP32/S2/C3 (~4 KB heap, #10705), RP2040/RP2350, and anything
-// defaulting large. RAM-rich targets should opt up explicitly in their variant.
+// unclassified - fail small rather than defaulting large (2.8.0 lesson).
 #define TRAFFIC_MANAGEMENT_CACHE_SIZE 400
 #endif
 #endif // TRAFFIC_MANAGEMENT_CACHE_SIZE
 
+// Enforce the per-class boot-cache budget (memory/MemClass.h) over the three big
+// boot-allocated mesh caches. Entry sizes are pinned by static_asserts at their
+// definitions: TMM UnifiedCacheEntry = 10 B (TrafficManagementModule.h),
+// WarmNodeEntry = 40 B (WarmNodeStore.h), PacketHistory::PacketRecord = 20 B
+// (PacketHistory.h; its optional hash index is excluded here). Skipped where
+// MAX_NUM_NODES is not a compile-time constant: ESP32-S3 (runtime, from flash
+// size) and portduino (runtime, from portduino_config.MaxNodes via variant.h).
+// If this fires on a new feature: shrink the cache for this class, or raise the
+// class budget in MemClass.h as a visible, reviewable decision.
+#if !defined(CONFIG_IDF_TARGET_ESP32S3) && !defined(ARCH_PORTDUINO)
+static_assert((uint32_t)TRAFFIC_MANAGEMENT_CACHE_SIZE * 10u + (uint32_t)WARM_NODE_COUNT * 40u + 20u * PACKETHISTORY_MAX <=
+                  MESHTASTIC_BOOT_CACHE_BUDGET,
+              "Boot-allocated mesh caches exceed this memory class's budget - shrink a tier or consciously raise "
+              "MESHTASTIC_BOOT_CACHE_BUDGET in memory/MemClass.h");
+#endif
+
 /// helper function for encoding a record as a protobuf, any failures to encode are fatal and we will panic
 /// returns the encoded packet size
 size_t pb_encode_to_bytes(uint8_t *destbuf, size_t destbufsize, const pb_msgdesc_t *fields, const void *src_struct);

variants/nrf52840/heltec_mesh_node_t114/platformio.ini

@@ -16,6 +16,7 @@ debug_tool = jlink
 
 # add -DCFG_SYSVIEW if you want to use the Segger systemview tool for OS profiling.
 build_flags = ${nrf52840_base.build_flags}
+  -UOLEDDISPLAY_REDUCE_MEMORY ; TFTDisplay.cpp needs the lib's buffer_back for dirty-window diffing
   -Ivariants/nrf52840/heltec_mesh_node_t114
   -DHELTEC_T114
 

variants/nrf52840/heltec_mesh_solar/platformio.ini

@@ -105,6 +105,7 @@ build_flags = ${heltec_mesh_solar_base.build_flags}
 [env:heltec-mesh-solar-tft]
 extends = heltec_mesh_solar_base
 build_flags = ${heltec_mesh_solar_base.build_flags}
+  -UOLEDDISPLAY_REDUCE_MEMORY ; TFTDisplay.cpp needs the lib's buffer_back for dirty-window diffing
   -DHELTEC_MESH_SOLAR_TFT
   -DSPI_INTERFACES_COUNT=2
   -DHAS_SPI_TFT=1