kevin/meshtastic-firmware
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

Pr1 nodedb warmstore (#10705)

Browse Source 
* NodeDB: 3-tier node store with persistent warm tier (long-tail identity retention)

Introduces a tiered NodeDB so the device retains identity (public key,
last_heard) for far more nodes than fit in the full-record hot store,
without growing heap or the persisted nodes.proto unboundedly.

- Hot store: full NodeInfoLite, MAX_NUM_NODES (120 on nRF52).
- Satellite maps: position/telemetry/environment/status capped at
  MAX_SATELLITE_NODES (40 freshest); eviction via enforceSatelliteCaps /
  evictSatelliteOverCap.
- Warm tier (WarmNodeStore): 40 B {num,last_heard,public_key} records for
  evicted nodes so DMs to/from long-tail nodes keep encrypting/decrypting.
  Persisted to /prefs/warm.dat, or on nRF52840 a dedicated 12 KB raw-flash
  record-ring below LittleFS (3x4 KB pages; see linker scripts + the
  nrf52_warm_region.py post-link guard).

NodeDB::getOrCreateMeshNode now demotes evicted nodes into the warm tier and
re-admits them (restoring key/last_heard). Router PKI decrypt/encode resolve
the peer key via NodeDB::copyPublicKey (hot store, then warm tier).

NodeInfoLite gains snr_q4 (sint32, Q4-encoded dB); the float snr is zeroed on
disk. NodeInfoLite grows 105 -> 112 B; backup 2432 -> 2468 B.

Note: the snr_q4 .proto change still needs to land in the protobufs submodule
(generated header is updated here; submodule pointer left at upstream).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* NodeDB: robust receive + retention for blocked (ignored) nodes

Hardens how ignored/favourite nodes are received over admin and retained,
closing paths where a block could be lost or accidentally cleared.

- Blocking keeps the node's public key (admin set_ignored_node and
  addFromContact no longer zero it / drop the warm-tier key), so a blocked
  peer stays a verifiable identity.
- set_ignored_node creates the node if absent, so a block by node ID sticks
  even for a node we've never heard from (e.g. pushed by a remote admin) with
  no NodeInfo or key.
- Eviction protection (favourite/ignored/manually-verified) now also applies to
  the load-time hot-store migration and is never undone by cleanupMeshDB, which
  previously purged ignored nodes that lacked user info.
- The hot-store migration leaves our own node (index 0) in place and prefers to
  demote non-protected nodes, like the runtime eviction scan.

Caps the protected set (favourite + ignored + verified) at MAX_NUM_NODES-2 via
NodeDB::setProtectedFlag(), so at least two evictable slots always remain and
getOrCreateMeshNode can always make room — replacing the previous unconditional
append that could run off the end of the node vector when every node was
protected. A locally-set favourite/ignore that hits the cap reports back to the
phone via a ClientNotification.

Adds test_nodedb_blocked covering the migration, favourite/ignored eviction
protection, ignored-survives-cleanup, and the protected-node cap. The
maintenance methods stay private in production; the test reaches them through a
PIO_UNIT_TESTING-guarded friend shim.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

# Conflicts:
#	src/mesh/NodeDB.h

* fix copilot comments

* once again

* WarmNodeStore: fix cppcheck warnings (uninitvar, constVariablePointer)

Zero-initialise `stranded[]` and `seqs[]/order[]` VLAs so cppcheck can
verify there are no unguarded reads of uninitialised memory (the guards
exist but are not visible to static analysis). Mark two local pointers
`const` where the pointed-to entry is never mutated after assignment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* self-care added to assist 2.7 and 2.8 nodedb migration

* Tidy warm-store/self-care: comments, guards, log + flash cleanup

Style/cleanup pass over the branch (no behavior change except the noted
preprocessor simplifications, which are semantically identical):

- Comments: move function descriptions to the headers, cap in-function
  comments at ~3-4 lines, drop leading-number step markers, label stacked
  #endif blocks, de-decorate banner comments.
- dumpToLog: fully gate decl + definition + AdminModule call site behind
  MESHTASTIC_NODEDB_MIGRATION_VERBOSE so it compiles out when disabled
  (~1.2 KB when off).
- mesh-pb-constants: drop the dead nRF52832 WARM_NODE_COUNT branch and trim
  the macro docs.
- WarmNodeStore: simplify the redundant `ARCH_NRF52 && NRF52840_XXAA` guards
  to `NRF52840_XXAA`, add a kNoPage sentinel for the ring page state.
- Shorten the always-on LOG_WARN strings (~120 B flash).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* more tidying up, aligning with docs and undoing other-arch regressions

* Update protobufs (#19)

Co-authored-by: NomDeTom <116762865+NomDeTom@users.noreply.github.com>

* made the migration pathway cleareer

* address copilot review

* fixed a copilot review on a downstream PR.

* Address Copilot review comments for PR #10705 (warmstore/nodedb)

- WarmNodeStore.h: default MIGRATION_VERBOSE to 0 (suppress info-level
  chatter on production builds; opt in with =1)
- WarmNodeStore.cpp load(): move memset to top of function so all
  failure paths (header-read fail, invalid header) leave entries clear
- WarmNodeStore.cpp save(): replace manual spiLock lock/unlock around
  mkdir with LockGuard covering the full SafeFile sequence, matching
  the lock discipline in load()
- Router.cpp: memcpy(&p->public_key.bytes, ...) -> memcpy(p->public_key.bytes,
  ...) — pass decayed uint8_t* rather than pointer-to-array
- AdminModule.cpp: check setProtectedFlag return for PKC auto-favorite;
  log cap-refusal warning instead of unconditional "auto-favoriting"
- nrf52_warm_region.py: error message references both v6.ld and v7.ld

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* NodeDB: formatting cleanup (blank lines after preprocessor blocks)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Lukewarm store

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ben Meadors <benmmeadors@gmail.com>
This commit is contained in:
Tom
2026-06-18 08:41:34 +01:00
committed by GitHub
co-authored by GitHub Claude Opus 4.8 github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Ben Meadors
parent b311e01ce0
commit 79a7dcc46c
17 changed files with 1848 additions and 87 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/copilot-instructions.md
+18 -1
View File
@@ -191,7 +191,24 @@ Writers go through `setNodeStatus`, `updatePosition`, `updateTelemetry` (which d
### Eviction
Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly.
Every code path that drops a node from the header table must also evict the satellites. The single chokepoint is `eraseNodeSatellites(NodeNum)`; it's already called from `getOrCreateMeshNode`'s oldest-boring eviction, `demoteOldestHotNodesToWarm` (the over-cap warm-tier migration), `removeNodeByNum`, both branches of `resetNodes`, `cleanupMeshDB`, `addFromContact`'s ignored-branch, and `AdminModule`'s `set_ignored_node`. Add new eviction sites here, not by calling `.erase()` directly. (Note: `enforceSatelliteCaps`/`evictSatelliteOverCap` call `.erase()` directly on purpose — that's a satellite-only cap trim where the node _stays_ in the header, a different operation from this chokepoint.)
### Warm tier (long-tail identity)
On every arch except STM32WL and bare nRF52832 (`WARM_NODE_COUNT > 0`), a node evicted from the header table is not forgotten outright: `WarmNodeStore` (`src/mesh/WarmNodeStore.{h,cpp}`) keeps a 40 B `{num, last_heard, public_key}` record per evicted node — primarily so PKI DMs to/from a long-tail node keep decrypting without re-running a NodeInfo exchange (the rest of `NodeInfoLite` rebuilds from traffic in seconds).
- **Write:** `getOrCreateMeshNode`'s eviction and `demoteOldestHotNodesToWarm` (the over-cap boot migration) call `warmStore.absorb(num, last_heard, key)` _before_ the node leaves the header.
- **Read-back:** `getOrCreateMeshNode` calls `warmStore.take()` to rehydrate `last_heard` + key when a warm node is re-admitted; `copyPublicKey()` falls back to the warm tier so the PKI send path finds keys for evicted peers.
- **Persistence:** nRF52840 uses a 12 KB raw-flash record-ring at `0xEA000` (below LittleFS; append + replay + compact-on-rotate, link-guarded by `nrf52840_s140_v7.ld` and `extra_scripts/nrf52_warm_region.py`). Everywhere else: a `/prefs/warm.dat` snapshot flushed by `saveIfDirty()` on the node-DB save cadence.
- **Tunables** (`mesh-pb-constants.h`): `WARM_NODE_COUNT` (per-arch; `0` disables the tier) and `MAX_NUM_NODES` (hot cap — 120 on nRF52840/generic ESP32 to fit the 28 KB LittleFS; ESP32-S3 keeps its flash-scaled 100/200/250, portduino 250). Verbose migration/self-care tracing routes through `LOG_MIGRATION`, gated by `MESHTASTIC_NODEDB_MIGRATION_VERBOSE`.
### Satellite caps
Only the freshest `MAX_SATELLITE_NODES` nodes keep satellite payloads; the rest of the header table carries just the `NodeInfoLite`. The cap is **per-platform**: 40 on RAM-constrained parts (nRF52840, generic ESP32) since the four maps live in internal SRAM (not PSRAM, ~408 B/node across the four), and 250 on flash-rich hosts (ESP32-S3, portduino) so every hot node can carry rich data as before the cap existed. `enforceSatelliteCaps()` trims each map to the cap on load (returns whether it trimmed); `evictSatelliteOverCap()` trims before each insert. Eviction is by the owning node's hot `last_heard` (stalest first, demoted/absent nodes rank as `last_heard==0`); self is never trimmed.
### On-boot self-care
`NodeDB::nodeDBSelfCare()` runs once identity is established (the constructor after key (re)gen, and `reloadFromDisk()` — _not_ inside `loadFromDisk`, where `getNodeNum()` is still 0). It confirms self is present (warns if a non-empty DB is missing us — a foreign/over-cap file), pins self to index 0, demotes/trims only **non-self** overflow into the warm tier, then rewrites `nodes.proto` **once** and only if it healed something — and never while encrypted storage is locked (it would persist placeholder defaults). `loadFromDisk` deliberately leaves the loaded store untrimmed for this pass.
### Sync flow: thin NodeInfo + post-COMPLETE_ID replay (no opt-in)
extra_scripts/nrf52_warm_region.py
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env python3
# trunk-ignore-all(ruff/F821)
# trunk-ignore-all(flake8/F821): For SConstruct imports
#
# Post-link guard for the warm-node-store raw-flash region on nRF52840.
#
# The 3 app-region pages below LittleFS (0xEA000-0xED000, reclaimed by whole-image
# LTO) are reserved for the WarmNodeStore record-ring (see WarmNodeStore.h). Our
# linker scripts (nrf52840_s140_v6.ld and nrf52840_s140_v7.ld) cap the image at
# 0xEA000, but boards on the framework-default script (FLASH ending at 0xED000) could
# silently place code in those pages — the first warm-store save would then brick the
# device. This turns that into a build failure.
#
# Image flash end = __etext + sizeof(.data) (loaded at LMA __etext); symbols from
# the framework's nrf52_common.ld.
import os
Import("env")
WARM_REGION_BASE = 0xEA000 # keep in sync with WARM_FLASH_REGION_BASE in WarmNodeStore.h (3 x 4 KB record-ring)
_tc = env.PioPlatform().get_package_dir("toolchain-gccarmnoneeabi") or ""
_NM = os.path.join(_tc, "bin", "arm-none-eabi-nm")
if not os.path.isfile(_NM):
_NM = "arm-none-eabi-nm" # fall back to PATH
def _assert_warm_region_clear(source, target, env):
import subprocess
import sys
try:
elf = env.subst("$BUILD_DIR/${PROGNAME}.elf")
out = subprocess.check_output([_NM, elf], universal_newlines=True)
except Exception as exc: # tooling hiccup: warn loudly, don't wedge the build
print("nrf52_warm_region: WARNING - guard skipped (nm failed: %s)" % exc)
return
syms = {}
for line in out.split("\n"):
f = line.split()
if len(f) >= 3 and f[-1] in ("__etext", "__data_start__", "__data_end__"):
syms[f[-1]] = int(f[0], 16)
if len(syms) != 3:
print("nrf52_warm_region: WARNING - guard skipped (linker symbols not found)")
return
flash_end = syms["__etext"] + (syms["__data_end__"] - syms["__data_start__"])
if flash_end > WARM_REGION_BASE:
sys.stderr.write(
"\n*** nrf52 warm-region guard: image ends at 0x%X, past the reserved "
"warm-store region at 0x%X ***\n"
"The 12 KB region at 0xEA000 holds the WarmNodeStore record-ring; a warm-store\n"
"save would overwrite this firmware's tail. Shrink the image, or shrink/move\n"
"the region (WARM_FLASH_REGION_BASE in src/mesh/WarmNodeStore.h, the FLASH\n"
"LENGTH in src/platform/nrf52/nrf52840_s140_v6.ld and _v7.ld, and this guard).\n\n"
% (flash_end, WARM_REGION_BASE)
)
from SCons.Script import Exit
Exit(1)
print(
"nrf52_warm_region: guard OK -- image ends at 0x%X, %d KB clear of the warm region"
% (flash_end, (WARM_REGION_BASE - flash_end) // 1024)
)
# Attach to the phony "buildprog" alias (not the .elf node) so the guard runs
# on incremental relinks too -- same reasoning as nrf52_lto.py's guard.
env.AddPostAction("buildprog", _assert_warm_region_clear)
src/graphics/draw/MenuHandler.cpp
+13 -4
View File
@@ -1572,6 +1572,7 @@ void menuHandler::manageNodeMenu()
nodeDB->set_favorite(false, menuHandler::pickedNodeNum);
} else {
LOG_INFO("Adding node %08X to favorites", menuHandler::pickedNodeNum);
// set_favorite() already logs PROTECTED_CAP_WARN_FMT on a cap refusal; don't double-log here.
nodeDB->set_favorite(true, menuHandler::pickedNodeNum);
}
screen->setFrames(graphics::Screen::FOCUS_PRESERVE);
@@ -1615,15 +1616,23 @@ void menuHandler::manageNodeMenu()
return;
}
bool changed = false;
if (nodeInfoLiteIsIgnored(n)) {
nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_IGNORED_MASK, false);
LOG_INFO("Unignoring node %08X", menuHandler::pickedNodeNum);
} else {
nodeInfoLiteSetBit(n, NODEINFO_BITFIELD_IS_IGNORED_MASK, true);
changed = true;
} else if (nodeDB->setProtectedFlag(n, NODEINFO_BITFIELD_IS_IGNORED_MASK, true)) {
LOG_INFO("Ignoring node %08X", menuHandler::pickedNodeNum);
changed = true;
} else {
LOG_WARN(NodeDB::PROTECTED_CAP_WARN_FMT, "ignore", menuHandler::pickedNodeNum, MAX_NUM_NODES - 2);
}
// Only persist/notify when the ignore bit actually moved; a cap
// refusal changed nothing and shouldn't trigger a prefs save.
if (changed) {
nodeDB->notifyObservers(true);
nodeDB->saveToDisk();
}
nodeDB->notifyObservers(true);
nodeDB->saveToDisk();
screen->setFrames(graphics::Screen::FOCUS_PRESERVE);
return;
}
src/mesh/NodeDB.cpp
+432 -31
View File
@@ -197,6 +197,8 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
if (ostream) {
const auto *vec = static_cast<const std::vector<meshtastic_NodeInfoLite> *>(iter->pData);
for (auto item : *vec) {
item.snr_q4 = (int32_t)(item.snr * 4.0f);
item.snr = 0.0f;
if (!pb_encode_tag_for_field(ostream, iter))
return false;
if (!pb_encode_submessage(ostream, meshtastic_NodeInfoLite_fields, &item))
@@ -206,8 +208,12 @@ bool meshtastic_NodeDatabase_callback(pb_istream_t *istream, pb_ostream_t *ostre
if (istream && istream->bytes_left) {
meshtastic_NodeInfoLite node = meshtastic_NodeInfoLite_init_zero;
auto *vec = static_cast<std::vector<meshtastic_NodeInfoLite> *>(iter->pData);
if (pb_decode(istream, meshtastic_NodeInfoLite_fields, &node))
if (pb_decode(istream, meshtastic_NodeInfoLite_fields, &node)) {
if (node.snr_q4)
node.snr = node.snr_q4 / 4.0f;
node.snr_q4 = 0;
vec->push_back(node);
}
}
return true;
}
@@ -482,9 +488,17 @@ NodeDB::NodeDB()
myNodeInfo.my_node_num = crc32Buffer(config.security.public_key.bytes, config.security.public_key.size);
}
#endif
// Include our owner in the node db under our nodenum
meshtastic_NodeInfoLite *info = getOrCreateMeshNode(getNodeNum());
TypeConversions::CopyUserToNodeInfoLite(info, owner);
// Identity is now established, so run the self-care pass on the store
// loadFromDisk() deliberately left untrimmed: confirm self, trim/demote only
// non-self overflow, pin self to index 0, rewrite once if healed.
nodeDBSelfCare();
// If we migrated from legacy during loadFromDisk(), persist the migrated DB
// only after identity and self-care are established.
if (migrationSavePending) {
saveNodeDatabaseToDisk();
migrationSavePending = false;
}
// If node database has not been saved for the first time, save it now
#ifdef FSCom
@@ -689,6 +703,39 @@ template <typename Map> std::vector<NodeNum> snapshotSatelliteNodeNums(const Map
}
return result;
}
// Drop the stalest entry of `map` (staleness proxied via the owner's
// last_heard; 0 = owner evicted, i.e. an orphan — first out). Never evicts our
// own node's entry. Caller holds satelliteMutex. Returns false if nothing
// could be evicted.
template <typename Map> bool evictStalestSatellite(NodeDB &db, Map &map)
{
auto victim = map.end();
uint32_t victimTs = UINT32_MAX;
for (auto it = map.begin(); it != map.end(); ++it) {
if (it->first == db.getNodeNum())
continue;
uint32_t ts = db.hotNodeLastHeard(it->first);
if (ts < victimTs) {
victimTs = ts;
victim = it;
}
}
if (victim == map.end())
return false;
map.erase(victim);
return true;
}
// Keep `map` within MAX_SATELLITE_NODES ahead of inserting `incoming` (the
// tier-1/tier-2 split: only the freshest MAX_SATELLITE_NODES nodes carry
// satellite payloads). Caller holds satelliteMutex.
template <typename Map> void evictSatelliteOverCap(NodeDB &db, Map &map, NodeNum incoming)
{
if (map.size() < MAX_SATELLITE_NODES || map.count(incoming))
return;
evictStalestSatellite(db, map);
}
} // namespace
void NodeDB::resetRadioConfig(bool is_fresh_install)
@@ -721,6 +768,7 @@ bool NodeDB::factoryReset(bool eraseBleBonds)
LOG_ERROR("Could not remove rangetest.csv file");
}
#endif
spiLock->unlock();
// rmDir above nuked the .dat file, but TransmitHistory's in-memory
@@ -731,6 +779,14 @@ bool NodeDB::factoryReset(bool eraseBleBonds)
#if HAS_SCREEN
messageStore.clearAllMessages();
#endif
#if WARM_NODE_COUNT > 0
// On nRF52840 the warm tier lives in raw flash outside /prefs, so rmDir
// didn't touch it; clear it and persist the empty store.
warmStore.clear();
warmStore.saveIfDirty();
#endif
// second, install default state (this will deal with the duplicate mac address issue)
installDefaultNodeDatabase();
installDefaultDeviceState();
@@ -745,6 +801,7 @@ bool NodeDB::factoryReset(bool eraseBleBonds)
// This will erase what's in NVS including ssl keys, persistent variables and ble pairing
nvs_flash_erase();
#endif
#ifdef ARCH_NRF52
LOG_INFO("Clear bluetooth bonds!");
bond_print_list(BLE_GAP_ROLE_PERIPH);
@@ -767,12 +824,15 @@ void NodeDB::installDefaultNodeDatabase()
#if !MESHTASTIC_EXCLUDE_POSITIONDB
nodePositions.clear();
#endif
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
nodeTelemetry.clear();
#endif
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
nodeEnvironment.clear();
#endif
#if !MESHTASTIC_EXCLUDE_STATUSDB
nodeStatus.clear();
#endif
@@ -835,34 +895,42 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
#else
config.lora.region = meshtastic_Config_LoRaConfig_RegionCode_UNSET;
#endif
#ifdef USERPREFS_LORACONFIG_MODEM_PRESET
config.lora.modem_preset = USERPREFS_LORACONFIG_MODEM_PRESET;
#else
config.lora.modem_preset = meshtastic_Config_LoRaConfig_ModemPreset_LONG_FAST;
#endif
#ifdef USERPREFS_LORACONFIG_USE_PRESET
config.lora.use_preset = USERPREFS_LORACONFIG_USE_PRESET;
#else
config.lora.use_preset = true;
#endif
#ifdef USERPREFS_LORACONFIG_BANDWIDTH
config.lora.bandwidth = USERPREFS_LORACONFIG_BANDWIDTH;
#endif
#ifdef USERPREFS_LORACONFIG_SPREAD_FACTOR
config.lora.spread_factor = USERPREFS_LORACONFIG_SPREAD_FACTOR;
#endif
#ifdef USERPREFS_LORACONFIG_CODING_RATE
config.lora.coding_rate = USERPREFS_LORACONFIG_CODING_RATE;
#endif
#ifdef USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY
config.lora.override_frequency = USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY;
#endif
config.lora.hop_limit = HOP_RELIABLE;
#ifdef USERPREFS_CONFIG_LORA_IGNORE_MQTT
config.lora.ignore_mqtt = USERPREFS_CONFIG_LORA_IGNORE_MQTT;
#else
config.lora.ignore_mqtt = false;
#endif
// Initialize admin_key_count to zero
byte numAdminKeys = 0;
@@ -892,6 +960,7 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
numAdminKeys++;
}
#endif
config.security.admin_key_count = numAdminKeys;
if (shouldPreserveKey) {
@@ -906,6 +975,7 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
#ifdef PIN_GPS_EN
config.position.gps_en_gpio = PIN_GPS_EN;
#endif
#if defined(USERPREFS_CONFIG_GPS_MODE)
config.position.gps_mode = USERPREFS_CONFIG_GPS_MODE;
#elif !HAS_GPS || GPS_DEFAULT_NOT_PRESENT
@@ -918,11 +988,13 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
#else
config.position.gps_mode = meshtastic_Config_PositionConfig_GpsMode_ENABLED;
#endif
#ifdef USERPREFS_CONFIG_SMART_POSITION_ENABLED
config.position.position_broadcast_smart_enabled = USERPREFS_CONFIG_SMART_POSITION_ENABLED;
#else
config.position.position_broadcast_smart_enabled = true;
#endif
config.position.broadcast_smart_minimum_distance = 100;
config.position.broadcast_smart_minimum_interval_secs = default_broadcast_smart_minimum_interval_secs;
if (config.device.role != meshtastic_Config_DeviceConfig_Role_ROUTER &&
@@ -942,6 +1014,7 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
// default to bluetooth capability of platform as default
config.bluetooth.enabled = true;
#endif
config.bluetooth.fixed_pin = defaultBLEPin;
#if defined(ST7735_CS) || defined(USE_EINK) || defined(ILI9341_DRIVER) || defined(ILI9342_DRIVER) || defined(ST7789_CS) || \
@@ -953,7 +1026,7 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
if (st7789_id == 0xFFFFFF) {
hasScreen = false;
}
#endif
#endif // HELTEC_MESH_NODE_T114
#elif ARCH_PORTDUINO
bool hasScreen = false;
if (portduino_config.displayPanel)
@@ -973,6 +1046,7 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
config.bluetooth.mode = hasScreen ? meshtastic_Config_BluetoothConfig_PairingMode_RANDOM_PIN
: meshtastic_Config_BluetoothConfig_PairingMode_FIXED_PIN;
#endif
// for backward compat, default position flags are ALT+MSL
config.position.position_flags =
(meshtastic_Config_PositionConfig_PositionFlags_ALTITUDE | meshtastic_Config_PositionConfig_PositionFlags_ALTITUDE_MSL |
@@ -985,8 +1059,8 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
config.network.enabled_protocols = USERPREFS_NETWORK_ENABLED_PROTOCOLS;
#else
config.network.enabled_protocols = 0;
#endif
#endif
#endif // Network enabled protocols
#endif // UDP Multicast
#ifdef USERPREFS_NETWORK_WIFI_ENABLED
config.network.wifi_enabled = USERPREFS_NETWORK_WIFI_ENABLED;
@@ -1013,16 +1087,20 @@ void NodeDB::installDefaultConfig(bool preserveKey = false)
#ifdef DISPLAY_FLIP_SCREEN
config.display.flip_screen = true;
#endif
#ifdef RAK4630
config.display.wake_on_tap_or_motion = true;
#endif
#if defined(T_WATCH_S3) || defined(SENSECAP_INDICATOR)
config.display.screen_on_secs = 30;
config.display.wake_on_tap_or_motion = true;
#endif
#ifdef COMPASS_ORIENTATION
config.display.compass_orientation = COMPASS_ORIENTATION;
#endif
#if defined(ARCH_ESP32) && !MESHTASTIC_EXCLUDE_WIFI
if (MeshtasticOTA::isUpdated()) {
MeshtasticOTA::recoverConfig(&config.network);
@@ -1046,6 +1124,7 @@ void NodeDB::initConfigIntervals()
#else
config.position.gps_update_interval = default_gps_update_interval;
#endif
#ifdef USERPREFS_CONFIG_POSITION_BROADCAST_INTERVAL
config.position.position_broadcast_secs = USERPREFS_CONFIG_POSITION_BROADCAST_INTERVAL;
#else
@@ -1082,22 +1161,26 @@ void NodeDB::installDefaultModuleConfig()
defined(NEOPIXEL_STATUS_NOTIFICATION_PIN)
moduleConfig.external_notification.enabled = true;
#endif
#if defined(PIN_BUZZER)
moduleConfig.external_notification.output_buzzer = PIN_BUZZER;
moduleConfig.external_notification.use_pwm = true;
moduleConfig.external_notification.alert_message_buzzer = true;
#endif
#if defined(PIN_VIBRATION)
moduleConfig.external_notification.output_vibra = PIN_VIBRATION;
moduleConfig.external_notification.alert_message_vibra = true;
moduleConfig.external_notification.output_ms = 500;
#endif
#if defined(LED_NOTIFICATION)
moduleConfig.external_notification.output = LED_NOTIFICATION;
moduleConfig.external_notification.active = LED_STATE_ON;
moduleConfig.external_notification.alert_message = true;
moduleConfig.external_notification.output_ms = 1000;
#endif
#if defined(PIN_VIBRATION)
moduleConfig.external_notification.nag_timeout = 2;
#elif defined(PIN_BUZZER) || defined(LED_NOTIFICATION) || defined(NEOPIXEL_STATUS_NOTIFICATION_PIN)
@@ -1114,14 +1197,16 @@ void NodeDB::installDefaultModuleConfig()
moduleConfig.external_notification.nag_timeout = 0;
#else
moduleConfig.external_notification.nag_timeout = default_ringtone_nag_secs;
#endif
#endif
#endif // HAS_TFT
#endif // HAS_I2S
#ifdef NANO_G2_ULTRA
moduleConfig.external_notification.enabled = true;
moduleConfig.external_notification.alert_message = true;
moduleConfig.external_notification.output_ms = 100;
moduleConfig.external_notification.active = true;
#endif
#endif // NANO_G2_ULTRA
#ifdef T_LORA_PAGER
moduleConfig.canned_message.updown1_enabled = true;
moduleConfig.canned_message.inputbroker_pin_a = ROTARY_A;
@@ -1131,35 +1216,43 @@ void NodeDB::installDefaultModuleConfig()
moduleConfig.canned_message.inputbroker_event_ccw = meshtastic_ModuleConfig_CannedMessageConfig_InputEventChar(29);
moduleConfig.canned_message.inputbroker_event_press = meshtastic_ModuleConfig_CannedMessageConfig_InputEventChar_SELECT;
#endif
moduleConfig.has_canned_message = true;
#if USERPREFS_MQTT_ENABLED && !MESHTASTIC_EXCLUDE_MQTT
moduleConfig.mqtt.enabled = true;
#endif
#ifdef USERPREFS_MQTT_ADDRESS
strncpy(moduleConfig.mqtt.address, USERPREFS_MQTT_ADDRESS, sizeof(moduleConfig.mqtt.address));
#else
strncpy(moduleConfig.mqtt.address, default_mqtt_address, sizeof(moduleConfig.mqtt.address));
#endif
#ifdef USERPREFS_MQTT_USERNAME
strncpy(moduleConfig.mqtt.username, USERPREFS_MQTT_USERNAME, sizeof(moduleConfig.mqtt.username));
#else
strncpy(moduleConfig.mqtt.username, default_mqtt_username, sizeof(moduleConfig.mqtt.username));
#endif
#ifdef USERPREFS_MQTT_PASSWORD
strncpy(moduleConfig.mqtt.password, USERPREFS_MQTT_PASSWORD, sizeof(moduleConfig.mqtt.password));
#else
strncpy(moduleConfig.mqtt.password, default_mqtt_password, sizeof(moduleConfig.mqtt.password));
#endif
#ifdef USERPREFS_MQTT_ROOT_TOPIC
strncpy(moduleConfig.mqtt.root, USERPREFS_MQTT_ROOT_TOPIC, sizeof(moduleConfig.mqtt.root));
#else
strncpy(moduleConfig.mqtt.root, default_mqtt_root, sizeof(moduleConfig.mqtt.root));
#endif
#ifdef USERPREFS_MQTT_ENCRYPTION_ENABLED
moduleConfig.mqtt.encryption_enabled = USERPREFS_MQTT_ENCRYPTION_ENABLED;
#else
moduleConfig.mqtt.encryption_enabled = default_mqtt_encryption_enabled;
#endif
#ifdef USERPREFS_MQTT_TLS_ENABLED
moduleConfig.mqtt.tls_enabled = USERPREFS_MQTT_TLS_ENABLED;
#else
@@ -1253,11 +1346,13 @@ void NodeDB::initModuleConfigIntervals()
#else
moduleConfig.telemetry.device_update_interval = MAX_INTERVAL;
#endif
#ifdef USERPREFS_CONFIG_ENV_TELEM_UPDATE_INTERVAL
moduleConfig.telemetry.environment_update_interval = USERPREFS_CONFIG_ENV_TELEM_UPDATE_INTERVAL;
#else
moduleConfig.telemetry.environment_update_interval = 0;
#endif
#ifdef USERPREFS_CONFIG_ENVIRONMENT_MEASUREMENT_ENABLED
moduleConfig.telemetry.environment_measurement_enabled = USERPREFS_CONFIG_ENVIRONMENT_MEASUREMENT_ENABLED;
#endif
@@ -1302,6 +1397,10 @@ void NodeDB::resetNodes(bool keepFavorites)
std::fill(nodeDatabase.nodes.begin() + 1, nodeDatabase.nodes.end(), meshtastic_NodeInfoLite());
}
(void)ourNum;
#if WARM_NODE_COUNT > 0
warmStore.clear(); // warm entries are never favorites; a DB reset clears them too
#endif
devicestate.has_rx_waypoint = false;
saveNodeDatabaseToDisk();
saveDeviceStateToDisk();
@@ -1323,6 +1422,11 @@ void NodeDB::removeNodeByNum(NodeNum nodeNum)
meshtastic_NodeInfoLite());
if (removed)
eraseNodeSatellites(nodeNum);
#if WARM_NODE_COUNT > 0
// Explicit user removal: don't let the warm tier resurrect the node
warmStore.remove(nodeNum);
#endif
LOG_DEBUG("NodeDB::removeNodeByNum purged %d entries. Save changes", removed);
saveNodeDatabaseToDisk();
}
@@ -1436,6 +1540,7 @@ void NodeDB::setNodeStatus(NodeNum n, const meshtastic_StatusMessage &status)
(void)status;
#else
concurrency::LockGuard guard(&satelliteMutex);
evictSatelliteOverCap(*this, nodeStatus, n);
nodeStatus[n] = status;
#endif
}
@@ -1447,6 +1552,7 @@ void NodeDB::touchNodePositionTime(NodeNum n, uint32_t time)
(void)time;
#else
concurrency::LockGuard guard(&satelliteMutex);
evictSatelliteOverCap(*this, nodePositions, n);
nodePositions[n].time = time;
#endif
}
@@ -1457,23 +1563,65 @@ void NodeDB::eraseNodeSatellites(NodeNum n)
#if !MESHTASTIC_EXCLUDE_POSITIONDB
nodePositions.erase(n);
#endif
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
nodeTelemetry.erase(n);
#endif
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
nodeEnvironment.erase(n);
#endif
#if !MESHTASTIC_EXCLUDE_STATUSDB
nodeStatus.erase(n);
#endif
}
bool NodeDB::enforceSatelliteCaps()
{
concurrency::LockGuard guard(&satelliteMutex);
bool trimmedAny = false;
auto trim = [this, &trimmedAny](auto &map, const char *name) {
const size_t before = map.size();
while (map.size() > MAX_SATELLITE_NODES) {
if (!evictStalestSatellite(*this, map))
break;
}
if (map.size() != before) {
trimmedAny = true;
LOG_MIGRATION("Trimmed %s satellites %u -> %u (cap %d)", name, (unsigned)before, (unsigned)map.size(),
MAX_SATELLITE_NODES);
}
};
#if !MESHTASTIC_EXCLUDE_POSITIONDB
trim(nodePositions, "position");
#endif
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
trim(nodeTelemetry, "telemetry");
#endif
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
trim(nodeEnvironment, "environment");
#endif
#if !MESHTASTIC_EXCLUDE_STATUSDB
trim(nodeStatus, "status");
#endif
(void)trim; // all four maps may be compiled out
return trimmedAny;
}
void NodeDB::cleanupMeshDB()
{
int newPos = 0, removed = 0;
for (int i = 0; i < numMeshNodes; i++) {
meshtastic_NodeInfoLite &n = meshNodes->at(i);
if (nodeInfoLiteHasUser(&n)) {
// Keep ignored (blocked) nodes even without user info: a block set by
// bare node ID has no NodeInfo and would otherwise be purged here,
// silently dropping the block.
if (nodeInfoLiteHasUser(&n) || nodeInfoLiteIsIgnored(&n)) {
if (n.public_key.size > 0) {
if (memfll(n.public_key.bytes, 0, n.public_key.size)) {
n.public_key.size = 0;
@@ -1486,8 +1634,16 @@ void NodeDB::cleanupMeshDB()
} else {
// No user info - drop this node and its satellites
const NodeNum gone = n.num;
if (gone)
if (gone) {
#if WARM_NODE_COUNT > 0
// Keep any key we learned (e.g. via a DM before the NodeInfo
// exchange completed) rather than losing it with the purge.
if (n.public_key.size == 32)
warmStore.absorb(gone, n.last_heard, n.public_key.bytes);
#endif
eraseNodeSatellites(gone);
}
removed++;
}
}
@@ -1518,12 +1674,15 @@ void NodeDB::installDefaultDeviceState()
#else
snprintf(owner.long_name, sizeof(owner.long_name), "Meshtastic %04x", getNodeNum() & 0x0ffff);
#endif
clampLongName(owner.long_name); // vendor userprefs may exceed the local cap
#ifdef USERPREFS_CONFIG_OWNER_SHORT_NAME
snprintf(owner.short_name, sizeof(owner.short_name), (const char *)USERPREFS_CONFIG_OWNER_SHORT_NAME);
#else
snprintf(owner.short_name, sizeof(owner.short_name), "%04x", getNodeNum() & 0x0ffff);
#endif
snprintf(owner.id, sizeof(owner.id), "!%08x", getNodeNum()); // Default node ID now based on nodenum
memcpy(owner.macaddr, ourMacAddr, sizeof(owner.macaddr));
owner.has_is_unmessagable = true;
@@ -1637,6 +1796,114 @@ LoadFileResult NodeDB::loadProto(const char *filename, size_t protoSize, size_t
return state;
}
#if WARM_NODE_COUNT > 0
void NodeDB::demoteOldestHotNodesToWarm()
{
const int keep = MAX_NUM_NODES;
if (numMeshNodes <= keep)
return;
// Protected nodes (favorite/ignored/verified) outrank recency and are demoted
// only when the store is full of them; within a class, most-recently-heard
// wins. Index 0 is self and stays put (sort from +1), as in runtime eviction.
std::sort(meshNodes->begin() + 1, meshNodes->begin() + numMeshNodes,
[](const meshtastic_NodeInfoLite &a, const meshtastic_NodeInfoLite &b) {
const bool ka = nodeInfoLiteIsProtected(&a);
const bool kb = nodeInfoLiteIsProtected(&b);
if (ka != kb)
return ka;
return a.last_heard > b.last_heard;
});
int demoted = 0;
for (int i = keep; i < numMeshNodes; i++) {
const meshtastic_NodeInfoLite &n = (*meshNodes)[i];
if (n.num == 0)
continue;
// Keep the public key if we have one (40 B warm record); keyless nodes
// still get a placeholder so re-admission restores last_heard.
warmStore.absorb(n.num, n.last_heard, n.public_key.size > 0 ? n.public_key.bytes : nullptr);
// Demotion drops the node from the header table, so drop its satellites
// too (the eviction chokepoint) — they'd otherwise orphan until the next
// enforceSatelliteCaps pass.
eraseNodeSatellites(n.num);
demoted++;
}
numMeshNodes = keep; // the resize() in loadFromDisk reclaims the demoted tail
LOG_MIGRATION("NodeDB migration: demoted %d node(s) over %d into the warm tier (keepers preferred)", demoted, keep);
}
#endif
void NodeDB::nodeDBSelfCare()
{
if (!meshNodes)
return;
const NodeNum self = getNodeNum();
const bool nodesOverCap = numMeshNodes > MAX_NUM_NODES;
// Confirm self is present and its key matches what we just (re)derived. A
// non-empty DB that doesn't contain us means a foreign/over-cap or corrupt
// nodes.proto was loaded; an empty DB is just a fresh device (no warning).
meshtastic_NodeInfoLite *selfNode = getMeshNode(self);
if (!selfNode && numMeshNodes > 0) {
LOG_WARN("NodeDB self-care: self 0x%08x absent from DB, re-adding", (unsigned)self);
} else if (selfNode && owner.public_key.size == 32 && selfNode->public_key.size == 32 &&
memcmp(selfNode->public_key.bytes, owner.public_key.bytes, 32) != 0) {
LOG_WARN("NodeDB self-care: self 0x%08x key mismatch, refreshing", (unsigned)self);
}
// Maintenance that must never touch self. Pin self to index 0 first so
// the positional demote/eviction scans (which skip index 0) provably exclude
// us, wherever the loaded file happened to place our row.
if (selfNode && numMeshNodes > 0 && selfNode != &meshNodes->at(0)) {
std::swap(meshNodes->at(0), *selfNode);
}
#if WARM_NODE_COUNT > 0
if (nodesOverCap)
demoteOldestHotNodesToWarm(); // demotes oldest NON-self overflow; index 0 (us) left in place
#endif
if (numMeshNodes > MAX_NUM_NODES) {
LOG_WARN("NodeDB self-care: %d over cap %d, truncating", numMeshNodes, MAX_NUM_NODES);
numMeshNodes = MAX_NUM_NODES;
}
// Normalise the backing store to the hot cap so getOrCreateMeshNode always
// has spare slots to append into (it indexes meshNodes->at(numMeshNodes++)).
meshNodes->resize(MAX_NUM_NODES);
const bool satsTrimmed = enforceSatelliteCaps();
// Ensure self exists, sits at index 0, and carries current owner info — after
// any demotion has freed a slot. Covers the foreign/fixture case where the
// loaded file did not contain us at all.
meshtastic_NodeInfoLite *info = getOrCreateMeshNode(self);
if (info) {
TypeConversions::CopyUserToNodeInfoLite(info, owner);
if (info != &meshNodes->at(0))
std::swap(meshNodes->at(0), *info);
}
// One-shot rewrite: only when we healed something, and never while storage
// is locked — a locked boot loads placeholder defaults that must not be written
// over the encrypted store; reloadFromDisk() re-runs self-care once unlocked.
#ifdef MESHTASTIC_ENCRYPTED_STORAGE
const bool storageLocked = EncryptedStorage::isLockdownActive() && !EncryptedStorage::isUnlocked();
#else
const bool storageLocked = false;
#endif
if ((nodesOverCap || satsTrimmed) && !storageLocked) {
LOG_MIGRATION("NodeDB self-care: healed store (nodes-over-cap:%s sats-trimmed:%s); rewriting nodes.proto once",
nodesOverCap ? "yes" : "no", satsTrimmed ? "yes" : "no");
saveNodeDatabaseToDisk();
#if WARM_NODE_COUNT > 0
warmStore.saveIfDirty();
#endif
}
}
void NodeDB::loadFromDisk()
{
// Mark the current device state as completely unusable, so that if we fail reading the entire file from
@@ -1650,6 +1917,8 @@ void NodeDB::loadFromDisk()
storageCorruptThisLoad = false;
#endif
migrationSavePending = false;
meshtastic_Config_SecurityConfig backupSecurity = meshtastic_Config_SecurityConfig_init_zero;
#ifdef ARCH_ESP32
@@ -1659,6 +1928,7 @@ void NodeDB::loadFromDisk()
rmDir("/static/static"); // Remove bad static web files bundle from initial 2.5.13 release
spiLock->unlock();
#endif
#ifdef FSCom
#if defined(FACTORY_INSTALL) && !defined(ARCH_PORTDUINO)
spiLock->lock();
@@ -1673,7 +1943,7 @@ void NodeDB::loadFromDisk()
}
}
spiLock->unlock();
#endif
#endif // FACTORY_INSTALL, not PORTDUINO
spiLock->lock();
if (FSCom.exists(legacyPrefFileName)) {
spiLock->unlock();
@@ -1691,7 +1961,8 @@ void NodeDB::loadFromDisk()
spiLock->unlock();
}
#endif
#endif // FSCom
#ifdef MESHTASTIC_ENCRYPTED_STORAGE
// Only take the locked-boot defaults path when lockdown is ACTIVE (the
// device is provisioned) AND storage is still locked. A lockdown-capable
@@ -1745,7 +2016,7 @@ void NodeDB::loadFromDisk()
installDefaultNodeDatabase();
} else if (nodeDatabase.version < DEVICESTATE_CUR_VER) {
if (migrateLegacyNodeDatabase())
saveNodeDatabaseToDisk();
migrationSavePending = true;
else
installDefaultNodeDatabase();
} else {
@@ -1758,33 +2029,40 @@ void NodeDB::loadFromDisk()
#else
0u;
#endif
const unsigned telCount =
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
(unsigned)nodeTelemetry.size();
#else
0u;
#endif
const unsigned envCount =
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
(unsigned)nodeEnvironment.size();
#else
0u;
#endif
const unsigned statusCount =
#if !MESHTASTIC_EXCLUDE_STATUSDB
(unsigned)nodeStatus.size();
#else
0u;
#endif
LOG_INFO("Loaded saved nodedatabase v%d: %d nodes, %u pos, %u tel, %u env, %u status", nodeDatabase.version,
nodeDatabase.nodes.size(), posCount, telCount, envCount, statusCount);
}
if (numMeshNodes > MAX_NUM_NODES) {
LOG_WARN("Node count %d exceeds MAX_NUM_NODES %d, truncating", numMeshNodes, MAX_NUM_NODES);
numMeshNodes = MAX_NUM_NODES;
}
meshNodes->resize(MAX_NUM_NODES);
// Left UNTRIMMED on purpose: trim/demote/satellite-cap/self-pin/rewrite all
// run in nodeDBSelfCare() once getNodeNum() is valid (still 0 here on a cold
// boot, so we could only assume index 0 == self — the very bug being fixed).
#if WARM_NODE_COUNT > 0
// Load the warm tier so its on-disk snapshot is available before the node DB
// is exercised (and before nodeDBSelfCare() demotes any overflow into it).
warmStore.load();
#endif
// static DeviceState scratch; We no longer read into a tempbuf because this structure is 15KB of valuable RAM
state = loadProto(deviceStateFileName, meshtastic_DeviceState_size, sizeof(meshtastic_DeviceState),
@@ -1856,18 +2134,23 @@ void NodeDB::loadFromDisk()
#ifdef USERPREFS_CONFIG_LORA_REGION
config.lora.region = USERPREFS_CONFIG_LORA_REGION;
#endif
#ifdef USERPREFS_LORACONFIG_USE_PRESET
config.lora.use_preset = USERPREFS_LORACONFIG_USE_PRESET;
#endif
#ifdef USERPREFS_LORACONFIG_BANDWIDTH
config.lora.bandwidth = USERPREFS_LORACONFIG_BANDWIDTH;
#endif
#ifdef USERPREFS_LORACONFIG_SPREAD_FACTOR
config.lora.spread_factor = USERPREFS_LORACONFIG_SPREAD_FACTOR;
#endif
#ifdef USERPREFS_LORACONFIG_CODING_RATE
config.lora.coding_rate = USERPREFS_LORACONFIG_CODING_RATE;
#endif
#ifdef USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY
config.lora.override_frequency = USERPREFS_LORACONFIG_OVERRIDE_FREQUENCY;
#endif
@@ -1884,6 +2167,7 @@ void NodeDB::loadFromDisk()
#if defined(USERPREFS_USE_ADMIN_KEY_0) || defined(USERPREFS_USE_ADMIN_KEY_1) || defined(USERPREFS_USE_ADMIN_KEY_2)
uint16_t sum = 0;
#endif
#ifdef USERPREFS_USE_ADMIN_KEY_0
for (uint8_t b = 0; b < 32; b++) {
@@ -2076,6 +2360,17 @@ bool NodeDB::reloadFromDisk()
return false;
}
// loadFromDisk() leaves the store untrimmed; run self-care now (getNodeNum()
// is valid at runtime) to trim/demote non-self overflow, pin self to index 0
// and normalise the backing store before the node DB is exercised again.
nodeDBSelfCare();
// Preserve constructor ordering: persist any migration only after self-care.
if (migrationSavePending) {
saveNodeDatabaseToDisk();
migrationSavePending = false;
}
// Push the now-real config to the radio.
if (rIface) {
channels.onConfigChanged();
@@ -2196,6 +2491,7 @@ bool NodeDB::saveChannelsToDisk()
FSCom.mkdir("/prefs");
spiLock->unlock();
#endif
return saveProto(channelFileName, meshtastic_ChannelFile_size, &meshtastic_ChannelFile_msg, &channelFile);
}
@@ -2251,6 +2547,7 @@ bool NodeDB::saveNodeDatabaseToDisk()
FSCom.mkdir("/prefs");
spiLock->unlock();
#endif
// Project the maps into the on-disk vectors just before encoding; cleared
// again on the way out so we don't carry duplicate state.
concurrency::LockGuard guard(&satelliteMutex);
@@ -2267,6 +2564,7 @@ bool NodeDB::saveNodeDatabaseToDisk()
#else
nodeDatabase.positions.clear();
#endif
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
nodeDatabase.telemetry.clear();
nodeDatabase.telemetry.reserve(nodeTelemetry.size());
@@ -2280,6 +2578,7 @@ bool NodeDB::saveNodeDatabaseToDisk()
#else
nodeDatabase.telemetry.clear();
#endif
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
nodeDatabase.environment.clear();
nodeDatabase.environment.reserve(nodeEnvironment.size());
@@ -2293,6 +2592,7 @@ bool NodeDB::saveNodeDatabaseToDisk()
#else
nodeDatabase.environment.clear();
#endif
#if !MESHTASTIC_EXCLUDE_STATUSDB
nodeDatabase.status.clear();
nodeDatabase.status.reserve(nodeStatus.size());
@@ -2319,6 +2619,11 @@ bool NodeDB::saveNodeDatabaseToDisk()
nodeDatabase.environment.shrink_to_fit();
nodeDatabase.status.clear();
nodeDatabase.status.shrink_to_fit();
#if WARM_NODE_COUNT > 0
// Same cadence as the node DB; failure is logged but must not propagate —
// a false return from here would trigger saveToDisk()'s fsFormat() path.
warmStore.saveIfDirty();
#endif
return ok;
}
@@ -2354,6 +2659,7 @@ bool NodeDB::saveToDiskNoRetry(int saveWhat)
FSCom.mkdir("/prefs");
spiLock->unlock();
#endif
if (saveWhat & SEGMENT_CONFIG) {
config.has_device = true;
config.has_display = true;
@@ -2555,6 +2861,7 @@ void NodeDB::updatePosition(uint32_t nodeId, const meshtastic_Position &p, RxSou
#else
{
concurrency::LockGuard guard(&satelliteMutex);
evictSatelliteOverCap(*this, nodePositions, nodeId);
meshtastic_PositionLite &slot = nodePositions[nodeId]; // creates default-zero entry if missing
if (src == RX_SRC_LOCAL) {
@@ -2612,8 +2919,10 @@ void NodeDB::updateTelemetry(uint32_t nodeId, const meshtastic_Telemetry &t, RxS
}
#if !MESHTASTIC_EXCLUDE_TELEMETRYDB
concurrency::LockGuard guard(&satelliteMutex);
evictSatelliteOverCap(*this, nodeTelemetry, nodeId);
nodeTelemetry[nodeId] = t.variant.device_metrics;
#endif
} else if (t.which_variant == meshtastic_Telemetry_environment_metrics_tag) {
if (src == RX_SRC_LOCAL) {
LOG_DEBUG("updateTelemetry LOCAL env");
@@ -2622,8 +2931,10 @@ void NodeDB::updateTelemetry(uint32_t nodeId, const meshtastic_Telemetry &t, RxS
}
#if !MESHTASTIC_EXCLUDE_ENVIRONMENTDB
concurrency::LockGuard guard(&satelliteMutex);
evictSatelliteOverCap(*this, nodeEnvironment, nodeId);
nodeEnvironment[nodeId] = t.variant.environment_metrics;
#endif
} else {
return; // air_quality / power / local_stats / health: not stored per-node
}
@@ -2652,13 +2963,13 @@ void NodeDB::addFromContact(meshtastic_SharedContact contact)
info->num = contact.node_num;
TypeConversions::CopyUserToNodeInfoLite(info, contact.user);
if (contact.should_ignore) {
// If should_ignore is set,
// we need to clear the public key and other cruft, in addition to setting the node as ignored
nodeInfoLiteSetBit(info, NODEINFO_BITFIELD_IS_IGNORED_MASK, true);
// Block the contact and drop its rich satellite data, but keep the
// public key copied above — an ignored peer keeps a usable identity
// (a verifiable target) rather than a bare node number.
if (!setProtectedFlag(info, NODEINFO_BITFIELD_IS_IGNORED_MASK, true))
LOG_WARN(PROTECTED_CAP_WARN_FMT, "ignore", contact.node_num, MAX_NUM_NODES - 2);
nodeInfoLiteSetBit(info, NODEINFO_BITFIELD_IS_FAVORITE_MASK, false);
eraseNodeSatellites(contact.node_num);
info->public_key.size = 0;
memset(info->public_key.bytes, 0, sizeof(info->public_key.bytes));
} else {
/* Clients are sending add_contact before every text message DM (because clients may hold a larger node database with
* public keys than the radio holds). However, we don't want to update last_heard just because we sent someone a DM!
@@ -2677,12 +2988,16 @@ void NodeDB::addFromContact(meshtastic_SharedContact contact)
} else {
// Normal case: set is_favorite to prevent expiration.
// last_heard will remain as-is (or remain 0 if this entry wasn't in the nodeDB).
nodeInfoLiteSetBit(info, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true);
// If the protected cap refuses the favorite, fall back to stamping last_heard so the
// contact still isn't the first eviction victim.
if (!setProtectedFlag(info, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true))
info->last_heard = getTime();
}
// As the clients will begin sending the contact with DMs, we want to strictly check if the node is manually verified
if (contact.manually_verified) {
nodeInfoLiteSetBit(info, NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_MASK, true);
if (!setProtectedFlag(info, NODEINFO_BITFIELD_IS_KEY_MANUALLY_VERIFIED_MASK, true))
LOG_WARN(PROTECTED_CAP_WARN_FMT, "verify", contact.node_num, MAX_NUM_NODES - 2);
}
// Mark the node's key as manually verified to indicate trustworthiness.
updateGUIforNode = info;
@@ -2815,14 +3130,48 @@ void NodeDB::updateFrom(const meshtastic_MeshPacket &mp)
}
}
void NodeDB::set_favorite(bool is_favorite, uint32_t nodeId)
int NodeDB::numProtectedNodes() const
{
int count = 0;
for (int i = 0; i < numMeshNodes; i++)
if (nodeInfoLiteIsProtected(&meshNodes->at(i)))
count++;
return count;
}
bool NodeDB::setProtectedFlag(meshtastic_NodeInfoLite *node, uint32_t mask, bool on)
{
if (!node)
return false;
if (!on) {
nodeInfoLiteSetBit(node, mask, false);
return true;
}
// Adding a flag to a node that is already protected doesn't grow the
// protected set, so it's always allowed. A newly-protected node is refused
// once the protected set has reached MAX_NUM_NODES-2, leaving two evictable
// slots so getOrCreateMeshNode can always make room.
if (nodeInfoLiteIsProtected(node) || numProtectedNodes() < MAX_NUM_NODES - 2) {
nodeInfoLiteSetBit(node, mask, true);
return true;
}
return false;
}
bool NodeDB::set_favorite(bool is_favorite, uint32_t nodeId)
{
meshtastic_NodeInfoLite *lite = getMeshNode(nodeId);
if (lite && nodeInfoLiteIsFavorite(lite) != is_favorite) {
nodeInfoLiteSetBit(lite, NODEINFO_BITFIELD_IS_FAVORITE_MASK, is_favorite);
if (!lite)
return false;
if (nodeInfoLiteIsFavorite(lite) == is_favorite)
return true; // already in the requested state
if (setProtectedFlag(lite, NODEINFO_BITFIELD_IS_FAVORITE_MASK, is_favorite)) {
sortMeshDB();
saveNodeDatabaseToDisk();
return true;
}
LOG_WARN(PROTECTED_CAP_WARN_FMT, "favorite", nodeId, MAX_NUM_NODES - 2);
return false;
}
bool NodeDB::isFavorite(uint32_t nodeId)
@@ -2952,6 +3301,30 @@ bool NodeDB::isFull()
return (numMeshNodes >= MAX_NUM_NODES) || (memGet.getFreeHeap() < MINIMUM_SAFE_FREE_HEAP);
}
uint32_t NodeDB::hotNodeLastHeard(NodeNum n) const
{
for (int i = 0; i < numMeshNodes; i++)
if (meshNodes->at(i).num == n)
return meshNodes->at(i).last_heard;
return 0;
}
bool NodeDB::copyPublicKey(NodeNum n, meshtastic_NodeInfoLite_public_key_t &out)
{
const meshtastic_NodeInfoLite *info = getMeshNode(n);
if (info && info->public_key.size == 32) {
out = info->public_key;
return true;
}
#if WARM_NODE_COUNT > 0
if (warmStore.copyKey(n, out.bytes)) {
out.size = 32;
return true;
}
#endif
return false;
}
/// Find a node in our DB, create an empty NodeInfo if missing
meshtastic_NodeInfoLite *NodeDB::getOrCreateMeshNode(NodeNum n)
{
@@ -2988,7 +3361,14 @@ meshtastic_NodeInfoLite *NodeDB::getOrCreateMeshNode(NodeNum n)
}
if (oldestIndex != -1) {
eraseNodeSatellites(meshNodes->at(oldestIndex).num);
const meshtastic_NodeInfoLite &evicted = meshNodes->at(oldestIndex);
#if WARM_NODE_COUNT > 0
// Demote to the warm tier so the identity (and crucially the
// PKI key) outlives the hot-store slot.
warmStore.absorb(evicted.num, evicted.last_heard,
evicted.public_key.size == 32 ? evicted.public_key.bytes : NULL);
#endif
eraseNodeSatellites(evicted.num);
// Shove the remaining nodes down the chain
for (int i = oldestIndex; i < numMeshNodes - 1; i++) {
meshNodes->at(i) = meshNodes->at(i + 1);
@@ -2996,12 +3376,33 @@ meshtastic_NodeInfoLite *NodeDB::getOrCreateMeshNode(NodeNum n)
(numMeshNodes)--;
}
}
// Don't append past the end of the vector. The protected-node cap
// (numProtectedNodes() <= MAX_NUM_NODES-2) means the eviction above frees
// a slot in normal operation; this guards the legacy case of a pre-cap
// database that is full of protected nodes — refuse rather than overrun.
if (numMeshNodes >= MAX_NUM_NODES)
return NULL;
// Pre-size before append when run before nodeDBSelfCare() (boot keygen); else at() aborts on nRF52.
if (static_cast<size_t>(numMeshNodes) >= meshNodes->size())
meshNodes->resize(numMeshNodes + 1);
// add the node at the end
lite = &meshNodes->at((numMeshNodes)++);
// everything is missing except the nodenum
memset(lite, 0, sizeof(*lite));
lite->num = n;
#if WARM_NODE_COUNT > 0
// Re-admission: restore what the warm tier kept for this node
WarmNodeEntry warm;
if (warmStore.take(n, warm)) {
lite->last_heard = warm.last_heard;
if (!memfll(warm.public_key, 0, sizeof(warm.public_key))) {
lite->public_key.size = 32;
memcpy(lite->public_key.bytes, warm.public_key, 32);
}
LOG_MIGRATION("Rehydrated node 0x%x from warm tier (key=%d)", n, lite->public_key.size == 32);
}
#endif
LOG_INFO("Adding node to database with %i nodes and %u bytes free!", numMeshNodes, memGet.getFreeHeap());
}
src/mesh/NodeDB.h
+77 -9
View File
@@ -11,6 +11,7 @@
#include "MeshTypes.h"
#include "NodeStatus.h"
#include "WarmNodeStore.h"
#include "concurrency/Lock.h"
#include "configuration.h"
#include "mesh-pb-constants.h"
@@ -226,9 +227,25 @@ class NodeDB
bool updateUser(uint32_t nodeId, meshtastic_User &p, uint8_t channelIndex = 0);
/*
* Sets a node either favorite or unfavorite
* Sets a node either favorite or unfavorite. Returns true if the node ends
* up in the requested state; false if the node is unknown or favouriting
* was refused by the protected-node cap (MAX_NUM_NODES - 2).
*/
void set_favorite(bool is_favorite, uint32_t nodeId);
bool set_favorite(bool is_favorite, uint32_t nodeId);
/// Count of eviction-protected (favourite/ignored/manually-verified) nodes.
int numProtectedNodes() const;
/// printf-style warning emitted when setProtectedFlag() refuses a node at
/// the cap. %s = verb (favorite/ignore), 0x%08x = node, %d = cap. Shared by
/// LOG_WARN here and AdminModule::sendWarning so the wording stays in sync.
static constexpr const char *PROTECTED_CAP_WARN_FMT = "Can't %s 0x%08x: protected-node limit (%d) reached";
/// Turn an eviction-protection flag (favourite/ignored/verified) on/off. Off
/// always succeeds; on returns false (no change) once the protected set hits
/// the cap (MAX_NUM_NODES-2), keeping >=2 always-evictable slots. Callers
/// surface the refusal to the user.
bool setProtectedFlag(meshtastic_NodeInfoLite *node, uint32_t mask, bool on);
/*
* Returns true if the node is in the NodeDB and marked as favorite
@@ -295,6 +312,24 @@ class NodeDB
virtual meshtastic_NodeInfoLite *getMeshNode(NodeNum n);
size_t getNumMeshNodes() { return numMeshNodes; }
/// Find a node in our DB, create an empty NodeInfoLite if missing (evicting
/// the oldest non-protected node when full). Public so admin handlers can
/// register a node we have not heard from yet (e.g. to block it by ID).
meshtastic_NodeInfoLite *getOrCreateMeshNode(NodeNum n);
#if WARM_NODE_COUNT > 0
// Warm ("long-tail") tier: minimal {num, last_heard, public_key} records
// for nodes evicted from the hot store. See WarmNodeStore.h.
WarmNodeStore warmStore;
#endif
/// Copy the 32-byte public key for node n — hot store first, then the warm
/// tier. Returns false if we don't know a key for n.
bool copyPublicKey(NodeNum n, meshtastic_NodeInfoLite_public_key_t &out);
/// last_heard of a hot-store node, or 0 if absent. Plain scan of meshNodes
/// with no allocation side effects (unlike getOrCreateMeshNode).
uint32_t hotNodeLastHeard(NodeNum n) const;
// Thread-safe satellite-map accessors. Return false if absent or the
// corresponding DB is compiled out.
@@ -341,11 +376,15 @@ class NodeDB
emptyNodeDatabase.version = DEVICESTATE_CUR_VER;
size_t nodeDatabaseSize;
pb_get_encoded_size(&nodeDatabaseSize, meshtastic_NodeDatabase_fields, &emptyNodeDatabase);
// Always include satellite slots so backups from higher-cap peers
// decode without truncation, even when our build excludes the DBs.
return nodeDatabaseSize + (MAX_NUM_NODES * meshtastic_NodeInfoLite_size) +
(MAX_NUM_NODES * meshtastic_NodePositionEntry_size) + (MAX_NUM_NODES * meshtastic_NodeTelemetryEntry_size) +
(MAX_NUM_NODES * meshtastic_NodeEnvironmentEntry_size) + (MAX_NUM_NODES * meshtastic_NodeStatusEntry_size);
// Decode-stream size ceiling only — no buffer this big is allocated (load
// streams from the file). Sized for the largest file any prior firmware
// could write (250-node ESP32-S3, satellites uncapped) so capacity
// downgrades / peer backups still decode; excess is trimmed after load.
// (not constexpr: portduino resolves MAX_NUM_NODES from runtime config)
const size_t loadCeiling = ((size_t)MAX_NUM_NODES > 250) ? (size_t)MAX_NUM_NODES : 250;
return nodeDatabaseSize + (loadCeiling * meshtastic_NodeInfoLite_size) +
(loadCeiling * meshtastic_NodePositionEntry_size) + (loadCeiling * meshtastic_NodeTelemetryEntry_size) +
(loadCeiling * meshtastic_NodeEnvironmentEntry_size) + (loadCeiling * meshtastic_NodeStatusEntry_size);
}
// returns true if the maximum number of nodes is reached or we are running low on memory
@@ -430,11 +469,10 @@ class NodeDB
mutable concurrency::Lock satelliteMutex;
bool duplicateWarned = false;
bool localPositionUpdatedSinceBoot = false;
bool migrationSavePending = false;
uint32_t lastNodeDbSave = 0; // when we last saved our db to flash
uint32_t lastBackupAttempt = 0; // when we last tried a backup automatically or manually
uint32_t lastSort = 0; // When last sorted the nodeDB
/// Find a node in our DB, create an empty NodeInfoLite if missing
meshtastic_NodeInfoLite *getOrCreateMeshNode(NodeNum n);
/*
* Internal boolean to track sorting paused
@@ -447,9 +485,33 @@ class NodeDB
/// read our db from flash
void loadFromDisk();
#ifdef PIO_UNIT_TESTING
// Grant the unit-test shim access to the private maintenance paths below
// (migration / cleanup / eviction) without relaxing production access.
friend class NodeDBTestShim;
#endif
/// purge db entries without user info
void cleanupMeshDB();
/// Trim each satellite map down to MAX_SATELLITE_NODES, dropping the
/// stalest entries (used after loading files written before the cap, or by
/// a build with a larger cap). Returns true iff anything was trimmed.
bool enforceSatelliteCaps();
/// Node-DB self-care; call only once identity is established (getNodeNum()
/// valid). Confirms self is present, trims/demotes only NON-self overflow, and
/// rewrites the store once when something changed (never while storage locked).
void nodeDBSelfCare();
#if WARM_NODE_COUNT > 0
/// A database from a larger-cap build (e.g. the pre-fork 150-node nRF52 store)
/// can exceed MAX_NUM_NODES on load. Rank the hot store, demote the oldest
/// overflow into the warm tier preserving {num, last_heard, public_key} so PKI
/// DMs survive instead of dropping on truncation.
void demoteOldestHotNodesToWarm();
#endif
/// Reinit device state from scratch (not loading from disk)
void installDefaultDeviceState(), installDefaultNodeDatabase(), installDefaultChannels(),
installDefaultConfig(bool preserveKey), installDefaultModuleConfig();
@@ -570,6 +632,12 @@ inline bool nodeInfoLiteHasXeddsaSigned(const meshtastic_NodeInfoLite *n)
{
return n && (n->bitfield & NODEINFO_BITFIELD_HAS_XEDDSA_SIGNED_MASK);
}
/// A node that the eviction/migration paths must not drop: a favourite, an
/// ignored (blocked) node, or a manually-verified key.
inline bool nodeInfoLiteIsProtected(const meshtastic_NodeInfoLite *n)
{
return nodeInfoLiteIsFavorite(n) || nodeInfoLiteIsIgnored(n) || nodeInfoLiteIsKeyManuallyVerified(n);
}
inline void nodeInfoLiteSetBit(meshtastic_NodeInfoLite *n, uint32_t mask, bool value)
{
src/mesh/NodeDBLegacyMigration.cpp
+3
View File
@@ -14,6 +14,7 @@
#include "configuration.h"
#include "mesh-pb-constants.h"
#include "mesh/generated/meshtastic/deviceonly_legacy.pb.h"
#include "meshUtils.h"
#include <algorithm>
#include <cstring>
@@ -88,8 +89,10 @@ bool NodeDB::migrateLegacyNodeDatabase()
slim.bitfield |= NODEINFO_BITFIELD_HAS_USER_MASK;
strncpy(slim.long_name, legacy.user.long_name, sizeof(slim.long_name));
slim.long_name[sizeof(slim.long_name) - 1] = '\0';
sanitizeUtf8(slim.long_name, sizeof(slim.long_name)); // replace bad bytes so nanopb encode never fails
strncpy(slim.short_name, legacy.user.short_name, sizeof(slim.short_name));
slim.short_name[sizeof(slim.short_name) - 1] = '\0';
sanitizeUtf8(slim.short_name, sizeof(slim.short_name)); // same — v24 names may contain non-UTF-8 bytes
slim.hw_model = legacy.user.hw_model;
slim.role = legacy.user.role;
if (legacy.user.is_licensed)
src/mesh/Router.cpp
+17 -13
View File
@@ -485,14 +485,16 @@ DecodeState perhapsDecode(meshtastic_MeshPacket *p)
bool decrypted = false;
ChannelIndex chIndex = 0;
#if !(MESHTASTIC_EXCLUDE_PKI)
// Attempt PKI decryption first
if (p->channel == 0 && isToUs(p) && p->to > 0 && !isBroadcast(p->to) && nodeDB->getMeshNode(p->from) != nullptr &&
nodeDB->getMeshNode(p->from)->public_key.size > 0 && nodeDB->getMeshNode(p->to) != nullptr &&
nodeDB->getMeshNode(p->to)->public_key.size > 0 && rawSize > MESHTASTIC_PKC_OVERHEAD) {
// Attempt PKI decryption first. The sender's key may come from the hot
// store or the warm tier (nodes evicted from the hot store keep their key
// there), so DMs from long-tail nodes still decrypt.
meshtastic_NodeInfoLite_public_key_t fromKey = {0, {0}};
if (p->channel == 0 && isToUs(p) && p->to > 0 && !isBroadcast(p->to) && nodeDB->copyPublicKey(p->from, fromKey) &&
nodeDB->getMeshNode(p->to) != nullptr && nodeDB->getMeshNode(p->to)->public_key.size > 0 &&
rawSize > MESHTASTIC_PKC_OVERHEAD) {
LOG_DEBUG("Attempt PKI decryption");
if (crypto->decryptCurve25519(p->from, nodeDB->getMeshNode(p->from)->public_key, p->id, rawSize, p->encrypted.bytes,
bytes)) {
if (crypto->decryptCurve25519(p->from, fromKey, p->id, rawSize, p->encrypted.bytes, bytes)) {
LOG_INFO("PKI Decryption worked!");
meshtastic_Data decodedtmp;
@@ -503,7 +505,7 @@ DecodeState perhapsDecode(meshtastic_MeshPacket *p)
decrypted = true;
LOG_INFO("Packet decrypted using PKI!");
p->pki_encrypted = true;
memcpy(&p->public_key.bytes, nodeDB->getMeshNode(p->from)->public_key.bytes, 32);
memcpy(p->public_key.bytes, fromKey.bytes, 32);
p->public_key.size = 32;
p->decoded = decodedtmp;
p->which_payload_variant = meshtastic_MeshPacket_decoded_tag; // change type to decoded
@@ -720,7 +722,10 @@ meshtastic_Routing_Error perhapsEncode(meshtastic_MeshPacket *p)
ChannelIndex chIndex = p->channel; // keep as a local because we are about to change it
#if !(MESHTASTIC_EXCLUDE_PKI)
meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(p->to);
// Destination key from the hot store or the warm tier (evicted
// long-tail nodes keep their key there)
meshtastic_NodeInfoLite_public_key_t destKey = {0, {0}};
bool haveDestKey = nodeDB->copyPublicKey(p->to, destKey);
// We may want to retool things so we can send a PKC packet when the client specifies a key and nodenum, even if the node
// is not in the local nodedb
// First, only PKC encrypt packets we are originating
@@ -743,18 +748,17 @@ meshtastic_Routing_Error perhapsEncode(meshtastic_MeshPacket *p)
if (numbytes + MESHTASTIC_HEADER_LENGTH + MESHTASTIC_PKC_OVERHEAD > MAX_LORA_PAYLOAD_LEN)
return meshtastic_Routing_Error_TOO_LARGE;
// Check for a known public key for the destination
if (node == nullptr || node->public_key.size != 32) {
if (!haveDestKey) {
LOG_WARN("Unknown public key for destination node 0x%08x (portnum %d), refusing to send legacy DM", p->to,
p->decoded.portnum);
return meshtastic_Routing_Error_PKI_SEND_FAIL_PUBLIC_KEY;
}
if (p->pki_encrypted && !memfll(p->public_key.bytes, 0, 32) &&
memcmp(p->public_key.bytes, node->public_key.bytes, 32) != 0) {
if (p->pki_encrypted && !memfll(p->public_key.bytes, 0, 32) && memcmp(p->public_key.bytes, destKey.bytes, 32) != 0) {
LOG_WARN("Client public key differs from requested: 0x%02x, stored key begins 0x%02x", *p->public_key.bytes,
*node->public_key.bytes);
*destKey.bytes);
return meshtastic_Routing_Error_PKI_FAILED;
}
crypto->encryptCurve25519(p->to, getFrom(p), node->public_key, p->id, numbytes, bytes, p->encrypted.bytes);
crypto->encryptCurve25519(p->to, getFrom(p), destKey, p->id, numbytes, bytes, p->encrypted.bytes);
numbytes += MESHTASTIC_PKC_OVERHEAD;
p->channel = 0;
p->pki_encrypted = true;
src/mesh/WarmNodeStore.cpp
+548
View File
@@ -0,0 +1,548 @@
#include "WarmNodeStore.h"
#if WARM_NODE_COUNT > 0
#include "FSCommon.h"
#include "SPILock.h"
#include "SafeFile.h"
#include "configuration.h"
#include "power/PowerHAL.h"
#include <ErriezCRC32.h>
#include <vector>
#if defined(NRF52840_XXAA)
#include "flash/flash_nrf5x.h"
#define WARM_RING_MAGIC 0x474E5257u // "WRNG"
// A tombstone is an entry record whose last_heard is all-ones — getTime()
// (unix seconds) cannot reach 0xFFFFFFFF until 2106, and erased flash is
// detected via num == 0xFFFFFFFF before last_heard is ever inspected.
#define WARM_RING_TOMBSTONE 0xFFFFFFFFu
#else
// warm.dat layout: this header followed by count packed WarmNodeEntry records.
struct WarmStoreHeader {
uint32_t magic; // WARM_STORE_MAGIC
uint32_t reserved; // 0; kept so the header stays 16 B
uint16_t count; // entries persisted
uint16_t entrySize; // sizeof(WarmNodeEntry), format guard
uint32_t crc; // crc32 over count * entrySize bytes
};
static_assert(sizeof(WarmStoreHeader) == 16, "header layout is part of the persistence format");
#define WARM_STORE_MAGIC 0x314D5257u // "WRM1"
#ifdef FSCom
static const char *warmFileName = "/prefs/warm.dat";
#endif
#endif // NRF52840_XXAA
static inline bool keyIsSet(const uint8_t key[32])
{
for (int i = 0; i < 32; i++)
if (key[i])
return true;
return false;
}
WarmNodeStore::WarmNodeStore()
{
#if defined(ARCH_ESP32) && defined(BOARD_HAS_PSRAM)
entries = static_cast<WarmNodeEntry *>(ps_calloc(WARM_NODE_COUNT, sizeof(WarmNodeEntry)));
if (!entries) {
LOG_WARN("WarmStore: PSRAM alloc failed, using heap");
entries = static_cast<WarmNodeEntry *>(calloc(WARM_NODE_COUNT, sizeof(WarmNodeEntry)));
}
#else
entries = static_cast<WarmNodeEntry *>(calloc(WARM_NODE_COUNT, sizeof(WarmNodeEntry)));
#endif
#if defined(NRF52840_XXAA)
memset(pageOf, kNoPage, sizeof(pageOf));
#endif
}
WarmNodeStore::~WarmNodeStore()
{
free(entries); // always malloc-family (calloc / ps_calloc)
entries = nullptr;
}
WarmNodeEntry *WarmNodeStore::find(NodeNum num) const
{
if (!entries || !num)
return nullptr;
for (size_t i = 0; i < WARM_NODE_COUNT; i++)
if (entries[i].num == num)
return &entries[i];
return nullptr;
}
// Slot placement with the keyed-first admission policy. Shared by absorb()
// and the ring replay, so the policy is applied identically in both paths.
WarmNodeEntry *WarmNodeStore::place(NodeNum num, uint32_t lastHeard, const uint8_t *key32)
{
if (!entries || !num)
return nullptr;
const bool candidateKeyed = key32 && keyIsSet(key32);
WarmNodeEntry *slot = find(num);
const bool sameNode = slot != nullptr;
if (!slot) {
// Pick a victim: any empty slot, else the oldest keyless entry, else
// (only for keyed candidates) the oldest keyed entry.
WarmNodeEntry *oldestKeyless = nullptr, *oldestKeyed = nullptr;
for (size_t i = 0; i < WARM_NODE_COUNT; i++) {
WarmNodeEntry &e = entries[i];
if (!e.num) {
slot = &e;
break;
}
if (keyIsSet(e.public_key)) {
if (!oldestKeyed || e.last_heard < oldestKeyed->last_heard)
oldestKeyed = &e;
} else {
if (!oldestKeyless || e.last_heard < oldestKeyless->last_heard)
oldestKeyless = &e;
}
}
if (!slot)
slot = oldestKeyless ? oldestKeyless : (candidateKeyed ? oldestKeyed : nullptr);
if (!slot)
return nullptr; // store full of keyed entries and the candidate has no key
}
slot->num = num;
slot->last_heard = lastHeard;
if (candidateKeyed)
memcpy(slot->public_key, key32, 32);
else if (!sameNode)
// Repurposing a victim slot for a different node: clear its stale key.
// A keyless refresh of a node already here keeps the key we learned.
memset(slot->public_key, 0, 32);
return slot;
}
bool WarmNodeStore::absorb(NodeNum num, uint32_t lastHeard, const uint8_t *key32)
{
const WarmNodeEntry *slot = place(num, lastHeard, key32);
if (!slot)
return false;
persistEntry(*slot);
LOG_MIGRATION("WarmStore absorb 0x%08x key=%d last_heard=%u (now %u/%u)", (unsigned)num, keyIsSet(slot->public_key) ? 1 : 0,
(unsigned)lastHeard, (unsigned)count(), (unsigned)capacity());
return true;
}
bool WarmNodeStore::take(NodeNum num, WarmNodeEntry &out)
{
WarmNodeEntry *e = find(num);
if (!e)
return false;
out = *e;
const int idx = static_cast<int>(e - entries);
memset(e, 0, sizeof(*e));
persistRemove(num, idx);
LOG_MIGRATION("WarmStore take(rehydrate) 0x%08x key=%d (now %u/%u)", (unsigned)num, keyIsSet(out.public_key) ? 1 : 0,
(unsigned)count(), (unsigned)capacity());
return true;
}
#if MESHTASTIC_NODEDB_MIGRATION_VERBOSE
void WarmNodeStore::dumpToLog(const char *reason) const
{
if (!entries) {
LOG_MIGRATION("WarmStore dump (%s): backend not allocated", reason);
return;
}
LOG_MIGRATION("WarmStore dump (%s): %u live / %u cap ==>", reason, (unsigned)count(), (unsigned)capacity());
unsigned shown = 0;
for (size_t i = 0; i < WARM_NODE_COUNT; i++) {
const WarmNodeEntry &e = entries[i];
if (e.num == 0)
continue;
LOG_MIGRATION(" warm[%3u] 0x%08x last_heard=%u key=%d", (unsigned)i, (unsigned)e.num, (unsigned)e.last_heard,
keyIsSet(e.public_key) ? 1 : 0);
shown++;
}
LOG_MIGRATION("WarmStore dump (%s): <== end (%u entries)", reason, shown);
}
#endif // MESHTASTIC_NODEDB_MIGRATION_VERBOSE
bool WarmNodeStore::copyKey(NodeNum num, uint8_t out[32]) const
{
const WarmNodeEntry *e = find(num);
if (!e || !keyIsSet(e->public_key))
return false;
memcpy(out, e->public_key, 32);
return true;
}
bool WarmNodeStore::contains(NodeNum num) const
{
return find(num) != nullptr;
}
void WarmNodeStore::remove(NodeNum num)
{
WarmNodeEntry *e = find(num);
if (e) {
const int idx = static_cast<int>(e - entries);
memset(e, 0, sizeof(*e));
persistRemove(num, idx);
}
}
void WarmNodeStore::clear()
{
if (!entries)
return;
memset(entries, 0, WARM_NODE_COUNT * sizeof(WarmNodeEntry));
#if defined(NRF52840_XXAA)
memset(pageOf, kNoPage, sizeof(pageOf));
#endif
persistClear();
}
size_t WarmNodeStore::count() const
{
size_t n = 0;
if (entries)
for (size_t i = 0; i < WARM_NODE_COUNT; i++)
if (entries[i].num)
n++;
return n;
}
bool WarmNodeStore::saveIfDirty()
{
if (!dirty)
return true;
bool ok = save();
if (ok)
dirty = false;
return ok;
}
#if defined(NRF52840_XXAA)
// Raw-flash record-ring backend (nRF52840).
// 3 × 4 KB pages below LittleFS. Mutations append 40 B records (entry snapshot,
// or tombstone with last_heard == 0xFFFFFFFF) via the shared flash_nrf5x page
// cache; saveIfDirty() is the durability point. A full page reclaims the oldest
// (stranded live entries re-appended, then erased). Flash access holds spiLock —
// the page cache is shared with InternalFS/LittleFS.
bool WarmNodeStore::ringReadHeader(uint8_t page, WarmPageHeader &h) const
{
flash_nrf5x_read(&h, WARM_FLASH_PAGE_ADDR(page), sizeof(h));
return h.magic == WARM_RING_MAGIC && h.seq != 0xFFFFFFFFu;
}
// Caller holds spiLock.
void WarmNodeStore::ringOpenPage(uint8_t page)
{
// Drop any cached state for the page before the real erase, so a later
// cache flush can't resurrect stale bytes.
flash_nrf5x_flush();
flash_nrf5x_erase(WARM_FLASH_PAGE_ADDR(page));
WarmPageHeader h;
h.magic = WARM_RING_MAGIC;
h.seq = nextSeq++;
flash_nrf5x_write(WARM_FLASH_PAGE_ADDR(page), &h, sizeof(h));
activePage = page;
writeSlot = 0;
}
// Caller holds spiLock. May recurse once via ringAppend if the stranded set
// fills the fresh page exactly — bounded by WARM_NODE_COUNT <= 2*kRecordsPerPage.
void WarmNodeStore::ringRotate()
{
uint8_t target = 0;
if (activePage != kNoPage) {
// Lowest-seq valid page, preferring erased pages; never the active one
uint32_t bestSeq = 0;
bool found = false;
for (uint8_t p = 0; p < WARM_FLASH_PAGES; p++) {
if (p == activePage)
continue;
WarmPageHeader h;
if (!ringReadHeader(p, h)) {
target = p; // erased/invalid page: free real estate, take it
found = true;
break;
}
if (!found || static_cast<int32_t>(h.seq - bestSeq) < 0) {
target = p;
bestSeq = h.seq;
found = true;
}
}
}
// Capture live entries stranded in the page we're about to erase
int stranded[WARM_NODE_COUNT] = {};
int nStranded = 0;
for (size_t i = 0; i < WARM_NODE_COUNT; i++) {
if (entries[i].num && pageOf[i] == target)
stranded[nStranded++] = static_cast<int>(i);
if (pageOf[i] == target)
pageOf[i] = kNoPage;
}
ringOpenPage(target);
for (int k = 0; k < nStranded; k++)
ringAppend(entries[stranded[k]], stranded[k]);
}
// Caller holds spiLock.
void WarmNodeStore::ringAppend(const WarmNodeEntry &rec, int storeSlot)
{
if (activePage == kNoPage || writeSlot >= kRecordsPerPage)
ringRotate();
const uint32_t addr =
WARM_FLASH_PAGE_ADDR(activePage) + sizeof(WarmPageHeader) + static_cast<uint32_t>(writeSlot) * sizeof(WarmNodeEntry);
flash_nrf5x_write(addr, &rec, sizeof(rec));
writeSlot++;
if (storeSlot >= 0)
pageOf[storeSlot] = activePage;
dirty = true;
}
void WarmNodeStore::persistEntry(const WarmNodeEntry &e)
{
concurrency::LockGuard g(spiLock);
ringAppend(e, static_cast<int>(&e - entries));
}
void WarmNodeStore::persistRemove(NodeNum num, int storeSlot)
{
if (storeSlot >= 0 && storeSlot < static_cast<int>(WARM_NODE_COUNT))
pageOf[storeSlot] = 0xFF;
WarmNodeEntry tomb;
memset(&tomb, 0, sizeof(tomb));
tomb.num = num;
tomb.last_heard = WARM_RING_TOMBSTONE;
concurrency::LockGuard g(spiLock);
ringAppend(tomb, -1);
}
void WarmNodeStore::persistClear()
{
concurrency::LockGuard g(spiLock);
flash_nrf5x_flush();
for (uint8_t p = 0; p < WARM_FLASH_PAGES; p++)
flash_nrf5x_erase(WARM_FLASH_PAGE_ADDR(p));
activePage = 0xFF;
writeSlot = 0;
nextSeq = 1;
dirty = false; // the erased ring already reflects the empty store
}
void WarmNodeStore::load()
{
if (!entries)
return;
concurrency::LockGuard g(spiLock);
// Order valid pages by ascending seq so replay applies oldest first
uint8_t order[WARM_FLASH_PAGES] = {};
uint32_t seqs[WARM_FLASH_PAGES] = {};
uint8_t nValid = 0;
uint8_t nCorrupt = 0;
for (uint8_t p = 0; p < WARM_FLASH_PAGES; p++) {
WarmPageHeader h;
if (!ringReadHeader(p, h)) {
// An erased page reads back all-ones; any other magic is a
// partially-written or bit-rotted header we're dropping, so flag it
// rather than silently treating the loss as a clean empty ring.
if (h.magic != 0xFFFFFFFFu)
nCorrupt++;
continue;
}
uint8_t pos = nValid;
while (pos > 0 && static_cast<int32_t>(h.seq - seqs[pos - 1]) < 0) {
order[pos] = order[pos - 1];
seqs[pos] = seqs[pos - 1];
pos--;
}
order[pos] = p;
seqs[pos] = h.seq;
nValid++;
}
if (nValid == 0) {
activePage = 0xFF;
writeSlot = 0;
nextSeq = 1;
if (nCorrupt)
LOG_WARN("WarmStore: ring unreadable (%u bad page(s)), empty", nCorrupt);
else
LOG_INFO("WarmStore: ring empty, starting fresh");
return;
}
uint32_t replayed = 0;
for (uint8_t k = 0; k < nValid; k++) {
const uint8_t p = order[k];
uint16_t slot = 0;
for (; slot < kRecordsPerPage; slot++) {
WarmNodeEntry rec;
flash_nrf5x_read(&rec, WARM_FLASH_PAGE_ADDR(p) + sizeof(WarmPageHeader) + (uint32_t)slot * sizeof(rec), sizeof(rec));
if (rec.num == 0xFFFFFFFFu)
break; // erased space: end of this page's records (append-only)
if (rec.num == 0)
continue; // unexpected; skip defensively
replayed++;
if (rec.last_heard == WARM_RING_TOMBSTONE) {
WarmNodeEntry *e = find(rec.num);
if (e) {
pageOf[e - entries] = 0xFF;
memset(e, 0, sizeof(*e));
}
} else {
const WarmNodeEntry *e = place(rec.num, rec.last_heard, rec.public_key);
if (e)
pageOf[e - entries] = p;
}
}
if (k == nValid - 1) { // newest page becomes the active head
activePage = p;
writeSlot = slot;
nextSeq = seqs[k] + 1;
}
}
if (nCorrupt)
LOG_WARN("WarmStore: dropped %u corrupt ring page(s), some nodes lost", nCorrupt);
LOG_INFO("WarmStore: replayed %u ring records -> %u live nodes (page %u, slot %u)", (unsigned)replayed, (unsigned)count(),
activePage, writeSlot);
}
bool WarmNodeStore::save()
{
if (!powerHAL_isPowerLevelSafe()) {
LOG_ERROR("Error: trying to save WarmStore on unsafe device power level.");
return false;
}
concurrency::LockGuard g(spiLock);
flash_nrf5x_flush();
return true;
}
#else // !NRF52840_XXAA --------------------
void WarmNodeStore::persistEntry(const WarmNodeEntry &e)
{
(void)e;
dirty = true;
}
void WarmNodeStore::persistRemove(NodeNum num, int storeSlot)
{
(void)num;
(void)storeSlot;
dirty = true;
}
void WarmNodeStore::persistClear()
{
dirty = true;
}
#ifdef FSCom
// ---- File persistence: /prefs/warm.dat snapshots ----------------------------
// Compact occupied slots to the front of `dst`; returns the count.
static uint16_t packEntries(const WarmNodeEntry *src, WarmNodeEntry *dst)
{
uint16_t n = 0;
for (size_t i = 0; i < WARM_NODE_COUNT; i++)
if (src[i].num)
dst[n++] = src[i];
return n;
}
void WarmNodeStore::load()
{
if (!entries)
return;
// Clear first — all failure paths below then correctly represent "empty",
// even if load() is called on an already-used instance.
memset(entries, 0, WARM_NODE_COUNT * sizeof(WarmNodeEntry));
concurrency::LockGuard g(spiLock);
auto f = FSCom.open(warmFileName, FILE_O_READ);
if (!f)
return;
WarmStoreHeader h;
if ((size_t)f.read((uint8_t *)&h, sizeof(h)) != sizeof(h)) {
f.close();
LOG_WARN("WarmStore: %s header read failed, starting empty", warmFileName);
return;
}
if (h.magic != WARM_STORE_MAGIC || h.entrySize != sizeof(WarmNodeEntry) || h.count > WARM_NODE_COUNT) {
f.close();
LOG_WARN("WarmStore: %s header invalid (magic=0x%08x entrySize=%u count=%u), starting empty", warmFileName, h.magic,
h.entrySize, h.count);
return;
}
if (h.count) {
const size_t len = (size_t)h.count * sizeof(WarmNodeEntry);
const bool readOk = (size_t)f.read((uint8_t *)entries, len) == len;
f.close();
if (!readOk) {
LOG_WARN("WarmStore: %s entries read failed, starting empty", warmFileName);
return;
}
if (crc32Buffer(entries, len) != h.crc) {
LOG_WARN("WarmStore: %s CRC mismatch, starting empty", warmFileName);
return;
}
} else {
f.close();
}
LOG_INFO("WarmStore: loaded %u warm nodes from %s", h.count, warmFileName);
}
bool WarmNodeStore::save()
{
if (!entries)
return false;
if (!powerHAL_isPowerLevelSafe()) {
LOG_ERROR("Error: trying to save WarmStore on unsafe device power level.");
return false;
}
std::vector<WarmNodeEntry> packed(WARM_NODE_COUNT);
WarmStoreHeader h;
h.magic = WARM_STORE_MAGIC;
h.reserved = 0;
h.count = packEntries(entries, packed.data());
h.entrySize = sizeof(WarmNodeEntry);
h.crc = crc32Buffer(packed.data(), h.count * sizeof(WarmNodeEntry));
concurrency::LockGuard g(spiLock);
FSCom.mkdir("/prefs");
auto f = SafeFile(warmFileName, false);
f.write((const uint8_t *)&h, sizeof(h));
f.write((const uint8_t *)packed.data(), h.count * sizeof(WarmNodeEntry));
bool ok = f.close();
if (!ok)
LOG_ERROR("WarmStore: can't write %s", warmFileName);
else
LOG_DEBUG("WarmStore: saved %u warm nodes to %s", h.count, warmFileName);
return ok;
}
#else
void WarmNodeStore::load() {}
bool WarmNodeStore::save()
{
return true;
}
#endif // FSCom
#endif // NRF52840_XXAA
#endif // WARM_NODE_COUNT > 0
src/mesh/WarmNodeStore.h
+130
View File
@@ -0,0 +1,130 @@
#pragma once
#include "MeshTypes.h"
#include "mesh-pb-constants.h"
#include <stdint.h>
#include <string.h>
// Verbose tracing for the warm-store migration + NodeDB self-care. Per-event /
// per-boot chatter routes through this so it can be silenced in one place (set
// to 0) once they're proven; genuine LOG_WARN anomalies stay unconditional.
#ifndef MESHTASTIC_NODEDB_MIGRATION_VERBOSE
#define MESHTASTIC_NODEDB_MIGRATION_VERBOSE 0
#endif
#if MESHTASTIC_NODEDB_MIGRATION_VERBOSE
#define LOG_MIGRATION(...) LOG_INFO(__VA_ARGS__)
#else
#define LOG_MIGRATION(...) ((void)0)
#endif
#if WARM_NODE_COUNT > 0
/**
* Warm ("long-tail") node tier.
*
* Minimal identity record (NodeNum, last_heard, Curve25519 public key) for nodes
* evicted from the hot NodeInfoLite store, so DMs to/from them keep encrypting —
* the key is expensive to re-learn, the rest rebuilds from traffic in seconds.
* Flat fixed array, linear scan (only on hot-store misses), LRU by last_heard
* with keyed entries outranking keyless.
*
* Persistence: nRF52840 uses a 12 KB raw-flash record-ring below LittleFS
* (append + replay + compact-on-rotate — see the backend in WarmNodeStore.cpp,
* link-guarded by nrf52840_s140_v7.ld). Everywhere else: /prefs/warm.dat.
*/
struct WarmNodeEntry {
NodeNum num; // 0 = empty slot
uint32_t last_heard; // recency for LRU ordering
uint8_t public_key[32]; // all-zero = no key (a real key is never all-zero)
};
static_assert(sizeof(WarmNodeEntry) == 40, "WarmNodeEntry must stay 40 B — persistence format depends on it");
// Gated on NRF52840_XXAA: the ring sits at 0xEA000
// valid only on the 1 MB-flash nRF52840.
#if defined(NRF52840_XXAA)
#define WARM_FLASH_PAGE_SIZE 4096u
#define WARM_FLASH_PAGES 3u
#define WARM_FLASH_REGION_BASE (0xED000u - WARM_FLASH_PAGES * WARM_FLASH_PAGE_SIZE) // 0xEA000
#define WARM_FLASH_PAGE_ADDR(i) (WARM_FLASH_REGION_BASE + (i)*WARM_FLASH_PAGE_SIZE)
#endif
class WarmNodeStore
{
public:
WarmNodeStore();
~WarmNodeStore();
WarmNodeStore(const WarmNodeStore &) = delete;
WarmNodeStore &operator=(const WarmNodeStore &) = delete;
/// Remember an evicted hot node. Keyless candidates never displace keyed
/// entries; otherwise the oldest (keyless-first) entry is replaced.
/// @return true if the node was stored or updated
bool absorb(NodeNum num, uint32_t lastHeard, const uint8_t *key32 /* may be NULL */);
/// Find and remove an entry (used when the node is re-admitted to the hot store).
bool take(NodeNum num, WarmNodeEntry &out);
/// Copy the 32-byte public key for a node, if we have one.
bool copyKey(NodeNum num, uint8_t out[32]) const;
bool contains(NodeNum num) const;
void remove(NodeNum num);
void clear();
size_t count() const;
size_t capacity() const { return entries ? WARM_NODE_COUNT : 0; }
#if MESHTASTIC_NODEDB_MIGRATION_VERBOSE
/// Debug: dump every live warm entry (num / last_heard / has-key) to the
/// console. Compiled out unless MESHTASTIC_NODEDB_MIGRATION_VERBOSE.
void dumpToLog(const char *reason = "dump") const;
#endif
/// Load persisted entries (called once at boot, after the node DB loads).
void load();
/// Durability point, piggybacked on the node-database save cadence. On the
/// ring backend this flushes the shared flash page cache; on the file
/// backend it writes the warm.dat snapshot.
bool saveIfDirty();
private:
WarmNodeEntry *entries = nullptr; // WARM_NODE_COUNT slots; PSRAM on ESP32 when available
bool dirty = false;
WarmNodeEntry *find(NodeNum num) const;
// Internal slot-placement shared by absorb() and ring replay: applies the
// keyed-first admission policy without touching persistence.
WarmNodeEntry *place(NodeNum num, uint32_t lastHeard, const uint8_t *key32);
// Persistence hooks called from the mutation paths. File backend: mark
// dirty. Ring backend: append an upsert/tombstone record (+ mark dirty).
void persistEntry(const WarmNodeEntry &e); // e must point into entries[]
void persistRemove(NodeNum num, int storeSlot);
void persistClear();
#if defined(NRF52840_XXAA)
// nRF52840 raw-flash record-ring state.
struct WarmPageHeader {
uint32_t magic; // WARM_RING_MAGIC
uint32_t seq; // page generation; 0xFFFFFFFF = erased/unused
};
static_assert(sizeof(WarmPageHeader) == 8, "page header is part of the flash format");
static constexpr uint16_t kRecordsPerPage = (WARM_FLASH_PAGE_SIZE - sizeof(WarmPageHeader)) / sizeof(WarmNodeEntry); // 102
static_assert(WARM_NODE_COUNT <= 2 * ((WARM_FLASH_PAGE_SIZE - 8) / 40), "live set must fit the ring with one page reclaimed");
static constexpr uint8_t kNoPage = 0xFF; // "no page" sentinel for activePage / pageOf[]
uint8_t activePage = kNoPage; // no page opened yet (fresh/erased ring)
uint16_t writeSlot = 0; // next free record slot in the active page
uint32_t nextSeq = 1; // seq for the next page opened
uint8_t pageOf[WARM_NODE_COUNT]; // flash page holding each RAM slot's newest record; kNoPage = none
void ringAppend(const WarmNodeEntry &rec, int storeSlot /* -1 for tombstones */);
void ringRotate(); // reclaim oldest page, compacting stranded live entries
void ringOpenPage(uint8_t page); // erase + write header (seq = nextSeq++)
bool ringReadHeader(uint8_t page, WarmPageHeader &h) const;
#endif
bool save();
};
#endif // WARM_NODE_COUNT > 0
src/mesh/mesh-pb-constants.h
+50 -16
View File
@@ -48,39 +48,40 @@ static_assert(sizeof(meshtastic_NodeInfoLite) <= 130, "NodeInfoLite size increas
#define MESHTASTIC_EXCLUDE_POSITIONDB 1
#else
#define MESHTASTIC_EXCLUDE_POSITIONDB 0
#endif
#endif
#endif // STM32WL
#endif // MESHTASTIC_EXCLUDE_POSITIONDB
#ifndef MESHTASTIC_EXCLUDE_TELEMETRYDB
#if defined(ARCH_STM32WL)
#define MESHTASTIC_EXCLUDE_TELEMETRYDB 1
#else
#define MESHTASTIC_EXCLUDE_TELEMETRYDB 0
#endif
#endif
#endif // STM32WL
#endif // MESHTASTIC_EXCLUDE_TELEMETRYDB
#ifndef MESHTASTIC_EXCLUDE_ENVIRONMENTDB
#if defined(ARCH_STM32WL)
#define MESHTASTIC_EXCLUDE_ENVIRONMENTDB 1
#else
#define MESHTASTIC_EXCLUDE_ENVIRONMENTDB 0
#endif
#endif
#endif // STM32WL
#endif // MESHTASTIC_EXCLUDE_ENVIRONMENTDB
#ifndef MESHTASTIC_EXCLUDE_STATUSDB
#if defined(ARCH_STM32WL) || defined(MESHTASTIC_EXCLUDE_STATUS)
#define MESHTASTIC_EXCLUDE_STATUSDB 1
#else
#define MESHTASTIC_EXCLUDE_STATUSDB 0
#endif
#endif
#endif // STM32WL
#endif // MESHTASTIC_EXCLUDE_STATUSDB
/// max number of nodes allowed in the nodeDB
/// Max nodes in the hot store (full NodeInfoLite). Evicted nodes' identities
/// live in the warm tier (WARM_NODE_COUNT). nRF52840 caps at 120 to keep
/// nodes.proto inside the stock 28 KB LittleFS; flash-rich platforms (ESP32-S3,
/// portduino) keep their larger hot store and lean on warm only for the tail.
#ifndef MAX_NUM_NODES
#if defined(ARCH_STM32WL)
#define MAX_NUM_NODES 10
#elif defined(ARCH_NRF52)
#define MAX_NUM_NODES 150
#elif defined(CONFIG_IDF_TARGET_ESP32S3)
#include "Esp.h"
static inline int get_max_num_nodes()
@@ -95,10 +96,43 @@ static inline int get_max_num_nodes()
}
}
#define MAX_NUM_NODES get_max_num_nodes()
#elif defined(ARCH_PORTDUINO)
#define MAX_NUM_NODES 250 // native host: no flash/RAM constraint; match the ESP32-S3 top tier
#else
#define MAX_NUM_NODES 100
#endif
#endif
#define MAX_NUM_NODES 120 // nRF52840 (28 KB LittleFS) and generic ESP32
#endif // platform
#endif // MAX_NUM_NODES
/// Per-map cap (position/telemetry/environment/status): only the freshest
/// MAX_SATELLITE_NODES nodes keep satellite payloads, the rest just the
/// NodeInfoLite header. RAM-bound (the maps are internal-SRAM, not PSRAM), so
/// flash-rich hosts get a cap >= their hot store (satellites for every node, as
/// before the cap existed) while constrained parts stay at 40.
#ifndef MAX_SATELLITE_NODES
#if defined(CONFIG_IDF_TARGET_ESP32S3) || defined(ARCH_PORTDUINO)
#define MAX_SATELLITE_NODES 250
#else
#define MAX_SATELLITE_NODES 40 // nRF52840 (28 KB LittleFS) and generic ESP32
#endif // platform
#endif // MAX_SATELLITE_NODES
/// Warm tier: 40 B {num, last_heard, public_key} records kept for evicted nodes
/// so DMs to/from them keep decrypting. 0 disables it; size is per-platform
/// below, persisted to /prefs/warm.dat (or the nRF52840 raw-flash ring).
#ifndef WARM_NODE_COUNT
#if defined(ARCH_STM32WL)
#define WARM_NODE_COUNT 0
#elif defined(NRF52840_XXAA)
// Keyed on the NRF52840_XXAA build flag, not ARCH_NRF52: the latter (from
// architecture.h via configuration.h) isn't defined this early in every include
// chain. Backed by the raw-flash ring below LittleFS — see WarmNodeStore.h.
#define WARM_NODE_COUNT 200
#elif defined(CONFIG_IDF_TARGET_ESP32S3)
#define WARM_NODE_COUNT 2000 // PSRAM-backed when available; warm.dat ~80 KB
#else
#define WARM_NODE_COUNT 320
#endif // platform
#endif // WARM_NODE_COUNT
/// Max number of channels allowed
#define MAX_NUM_CHANNELS (member_size(meshtastic_ChannelFile, channels) / member_size(meshtastic_ChannelFile, channels[0]))
@@ -125,8 +159,8 @@ static inline int get_max_num_nodes()
#define TRAFFIC_MANAGEMENT_CACHE_SIZE 1000
#else
#define TRAFFIC_MANAGEMENT_CACHE_SIZE 0
#endif
#endif
#endif // HAS_TRAFFIC_MANAGEMENT
#endif // TRAFFIC_MANAGEMENT_CACHE_SIZE
/// helper function for encoding a record as a protobuf, any failures to encode are fatal and we will panic
/// returns the encoded packet size
src/modules/AdminModule.cpp
+29 -12
View File
@@ -181,8 +181,11 @@ bool AdminModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshta
// without the user doing so deliberately.
LOG_INFO("PKC admin valid, but not auto-favoriting node %x because role==CLIENT_BASE", mp.from);
} else {
LOG_INFO("PKC admin valid. Auto-favoriting node %x", mp.from);
nodeInfoLiteSetBit(remoteNode, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true);
if (nodeDB->setProtectedFlag(remoteNode, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true)) {
LOG_INFO("PKC admin valid. Auto-favoriting node %x", mp.from);
} else {
LOG_WARN("PKC admin valid, but auto-favorite refused for node %x (protected-node cap)", mp.from);
}
}
}
} else {
@@ -472,10 +475,13 @@ bool AdminModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshta
LOG_INFO("Client received set_favorite_node command");
meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(r->set_favorite_node);
if (node != NULL) {
nodeInfoLiteSetBit(node, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true);
saveChanges(SEGMENT_NODEDATABASE, false);
if (screen)
screen->setFrames(graphics::Screen::FOCUS_PRESERVE); // <-- Rebuild screens
if (nodeDB->setProtectedFlag(node, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true)) {
saveChanges(SEGMENT_NODEDATABASE, false);
if (screen)
screen->setFrames(graphics::Screen::FOCUS_PRESERVE); // <-- Rebuild screens
} else if (mp.from == 0) { // local request from the phone — tell the user why it didn't take
sendWarning(NodeDB::PROTECTED_CAP_WARN_FMT, "favorite", r->set_favorite_node, MAX_NUM_NODES - 2);
}
}
break;
}
@@ -492,13 +498,17 @@ bool AdminModule::handleReceivedProtobuf(const meshtastic_MeshPacket &mp, meshta
}
case meshtastic_AdminMessage_set_ignored_node_tag: {
LOG_INFO("Client received set_ignored_node command");
meshtastic_NodeInfoLite *node = nodeDB->getMeshNode(r->set_ignored_node);
// Unlike the sibling node-targeted admin commands, create the entry if
// it's absent so the block sticks for a node we've not heard from yet
// (e.g. one a remote admin asks us to block) with no NodeInfo or key.
meshtastic_NodeInfoLite *node = nodeDB->getOrCreateMeshNode(r->set_ignored_node);
if (node != NULL) {
nodeInfoLiteSetBit(node, NODEINFO_BITFIELD_IS_IGNORED_MASK, true);
nodeDB->eraseNodeSatellites(node->num);
node->public_key.size = 0;
memset(node->public_key.bytes, 0, sizeof(node->public_key.bytes));
saveChanges(SEGMENT_NODEDATABASE, false);
if (nodeDB->setProtectedFlag(node, NODEINFO_BITFIELD_IS_IGNORED_MASK, true)) {
nodeDB->eraseNodeSatellites(node->num);
saveChanges(SEGMENT_NODEDATABASE, false);
} else if (mp.from == 0) { // local request from the phone — tell the user why it didn't take
sendWarning(NodeDB::PROTECTED_CAP_WARN_FMT, "ignore", r->set_ignored_node, MAX_NUM_NODES - 2);
}
}
break;
}
@@ -1405,6 +1415,13 @@ void AdminModule::handleGetNodeRemoteHardwarePins(const meshtastic_MeshPacket &r
void AdminModule::handleGetDeviceMetadata(const meshtastic_MeshPacket &req)
{
#if WARM_NODE_COUNT > 0 && MESHTASTIC_NODEDB_MIGRATION_VERBOSE
// Debug aid: dump the warm tier to the console on a local metadata request
// (e.g. `meshtastic --info` over USB/BLE). Gated to req.from == 0 so remote
// or admin polling can't spam the console.
if (nodeDB && req.from == 0)
nodeDB->warmStore.dumpToLog("admin get_metadata");
#endif
meshtastic_AdminMessage r = meshtastic_AdminMessage_init_default;
r.get_device_metadata_response = getDeviceMetadata();
r.which_payload_variant = meshtastic_AdminMessage_get_device_metadata_response_tag;
src/platform/nrf52/nrf52840_s140_v6.ld
+46
View File
@@ -0,0 +1,46 @@
/* Linker script to configure memory regions. */
SEARCH_DIR(.)
GROUP(-lgcc -lc -lnosys)
MEMORY
{
/* App region ends at 0xEA000, not 0xED000: the 12 KB warm-node-store
* record-ring (3 x 4 KB pages, src/mesh/WarmNodeStore.h) occupies
* 0xEA000-0xED000, directly below the stock LittleFS partition. Boards on
* the framework-default linker script are covered by the post-link guard
* in extra_scripts/nrf52_warm_region.py instead.
*
* S140 v6.x app region starts at 0x26000 (152 KB SoftDevice); v7.x uses
* 0x27000. All other boundaries are identical — see nrf52840_s140_v7.ld. */
FLASH (rx) : ORIGIN = 0x26000, LENGTH = 0xEA000 - 0x26000
/* SRAM required by Softdevice depend on
* - Attribute Table Size (Number of Services and Characteristics)
* - Vendor UUID count
* - Max ATT MTU
* - Concurrent connection peripheral + central + secure links
* - Event Len, HVN queue, Write CMD queue
*/
RAM (rwx) : ORIGIN = 0x20006000, LENGTH = 0x20040000 - 0x20006000
}
SECTIONS
{
. = ALIGN(4);
.svc_data :
{
PROVIDE(__start_svc_data = .);
KEEP(*(.svc_data))
PROVIDE(__stop_svc_data = .);
} > RAM
.fs_data :
{
PROVIDE(__start_fs_data = .);
KEEP(*(.fs_data))
PROVIDE(__stop_fs_data = .);
} > RAM
} INSERT AFTER .data;
INCLUDE "nrf52_common.ld"
src/platform/nrf52/nrf52840_s140_v7.ld
+6 -1
View File
@@ -5,7 +5,12 @@ GROUP(-lgcc -lc -lnosys)
MEMORY
{
FLASH (rx) : ORIGIN = 0x27000, LENGTH = 0xED000 - 0x27000
/* App region ends at 0xEA000, not 0xED000: the 12 KB warm-node-store
* record-ring (3 x 4 KB pages, src/mesh/WarmNodeStore.h) occupies
* 0xEA000-0xED000, directly below the stock LittleFS partition. Boards on
* the framework-default linker script are covered by the post-link guard
* in extra_scripts/nrf52_warm_region.py instead. */
FLASH (rx) : ORIGIN = 0x27000, LENGTH = 0xEA000 - 0x27000
/* SRAM required by Softdevice depend on
* - Attribute Table Size (Number of Services and Characteristics)
test/test_nodedb_blocked/test_main.cpp
+193
View File
@@ -0,0 +1,193 @@
// Tests for the NodeDB hot-store migration and favourite/ignored (blocked)
// retention paths — src/mesh/NodeDB.cpp.
#include "MeshTypes.h" // BEFORE TestUtil.h — provides WARM_NODE_COUNT / MAX_NUM_NODES via mesh-pb-constants.h
#include "TestUtil.h"
#include <unity.h>
#if defined(ARCH_PORTDUINO)
#define NDB_TEST_ENTRY extern "C"
#else
#define NDB_TEST_ENTRY
#endif
// The migration demotes overflow into the warm tier, so these tests need it.
#if WARM_NODE_COUNT > 0
#include "mesh/NodeDB.h"
#include <cstring>
// Subclass shim: exposes the private maintenance paths (via the friend
// declaration in NodeDB.h) and lets a test own the hot store directly
// (meshNodes/numMeshNodes are public). Declared at global scope so it matches
// `friend class NodeDBTestShim` — an anonymous-namespace class would not.
class NodeDBTestShim : public NodeDB
{
public:
void runDemote() { demoteOldestHotNodesToWarm(); }
void runCleanup() { cleanupMeshDB(); }
void clearHot()
{
meshNodes->clear();
numMeshNodes = 0;
}
void push(NodeNum num, uint32_t lastHeard, bool favorite, bool ignored, bool withUser, bool withKey)
{
meshtastic_NodeInfoLite n = meshtastic_NodeInfoLite_init_zero;
n.num = num;
n.last_heard = lastHeard;
if (favorite)
nodeInfoLiteSetBit(&n, NODEINFO_BITFIELD_IS_FAVORITE_MASK, true);
if (ignored)
nodeInfoLiteSetBit(&n, NODEINFO_BITFIELD_IS_IGNORED_MASK, true);
if (withUser)
nodeInfoLiteSetBit(&n, NODEINFO_BITFIELD_HAS_USER_MASK, true);
if (withKey) {
n.public_key.size = 32;
memset(n.public_key.bytes, static_cast<uint8_t>(num & 0xff), 32);
n.public_key.bytes[0] = 0x01; // ensure non-zero (all-zero == "no key")
}
meshNodes->push_back(n);
numMeshNodes = meshNodes->size();
}
// Index 0 is our own node; the eviction/migration scans treat it as self.
void seedSelf() { push(0x0BADF00D, 0xFFFFFFFFu, false, false, /*withUser=*/true, /*withKey=*/false); }
};
namespace
{
NodeDBTestShim *db = nullptr;
bool warmHasKey(NodeNum n)
{
meshtastic_NodeInfoLite_public_key_t k = {0, {0}};
return db->copyPublicKey(n, k) && k.size == 32;
}
} // namespace
void setUp(void)
{
db->clearHot();
}
void tearDown(void) {}
// Migration: a database from a larger-cap build trims to MAX_NUM_NODES; the
// oldest non-protected nodes are demoted into the warm tier (keys preserved),
// while self, favourites and ignored survive even when they are the oldest.
static void test_migration_demotesOldestKeepsKeepersAndSelf(void)
{
db->seedSelf();
const int extra = MAX_NUM_NODES + 30; // overflow well past the MAX-2 cap
for (int i = 1; i <= extra; i++) {
const bool fav = (i == 1); // oldest, but a favourite
const bool ign = (i == 2); // 2nd-oldest, but blocked
db->push(2000 + i, /*last_heard=*/i, fav, ign, /*withUser=*/true, /*withKey=*/true);
}
db->runDemote();
TEST_ASSERT_EQUAL_INT(MAX_NUM_NODES, (int)db->getNumMeshNodes());
TEST_ASSERT_NOT_NULL(db->getMeshNode(0x0BADF00D)); // self retained
TEST_ASSERT_NOT_NULL(db->getMeshNode(2000 + 1)); // oldest favourite retained
TEST_ASSERT_NOT_NULL(db->getMeshNode(2000 + 2)); // oldest ignored retained
TEST_ASSERT_NOT_NULL(db->getMeshNode(2000 + extra)); // freshest retained
TEST_ASSERT_NULL(db->getMeshNode(2000 + 3)); // oldest non-protected demoted out of hot
TEST_ASSERT_TRUE(warmHasKey(2000 + 3)); // ...but its key kept in the warm tier
}
// Favourite handling: a favourite is never the eviction victim, even when it is
// the oldest node in a full hot store.
static void test_eviction_preservesFavorite(void)
{
db->seedSelf();
for (int i = 1; i < MAX_NUM_NODES; i++) { // fill to MAX_NUM_NODES total (incl. self)
const bool fav = (i == 1); // oldest non-self, favourite
db->push(3000 + i, /*last_heard=*/i, fav, false, /*withUser=*/true, /*withKey=*/true);
}
TEST_ASSERT_EQUAL_INT(MAX_NUM_NODES, (int)db->getNumMeshNodes()); // full
TEST_ASSERT_NOT_NULL(db->getOrCreateMeshNode(0x99990000)); // forces an eviction
TEST_ASSERT_NOT_NULL(db->getMeshNode(3000 + 1)); // favourite survived despite being oldest
TEST_ASSERT_NULL(db->getMeshNode(3000 + 2)); // oldest non-favourite evicted
TEST_ASSERT_NOT_NULL(db->getMeshNode(0x99990000));
}
// Ignored handling: an ignored node survives eviction (like a favourite), and is
// never purged by cleanupMeshDB even with no user info (a block set by bare ID).
static void test_ignored_survivesEvictionAndCleanup(void)
{
// (a) eviction protection
db->clearHot();
db->seedSelf();
for (int i = 1; i < MAX_NUM_NODES; i++) {
const bool ign = (i == 1); // oldest non-self, blocked
db->push(4000 + i, /*last_heard=*/i, false, ign, /*withUser=*/true, /*withKey=*/true);
}
TEST_ASSERT_NOT_NULL(db->getOrCreateMeshNode(0x88880000));
TEST_ASSERT_NOT_NULL(db->getMeshNode(4000 + 1)); // blocked node survived
TEST_ASSERT_NULL(db->getMeshNode(4000 + 2)); // oldest non-blocked evicted
// (b) cleanup protection — ignored kept without user info, plain no-user purged
db->clearHot();
db->seedSelf();
db->push(5000, 100, false, /*ignored=*/true, /*withUser=*/false, false);
db->push(5001, 100, false, false, /*withUser=*/false, false);
db->runCleanup();
TEST_ASSERT_NOT_NULL(db->getMeshNode(5000)); // blocked-by-ID kept despite no user info
TEST_ASSERT_NULL(db->getMeshNode(5001)); // ordinary no-user node purged
}
// Protected-node cap: at most MAX_NUM_NODES-2 nodes may be protected, so >=2
// evictable slots always remain. setProtectedFlag refuses once the cap is hit.
static void test_protectedCap_refusesBeyondLimit(void)
{
db->seedSelf();
for (int i = 0; i < MAX_NUM_NODES - 2; i++)
db->push(6000 + i, 100, /*favorite=*/true, false, /*withUser=*/true, false);
TEST_ASSERT_EQUAL_INT(MAX_NUM_NODES - 2, db->numProtectedNodes());
db->push(7000, 100, false, false, /*withUser=*/true, false);
meshtastic_NodeInfoLite *fresh = db->getMeshNode(7000);
TEST_ASSERT_NOT_NULL(fresh);
TEST_ASSERT_FALSE(db->setProtectedFlag(fresh, NODEINFO_BITFIELD_IS_IGNORED_MASK, true)); // refused at cap
TEST_ASSERT_FALSE(nodeInfoLiteIsIgnored(fresh)); // unchanged
TEST_ASSERT_EQUAL_INT(MAX_NUM_NODES - 2, db->numProtectedNodes());
// Adding another flag to an already-protected node doesn't grow the set, so
// it's still allowed at the cap.
meshtastic_NodeInfoLite *already = db->getMeshNode(6000);
TEST_ASSERT_TRUE(db->setProtectedFlag(already, NODEINFO_BITFIELD_IS_IGNORED_MASK, true));
}
NDB_TEST_ENTRY void setup()
{
initializeTestEnvironment();
db = new NodeDBTestShim();
nodeDB = db;
UNITY_BEGIN();
RUN_TEST(test_migration_demotesOldestKeepsKeepersAndSelf);
RUN_TEST(test_eviction_preservesFavorite);
RUN_TEST(test_ignored_survivesEvictionAndCleanup);
RUN_TEST(test_protectedCap_refusesBeyondLimit);
exit(UNITY_END());
}
NDB_TEST_ENTRY void loop() {}
#else // WARM_NODE_COUNT == 0 — nothing to exercise here
void setUp(void) {}
void tearDown(void) {}
NDB_TEST_ENTRY void setup()
{
UNITY_BEGIN();
exit(UNITY_END());
}
NDB_TEST_ENTRY void loop() {}
#endif
test/test_warm_store/test_main.cpp
+211
View File
@@ -0,0 +1,211 @@
// Unit tests for the warm ("long-tail") node tier — src/mesh/WarmNodeStore.cpp.
// Covers admission/eviction policy (keyed entries outrank keyless), take()
// rehydration semantics, and a tolerant persistence round trip.
#include "MeshTypes.h" // BEFORE TestUtil.h — provides WARM_NODE_COUNT via mesh-pb-constants.h
#include "TestUtil.h"
#include <unity.h>
#if defined(ARCH_PORTDUINO)
#define WS_TEST_ENTRY extern "C"
#else
#define WS_TEST_ENTRY
#endif
#if WARM_NODE_COUNT > 0
#include "mesh/WarmNodeStore.h"
#include <cstring>
namespace
{
void makeKey(uint8_t out[32], uint8_t seed)
{
memset(out, 0, 32);
out[0] = seed;
out[31] = seed ^ 0xA5;
}
} // namespace
void setUp(void) {}
void tearDown(void) {}
void test_ws_absorb_and_copyKey_roundTrip()
{
WarmNodeStore ws;
uint8_t key[32], got[32];
makeKey(key, 7);
TEST_ASSERT_TRUE(ws.absorb(0x100, 1000, key));
TEST_ASSERT_TRUE(ws.contains(0x100));
TEST_ASSERT_TRUE(ws.copyKey(0x100, got));
TEST_ASSERT_EQUAL_MEMORY(key, got, 32);
TEST_ASSERT_EQUAL(1, ws.count());
}
void test_ws_keylessEntry_hasNoKey()
{
WarmNodeStore ws;
uint8_t got[32];
TEST_ASSERT_TRUE(ws.absorb(0x200, 1000, NULL));
TEST_ASSERT_TRUE(ws.contains(0x200));
TEST_ASSERT_FALSE(ws.copyKey(0x200, got));
}
void test_ws_absorb_rejectsNodeNumZero()
{
WarmNodeStore ws;
TEST_ASSERT_FALSE(ws.absorb(0, 1000, NULL));
TEST_ASSERT_EQUAL(0, ws.count());
}
void test_ws_absorb_updatesExistingEntry()
{
WarmNodeStore ws;
uint8_t key[32], got[32];
makeKey(key, 9);
TEST_ASSERT_TRUE(ws.absorb(0x300, 1000, NULL));
TEST_ASSERT_TRUE(ws.absorb(0x300, 2000, key)); // later eviction learned a key
TEST_ASSERT_EQUAL(1, ws.count());
TEST_ASSERT_TRUE(ws.copyKey(0x300, got));
TEST_ASSERT_EQUAL_MEMORY(key, got, 32);
}
void test_ws_take_removesEntry()
{
WarmNodeStore ws;
uint8_t key[32];
makeKey(key, 3);
ws.absorb(0x400, 1234, key);
WarmNodeEntry e;
TEST_ASSERT_TRUE(ws.take(0x400, e));
TEST_ASSERT_EQUAL(0x400, e.num);
TEST_ASSERT_EQUAL(1234, e.last_heard);
TEST_ASSERT_EQUAL_MEMORY(key, e.public_key, 32);
TEST_ASSERT_FALSE(ws.contains(0x400));
TEST_ASSERT_FALSE(ws.take(0x400, e));
TEST_ASSERT_EQUAL(0, ws.count());
}
void test_ws_keylessCandidate_neverEvictsKeyedEntries()
{
WarmNodeStore ws;
uint8_t key[32];
// Fill the store entirely with keyed entries
for (size_t i = 0; i < ws.capacity(); i++) {
makeKey(key, (uint8_t)i);
TEST_ASSERT_TRUE(ws.absorb(0x1000 + i, 100 + i, key));
}
TEST_ASSERT_EQUAL(ws.capacity(), ws.count());
// A keyless candidate (even a fresh one) must be rejected
TEST_ASSERT_FALSE(ws.absorb(0x9999, 999999, NULL));
TEST_ASSERT_FALSE(ws.contains(0x9999));
}
void test_ws_keyedCandidate_evictsOldestKeylessFirst()
{
WarmNodeStore ws;
uint8_t key[32];
makeKey(key, 0x42);
// Fill with keyed entries except two keyless ones in the middle
for (size_t i = 0; i < ws.capacity(); i++) {
const bool keyless = (i == 5 || i == 10);
TEST_ASSERT_TRUE(ws.absorb(0x1000 + i, keyless ? (i == 10 ? 50 : 60) : 10, keyless ? NULL : key));
}
// Keyed candidate must displace the OLDEST KEYLESS entry (0x100A, ts=50),
// even though every keyed entry is older (ts=10)
uint8_t k2[32];
makeKey(k2, 0x43);
TEST_ASSERT_TRUE(ws.absorb(0x8888, 70, k2));
TEST_ASSERT_FALSE(ws.contains(0x1000 + 10));
TEST_ASSERT_TRUE(ws.contains(0x1000 + 5));
TEST_ASSERT_TRUE(ws.contains(0x8888));
}
void test_ws_keyedCandidate_evictsOldestKeyedWhenNoKeyless()
{
WarmNodeStore ws;
uint8_t key[32];
for (size_t i = 0; i < ws.capacity(); i++) {
makeKey(key, (uint8_t)i);
TEST_ASSERT_TRUE(ws.absorb(0x1000 + i, 1000 + i, key)); // 0x1000 is the oldest
}
uint8_t k2[32];
makeKey(k2, 0x44);
TEST_ASSERT_TRUE(ws.absorb(0x7777, 999999, k2));
TEST_ASSERT_TRUE(ws.contains(0x7777));
TEST_ASSERT_FALSE(ws.contains(0x1000)); // oldest keyed evicted
TEST_ASSERT_EQUAL(ws.capacity(), ws.count());
}
void test_ws_remove_and_clear()
{
WarmNodeStore ws;
ws.absorb(0x500, 1, NULL);
ws.absorb(0x501, 2, NULL);
ws.remove(0x500);
TEST_ASSERT_FALSE(ws.contains(0x500));
TEST_ASSERT_EQUAL(1, ws.count());
ws.clear();
TEST_ASSERT_EQUAL(0, ws.count());
}
void test_ws_persistence_roundTrip()
{
WarmNodeStore a;
uint8_t key[32], got[32];
makeKey(key, 0x55);
a.absorb(0x600, 4242, key);
a.absorb(0x601, 4243, NULL);
if (!a.saveIfDirty()) {
TEST_IGNORE_MESSAGE("Filesystem not available in this test environment");
return;
}
WarmNodeStore b;
b.load();
TEST_ASSERT_TRUE(b.contains(0x600));
TEST_ASSERT_TRUE(b.contains(0x601));
TEST_ASSERT_TRUE(b.copyKey(0x600, got));
TEST_ASSERT_EQUAL_MEMORY(key, got, 32);
// Cleanup so reruns start fresh
b.clear();
b.saveIfDirty();
}
WS_TEST_ENTRY void setup()
{
initializeTestEnvironment();
UNITY_BEGIN();
RUN_TEST(test_ws_absorb_and_copyKey_roundTrip);
RUN_TEST(test_ws_keylessEntry_hasNoKey);
RUN_TEST(test_ws_absorb_rejectsNodeNumZero);
RUN_TEST(test_ws_absorb_updatesExistingEntry);
RUN_TEST(test_ws_take_removesEntry);
RUN_TEST(test_ws_keylessCandidate_neverEvictsKeyedEntries);
RUN_TEST(test_ws_keyedCandidate_evictsOldestKeylessFirst);
RUN_TEST(test_ws_keyedCandidate_evictsOldestKeyedWhenNoKeyless);
RUN_TEST(test_ws_remove_and_clear);
RUN_TEST(test_ws_persistence_roundTrip);
exit(UNITY_END());
}
WS_TEST_ENTRY void loop() {}
#else
void setUp(void) {}
void tearDown(void) {}
WS_TEST_ENTRY void setup()
{
initializeTestEnvironment();
UNITY_BEGIN();
exit(UNITY_END());
}
WS_TEST_ENTRY void loop() {}
#endif
variants/nrf52840/nrf52.ini
+1
View File
@@ -15,6 +15,7 @@ extra_scripts =
${env.extra_scripts}
extra_scripts/nrf52_extra.py
pre:extra_scripts/nrf52_lto.py
extra_scripts/nrf52_warm_region.py ; post-link guard: image must end below the 12 KB warm-store raw-flash region at 0xEA000-0xED000
build_type = release
build_flags =
variants/nrf52840/nrf52840.ini
+4
View File
@@ -1,6 +1,10 @@
[nrf52840_base]
extends = nrf52_base
; Cap the app image at 0xEA000 (below the WarmNodeStore raw-flash region).
; Boards that have upgraded to S140 v7 override this in their own platformio.ini.
board_build.ldscript = src/platform/nrf52/nrf52840_s140_v6.ld
build_flags =
${nrf52_base.build_flags}
-DSERIAL_BUFFER_SIZE=4096