Automatic variable hop limits based on mesh activity and size estimation (#10176)
* asdf * Implement SphereOfInfluenceModule for traffic management and eviction tracking * Implement Sphere of Influence module for dynamic hop limit adjustment and role-based floor * Update SAMPLING_DENOMINATOR to improve mesh size estimation accuracy * Add debug logging for scale factor estimation and per-hop node counts in SphereOfInfluenceModule * Enable variable hop limits and role-based hop floors in Sphere of Influence module * Respond to copilot review * Disable variable hop limits and role-based hop floors in Sphere of Influence module * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Implement adaptive sampling for unique node ID tracking in Sphere of Influence module * Add state persistence for Sphere of Influence module * Enhance Sphere of Influence module with state management and adaptive sampling adjustments * Refactor hop scaling functionality: remove SphereOfInfluenceModule and introduce HopScalingModule - Deleted SphereOfInfluenceModule.h, consolidating its responsibilities into the new HopScalingModule. - Added HopScalingModule.h and HopScalingModule.cpp to manage hop scaling logic, including eviction tracking and sampling-based mesh size estimation. - Implemented methods for recording evictions and packet senders, estimating scale factors, and computing required hops based on node activity. - Introduced state persistence for hop scaling parameters to maintain continuity across reboots. - Enhanced thread safety and modularity by utilizing concurrency features. * Guard out STM32. Sowwy. * Refactor HopScalingModule: enhance sampling logic and improve state management * Add unit tests for HopScalingModule: implement mock database and various test scenarios * Refactor test output in HopScalingModule tests: replace printf with TEST_MESSAGE for better integration with Unity * Refactor HopScalingModule logging: replace lastStatusMode with descriptive mode names for improved readability * Refactor test_main.cpp: change NodeNum variable to static and improve comments for clarity * Remove unnecessary delay in setup function for improved test performance * Add missing include for MeshTypes in test_main.cpp * Refactor HopScalingModule tests: enhance mesh topology scenarios and improve test clarity * Update HopScalingModule tests: flesh out node scenarios and improve clarity for dense and sparse mesh cases * Fix politeness factor calculations in HopScalingModule and update related test scenarios for clarity. Remove outdated design doc. * Enhance HopScalingModule: add sampled estimate for scaling decisions and refactor initial run state management * Add sample traffic injection for HopScaling tests to enhance sampledEst visibility * Enhance HopScalingModule: adjust windowFraction calculation for early triggers and improve test output formatting * Enhance HopScalingModule: add jitter functionality to sampling denominator and update tests for consistent behavior * Enhance HopScalingModule: implement adaptive sampling denominator adjustment and add reset functionality for tests * Enhance HopScalingTestShim: add test-only clock and window helpers, update injectSampleTraffic for adaptive sampling, and improve scenario summary output * Enhance HopScalingModule: add detailed documentation for functions, improve clarity of jitter and sampling logic, and reset functionality in tests * Enhance HopScalingModule: add evictionEstimate parameter to estimateScaleFactor and update related logging for improved mesh size estimation * Enhance HopScalingModule: adjust effective rolls calculation for improved accuracy, add eviction estimate logic, responding to all copilot review points * Implement CompactHistogram for parallel hop scaling sampling - Added CompactHistogram class to track node hop distances with bitwise sampling. - Integrated CompactHistogram into HopScalingModule for independent packet sampling. - Updated NodeDB to feed both the hop scaling module and the new histogram sampler. - Enhanced HopScalingModule with methods to sample packets for the histogram and retrieve hop distribution statistics. - Implemented tests for CompactHistogram functionality, including sampling, window rolling, and adaptive denominator scaling. - Updated existing tests to validate the integration of the new histogram sampling mechanism. * Enhance CompactHistogram and HopScalingModule: add per-hop distribution functionality, improve time handling for unit tests, and refine test setup for deterministic behavior * CompactHistogram: add mesh size estimation, improve entry replacement logic, and update logging for per-hop distribution * Refactor HopScalingModule and CompactHistogram integration - Removed the suggestedHopFromCompactHistogram function to streamline hop suggestion logic. - Updated HopScalingModule to directly utilize CompactHistogram's internal methods for hop suggestions and sampling. - Enhanced logging in HopScalingModule to provide detailed histogram statistics. - Modified test cases to ensure comprehensive coverage of new histogram behaviors and sampling logic. - Improved node ID distribution in tests to better exercise sampling mechanisms. - Ensured that filtering denominators are held for 12 hours before dropping, enhancing stability in sampling. * Refactor CompactHistogram to support 13-hour activity tracking and introduce politeness regimes - Updated the bitfield structure to accommodate 13-hour seen tracking. - Changed the logic in rollHour() to analyze activity over the last 0-2 hours vs. 1-3 hours for politeness factor calculation. - Introduced three politeness levels: GENEROUS, DEFAULT, and STRICT based on recent activity ratios. - Adjusted filtering and sampling logic to reflect the new 13-hour tracking period. - Updated unit tests to validate new behavior and ensure proper functionality of politeness regimes. * Enhance CompactHistogram and HopScalingModule for improved sampling and decision-making - Introduced a session-specific hash seed in CompactHistogram to reduce bias in node ID sampling. - Updated sampling logic to use hashed node IDs instead of raw IDs for filtering and entry management. - Added histogram rollover tracking in HopScalingModule to ensure proper decision-making after initial data collection. - Adjusted logging to reflect the active state of the histogram and its comparison with NodeDB advisory hops. - Enhanced unit tests to validate new sampling logic and memory layout changes. * Expose hashNodeId for testing in CompactHistogram * Add mesh trend statistics to CompactHistogram for enhanced activity tracking * Implement histogram state persistence in CompactHistogram with save and load functions * Refactor CompactHistogram to improve entry management and enhance rollHour logging * feat: add HopScalingModule for adaptive hop limit recommendations Introduces HopScalingModule, a sampled hop-distance histogram that recommends the minimum hop limit needed to reach ~40 nodes, and automatically reducing the hops as the mesh grows. Key design: - 512-byte packed histogram (128 × 4-byte Record entries) embedded in a new HopScalingModule. - Each Record: 16-bit node hash, 3-bit hop distance, 13-bit seen bitmap - Sampling filter: only nodes where (hash & (denom-1)) == 0 are kept; denominator doubles on overflow and halves when utilisation is low - Hourly rollHour(): tallies per-hop counts, walks scaled buckets to find the minimum hop satisfying TARGET_AFFECTED_NODES (40), applies a politeness extension based on recent/older activity ratio, shifts all seen bitmaps, and persists state to /prefs/hopScalingState.bin - Hop recommendation gated by bootstrap (requires >=1 rollHour before overriding HOP_MAX) - NodeDB calls samplePacketForHistogram() on every non-MQTT rx packet - Module also estimates total mesh size and logs useful information about mesh characteristics. Changes: - src/modules/HopScalingModule.h/.cpp: new module - src/mesh/NodeDB.cpp: wire up samplePacketForHistogram - src/mesh/Router.cpp: consume getLastRequiredHop() - test/test_hop_scaling/: 12-test suite covering all mesh topologies and anticipated operational requirements * test: increase run iterations in sparse to dense transition test * feat: refactor HopScalingModule to use RUNS_PER_HOUR constant and improve logging * feat: enhance HopScalingModule with filtering denominator management and add tests for state transitions * refactor: remove CompactHistogram module and related files * address copilot review comments * Tweak: packet sampling only lora * ove role-based hop floor logic and related definitions into the module - keep it in one place. * Refactor MockNodeDB to use nodeInfoLiteSetBit for MQTT flag setting * Refactor hop scaling parameters and logic to integer maths and put default values in defaults.h. Small flash size reduction, no functional impact. * Update unit test preprocessor directives to PIO_UNIT_TESTING for consistency * refactor: improve test output organization and clarity in hop scaling tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Tom
GitHub
Copilot
@@ -191,10 +191,12 @@ echo
|
||||
# PASS/FAIL — every hardware test would SKIP with "role not present". We
|
||||
# narrow to tests/unit explicitly so the summary reads as "no hardware,
|
||||
# unit suite only" instead of "big skip count looks suspicious".
|
||||
# Keep terminal output condensed (`-q -r fE`) so skip-heavy runs do not print
|
||||
# each skipped test in full; skip counts still appear in pytest's summary.
|
||||
if [[ -z $DETECTED && $# -eq 0 ]]; then
|
||||
echo "[pre-flight] no supported devices detected; running unit tier only."
|
||||
echo
|
||||
exec "$VENV_PY" -m pytest tests/unit -v --report-log=tests/reportlog.jsonl
|
||||
exec "$VENV_PY" -m pytest tests/unit -q -r fE --report-log=tests/reportlog.jsonl
|
||||
fi
|
||||
|
||||
# Default pytest args when the user passed none. Power users can invoke
|
||||
@@ -210,11 +212,13 @@ fi
|
||||
# skipping half the hardware tests with "not baked with session profile"
|
||||
# errors. Power users who know their hardware is current and want to shave
|
||||
# those seconds can pass `--assume-baked` explicitly.
|
||||
# Defaults also use condensed reporting (`-q -r fE`) to avoid listing every
|
||||
# skipped test verbatim while still surfacing failures/errors and summary data.
|
||||
if [[ $# -eq 0 ]]; then
|
||||
set -- tests/ \
|
||||
--html=tests/report.html --self-contained-html \
|
||||
--junitxml=tests/junit.xml \
|
||||
-v --tb=short
|
||||
-q -r fE --tb=short
|
||||
fi
|
||||
|
||||
# UI tier requires opencv-python-headless (and ideally easyocr). If it's
|
||||
|
||||
@@ -37,6 +37,12 @@ enum class TrafficType { POSITION, TELEMETRY };
|
||||
#define default_traffic_mgmt_position_precision_bits 24 // ~10m grid cells
|
||||
#define default_traffic_mgmt_position_min_interval_secs (ONE_DAY / 2) // 12 hours between identical positions
|
||||
|
||||
// Hop scaling defaults
|
||||
#define default_hop_scaling_min_target_nodes 40 // walk threshold: first hop reaching this cumulative count
|
||||
#define default_hop_scaling_max_target_nodes 80 // generous extension ceiling (2 × min)
|
||||
#define default_hop_scaling_min_target_nodes_floor 5 // minimum allowed min_target_nodes
|
||||
#define default_hop_scaling_max_target_nodes_ceiling 512 // maximum allowed max_target_nodes
|
||||
|
||||
#ifdef USERPREFS_RINGTONE_NAG_SECS
|
||||
#define default_ringtone_nag_secs USERPREFS_RINGTONE_NAG_SECS
|
||||
#else
|
||||
|
||||
@@ -25,6 +25,9 @@
|
||||
#include "mesh/generated/meshtastic/deviceonly_legacy.pb.h"
|
||||
#include "meshUtils.h"
|
||||
#include "modules/NeighborInfoModule.h"
|
||||
#if HAS_VARIABLE_HOPS
|
||||
#include "modules/HopScalingModule.h"
|
||||
#endif
|
||||
#include "xmodem.h"
|
||||
#include <ErriezCRC32.h>
|
||||
#include <algorithm>
|
||||
@@ -2538,6 +2541,14 @@ void NodeDB::updateFrom(const meshtastic_MeshPacket &mp)
|
||||
nodeInfoLiteSetBit(info, NODEINFO_BITFIELD_VIA_MQTT_MASK,
|
||||
mp.via_mqtt); // Store if we received this packet via MQTT
|
||||
|
||||
#if HAS_VARIABLE_HOPS
|
||||
// Only sample packets that arrived over LoRa.
|
||||
if (mp.transport_mechanism == meshtastic_MeshPacket_TransportMechanism_TRANSPORT_LORA && hopScalingModule) {
|
||||
uint8_t hopCount = std::max(int8_t(0), getHopsAway(mp));
|
||||
hopScalingModule->samplePacketForHistogram(mp.from, hopCount);
|
||||
}
|
||||
#endif
|
||||
|
||||
// If hopStart was set and there wasn't someone messing with the limit in the middle, add hopsAway
|
||||
const int8_t hopsAway = getHopsAway(mp);
|
||||
if (hopsAway >= 0) {
|
||||
|
||||
@@ -15,6 +15,9 @@
|
||||
#if HAS_TRAFFIC_MANAGEMENT
|
||||
#include "modules/TrafficManagementModule.h"
|
||||
#endif
|
||||
#if HAS_VARIABLE_HOPS
|
||||
#include "modules/HopScalingModule.h"
|
||||
#endif
|
||||
#if !MESHTASTIC_EXCLUDE_MQTT
|
||||
#include "mqtt/MQTT.h"
|
||||
#endif
|
||||
@@ -355,6 +358,30 @@ ErrorCode Router::send(meshtastic_MeshPacket *p)
|
||||
p->from = getFrom(p);
|
||||
|
||||
p->relay_node = nodeDB->getLastByteOfNodeNum(getNodeNum()); // set the relayer to us
|
||||
|
||||
#if HAS_VARIABLE_HOPS
|
||||
// Apply HopScaling hop recommendation to routine outgoing broadcasts
|
||||
if (isFromUs(p) && isBroadcast(p->to) && hopScalingModule && p->which_payload_variant == meshtastic_MeshPacket_decoded_tag) {
|
||||
switch (p->decoded.portnum) {
|
||||
case meshtastic_PortNum_POSITION_APP:
|
||||
case meshtastic_PortNum_TELEMETRY_APP:
|
||||
case meshtastic_PortNum_NODEINFO_APP:
|
||||
case meshtastic_PortNum_NEIGHBORINFO_APP: {
|
||||
uint8_t variableHopLimit = hopScalingModule->getLastRequiredHop();
|
||||
|
||||
// Never exceed user-configured hop_limit
|
||||
if (variableHopLimit < p->hop_limit) {
|
||||
LOG_DEBUG("[HOPSCALE] hop_limit %u -> %u for portnum %u", p->hop_limit, variableHopLimit, p->decoded.portnum);
|
||||
p->hop_limit = variableHopLimit;
|
||||
}
|
||||
break;
|
||||
}
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
// If we are the original transmitter, set the hop limit with which we start
|
||||
if (isFromUs(p))
|
||||
p->hop_start = p->hop_limit;
|
||||
|
||||
@@ -109,6 +109,15 @@ static inline int get_max_num_nodes()
|
||||
#define HAS_TRAFFIC_MANAGEMENT 0
|
||||
#endif
|
||||
|
||||
// HopScalingModule - variable hop module: dynamically adjusts broadcast hop_limit based on mesh density
|
||||
// Enable per-variant by defining HAS_VARIABLE_HOPS=1 in variant.h
|
||||
#ifdef ARCH_STM32WL
|
||||
#define HAS_VARIABLE_HOPS 0
|
||||
#endif
|
||||
#ifndef HAS_VARIABLE_HOPS
|
||||
#define HAS_VARIABLE_HOPS 1
|
||||
#endif
|
||||
|
||||
// Cache size for traffic management (number of nodes to track)
|
||||
// Can be overridden per-variant based on available memory
|
||||
#ifndef TRAFFIC_MANAGEMENT_CACHE_SIZE
|
||||
|
||||
@@ -66,4 +66,10 @@ inline uint32_t pow_of_2(uint32_t n)
|
||||
return 1 << n;
|
||||
}
|
||||
|
||||
/// Returns true if n is a power of two (n >= 1).
|
||||
template <typename T> constexpr bool is_pow_of_2(T n)
|
||||
{
|
||||
return n >= T(1) && (n & (n - T(1))) == T(0);
|
||||
}
|
||||
|
||||
#define IS_ONE_OF(item, ...) isOneOf(item, sizeof((int[]){__VA_ARGS__}) / sizeof(int), __VA_ARGS__)
|
||||
|
||||
@@ -0,0 +1,493 @@
|
||||
#include "HopScalingModule.h"
|
||||
#include "SafeFile.h"
|
||||
#include "meshUtils.h"
|
||||
|
||||
#if HAS_VARIABLE_HOPS
|
||||
|
||||
#include "FSCommon.h"
|
||||
#include "NodeDB.h"
|
||||
#include "SPILock.h"
|
||||
#include "concurrency/LockGuard.h"
|
||||
#include "mesh-pb-constants.h"
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <cstring>
|
||||
|
||||
namespace
|
||||
{
|
||||
// Module scheduling
|
||||
constexpr uint32_t INITIAL_DELAY_MS = 30 * 1000UL; // Startup grace period before first run
|
||||
constexpr uint32_t RUN_INTERVAL_MS = 5 * 60 * 1000UL; // Emit micro-summary every 5 minutes
|
||||
// RUNS_PER_HOUR is a public class constant in HopScalingModule.h
|
||||
|
||||
// Persistence
|
||||
// Note: this only needs incrementing if the published arrangement changes. For testing purposes, or prior to widespread release,
|
||||
// it can stay the same even if the internal layout changes.
|
||||
constexpr uint32_t HISTOGRAM_STATE_MAGIC = 0x48535432; // 'HST2' — layout v2
|
||||
constexpr uint8_t HISTOGRAM_STATE_VERSION = 1;
|
||||
constexpr const char *HISTOGRAM_STATE_FILE = "/prefs/hopScalingState.bin";
|
||||
|
||||
#pragma pack(push, 1)
|
||||
struct PersistedHistogram {
|
||||
uint32_t magic;
|
||||
uint8_t version;
|
||||
uint8_t samplingDenominator;
|
||||
uint8_t filteringDenominator;
|
||||
uint8_t filterDenomHoldRollsRemaining; // rollHour() calls remaining in the hold; 0 when expired/not active
|
||||
uint16_t hashSeed;
|
||||
Record entries[HopScalingModule::CAPACITY]; // full 512-byte array; count derived on load
|
||||
};
|
||||
#pragma pack(pop)
|
||||
|
||||
} // namespace
|
||||
|
||||
HopScalingModule *hopScalingModule;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Lifecycle
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
HopScalingModule::HopScalingModule() : concurrency::OSThread("HopScaling")
|
||||
{
|
||||
clear();
|
||||
loadFromDisk();
|
||||
setIntervalFromNow(INITIAL_DELAY_MS);
|
||||
}
|
||||
|
||||
void HopScalingModule::clear()
|
||||
{
|
||||
memset(entries, 0, sizeof(entries));
|
||||
count = 0;
|
||||
samplingDenominator = DENOM_MIN;
|
||||
filteringDenominator = DENOM_MIN;
|
||||
filteringDenomHoldRollsRemaining = 0;
|
||||
lastPerHopCounts = {};
|
||||
lastSuggestedHop = MAX_HOP;
|
||||
lastPoliteNumer = POLITENESS_DEFAULT;
|
||||
lastTrendStats = {};
|
||||
memset(denominatorHistory, DENOM_MIN, sizeof(denominatorHistory));
|
||||
#ifndef PIO_UNIT_TESTING
|
||||
hashSeed = static_cast<uint16_t>(random());
|
||||
#else
|
||||
hashSeed = 0; // deterministic in unit tests
|
||||
#endif
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Persistence
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
void HopScalingModule::saveToDisk() const
|
||||
{
|
||||
#ifdef FSCom
|
||||
FSCom.mkdir("/prefs");
|
||||
PersistedHistogram state{};
|
||||
state.magic = HISTOGRAM_STATE_MAGIC;
|
||||
state.version = HISTOGRAM_STATE_VERSION;
|
||||
state.samplingDenominator = samplingDenominator;
|
||||
state.filteringDenominator = filteringDenominator;
|
||||
state.filterDenomHoldRollsRemaining = filteringDenomHoldRollsRemaining;
|
||||
state.hashSeed = hashSeed;
|
||||
// Save all CAPACITY slots; count is reconstructed on load by scanning seenHoursAgo.
|
||||
memcpy(state.entries, entries, sizeof(state.entries));
|
||||
auto file = SafeFile(HISTOGRAM_STATE_FILE, true);
|
||||
const size_t written = file.write(reinterpret_cast<const uint8_t *>(&state), sizeof(state));
|
||||
if (file.close() && written == sizeof(state)) {
|
||||
LOG_DEBUG("[HOPSCALE] Saved: count=%u samp=1/%u filt=1/%u holdRollsRemaining=%u", count, samplingDenominator,
|
||||
filteringDenominator, state.filterDenomHoldRollsRemaining);
|
||||
} else {
|
||||
LOG_WARN("[HOPSCALE] Failed to write %s (%u of %u bytes)", HISTOGRAM_STATE_FILE, static_cast<unsigned>(written),
|
||||
static_cast<unsigned>(sizeof(state)));
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
void HopScalingModule::loadFromDisk()
|
||||
{
|
||||
#ifdef FSCom
|
||||
concurrency::LockGuard g(spiLock);
|
||||
auto file = FSCom.open(HISTOGRAM_STATE_FILE, FILE_O_READ);
|
||||
if (!file)
|
||||
return;
|
||||
PersistedHistogram state{};
|
||||
const bool readOk = (file.read(reinterpret_cast<uint8_t *>(&state), sizeof(state)) == sizeof(state));
|
||||
file.close();
|
||||
// Validate magic, version, denom range, denom power-of-two invariant, and hold counter.
|
||||
if (!readOk || state.magic != HISTOGRAM_STATE_MAGIC || state.version != HISTOGRAM_STATE_VERSION ||
|
||||
state.samplingDenominator < DENOM_MIN || state.samplingDenominator > DENOM_MAX ||
|
||||
state.filteringDenominator < state.samplingDenominator || state.filteringDenominator > DENOM_MAX ||
|
||||
!is_pow_of_2(state.samplingDenominator) || !is_pow_of_2(state.filteringDenominator) ||
|
||||
state.filterDenomHoldRollsRemaining > FILTER_DENOM_HOLD_ROLLS) {
|
||||
LOG_DEBUG("[HOPSCALE] No valid persisted state (magic=%08x ver=%u samp=%u filt=%u hold=%u), starting fresh", state.magic,
|
||||
state.version, state.samplingDenominator, state.filteringDenominator, state.filterDenomHoldRollsRemaining);
|
||||
return;
|
||||
}
|
||||
// Derive count by scanning: active entries have seenHoursAgo != 0; pack them to the front.
|
||||
|
||||
uint8_t restored = 0;
|
||||
for (uint8_t i = 0; i < CAPACITY && restored < CAPACITY; i++) {
|
||||
if (state.entries[i].seenHoursAgo != 0u) {
|
||||
entries[restored++] = state.entries[i];
|
||||
}
|
||||
}
|
||||
count = restored;
|
||||
samplingDenominator = state.samplingDenominator;
|
||||
filteringDenominator = state.filteringDenominator;
|
||||
filteringDenomHoldRollsRemaining = state.filterDenomHoldRollsRemaining;
|
||||
// denominatorHistory can't be recovered; initialise all slots to filteringDenominator so
|
||||
// the first few post-reboot scaledPerHour values use a safe (slightly conservative) multiplier.
|
||||
memset(denominatorHistory, filteringDenominator, sizeof(denominatorHistory));
|
||||
hashSeed = state.hashSeed;
|
||||
LOG_INFO("[HOPSCALE] Restored: count=%u samp=1/%u filt=1/%u holdRollsRemaining=%u", count, samplingDenominator,
|
||||
filteringDenominator, state.filterDenomHoldRollsRemaining);
|
||||
#endif
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Core API
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
void HopScalingModule::samplePacketForHistogram(uint32_t nodeId, uint8_t hopCount)
|
||||
{
|
||||
const uint16_t hash = hashNodeId(nodeId);
|
||||
|
||||
if (!passesFilter(hash, samplingDenominator))
|
||||
return;
|
||||
|
||||
hopCount = std::min(hopCount, MAX_HOP);
|
||||
|
||||
// Update an existing entry
|
||||
Record *entry = nullptr;
|
||||
for (uint8_t i = 0; i < count; i++) {
|
||||
if (entries[i].nodeHash == hash) {
|
||||
entry = &entries[i];
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (entry) {
|
||||
entry->hops_away = hopCount;
|
||||
markCurrentHour(*entry);
|
||||
return;
|
||||
}
|
||||
|
||||
// New node: trim if necessary before allocating a slot
|
||||
if (getFillPercentage() >= FILL_HIGH_PCT) {
|
||||
trimIfNeeded();
|
||||
}
|
||||
|
||||
if (count < CAPACITY) {
|
||||
entries[count].nodeHash = hash;
|
||||
entries[count].hops_away = hopCount;
|
||||
entries[count].seenHoursAgo = 1u; // mark current hour
|
||||
count++;
|
||||
} else {
|
||||
LOG_WARN("[HOPSCALE] Histogram full at samp=1/%u (DENOM_MAX=%u); dropping node hash=0x%04x; hop recommendation may be "
|
||||
"skewed!!!",
|
||||
samplingDenominator, DENOM_MAX, hash);
|
||||
}
|
||||
}
|
||||
|
||||
void HopScalingModule::rollHour()
|
||||
{
|
||||
// Advance denominatorHistory before the tally so each slot h holds the filteringDenominator
|
||||
// that was active when seenHoursAgo bit h was set. hourlyRaw[h] is then gated per-slot by
|
||||
// denominatorHistory[h], giving a correct population estimate for each historical hour even
|
||||
// when filteringDenominator changes between rolls. Scale-up backfills the entire array so
|
||||
// the invariant holds retroactively (see trimIfNeeded()).
|
||||
for (uint8_t h = 12; h > 0; h--)
|
||||
denominatorHistory[h] = denominatorHistory[h - 1];
|
||||
denominatorHistory[0] = filteringDenominator;
|
||||
|
||||
// 1. Tally per-hop counts and per-slot hourly activity in one pass.
|
||||
// hourlyRaw[h]: gated per-slot by denominatorHistory[h] so the raw count and its
|
||||
// multiplier are always consistent, even across filteringDenominator transitions.
|
||||
// counts.*: gated uniformly by the current filteringDenominator for a consistent
|
||||
// population estimate used by the hop-walk recommendation (step 2).
|
||||
PerHopCounts counts{};
|
||||
uint16_t hourlyRaw[13] = {};
|
||||
uint16_t trendNewThisHour = 0;
|
||||
uint16_t trendReturning = 0;
|
||||
uint16_t trendLapsed = 0;
|
||||
uint16_t trendOlderThan4h = 0;
|
||||
uint16_t trendAgingOut = 0;
|
||||
for (uint8_t i = 0; i < count; i++) {
|
||||
const uint16_t hash = entries[i].nodeHash;
|
||||
const uint32_t seen = entries[i].seenHoursAgo;
|
||||
|
||||
// Per-slot hourly activity: gate each slot by its own denominator.
|
||||
for (uint8_t h = 0; h < 13; h++) {
|
||||
if ((seen & (1u << h)) && passesFilter(hash, denominatorHistory[h]))
|
||||
hourlyRaw[h]++;
|
||||
}
|
||||
|
||||
// Hop counts and trend stats: uniform current-denominator gate.
|
||||
if (!passesFilter(hash, filteringDenominator))
|
||||
continue;
|
||||
|
||||
if (seenInLast13h(entries[i])) {
|
||||
counts.perHop[entries[i].hops_away]++;
|
||||
counts.total++;
|
||||
}
|
||||
const bool heardThisHour = (seen & 1u) != 0u;
|
||||
const bool heardLastHour = (seen & 2u) != 0u;
|
||||
const bool hasOlderHistory = (seen >> 1u) != 0u;
|
||||
const bool recentlySilent = (seen & 0xFu) == 0u;
|
||||
if (heardThisHour && !hasOlderHistory)
|
||||
trendNewThisHour++;
|
||||
else if (heardThisHour && hasOlderHistory)
|
||||
trendReturning++;
|
||||
if (!heardThisHour && heardLastHour)
|
||||
trendLapsed++;
|
||||
if (recentlySilent && (seen & 0x1FF0u) != 0u)
|
||||
trendOlderThan4h++;
|
||||
if (seen == (1u << 12u))
|
||||
trendAgingOut++;
|
||||
}
|
||||
lastPerHopCounts = counts;
|
||||
|
||||
// 1b. Compute politeness factor from the 0-2 h vs 1-3 h activity ratio.
|
||||
{
|
||||
const uint32_t recent = static_cast<uint32_t>(hourlyRaw[0]) + hourlyRaw[1];
|
||||
const uint32_t older = static_cast<uint32_t>(hourlyRaw[1]) + hourlyRaw[2];
|
||||
if (older > 1 && recent > 1) {
|
||||
const uint32_t r = static_cast<uint32_t>(recent) * ACTIVITY_WEIGHT_SCALE;
|
||||
const uint32_t o = static_cast<uint32_t>(older);
|
||||
if (r < o * ACTIVITY_WEIGHT_GENEROUS_MAX_NUMER)
|
||||
lastPoliteNumer = POLITENESS_GENEROUS;
|
||||
else if (r > o * ACTIVITY_WEIGHT_STRICT_MIN_NUMER)
|
||||
lastPoliteNumer = POLITENESS_STRICT;
|
||||
else
|
||||
lastPoliteNumer = POLITENESS_DEFAULT;
|
||||
} else {
|
||||
lastPoliteNumer = POLITENESS_DEFAULT;
|
||||
}
|
||||
}
|
||||
|
||||
// 1c. Scale and cache trend stats (denominatorHistory already advanced above).
|
||||
{
|
||||
MeshTrendStats t{};
|
||||
for (uint8_t h = 0; h < 13; h++) {
|
||||
const uint32_t s = static_cast<uint32_t>(hourlyRaw[h]) * denominatorHistory[h];
|
||||
t.scaledPerHour[h] = static_cast<uint16_t>(std::min<uint32_t>(s, UINT16_MAX));
|
||||
}
|
||||
auto scale = [&](uint16_t raw) -> uint16_t {
|
||||
return static_cast<uint16_t>(std::min<uint32_t>(static_cast<uint32_t>(raw) * filteringDenominator, UINT16_MAX));
|
||||
};
|
||||
t.newThisHour = scale(trendNewThisHour);
|
||||
t.returningThisHour = scale(trendReturning);
|
||||
t.lapsedSinceLastHour = scale(trendLapsed);
|
||||
t.olderThan4h = scale(trendOlderThan4h);
|
||||
t.agingOut = scale(trendAgingOut);
|
||||
lastTrendStats = t;
|
||||
}
|
||||
|
||||
// 2. Walk scaled hop buckets to produce a hop-limit recommendation.
|
||||
// effectiveMin: walk threshold — first hop whose cumulative count reaches this.
|
||||
// effectiveMax: ceiling on the one-hop extension check with GENEROUS politeness.
|
||||
const uint16_t effectiveMin = TARGET_AFFECTED_NODES;
|
||||
const uint16_t effectiveMax = MAX_TARGET_NODES;
|
||||
uint8_t suggested = MAX_HOP;
|
||||
if (counts.total > 0) {
|
||||
uint32_t cumulative = 0;
|
||||
for (uint8_t hop = 0; hop <= MAX_HOP; hop++) {
|
||||
cumulative += static_cast<uint32_t>(counts.perHop[hop]) * filteringDenominator;
|
||||
if (cumulative >= effectiveMin) {
|
||||
suggested = hop;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (suggested < MAX_HOP) {
|
||||
const uint32_t atNext = static_cast<uint32_t>(counts.perHop[suggested + 1]) * filteringDenominator;
|
||||
// politeLimit = effectiveMin + gap * politeNumer / POLITENESS_DENOM
|
||||
// Multiply both sides by POLITENESS_DENOM to stay in integers.
|
||||
const uint32_t gap = static_cast<uint32_t>(effectiveMax) - static_cast<uint32_t>(effectiveMin);
|
||||
if ((cumulative + atNext) * POLITENESS_DENOM <=
|
||||
static_cast<uint32_t>(effectiveMin) * POLITENESS_DENOM + gap * lastPoliteNumer) {
|
||||
suggested++;
|
||||
}
|
||||
}
|
||||
}
|
||||
lastSuggestedHop = suggested;
|
||||
|
||||
// 3. Log scaled per-hop counts and recommendation.
|
||||
{
|
||||
uint16_t scaled[MAX_HOP + 1];
|
||||
for (uint8_t h = 0; h <= MAX_HOP; h++) {
|
||||
const uint32_t s = static_cast<uint32_t>(counts.perHop[h]) * filteringDenominator;
|
||||
scaled[h] = static_cast<uint16_t>(std::min<uint32_t>(s, UINT16_MAX));
|
||||
}
|
||||
const uint32_t scaledTotal = static_cast<uint32_t>(counts.total) * filteringDenominator;
|
||||
memcpy(lastScaledPerHop, scaled, sizeof(lastScaledPerHop));
|
||||
LOG_INFO("[HOPSCALE] rollHour: entries=%u/128 samp=1/%u filt=1/%u counted=%u est=%u suggestedHop=%u polite=%u/4", count,
|
||||
samplingDenominator, filteringDenominator, counts.total, static_cast<unsigned>(scaledTotal), suggested,
|
||||
lastPoliteNumer);
|
||||
|
||||
const auto &ts = lastTrendStats;
|
||||
LOG_INFO("[HOPSCALE] scaledSeenPerHour (h0=now): [%u %u %u %u %u %u %u %u %u %u %u %u %u]", ts.scaledPerHour[0],
|
||||
ts.scaledPerHour[1], ts.scaledPerHour[2], ts.scaledPerHour[3], ts.scaledPerHour[4], ts.scaledPerHour[5],
|
||||
ts.scaledPerHour[6], ts.scaledPerHour[7], ts.scaledPerHour[8], ts.scaledPerHour[9], ts.scaledPerHour[10],
|
||||
ts.scaledPerHour[11], ts.scaledPerHour[12]);
|
||||
LOG_INFO("[HOPSCALE] trend: new=%u returning=%u lapsed=%u olderThan4h=%u agingOut=%u", ts.newThisHour,
|
||||
ts.returningThisHour, ts.lapsedSinceLastHour, ts.olderThan4h, ts.agingOut);
|
||||
}
|
||||
|
||||
// 4. Scale-down check: if fewer than FILL_LOW_PCT% of capacity pass the filteringDenominator
|
||||
// gate and are active, halve samplingDenominator to admit more nodes.
|
||||
// Note: during a filteringDenominator hold period, lowering samplingDenominator does not
|
||||
// immediately improve counts.total (new admissions don't pass the elevated
|
||||
// filteringDenominator). On a genuinely quieting mesh this check can therefore fire on
|
||||
// consecutive hours, cascading samplingDenominator toward DENOM_MIN. This is intentional:
|
||||
// rapid re-admission allows quick recovery if the mesh returns. The hop recommendation
|
||||
// stays conservative (MAX_HOP) throughout because filteringDenominator remains elevated;
|
||||
// step 5 below re-synchronises the denominators once the hold expires.
|
||||
if (counts.total * 100u < static_cast<uint32_t>(CAPACITY) * FILL_LOW_PCT) {
|
||||
if (samplingDenominator > DENOM_MIN) {
|
||||
samplingDenominator = static_cast<uint8_t>(samplingDenominator / 2u);
|
||||
LOG_INFO("[HOPSCALE] Scale-down: sampling denom halved to %u (filter denom=%u)", samplingDenominator,
|
||||
filteringDenominator);
|
||||
}
|
||||
}
|
||||
|
||||
// 5. Tick down the hold counter; once it reaches zero, halve filteringDenominator toward
|
||||
// samplingDenominator once per rollHour() (= once per hour) rather than a single jump:
|
||||
// avoids a sudden large change in the hop-walk count when samplingDenominator cascaded
|
||||
// down significantly during the hold period. No new hold is placed on each step — the
|
||||
// 13-roll hold already guaranteed that re-admitted nodes have full seenHoursAgo history;
|
||||
// further pacing is provided naturally by the 1-step-per-hour rate. denominatorHistory
|
||||
// is updated automatically by the shift at the top of rollHour(), so no backfill here.
|
||||
if (filteringDenominator > samplingDenominator) {
|
||||
if (filteringDenomHoldRollsRemaining > 0)
|
||||
filteringDenomHoldRollsRemaining--;
|
||||
if (filteringDenomHoldRollsRemaining == 0) {
|
||||
const uint8_t stepped = static_cast<uint8_t>(filteringDenominator / 2u);
|
||||
filteringDenominator = (stepped > samplingDenominator) ? stepped : samplingDenominator;
|
||||
LOG_INFO("[HOPSCALE] Filter denom stepped to %u (samp=1/%u)", filteringDenominator, samplingDenominator);
|
||||
}
|
||||
}
|
||||
|
||||
// 6. Shift all seen bitmaps left by one slot (opens a fresh slot for the new hour).
|
||||
for (uint8_t i = 0; i < count; i++) {
|
||||
rollSeenBits(entries[i]);
|
||||
}
|
||||
|
||||
if (histogramRollCount < 255)
|
||||
histogramRollCount++;
|
||||
|
||||
saveToDisk();
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Internal helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
void HopScalingModule::trimIfNeeded()
|
||||
{
|
||||
// Step 1: evict stale entries (not seen in any of the past 13 hours).
|
||||
uint8_t newCount = 0;
|
||||
for (uint8_t i = 0; i < count; i++) {
|
||||
if (seenInLast13h(entries[i])) {
|
||||
if (i != newCount) {
|
||||
entries[newCount] = entries[i];
|
||||
}
|
||||
newCount++;
|
||||
}
|
||||
}
|
||||
count = newCount;
|
||||
|
||||
// Step 2: if still too full, double the sampling denominator and remove non-matching entries.
|
||||
if (getFillPercentage() >= FILL_HIGH_PCT && samplingDenominator < DENOM_MAX) {
|
||||
samplingDenominator = static_cast<uint8_t>(
|
||||
std::min<uint16_t>(static_cast<uint16_t>(samplingDenominator) * 2u, static_cast<uint16_t>(DENOM_MAX)));
|
||||
filteringDenominator = std::max(filteringDenominator, samplingDenominator);
|
||||
filteringDenomHoldRollsRemaining = FILTER_DENOM_HOLD_ROLLS;
|
||||
// Raise any denominatorHistory slot that is below the new filteringDenominator.
|
||||
// Slots already above it (recorded during a prior scale-up that hasn't fully stepped
|
||||
// down yet) are left untouched: eviction at samplingDenominator retains exactly those
|
||||
// entries, so the old higher gate remains accurate for those historical hours.
|
||||
// Slots below the new value must be raised because the eviction removed entries that
|
||||
// had been admitted at the looser old gate — the remaining entries represent a 1/N
|
||||
// subsample where N is the new filteringDenominator, not the old smaller value.
|
||||
for (uint8_t h = 0; h < 13; h++)
|
||||
denominatorHistory[h] = std::max(denominatorHistory[h], filteringDenominator);
|
||||
LOG_INFO("[HOPSCALE] Scale-up: samp denom doubled to %u (filt=%u)", samplingDenominator, filteringDenominator);
|
||||
|
||||
newCount = 0;
|
||||
for (uint8_t i = 0; i < count; i++) {
|
||||
if (passesFilter(entries[i].nodeHash, samplingDenominator)) {
|
||||
if (i != newCount) {
|
||||
entries[newCount] = entries[i];
|
||||
}
|
||||
newCount++;
|
||||
}
|
||||
}
|
||||
count = newCount;
|
||||
}
|
||||
}
|
||||
|
||||
void HopScalingModule::logStatusReport(bool didHourlyUpdate) const
|
||||
{
|
||||
const bool histActive = (histogramRollCount > 0 && count > 0);
|
||||
const auto &histCounts = lastPerHopCounts;
|
||||
const uint8_t runsRemaining = didHourlyUpdate ? RUNS_PER_HOUR : (RUNS_PER_HOUR - runsSinceLastHourlyUpdate);
|
||||
const uint8_t minsUntilRollover = runsRemaining * (RUN_INTERVAL_MS / (60 * 1000UL));
|
||||
|
||||
LOG_INFO("[HOPSCALE] hop=%u histActive=%u fill=%u%% samp=1/%u filt=1/%u entries=%u lastCounted=%u polite=%u/4 "
|
||||
"nextRoll=%umin",
|
||||
lastRequiredHop, histActive ? 1u : 0u, getFillPercentage(), samplingDenominator, filteringDenominator, count,
|
||||
histCounts.total, lastPoliteNumer, minsUntilRollover);
|
||||
|
||||
LOG_INFO("[HOPSCALE] nodes perHop: [%u %u %u %u %u %u %u %u]", histCounts.perHop[0], histCounts.perHop[1],
|
||||
histCounts.perHop[2], histCounts.perHop[3], histCounts.perHop[4], histCounts.perHop[5], histCounts.perHop[6],
|
||||
histCounts.perHop[7]);
|
||||
LOG_INFO("[HOPSCALE] last scaled perHop: [%u %u %u %u %u %u %u %u]", lastScaledPerHop[0], lastScaledPerHop[1],
|
||||
lastScaledPerHop[2], lastScaledPerHop[3], lastScaledPerHop[4], lastScaledPerHop[5], lastScaledPerHop[6],
|
||||
lastScaledPerHop[7]);
|
||||
}
|
||||
|
||||
int32_t HopScalingModule::runOnce()
|
||||
{
|
||||
const bool isFirstRun = !hasCompletedInitialRun;
|
||||
bool didHourlyUpdate = false;
|
||||
|
||||
if (isFirstRun) {
|
||||
hasCompletedInitialRun = true;
|
||||
runsSinceLastHourlyUpdate = 0;
|
||||
didHourlyUpdate = true;
|
||||
} else {
|
||||
runsSinceLastHourlyUpdate++;
|
||||
if (runsSinceLastHourlyUpdate >= RUNS_PER_HOUR) {
|
||||
runsSinceLastHourlyUpdate = 0;
|
||||
didHourlyUpdate = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (didHourlyUpdate && !isFirstRun) {
|
||||
rollHour();
|
||||
}
|
||||
|
||||
if (didHourlyUpdate) {
|
||||
uint8_t suggested = (histogramRollCount > 0 && count > 0) ? lastSuggestedHop : HOP_MAX;
|
||||
// Role-based hop floor: TRACKER/TAK_TRACKER always reach at least 2 hops,
|
||||
// SENSOR reaches at least 1, so these reporting roles remain reachable even
|
||||
// on a dense mesh where the histogram recommends a lower hop count.
|
||||
uint8_t roleFloor = 0;
|
||||
switch (config.device.role) {
|
||||
case meshtastic_Config_DeviceConfig_Role_TRACKER:
|
||||
case meshtastic_Config_DeviceConfig_Role_TAK_TRACKER:
|
||||
roleFloor = 2;
|
||||
break;
|
||||
case meshtastic_Config_DeviceConfig_Role_SENSOR:
|
||||
roleFloor = 1;
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
lastRequiredHop = std::max(suggested, roleFloor);
|
||||
}
|
||||
|
||||
logStatusReport(didHourlyUpdate);
|
||||
|
||||
return RUN_INTERVAL_MS;
|
||||
}
|
||||
|
||||
#endif
|
||||
@@ -0,0 +1,351 @@
|
||||
#pragma once
|
||||
|
||||
#include "MeshTypes.h"
|
||||
#include "concurrency/OSThread.h"
|
||||
#include "configuration.h"
|
||||
#include "mesh/Default.h"
|
||||
#include "mesh/mesh-pb-constants.h"
|
||||
#include <algorithm>
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
|
||||
#if HAS_VARIABLE_HOPS
|
||||
|
||||
/**
|
||||
* HopScalingModule: Sampled hop-distance histogram for mesh-aware hop limit recommendations.
|
||||
*
|
||||
* Memory layout: 512 bytes total (128 entries × 4 bytes/entry, no padding)
|
||||
* - 16-bit XOR-fold hash of node ID
|
||||
* - 3-bit hops away (0–7)
|
||||
* - 13-bit hourly seen bitmap
|
||||
* All three fields are packed into a single 32-bit Record; sizeof(Record) == 4.
|
||||
*
|
||||
* Sampling:
|
||||
* - A node is added only when passesFilter(hashNodeId(nodeId), samplingDenominator),
|
||||
* i.e. (hash16(nodeId) & (samplingDenominator – 1)) == 0 (hash-space subsample, not raw ID)
|
||||
* - samplingDenominator starts at 1 (sample all), doubles when the list exceeds FILL_HIGH_PCT
|
||||
* - filteringDenominator tracks samplingDenominator upward immediately but does not drop back
|
||||
* down until FILTER_DENOM_HOLD_MS (13 h) have elapsed since the last scale-up
|
||||
*
|
||||
* Hourly rollover (rollHour()):
|
||||
* - Summarises per-hop node counts for entries matching filteringDenominator and seen in the
|
||||
* last 13 hours
|
||||
* - Scales each hop bucket by filteringDenominator and walks the buckets to recommend a hop
|
||||
* limit, matching the same algorithm used in HopScalingModule
|
||||
* - Shifts the 13-bit seen bitmap left by one slot to open a fresh slot for the new hour;
|
||||
* nodes not seen in 13 consecutive hours have all seen bits cleared (stale)
|
||||
* - Checks for scale-down: if fewer than FILL_LOW_PCT of capacity pass filteringDenominator,
|
||||
* samplingDenominator is halved (filteringDenominator is held until the 13-h lock expires)
|
||||
*
|
||||
* Thread-safety: all access is single-threaded via the main loop cooperative scheduler.
|
||||
*/
|
||||
|
||||
struct Record {
|
||||
uint32_t nodeHash : 16;
|
||||
uint32_t hops_away : 3;
|
||||
uint32_t seenHoursAgo : 13;
|
||||
};
|
||||
static_assert(sizeof(Record) == 4);
|
||||
|
||||
class HopScalingModule : private concurrency::OSThread
|
||||
{
|
||||
public:
|
||||
// -----------------------------------------------------------------------
|
||||
// Capacity and memory layout
|
||||
// -----------------------------------------------------------------------
|
||||
static constexpr size_t CAPACITY = 128;
|
||||
static constexpr size_t ENTRY_BYTES = sizeof(Record);
|
||||
static constexpr size_t TOTAL_BYTES = CAPACITY * ENTRY_BYTES;
|
||||
|
||||
// Denominator limits (must be powers of 2)
|
||||
static constexpr uint8_t DENOM_MIN = 1;
|
||||
static constexpr uint8_t DENOM_MAX = 128;
|
||||
|
||||
static constexpr uint8_t MAX_HOP = 7;
|
||||
|
||||
// Fill-level thresholds (percent of CAPACITY)
|
||||
static constexpr uint8_t FILL_HIGH_PCT = 80;
|
||||
static constexpr uint8_t FILL_LOW_PCT = 20;
|
||||
|
||||
// How long filteringDenominator is held at an elevated level before it may drop.
|
||||
//
|
||||
// This value is deliberately equal to the seenHoursAgo window (13 hours / 13 bits).
|
||||
// Invariant: every entry that existed when a scale-up fired had seenHoursAgo != 0 at
|
||||
// that moment (trimIfNeeded() evicts stale entries before doubling the denominator),
|
||||
// so it remains seenInLast13h for at most 13 more rollHour() calls — exactly the
|
||||
// hold duration. That means entries from the scale-up event keep counts.total above
|
||||
// the scale-down threshold for the entire hold period under normal (active) mesh
|
||||
// conditions. On a genuinely quieting mesh the scale-down CAN fire before the hold
|
||||
// expires — each firing halves samplingDenominator but filteringDenominator stays
|
||||
// elevated, so the hop recommendation correctly stays conservative (MAX_HOP) while
|
||||
// the cascade runs. The cascade is bounded at DENOM_MIN (7 halvings from DENOM_MAX);
|
||||
// when the hold finally expires, step 5 of rollHour() halves filteringDenominator
|
||||
// once per hour (rather than jumping directly to samplingDenominator) until the two
|
||||
// converge, giving the hop-walk a gradual, 1-step-per-hour descent.
|
||||
static constexpr uint32_t FILTER_DENOM_HOLD_MS = 13UL * 60UL * 60UL * 1000UL; // 13 h (documentation only)
|
||||
// Number of rollHour() calls the hold spans — equals the seenHoursAgo window width.
|
||||
// filteringDenomHoldRollsRemaining is initialised to this value on scale-up and
|
||||
// decremented once per rollHour(); step-down begins when it reaches zero.
|
||||
static constexpr uint8_t FILTER_DENOM_HOLD_ROLLS = 13u;
|
||||
|
||||
// Hop-walk: target cumulative affected-node count when choosing a hop limit
|
||||
static constexpr uint16_t TARGET_AFFECTED_NODES = default_hop_scaling_min_target_nodes;
|
||||
|
||||
// Clamp bounds enforced on min_target_nodes / max_target_nodes
|
||||
static constexpr uint16_t MIN_TARGET_NODES_FLOOR = default_hop_scaling_min_target_nodes_floor;
|
||||
static constexpr uint16_t MAX_TARGET_NODES_CEILING = default_hop_scaling_max_target_nodes_ceiling;
|
||||
static constexpr uint16_t MAX_TARGET_NODES = default_hop_scaling_max_target_nodes;
|
||||
|
||||
// Politeness factors for the one-hop extension check in the hop walk.
|
||||
// Stored as integer numerators over POLITENESS_DENOM (4):
|
||||
// politeLimit = min + gap * politeNumer / POLITENESS_DENOM
|
||||
// STRICT → min + 25% of gap; DEFAULT → midpoint; GENEROUS → max
|
||||
static constexpr uint8_t POLITENESS_DENOM = 4u;
|
||||
static constexpr uint8_t POLITENESS_GENEROUS = 4u; // 4/4 = 1.00
|
||||
static constexpr uint8_t POLITENESS_DEFAULT = 2u; // 2/4 = 0.50
|
||||
static constexpr uint8_t POLITENESS_STRICT = 1u; // 1/4 = 0.25
|
||||
|
||||
// Activity weight thresholds (ratio of 0-2 h window vs 1-3 h window).
|
||||
// Cross-multiply form: recent * ACTIVITY_WEIGHT_SCALE vs older * threshold_numer.
|
||||
// GENEROUS if recent*10 < older*9 (ratio < 0.9); STRICT if recent*10 > older*12 (ratio > 1.2)
|
||||
static constexpr uint8_t ACTIVITY_WEIGHT_SCALE = 10u;
|
||||
static constexpr uint8_t ACTIVITY_WEIGHT_GENEROUS_MAX_NUMER = 9u;
|
||||
static constexpr uint8_t ACTIVITY_WEIGHT_STRICT_MIN_NUMER = 12u;
|
||||
|
||||
// Scheduling: number of 5-minute runOnce() ticks that make up one hourly rollover
|
||||
static constexpr uint8_t RUNS_PER_HOUR = 12;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Types
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/// Per-hop node counts produced at each hourly rollover.
|
||||
struct PerHopCounts {
|
||||
uint16_t perHop[MAX_HOP + 1] = {};
|
||||
uint16_t total = 0;
|
||||
};
|
||||
|
||||
/// Mesh activity trend stats produced at each hourly rollover.
|
||||
/// All counts are scaled by filteringDenominator (i.e. estimated full-mesh population).
|
||||
///
|
||||
/// Bitmap interpretation (before the hourly shift): bit 0 = just-completed hour, bit 12 = 12 h ago.
|
||||
struct MeshTrendStats {
|
||||
/// Estimated node count per hour slot (h=0 is the just-completed hour, h=12 is 12 h ago).
|
||||
uint16_t scaledPerHour[13] = {};
|
||||
/// Nodes heard only this hour with no prior bitmap history — indicates new arrivals.
|
||||
uint16_t newThisHour = 0;
|
||||
/// Nodes heard this hour that also appeared in at least one older hour — stable regulars.
|
||||
uint16_t returningThisHour = 0;
|
||||
/// Nodes heard last hour but silent this hour — potential departures.
|
||||
uint16_t lapsedSinceLastHour = 0;
|
||||
/// Nodes absent from the last 4 hours but still present in some older hour (5–13 h) — quieting down.
|
||||
uint16_t olderThan4h = 0;
|
||||
/// Nodes whose only remaining history is the 13th hour (bit 12 only) — about to age out entirely.
|
||||
uint16_t agingOut = 0;
|
||||
};
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Lifecycle
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
HopScalingModule();
|
||||
~HopScalingModule() = default;
|
||||
|
||||
/// Reset all entries and state.
|
||||
void clear();
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Core API
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/// Record a received packet.
|
||||
/// Adds or updates an entry when passesFilter(hashNodeId(nodeId), samplingDenominator),
|
||||
/// i.e. when the 16-bit XOR-fold hash of the node ID falls in the 1/samplingDenominator
|
||||
/// subsample of the hash space. This is NOT a raw nodeId modulo check.
|
||||
/// Marks the current hour as seen and updates the stored hop count to the last observed value.
|
||||
/// Triggers a trim pass if the list exceeds FILL_HIGH_PCT after the insertion.
|
||||
void samplePacketForHistogram(uint32_t nodeId, uint8_t hopCount);
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Accessors
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
uint8_t getLastRequiredHop() const { return lastRequiredHop; }
|
||||
uint8_t getEntryCount() const { return count; }
|
||||
uint8_t getFillPercentage() const { return static_cast<uint8_t>((static_cast<uint16_t>(count) * 100u) / CAPACITY); }
|
||||
uint8_t getSamplingDenominator() const { return samplingDenominator; }
|
||||
uint8_t getFilteringDenominator() const { return filteringDenominator; }
|
||||
float getPoliteness() const { return lastPoliteNumer / static_cast<float>(POLITENESS_DENOM); }
|
||||
const PerHopCounts &getLastPerHopCounts() const { return lastPerHopCounts; }
|
||||
uint8_t getLastSuggestedHop() const { return lastSuggestedHop; }
|
||||
const MeshTrendStats &getLastTrendStats() const { return lastTrendStats; }
|
||||
|
||||
// Compatibility accessors used by tests
|
||||
uint8_t getCompactHistogramEntryCount() const { return getEntryCount(); }
|
||||
uint8_t getCompactHistogramDenominator() const { return getSamplingDenominator(); }
|
||||
uint8_t getCompactHistogramFilterDenominator() const { return getFilteringDenominator(); }
|
||||
uint8_t getCompactHistogramSuggestedHop() const { return getLastSuggestedHop(); }
|
||||
size_t getCompactHistogramAllSampleCount() const { return getEntryCount(); }
|
||||
|
||||
/// Force both sampling and filtering denominators to a specific value.
|
||||
/// Intended for unit tests that need a deterministic starting denominator.
|
||||
void setSamplingDenominator(uint8_t d)
|
||||
{
|
||||
samplingDenominator = (d < DENOM_MIN) ? DENOM_MIN : (d > DENOM_MAX ? DENOM_MAX : d);
|
||||
filteringDenominator = samplingDenominator;
|
||||
filteringDenomHoldRollsRemaining = 0;
|
||||
}
|
||||
|
||||
#ifdef PIO_UNIT_TESTING
|
||||
// Writable from tests as HopScalingModule::s_testNowMs; drives nowMs() in PIO_UNIT_TESTING builds.
|
||||
inline static uint32_t s_testNowMs = 0;
|
||||
/// Override the per-session hash seed. Use in tests that need a specific sampling distribution.
|
||||
void setHashSeed(uint16_t seed) { hashSeed = seed; }
|
||||
uint16_t getHashSeed() const { return hashSeed; }
|
||||
/// Expose hashNodeId for tests that need to compute which node IDs pass a given denominator.
|
||||
uint16_t hashNodeIdPublic(uint32_t nodeId) const { return hashNodeId(nodeId); }
|
||||
#endif
|
||||
|
||||
protected:
|
||||
int32_t runOnce() override;
|
||||
|
||||
private:
|
||||
#ifdef PIO_UNIT_TESTING
|
||||
friend class HopScalingTestShim;
|
||||
#endif
|
||||
|
||||
/// Perform hourly rollover.
|
||||
/// 1. Tallies per-hop counts for entries matching filteringDenominator and seen in 13 h.
|
||||
/// 2. Walks the scaled hop buckets and returns the recommended hop limit.
|
||||
/// 3. Logs scaled per-hop counts and recommendation.
|
||||
/// 4. Checks for scale-down (< FILL_LOW_PCT of capacity pass filteringDenominator).
|
||||
/// 5. Decrements filteringDenomHoldRollsRemaining (if > 0); once it reaches zero, halves
|
||||
/// filteringDenominator once toward samplingDenominator per rollHour() call.
|
||||
/// 6. Shifts all seen bitmaps left by one hour slot.
|
||||
void rollHour();
|
||||
// -----------------------------------------------------------------------
|
||||
// Persistence
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
/// Persist the histogram state (entries, denominators, hold-timer) to flash.
|
||||
/// No-op on platforms without a filesystem. Performs a full delete-and-rewrite of
|
||||
/// the state file on each call; avoid calling more frequently than once per rollHour().
|
||||
void saveToDisk() const;
|
||||
|
||||
/// Restore histogram state from flash. Safe to call even when no file exists.
|
||||
/// Call once after construction, before the first rollHour(), to warm-start the
|
||||
/// histogram across reboots without waiting 13 hours for data to re-accumulate.
|
||||
/// The restored entries are available immediately for sampling, but the first
|
||||
/// rollHour() (triggered by the second runOnce() tick) is needed before a warm-start
|
||||
/// recommendation replaces the HOP_MAX boot default.
|
||||
void loadFromDisk();
|
||||
|
||||
/// Remove stale entries (seen-bits all zero) and, if the list is still crowded,
|
||||
/// double samplingDenominator and filteringDenominator and remove non-matching entries.
|
||||
void trimIfNeeded();
|
||||
|
||||
void logStatusReport(bool didHourlyUpdate) const;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Histogram storage
|
||||
// -----------------------------------------------------------------------
|
||||
Record entries[CAPACITY] = {};
|
||||
uint8_t count = 0;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Denominator state
|
||||
//
|
||||
// Two separate denominators control two distinct gates:
|
||||
//
|
||||
// samplingDenominator — admission gate. A node is added/updated only when
|
||||
// passesFilter(hash, samplingDenominator). Lower value = more permissive =
|
||||
// more nodes enter = represents recent mesh state.
|
||||
//
|
||||
// filteringDenominator — counting gate. The hop-walk tally in rollHour() only
|
||||
// counts entries that pass passesFilter(hash, filteringDenominator). It moves
|
||||
// up with samplingDenominator immediately (scale-up) but is held at the
|
||||
// elevated value for FILTER_DENOM_HOLD_MS (13 h) after any scale-up before it
|
||||
// may drop back down (scale-down).
|
||||
//
|
||||
// Why the estimate is invariant: passesFilter uses a hash-based uniform subsample.
|
||||
// For any two powers-of-two denominators D ≤ F, the fraction of D-sampled entries
|
||||
// that also pass F is exactly D/F. Therefore:
|
||||
// raw_count × F = (total × D/F) × F = total × D
|
||||
// The population estimate is the same whether we count with D or with F.
|
||||
// The hold period is not about accuracy — it is about stability: it prevents the
|
||||
// hop recommendation from reacting to recently-admitted nodes that have not yet
|
||||
// accumulated enough seenHoursAgo history to be statistically reliable.
|
||||
//
|
||||
// denominatorHistory[h] — the filteringDenominator used to both gate and scale
|
||||
// hourlyRaw[h]. Invariant: denominatorHistory[h] always equals the
|
||||
// filteringDenominator that was active when seenHoursAgo bit h was set.
|
||||
// rollHour() advances the array at the very start (before the tally loop), then
|
||||
// gates hourlyRaw[h] per-slot by denominatorHistory[h] — each slot's raw count
|
||||
// and multiplier are therefore always consistent, even when filteringDenominator
|
||||
// changes between rolls (e.g. hold expiry). On scale-up (trimIfNeeded()), the
|
||||
// entire array is backfilled uniformly with the new filteringDenominator to
|
||||
// preserve the invariant retroactively for all 13 slots. Initialised to
|
||||
// DENOM_MIN (1); scaledPerHour slots that draw from a 1 entry are unscaled —
|
||||
// correct for a fresh instance with no prior history.
|
||||
// -----------------------------------------------------------------------
|
||||
uint8_t samplingDenominator = DENOM_MIN;
|
||||
uint8_t filteringDenominator = DENOM_MIN;
|
||||
uint8_t filteringDenomHoldRollsRemaining = 0; // counts down from FILTER_DENOM_HOLD_ROLLS to 0; step-down fires at 0
|
||||
uint8_t denominatorHistory[13] = {};
|
||||
uint16_t hashSeed = 0;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Cached hourly results
|
||||
// -----------------------------------------------------------------------
|
||||
PerHopCounts lastPerHopCounts = {};
|
||||
uint16_t lastScaledPerHop[MAX_HOP + 1] = {};
|
||||
uint8_t lastSuggestedHop = MAX_HOP;
|
||||
uint8_t lastPoliteNumer = POLITENESS_DEFAULT;
|
||||
MeshTrendStats lastTrendStats = {};
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Hop recommendation state
|
||||
// -----------------------------------------------------------------------
|
||||
uint8_t lastRequiredHop = HOP_MAX;
|
||||
uint8_t histogramRollCount = 0;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Scheduler state
|
||||
// -----------------------------------------------------------------------
|
||||
bool hasCompletedInitialRun = false;
|
||||
uint8_t runsSinceLastHourlyUpdate = 0;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Inline record helpers
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
// Record field semantics:
|
||||
// nodeHash → XOR-fold of full 32-bit node ID to 16 bits
|
||||
// hops_away → hop distance (0–7)
|
||||
// seenHoursAgo → 13-bit per-hour seen bitmap
|
||||
// bit 0 = seen in the current / most-recent hour
|
||||
// bit 12 = seen 12 hours ago
|
||||
// Shifts left on each rollHour(); 0 means not seen in 13 h.
|
||||
|
||||
/// XOR-fold + golden-ratio hash of a 32-bit node ID to 16 bits, mixed with the session seed.
|
||||
/// Multiplying by floor(2^32 / φ) gives uniform avalanche; XORing the seed ensures different
|
||||
/// devices (or the same device after a clear()) sample a different subset of node IDs.
|
||||
/// For seed=0 the function is deterministic, which is used in PIO_UNIT_TESTING builds.
|
||||
uint16_t hashNodeId(uint32_t nodeId) const { return static_cast<uint16_t>((nodeId * 2654435761u) >> 16) ^ hashSeed; }
|
||||
static bool seenInLast13h(const Record &r) { return r.seenHoursAgo != 0u; }
|
||||
static void markCurrentHour(Record &r) { r.seenHoursAgo |= 1u; }
|
||||
static void rollSeenBits(Record &r) { r.seenHoursAgo = (r.seenHoursAgo << 1u) & 0x1FFFu; }
|
||||
static bool passesFilter(uint16_t nodeHash, uint8_t denom) { return (nodeHash & static_cast<uint16_t>(denom - 1u)) == 0u; }
|
||||
|
||||
public:
|
||||
// Clock — public so tests can share the same timebase via HopScalingModule::s_testNowMs
|
||||
#ifdef PIO_UNIT_TESTING
|
||||
static uint32_t nowMs() { return s_testNowMs; }
|
||||
#else
|
||||
static uint32_t nowMs() { return millis(); }
|
||||
#endif
|
||||
};
|
||||
|
||||
extern HopScalingModule *hopScalingModule;
|
||||
|
||||
#endif
|
||||
@@ -41,6 +41,9 @@
|
||||
#if HAS_TRAFFIC_MANAGEMENT && !MESHTASTIC_EXCLUDE_TRAFFIC_MANAGEMENT
|
||||
#include "modules/TrafficManagementModule.h"
|
||||
#endif
|
||||
#if HAS_VARIABLE_HOPS
|
||||
#include "modules/HopScalingModule.h"
|
||||
#endif
|
||||
#include "modules/TextMessageModule.h"
|
||||
#if !MESHTASTIC_EXCLUDE_TRACEROUTE
|
||||
#include "modules/TraceRouteModule.h"
|
||||
@@ -131,6 +134,10 @@ void setupModules()
|
||||
}
|
||||
#endif
|
||||
|
||||
#if HAS_VARIABLE_HOPS
|
||||
hopScalingModule = new HopScalingModule();
|
||||
#endif
|
||||
|
||||
#if !MESHTASTIC_EXCLUDE_ADMIN
|
||||
adminModule = new AdminModule();
|
||||
#endif
|
||||
|
||||
+15
-8
@@ -200,7 +200,7 @@ class MockNodeDB : public NodeDB
|
||||
node.num = num;
|
||||
node.has_hops_away = hasHops;
|
||||
node.hops_away = hopsAway;
|
||||
node.via_mqtt = viaMqtt;
|
||||
nodeInfoLiteSetBit(&node, NODEINFO_BITFIELD_VIA_MQTT_MASK, viaMqtt);
|
||||
node.last_heard = getTime() - ageSecs;
|
||||
testNodes.push_back(node);
|
||||
meshNodes = &testNodes;
|
||||
@@ -358,6 +358,7 @@ PlatformIO defines `PIO_UNIT_TESTING` during `pio test` builds. Several producti
|
||||
- [ ] Set `nodeDB = mockNodeDB`
|
||||
- [ ] Delete persisted state files (`FSCom.remove(...)`)
|
||||
- [ ] Reset file-scope mutable globals
|
||||
- [ ] Reset mock clock to a safe base value (e.g. `mockTime = ONE_HOUR_MS`) — prevents unsigned subtraction underflow in time-dependent logic
|
||||
- [ ] Disable randomness/jitter flags
|
||||
- [ ] In `tearDown`: null the global singleton pointer, restore flags
|
||||
|
||||
@@ -375,15 +376,21 @@ A well-structured test suite follows this pattern:
|
||||
|
||||
| Suite | Module Under Test |
|
||||
| ---------------------------- | ----------------------------- |
|
||||
| `test_admin_radio` | Admin + LoRa region config |
|
||||
| `test_atak` | ATAK integration |
|
||||
| `test_crypto` | CryptoEngine |
|
||||
| `test_mqtt` | MQTT integration |
|
||||
| `test_radio` | Radio interface |
|
||||
| `test_default` | Default configuration helpers |
|
||||
| `test_hop_scaling` | Hop scaling algorithm |
|
||||
| `test_http_content_handler` | HTTP handling |
|
||||
| `test_mac_from_string` | MAC address parsing |
|
||||
| `test_mesh_module` | Module framework |
|
||||
| `test_meshpacket_serializer` | Packet serialization |
|
||||
| `test_transmit_history` | Retransmission tracking |
|
||||
| `test_atak` | ATAK integration |
|
||||
| `test_default` | Default configuration helpers |
|
||||
| `test_http_content_handler` | HTTP handling |
|
||||
| `test_mqtt` | MQTT integration |
|
||||
| `test_packet_history` | Packet history tracking |
|
||||
| `test_position_precision` | Position precision helpers |
|
||||
| `test_radio` | Radio interface |
|
||||
| `test_serial` | Serial communication |
|
||||
| `test_hop_scaling` | Hop scaling algorithm |
|
||||
| `test_traffic_management` | Traffic management |
|
||||
| `test_transmit_history` | Retransmission tracking |
|
||||
| `test_type_conversions` | NodeDB v25 type conversions |
|
||||
| `test_utf8` | UTF-8 utilities |
|
||||
|
||||
@@ -0,0 +1,738 @@
|
||||
#include "MeshTypes.h"
|
||||
#include "TestUtil.h"
|
||||
#include <unity.h>
|
||||
|
||||
#if HAS_VARIABLE_HOPS
|
||||
|
||||
#include "FSCommon.h"
|
||||
#include "gps/RTC.h"
|
||||
#include "mesh/NodeDB.h"
|
||||
#include "modules/HopScalingModule.h"
|
||||
#include <cstdio>
|
||||
#include <cstring>
|
||||
#include <memory>
|
||||
|
||||
// Unity only shows TEST_MESSAGE output. printf goes to stdout which the runner swallows.
|
||||
#define MSG_BUF_LEN 200
|
||||
#define TEST_MSG_FMT(fmt, ...) \
|
||||
do { \
|
||||
char _buf[MSG_BUF_LEN]; \
|
||||
snprintf(_buf, sizeof(_buf), fmt, __VA_ARGS__); \
|
||||
TEST_MESSAGE(_buf); \
|
||||
} while (0)
|
||||
|
||||
static constexpr NodeNum kLocalNode = 0x11111111;
|
||||
|
||||
// Shared mock clock — drives HopScalingModule::nowMs()
|
||||
static uint32_t &mockTime = HopScalingModule::s_testNowMs;
|
||||
static constexpr uint32_t ONE_HOUR_MS = 3600UL * 1000UL;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MockNodeDB — not used for hop decisions any more, kept for completeness
|
||||
// ---------------------------------------------------------------------------
|
||||
class MockNodeDB : public NodeDB
|
||||
{
|
||||
public:
|
||||
void clearTestNodes()
|
||||
{
|
||||
testNodes.clear();
|
||||
numMeshNodes = 0;
|
||||
}
|
||||
|
||||
void addTestNode(NodeNum num, uint8_t hopsAway, bool hasHops, uint32_t ageSecs, bool viaMqtt = false)
|
||||
{
|
||||
meshtastic_NodeInfoLite node = meshtastic_NodeInfoLite_init_zero;
|
||||
node.num = num;
|
||||
node.has_hops_away = hasHops;
|
||||
node.hops_away = hopsAway;
|
||||
nodeInfoLiteSetBit(&node, NODEINFO_BITFIELD_VIA_MQTT_MASK, viaMqtt);
|
||||
node.last_heard = getTime() - ageSecs;
|
||||
testNodes.push_back(node);
|
||||
meshNodes = &testNodes;
|
||||
numMeshNodes = testNodes.size();
|
||||
}
|
||||
|
||||
std::vector<meshtastic_NodeInfoLite> testNodes;
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Test shim — expose protected/private members for direct invocation
|
||||
// ---------------------------------------------------------------------------
|
||||
class HopScalingTestShim : public HopScalingModule
|
||||
{
|
||||
public:
|
||||
using HopScalingModule::runOnce;
|
||||
using HopScalingModule::samplePacketForHistogram;
|
||||
|
||||
using HopScalingModule::getLastRequiredHop;
|
||||
|
||||
// Test-only helpers (require UNIT_TEST friend access)
|
||||
void rollHourTest() { rollHour(); }
|
||||
void setHistogramDenominator(uint8_t d) { setSamplingDenominator(d); }
|
||||
|
||||
/// Directly set denominator state, bypassing any scale-up/down logic.
|
||||
/// Used by tests that need a specific pre-condition without triggering trim.
|
||||
void forceFilterDenomState(uint8_t samp, uint8_t filt, uint8_t holdRolls)
|
||||
{
|
||||
samplingDenominator = samp;
|
||||
filteringDenominator = filt;
|
||||
filteringDenomHoldRollsRemaining = holdRolls;
|
||||
}
|
||||
uint8_t getFilteringDenomHoldRollsRemaining() const { return filteringDenomHoldRollsRemaining; }
|
||||
|
||||
/// Insert an entry with an explicit hash, bypassing the sampling filter.
|
||||
/// Used to fill the histogram to a known state without depending on hashNodeId distribution.
|
||||
void forceInsertEntry(uint16_t hash, uint8_t hops)
|
||||
{
|
||||
if (count < CAPACITY) {
|
||||
entries[count].nodeHash = hash;
|
||||
entries[count].hops_away = hops;
|
||||
entries[count].seenHoursAgo = 1u;
|
||||
count++;
|
||||
}
|
||||
}
|
||||
|
||||
// Size introspection for test_memory_layout
|
||||
static constexpr size_t sizeofSelf() { return sizeof(HopScalingModule); }
|
||||
};
|
||||
|
||||
static MockNodeDB *mockNodeDB = nullptr;
|
||||
|
||||
// Create deterministic IDs that produce a broad spread of 16-bit hashes.
|
||||
// HopScalingModule admission uses passesFilter(hashNodeId(nodeId), denom), NOT a raw nodeId
|
||||
// modulo check — do not assume (nodeId & (denom-1)) == 0 determines whether a node is admitted.
|
||||
static uint32_t makeDistributedNodeId(uint32_t baseId, uint32_t ordinal, uint32_t salt = 0)
|
||||
{
|
||||
return baseId + salt + (ordinal * 33u);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers — mesh topology builders
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// Helper: add N nodes at a given hop with ages spread across a time range.
|
||||
static void addNodesAtHop(uint32_t baseId, uint8_t hop, uint32_t count, uint32_t ageSecs, uint32_t stride = 10)
|
||||
{
|
||||
for (uint32_t i = 0; i < count; i++) {
|
||||
const uint32_t nodeId = makeDistributedNodeId(baseId, i, static_cast<uint32_t>(hop) << 8);
|
||||
mockNodeDB->addTestNode(nodeId, hop, true, ageSecs + i * stride);
|
||||
}
|
||||
}
|
||||
|
||||
// Feed sampled traffic into the histogram.
|
||||
// Advances mock clock by one hour per roll and calls rollHour() so each roll produces data.
|
||||
static void injectSampleTraffic(HopScalingTestShim &shim, uint32_t baseId, const uint16_t hopDist[HOP_MAX + 1],
|
||||
uint8_t numRolls = 16)
|
||||
{
|
||||
shim.setHistogramDenominator(HopScalingModule::DENOM_MIN);
|
||||
|
||||
for (uint8_t roll = 0; roll < numRolls; ++roll) {
|
||||
mockTime += ONE_HOUR_MS;
|
||||
|
||||
uint16_t ordinal = 0;
|
||||
for (uint8_t hop = 0; hop <= HOP_MAX; ++hop) {
|
||||
for (uint16_t n = 0; n < hopDist[hop]; ++n) {
|
||||
const uint32_t nodeId = makeDistributedNodeId(baseId, ordinal);
|
||||
shim.samplePacketForHistogram(nodeId, hop);
|
||||
++ordinal;
|
||||
}
|
||||
}
|
||||
shim.rollHourTest();
|
||||
}
|
||||
}
|
||||
|
||||
static void assertCompactHistogramActive(HopScalingTestShim &shim)
|
||||
{
|
||||
TEST_ASSERT_GREATER_THAN_UINT8(0, shim.getCompactHistogramEntryCount());
|
||||
TEST_ASSERT_TRUE(shim.getCompactHistogramAllSampleCount() > 0);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Topology builders
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// Scenario A: Dense local mesh — 110 nodes, heavy at hops 0–2.
|
||||
static void buildDenseLocalMesh()
|
||||
{
|
||||
mockNodeDB->clearTestNodes();
|
||||
addNodesAtHop(0x1000, 0, 25, 120);
|
||||
addNodesAtHop(0x2000, 1, 30, 300);
|
||||
addNodesAtHop(0x3000, 2, 15, 600);
|
||||
addNodesAtHop(0x4000, 3, 5, 1200);
|
||||
addNodesAtHop(0x5000, 4, 10, 1800);
|
||||
addNodesAtHop(0x6000, 5, 15, 2400);
|
||||
addNodesAtHop(0x7000, 6, 10, 3000);
|
||||
}
|
||||
|
||||
// Scenario B: Spread sparse mesh — 76 nodes across hops 0–7.
|
||||
static void buildSpreadSparseMesh()
|
||||
{
|
||||
mockNodeDB->clearTestNodes();
|
||||
addNodesAtHop(0x1000, 0, 5, 120);
|
||||
addNodesAtHop(0x2000, 1, 8, 300);
|
||||
addNodesAtHop(0x3000, 2, 12, 600);
|
||||
addNodesAtHop(0x4000, 3, 15, 900);
|
||||
addNodesAtHop(0x5000, 4, 10, 1200);
|
||||
addNodesAtHop(0x6000, 5, 6, 1800);
|
||||
addNodesAtHop(0x7000, 6, 10, 3000);
|
||||
addNodesAtHop(0x8000, 7, 10, 3600);
|
||||
}
|
||||
|
||||
// Scenario C: Deep linear chain — 22 thin nodes, never reaches 40.
|
||||
static void buildDeepLinearChain()
|
||||
{
|
||||
mockNodeDB->clearTestNodes();
|
||||
addNodesAtHop(0x1000, 0, 2, 120);
|
||||
addNodesAtHop(0x2000, 1, 3, 300);
|
||||
addNodesAtHop(0x3000, 2, 3, 600);
|
||||
addNodesAtHop(0x4000, 3, 4, 900);
|
||||
addNodesAtHop(0x5000, 4, 3, 1200);
|
||||
addNodesAtHop(0x6000, 5, 2, 1800);
|
||||
addNodesAtHop(0x7000, 6, 2, 2400);
|
||||
addNodesAtHop(0x8000, 7, 3, 3600);
|
||||
}
|
||||
|
||||
// Scenario D: Router cluster — 71 nodes, 45 at hop 2.
|
||||
static void buildRouterCluster()
|
||||
{
|
||||
mockNodeDB->clearTestNodes();
|
||||
addNodesAtHop(0x1000, 0, 3, 120);
|
||||
addNodesAtHop(0x2000, 1, 5, 300);
|
||||
addNodesAtHop(0x3000, 2, 45, 600);
|
||||
addNodesAtHop(0x4000, 3, 8, 1200);
|
||||
addNodesAtHop(0x5000, 4, 3, 1200);
|
||||
addNodesAtHop(0x6000, 5, 2, 1800);
|
||||
addNodesAtHop(0x7000, 6, 2, 2400);
|
||||
addNodesAtHop(0x8000, 7, 3, 3600);
|
||||
}
|
||||
|
||||
// Scenario E: Megamesh — 199 nodes (DB near capacity).
|
||||
static void buildMegamesh()
|
||||
{
|
||||
mockNodeDB->clearTestNodes();
|
||||
addNodesAtHop(0x01000, 0, 30, 120);
|
||||
addNodesAtHop(0x02000, 1, 40, 300);
|
||||
addNodesAtHop(0x03000, 2, 35, 600);
|
||||
addNodesAtHop(0x04000, 3, 30, 900);
|
||||
addNodesAtHop(0x05000, 4, 20, 1200);
|
||||
addNodesAtHop(0x06000, 5, 15, 1800);
|
||||
addNodesAtHop(0x07000, 6, 14, 2400);
|
||||
addNodesAtHop(0x08000, 7, 15, 3600);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests — Topology-driven hop reduction scenarios
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
void test_dense_local_telemetry()
|
||||
{
|
||||
TEST_MESSAGE("=== Dense local mesh: telemetry broadcast ===");
|
||||
TEST_MESSAGE("Topology: 110 nodes with 25/30/15 nodes at hops 0/1/2 and a thinner tail to hop 6.");
|
||||
TEST_MESSAGE("Expectation: cumulative reaches 55 nodes by hop 1, result stays tightly constrained.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
buildDenseLocalMesh();
|
||||
const uint16_t distA[HOP_MAX + 1] = {25, 30, 15, 5, 10, 15, 10, 0};
|
||||
injectSampleTraffic(*shim, 0x91000000, distA);
|
||||
shim->runOnce();
|
||||
|
||||
TEST_MSG_FMT("Dense local: hop=%u", shim->getLastRequiredHop());
|
||||
TEST_ASSERT_TRUE(shim->getLastRequiredHop() <= 3);
|
||||
TEST_ASSERT_TRUE(shim->getLastRequiredHop() >= 1);
|
||||
assertCompactHistogramActive(*shim);
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_spread_sparse_position()
|
||||
{
|
||||
TEST_MESSAGE("=== Spread sparse mesh: position broadcast ===");
|
||||
TEST_MESSAGE("Topology: 76 nodes spread across all hops, reaching 40 nodes only when hop 3 is included.");
|
||||
TEST_MESSAGE("Expectation: hop settles in the 3-5 range.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
buildSpreadSparseMesh();
|
||||
const uint16_t distB[HOP_MAX + 1] = {5, 8, 12, 15, 10, 6, 10, 10};
|
||||
injectSampleTraffic(*shim, 0x92000000, distB);
|
||||
shim->runOnce();
|
||||
|
||||
TEST_MSG_FMT("Spread sparse: hop=%u", shim->getLastRequiredHop());
|
||||
TEST_ASSERT_TRUE(shim->getLastRequiredHop() >= 3);
|
||||
TEST_ASSERT_TRUE(shim->getLastRequiredHop() <= 5);
|
||||
assertCompactHistogramActive(*shim);
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_deep_chain_position()
|
||||
{
|
||||
TEST_MESSAGE("=== Deep linear chain: position broadcast ===");
|
||||
TEST_MESSAGE("Topology: 22 nodes spread thinly across hops 0-7, never reaching the 40-node floor.");
|
||||
TEST_MESSAGE("Expectation: module must keep HOP_MAX.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
buildDeepLinearChain();
|
||||
const uint16_t distC[HOP_MAX + 1] = {2, 3, 3, 4, 3, 2, 2, 3};
|
||||
injectSampleTraffic(*shim, 0x93000000, distC);
|
||||
shim->runOnce();
|
||||
|
||||
TEST_MSG_FMT("Deep chain: hop=%u", shim->getLastRequiredHop());
|
||||
TEST_ASSERT_EQUAL_UINT8(HOP_MAX, shim->getLastRequiredHop());
|
||||
assertCompactHistogramActive(*shim);
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_router_cluster_telemetry()
|
||||
{
|
||||
TEST_MESSAGE("=== Router cluster: telemetry broadcast ===");
|
||||
TEST_MESSAGE("Topology: 71 nodes with a concentrated 45-node cluster at hop 2.");
|
||||
TEST_MESSAGE("Expectation: result stays in the 2-4 range.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
buildRouterCluster();
|
||||
const uint16_t distD[HOP_MAX + 1] = {3, 5, 45, 8, 3, 2, 2, 3};
|
||||
injectSampleTraffic(*shim, 0x94000000, distD);
|
||||
shim->runOnce();
|
||||
|
||||
TEST_MSG_FMT("Router cluster: hop=%u", shim->getLastRequiredHop());
|
||||
TEST_ASSERT_TRUE(shim->getLastRequiredHop() >= 2);
|
||||
TEST_ASSERT_TRUE(shim->getLastRequiredHop() <= 4);
|
||||
assertCompactHistogramActive(*shim);
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_megamesh_eviction_scaling()
|
||||
{
|
||||
TEST_MESSAGE("=== Megamesh with eviction scaling ===");
|
||||
TEST_MESSAGE("Topology: NodeDB at capacity (199 nodes), ~2000-node mesh with sustained eviction pressure.");
|
||||
TEST_MESSAGE("Expectation: sustained evictions tracked in rolling average, hop stays well below HOP_MAX.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
buildMegamesh();
|
||||
|
||||
const uint16_t distE[HOP_MAX + 1] = {301, 402, 352, 301, 201, 151, 141, 151};
|
||||
injectSampleTraffic(*shim, 0x9B000000, distE);
|
||||
|
||||
shim->runOnce();
|
||||
uint8_t hopBefore = shim->getLastRequiredHop();
|
||||
TEST_MSG_FMT("Megamesh initial: hop=%u", hopBefore);
|
||||
|
||||
for (int hour = 0; hour < 3; hour++) {
|
||||
mockTime += ONE_HOUR_MS;
|
||||
{
|
||||
const uint16_t megaDist[HOP_MAX + 1] = {301, 402, 352, 301, 201, 151, 141, 151};
|
||||
uint16_t ordinal = 0;
|
||||
for (uint8_t hop = 0; hop <= HOP_MAX; ++hop) {
|
||||
for (uint16_t n = 0; n < megaDist[hop]; ++n) {
|
||||
const uint32_t nodeId = makeDistributedNodeId(0x9C000000u, ordinal, static_cast<uint32_t>(hour) * 0x10000u);
|
||||
shim->samplePacketForHistogram(nodeId, hop);
|
||||
++ordinal;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (int run = 0; run < 7; run++)
|
||||
shim->runOnce();
|
||||
|
||||
TEST_MSG_FMT("Megamesh hour %d: hop=%u", hour + 1, shim->getLastRequiredHop());
|
||||
}
|
||||
|
||||
TEST_MESSAGE("Assertion: hop stays well below HOP_MAX on a large-distribution mesh.");
|
||||
TEST_ASSERT_TRUE(shim->getLastRequiredHop() <= 3);
|
||||
assertCompactHistogramActive(*shim);
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_sparse_to_dense_transition()
|
||||
{
|
||||
TEST_MESSAGE("=== Sparse-to-dense transition ===");
|
||||
TEST_MESSAGE("Topology change: start with a 22-node deep chain, then inject 50 new neighbors at hops 0-1.");
|
||||
TEST_MESSAGE("Expectation: hop drops sharply once the local neighborhood becomes dense.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
|
||||
buildDeepLinearChain();
|
||||
const uint16_t distC2[HOP_MAX + 1] = {2, 3, 3, 4, 3, 2, 2, 3};
|
||||
injectSampleTraffic(*shim, 0x95000000, distC2);
|
||||
shim->runOnce();
|
||||
uint8_t hopSparse = shim->getLastRequiredHop();
|
||||
TEST_MSG_FMT("Phase 1 sparse: hop=%u (expect %u)", hopSparse, HOP_MAX);
|
||||
TEST_ASSERT_EQUAL_UINT8(HOP_MAX, hopSparse);
|
||||
|
||||
addNodesAtHop(0xA000, 0, 25, 120);
|
||||
addNodesAtHop(0xB000, 1, 25, 300);
|
||||
|
||||
for (uint32_t i = 0; i < 25; ++i)
|
||||
shim->samplePacketForHistogram(makeDistributedNodeId(0xA000, i, static_cast<uint32_t>(0) << 8), 0);
|
||||
for (uint32_t i = 0; i < 25; ++i)
|
||||
shim->samplePacketForHistogram(makeDistributedNodeId(0xB000, i, static_cast<uint32_t>(1) << 8), 1);
|
||||
|
||||
for (int run = 0; run < HopScalingModule::RUNS_PER_HOUR; run++)
|
||||
shim->runOnce();
|
||||
|
||||
uint8_t hopDense = shim->getLastRequiredHop();
|
||||
TEST_MSG_FMT("Phase 2 dense: hop=%u (expect <= 3)", hopDense);
|
||||
|
||||
TEST_ASSERT_TRUE(hopDense < hopSparse);
|
||||
TEST_ASSERT_TRUE(hopDense <= 3);
|
||||
assertCompactHistogramActive(*shim);
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_state_persistence()
|
||||
{
|
||||
TEST_MESSAGE("=== State persistence across restart ===");
|
||||
TEST_MESSAGE("Expectation: histogram entries survive instance teardown and reload.");
|
||||
|
||||
{
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
|
||||
const uint16_t dist[HOP_MAX + 1] = {5, 8, 12, 10, 5, 3, 2, 1};
|
||||
injectSampleTraffic(*shim, 0x9D000000, dist, 2);
|
||||
|
||||
TEST_MSG_FMT("Phase 1: entries=%u hop=%u", shim->getEntryCount(), shim->getLastRequiredHop());
|
||||
TEST_ASSERT_GREATER_THAN_UINT8(0, shim->getEntryCount());
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
{
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
shim->runOnce();
|
||||
|
||||
TEST_MSG_FMT("Phase 2 restored: entries=%u hop=%u", shim->getEntryCount(), shim->getLastRequiredHop());
|
||||
TEST_ASSERT_GREATER_THAN_UINT8(0, shim->getEntryCount());
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
void test_hourly_roll()
|
||||
{
|
||||
TEST_MESSAGE("=== Hourly roll cycle ===");
|
||||
TEST_MESSAGE("Expectation: histogram accumulates data and provides valid hop recommendation after multiple rolls.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
buildSpreadSparseMesh();
|
||||
shim->setHistogramDenominator(HopScalingModule::DENOM_MIN);
|
||||
|
||||
for (uint32_t i = 1; i <= 30; i++) {
|
||||
const uint32_t nodeId = makeDistributedNodeId(0x97000000, i, 0xAAu);
|
||||
shim->samplePacketForHistogram(nodeId, static_cast<uint8_t>(i % (HOP_MAX + 1)));
|
||||
}
|
||||
|
||||
for (int run = 0; run < 13; run++) {
|
||||
int32_t interval = shim->runOnce();
|
||||
TEST_ASSERT_GREATER_THAN(0, interval);
|
||||
}
|
||||
|
||||
TEST_MSG_FMT("Hourly roll: hop=%u entries=%u", shim->getLastRequiredHop(), shim->getEntryCount());
|
||||
assertCompactHistogramActive(*shim);
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_intermediate_status()
|
||||
{
|
||||
TEST_MESSAGE("=== Intermediate status (no recomputation) ===");
|
||||
TEST_MESSAGE("Expectation: runs between hourly updates leave hop unchanged.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
buildRouterCluster();
|
||||
const uint16_t distD[HOP_MAX + 1] = {3, 5, 45, 8, 3, 2, 2, 3};
|
||||
injectSampleTraffic(*shim, 0x98000000, distD);
|
||||
|
||||
shim->runOnce();
|
||||
uint8_t hopAfterInitial = shim->getLastRequiredHop();
|
||||
TEST_MSG_FMT("Initial: hop=%u", hopAfterInitial);
|
||||
|
||||
for (int run = 0; run < 3; run++) {
|
||||
shim->runOnce();
|
||||
TEST_ASSERT_EQUAL_UINT8(hopAfterInitial, shim->getLastRequiredHop());
|
||||
}
|
||||
|
||||
TEST_MSG_FMT("After 3 intermediate runs: hop=%u (unchanged)", shim->getLastRequiredHop());
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_startup_blank_state()
|
||||
{
|
||||
TEST_MESSAGE("=== Startup with blank state ===");
|
||||
TEST_MESSAGE("Expectation: fresh instance starts with zeroed rolling averages and a valid hop result.");
|
||||
|
||||
#ifdef FSCom
|
||||
FSCom.remove("/prefs/hopScalingState.bin");
|
||||
#endif
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
buildDeepLinearChain();
|
||||
|
||||
int32_t interval = shim->runOnce();
|
||||
|
||||
TEST_ASSERT_GREATER_THAN(0, interval);
|
||||
TEST_ASSERT_TRUE(shim->getLastRequiredHop() <= HOP_MAX);
|
||||
|
||||
TEST_MSG_FMT("Startup blank: hop=%u", shim->getLastRequiredHop());
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests — Denominator state machine
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
void test_denominator_rises_on_overflow()
|
||||
{
|
||||
TEST_MESSAGE("=== samplingDenominator doubles when histogram overflows ===");
|
||||
TEST_MESSAGE("Fill to > FILL_HIGH_PCT with forceInsertEntry, then trigger via samplePacketForHistogram.");
|
||||
TEST_MESSAGE("Expectation: samp/filt both double to 2, hold set to FILTER_DENOM_HOLD_ROLLS.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
|
||||
// Insert 103 entries with hashes 1..103 (all distinct, no sampling-filter skew).
|
||||
// 103 / 128 = 80.4% fill, which meets FILL_HIGH_PCT=80.
|
||||
// Odd hashes (1,3,...,103) will be evicted when denom doubles to 2; even ones survive.
|
||||
static constexpr uint8_t FILL_COUNT = 103u;
|
||||
for (uint8_t i = 1; i <= FILL_COUNT; i++)
|
||||
shim->forceInsertEntry(i, 2u);
|
||||
|
||||
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::DENOM_MIN, shim->getSamplingDenominator());
|
||||
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::DENOM_MIN, shim->getFilteringDenominator());
|
||||
TEST_ASSERT_EQUAL_UINT8(0u, shim->getFilteringDenomHoldRollsRemaining());
|
||||
TEST_ASSERT_EQUAL_UINT8(FILL_COUNT, shim->getEntryCount());
|
||||
|
||||
// A new node passes the denom=1 admission gate; fill ≥ 80% triggers trimIfNeeded → doubling.
|
||||
shim->samplePacketForHistogram(0xB0000000u, 1u);
|
||||
|
||||
TEST_MSG_FMT("After scale-up: samp=1/%u filt=1/%u holdRolls=%u entries=%u", shim->getSamplingDenominator(),
|
||||
shim->getFilteringDenominator(), shim->getFilteringDenomHoldRollsRemaining(), shim->getEntryCount());
|
||||
|
||||
TEST_ASSERT_EQUAL_UINT8(2u, shim->getSamplingDenominator());
|
||||
TEST_ASSERT_EQUAL_UINT8(2u, shim->getFilteringDenominator());
|
||||
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::FILTER_DENOM_HOLD_ROLLS, shim->getFilteringDenomHoldRollsRemaining());
|
||||
// After evicting entries with (hash & 1) != 0, roughly half the entries remain.
|
||||
TEST_ASSERT_LESS_THAN_UINT8(FILL_COUNT, shim->getEntryCount());
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_filtering_denom_hold_counts_down()
|
||||
{
|
||||
TEST_MESSAGE("=== filteringDenominator held while hold counter > 0 ===");
|
||||
TEST_MESSAGE("Force filt=4 samp=1 hold=3; verify no step for 2 rolls, then step fires on roll 3.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
|
||||
// samp=DENOM_MIN so scale-down in step 4 can't go lower; hold=3 for a short, fast test.
|
||||
shim->forceFilterDenomState(HopScalingModule::DENOM_MIN, 4u, 3u);
|
||||
|
||||
shim->rollHourTest(); // hold 3→2, no step
|
||||
TEST_ASSERT_EQUAL_UINT8(4u, shim->getFilteringDenominator());
|
||||
TEST_ASSERT_EQUAL_UINT8(2u, shim->getFilteringDenomHoldRollsRemaining());
|
||||
|
||||
shim->rollHourTest(); // hold 2→1, no step
|
||||
TEST_ASSERT_EQUAL_UINT8(4u, shim->getFilteringDenominator());
|
||||
TEST_ASSERT_EQUAL_UINT8(1u, shim->getFilteringDenomHoldRollsRemaining());
|
||||
|
||||
// Roll 3: hold 1→0, step fires — filteringDenominator halves to max(2, samp=1) = 2.
|
||||
shim->rollHourTest();
|
||||
TEST_MSG_FMT("After hold expires: filt=1/%u samp=1/%u holdRolls=%u", shim->getFilteringDenominator(),
|
||||
shim->getSamplingDenominator(), shim->getFilteringDenomHoldRollsRemaining());
|
||||
TEST_ASSERT_EQUAL_UINT8(2u, shim->getFilteringDenominator());
|
||||
TEST_ASSERT_EQUAL_UINT8(0u, shim->getFilteringDenomHoldRollsRemaining());
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_filtering_denom_steps_down_gradually()
|
||||
{
|
||||
TEST_MESSAGE("=== filteringDenominator descends one halving per rollHour() after hold expires ===");
|
||||
TEST_MESSAGE("Force filt=8 samp=1 hold=1; expect 8→4→2→1 over 3 rolls, then stable.");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
|
||||
shim->forceFilterDenomState(HopScalingModule::DENOM_MIN, 8u, 1u);
|
||||
|
||||
shim->rollHourTest(); // hold 1→0, step: 8/2=4 > 1, filt=4
|
||||
TEST_ASSERT_EQUAL_UINT8(4u, shim->getFilteringDenominator());
|
||||
|
||||
shim->rollHourTest(); // hold=0 (no decrement), step: 4/2=2 > 1, filt=2
|
||||
TEST_ASSERT_EQUAL_UINT8(2u, shim->getFilteringDenominator());
|
||||
|
||||
shim->rollHourTest(); // step: 2/2=1, not > samp=1, filt=samp=1 — converged
|
||||
TEST_ASSERT_EQUAL_UINT8(1u, shim->getFilteringDenominator());
|
||||
|
||||
shim->rollHourTest(); // filt==samp, outer if is false — no further change
|
||||
TEST_ASSERT_EQUAL_UINT8(1u, shim->getFilteringDenominator());
|
||||
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::DENOM_MIN, shim->getSamplingDenominator());
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_full_at_denom_max_drops_entry()
|
||||
{
|
||||
TEST_MESSAGE("=== Full histogram at DENOM_MAX drops new entries ===");
|
||||
TEST_MESSAGE("Fill CAPACITY entries, force samp=DENOM_MAX, sample admissible node.");
|
||||
TEST_MESSAGE("Expectation: entry count stays at CAPACITY (LOG_WARN fires; visible in test output).");
|
||||
|
||||
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
|
||||
hopScalingModule = shim.get();
|
||||
shim->setHashSeed(0); // deterministic hash for admissible-ID search
|
||||
|
||||
shim->forceFilterDenomState(HopScalingModule::DENOM_MAX, HopScalingModule::DENOM_MAX, 0u);
|
||||
|
||||
// Fill with odd hashes 1,3,5,...,(2*CAPACITY-1). None are multiples of 128, so none
|
||||
// collide with the admissible node's hash (which must be a multiple of 128).
|
||||
for (uint16_t i = 0; i < HopScalingModule::CAPACITY; i++)
|
||||
shim->forceInsertEntry(static_cast<uint16_t>(2u * i + 1u), 1u);
|
||||
|
||||
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::CAPACITY, shim->getEntryCount());
|
||||
|
||||
// Find a node ID whose hash passes DENOM_MAX, i.e. (hash & 127) == 0.
|
||||
uint32_t admissibleId = 0;
|
||||
for (uint32_t id = 1u; id < 0x10000u; id++) {
|
||||
if ((shim->hashNodeIdPublic(id) & (HopScalingModule::DENOM_MAX - 1u)) == 0u) {
|
||||
admissibleId = id;
|
||||
break;
|
||||
}
|
||||
}
|
||||
TEST_ASSERT_NOT_EQUAL(0u, admissibleId); // sanity: the hash space is dense enough to find one quickly
|
||||
|
||||
shim->samplePacketForHistogram(admissibleId, 3u);
|
||||
|
||||
TEST_MSG_FMT("After drop attempt: entries=%u CAPACITY=%u admissibleId=0x%08x hash=0x%04x", shim->getEntryCount(),
|
||||
static_cast<unsigned>(HopScalingModule::CAPACITY), admissibleId,
|
||||
static_cast<unsigned>(shim->hashNodeIdPublic(admissibleId)));
|
||||
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::CAPACITY, shim->getEntryCount());
|
||||
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void test_scenario_summary_output()
|
||||
{
|
||||
TEST_MESSAGE("=== Scenario summary ===");
|
||||
TEST_MESSAGE("Scenario | Nodes | Distribution | Hop | Why");
|
||||
TEST_MESSAGE("A: Dense local | 110 | 25/30/15/5/10/15/10 h0-6 | 1-2 | 55 nodes at h1 >> 40");
|
||||
TEST_MESSAGE("B: Spread | 76 | 5/8/12/15/10/6/10/10 h0-7 | 3-4 | Need h3 to reach 40");
|
||||
TEST_MESSAGE("C: Deep chain | 22 | 2/3/3/4/3/2/2/3 h0-7 | 7 | Never reaches 40");
|
||||
TEST_MESSAGE("D: Router | 71 | 3/5/45/8/3/2/2/3 h0-7 | 2-3 | 45-node hop-2 cluster");
|
||||
TEST_MESSAGE("E: Megamesh | 199 | 30/40/35/30/20/15/14/15 h0-7 | 0-1 | Dense low-hop histogram");
|
||||
TEST_MESSAGE("F: Transition | 22->72 | Chain -> dense local | 7-><=3 | Adapts to new neighbors");
|
||||
TEST_MESSAGE("G: Persistence | -- | -- | -- | Eviction avg survives reboot");
|
||||
TEST_MESSAGE("");
|
||||
TEST_MESSAGE("=== Denominator state machine summary ===");
|
||||
TEST_MESSAGE("Test | Pre-condition | Expectation");
|
||||
TEST_MESSAGE("H: Rises on overflow | 103 entries forced, denom=1 | samp/filt→2, holdRolls=13");
|
||||
TEST_MESSAGE(
|
||||
"I: Hold counts down | filt=4 samp=1 hold=3 | no step for 2 rolls, step on roll 3: filt→2");
|
||||
TEST_MESSAGE("J: Steps down gradually | filt=8 samp=1 hold=1 | 8→4→2→1 over 3 rolls, stable on 4th");
|
||||
TEST_MESSAGE("K: Full at DENOM_MAX drops entry | 128 entries, samp=filt=128 | count stays 128, LOG_WARN emitted");
|
||||
}
|
||||
|
||||
static void test_memory_layout()
|
||||
{
|
||||
TEST_MSG_FMT("%-35s %6s %s", "Type", "bytes", "Notes");
|
||||
TEST_MSG_FMT("%-35s %6zu %s", "Record", sizeof(Record), "nodeHash:16 + hops:3 + seen:13 (32-bit packed)");
|
||||
TEST_MSG_FMT("%-35s %6zu %s", "HopScalingModule::PerHopCounts", sizeof(HopScalingModule::PerHopCounts),
|
||||
"perHop[8](16) + total(2)");
|
||||
TEST_MSG_FMT("%-35s %6zu %s", "HopScalingModule (instance)", HopScalingTestShim::sizeofSelf(),
|
||||
"entries[128](512) + denom state + cached results + OSThread overhead");
|
||||
|
||||
TEST_PASS();
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Unity setup / teardown / main
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
void setUp(void)
|
||||
{
|
||||
if (!mockNodeDB)
|
||||
mockNodeDB = new MockNodeDB();
|
||||
mockNodeDB->clearTestNodes();
|
||||
|
||||
config = meshtastic_LocalConfig_init_zero;
|
||||
moduleConfig = meshtastic_LocalModuleConfig_init_zero;
|
||||
myNodeInfo.my_node_num = kLocalNode;
|
||||
nodeDB = mockNodeDB;
|
||||
|
||||
#ifdef FSCom
|
||||
FSCom.remove("/prefs/hopScalingState.bin");
|
||||
#endif
|
||||
|
||||
// Reset mock clock to a known base (1 hour in so subtraction never underflows)
|
||||
mockTime = ONE_HOUR_MS;
|
||||
}
|
||||
|
||||
void tearDown(void)
|
||||
{
|
||||
hopScalingModule = nullptr;
|
||||
}
|
||||
|
||||
void setup()
|
||||
{
|
||||
initializeTestEnvironment();
|
||||
nodeDB = mockNodeDB;
|
||||
|
||||
UNITY_BEGIN();
|
||||
|
||||
printf("\n=== Topology-driven hop reduction ===\n");
|
||||
RUN_TEST(test_dense_local_telemetry);
|
||||
RUN_TEST(test_spread_sparse_position);
|
||||
RUN_TEST(test_deep_chain_position);
|
||||
RUN_TEST(test_router_cluster_telemetry);
|
||||
RUN_TEST(test_megamesh_eviction_scaling);
|
||||
RUN_TEST(test_sparse_to_dense_transition);
|
||||
|
||||
printf("\n=== Lifecycle ===\n");
|
||||
RUN_TEST(test_state_persistence);
|
||||
RUN_TEST(test_hourly_roll);
|
||||
RUN_TEST(test_intermediate_status);
|
||||
RUN_TEST(test_startup_blank_state);
|
||||
|
||||
printf("\n=== Denominator state machine ===\n");
|
||||
RUN_TEST(test_denominator_rises_on_overflow);
|
||||
RUN_TEST(test_filtering_denom_hold_counts_down);
|
||||
RUN_TEST(test_filtering_denom_steps_down_gradually);
|
||||
RUN_TEST(test_full_at_denom_max_drops_entry);
|
||||
|
||||
printf("\n=== Summary ===\n");
|
||||
RUN_TEST(test_memory_layout);
|
||||
RUN_TEST(test_scenario_summary_output);
|
||||
|
||||
exit(UNITY_END());
|
||||
}
|
||||
|
||||
void loop() {}
|
||||
|
||||
#else // !HAS_VARIABLE_HOPS
|
||||
|
||||
void setUp(void) {}
|
||||
void tearDown(void) {}
|
||||
|
||||
void setup()
|
||||
{
|
||||
initializeTestEnvironment();
|
||||
UNITY_BEGIN();
|
||||
exit(UNITY_END());
|
||||
}
|
||||
|
||||
void loop() {}
|
||||
|
||||
#endif
|
||||
@@ -13,6 +13,9 @@
|
||||
#ifndef HAS_TRAFFIC_MANAGEMENT
|
||||
#define HAS_TRAFFIC_MANAGEMENT 1
|
||||
#endif
|
||||
#ifndef HAS_VARIABLE_HOPS
|
||||
#define HAS_VARIABLE_HOPS 1
|
||||
#endif
|
||||
#ifndef TRAFFIC_MANAGEMENT_CACHE_SIZE
|
||||
#define TRAFFIC_MANAGEMENT_CACHE_SIZE 2048
|
||||
#endif
|
||||
|
||||
Reference in New Issue
Block a user