kevin/meshtastic-firmware
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

Automatic variable hop limits based on mesh activity and size estimation (#10176)

Browse Source 
* asdf

* Implement SphereOfInfluenceModule for traffic management and eviction tracking

* Implement Sphere of Influence module for dynamic hop limit adjustment and role-based floor

* Update SAMPLING_DENOMINATOR to improve mesh size estimation accuracy

* Add debug logging for scale factor estimation and per-hop node counts in SphereOfInfluenceModule

* Enable variable hop limits and role-based hop floors in Sphere of Influence module

* Respond to copilot review

* Disable variable hop limits and role-based hop floors in Sphere of Influence module

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Implement adaptive sampling for unique node ID tracking in Sphere of Influence module

* Add state persistence for Sphere of Influence module

* Enhance Sphere of Influence module with state management and adaptive sampling adjustments

* Refactor hop scaling functionality: remove SphereOfInfluenceModule and introduce HopScalingModule

- Deleted SphereOfInfluenceModule.h, consolidating its responsibilities into the new HopScalingModule.
- Added HopScalingModule.h and HopScalingModule.cpp to manage hop scaling logic, including eviction tracking and sampling-based mesh size estimation.
- Implemented methods for recording evictions and packet senders, estimating scale factors, and computing required hops based on node activity.
- Introduced state persistence for hop scaling parameters to maintain continuity across reboots.
- Enhanced thread safety and modularity by utilizing concurrency features.

* Guard out STM32. Sowwy.

* Refactor HopScalingModule: enhance sampling logic and improve state management

* Add unit tests for HopScalingModule: implement mock database and various test scenarios

* Refactor test output in HopScalingModule tests: replace printf with TEST_MESSAGE for better integration with Unity

* Refactor HopScalingModule logging: replace lastStatusMode with descriptive mode names for improved readability

* Refactor test_main.cpp: change NodeNum variable to static and improve comments for clarity

* Remove unnecessary delay in setup function for improved test performance

* Add missing include for MeshTypes in test_main.cpp

* Refactor HopScalingModule tests: enhance mesh topology scenarios and improve test clarity

* Update HopScalingModule tests: flesh out node scenarios and improve clarity for dense and sparse mesh cases

* Fix politeness factor calculations in HopScalingModule and update related test scenarios for clarity. Remove outdated design doc.

* Enhance HopScalingModule: add sampled estimate for scaling decisions and refactor initial run state management

* Add sample traffic injection for HopScaling tests to enhance sampledEst visibility

* Enhance HopScalingModule: adjust windowFraction calculation for early triggers and improve test output formatting

* Enhance HopScalingModule: add jitter functionality to sampling denominator and update tests for consistent behavior

* Enhance HopScalingModule: implement adaptive sampling denominator adjustment and add reset functionality for tests

* Enhance HopScalingTestShim: add test-only clock and window helpers, update injectSampleTraffic for adaptive sampling, and improve scenario summary output

* Enhance HopScalingModule: add detailed documentation for functions, improve clarity of jitter and sampling logic, and reset functionality in tests

* Enhance HopScalingModule: add evictionEstimate parameter to estimateScaleFactor and update related logging for improved mesh size estimation

* Enhance HopScalingModule: adjust effective rolls calculation for improved accuracy, add eviction estimate logic, responding to all copilot review points

* Implement CompactHistogram for parallel hop scaling sampling

- Added CompactHistogram class to track node hop distances with bitwise sampling.
- Integrated CompactHistogram into HopScalingModule for independent packet sampling.
- Updated NodeDB to feed both the hop scaling module and the new histogram sampler.
- Enhanced HopScalingModule with methods to sample packets for the histogram and retrieve hop distribution statistics.
- Implemented tests for CompactHistogram functionality, including sampling, window rolling, and adaptive denominator scaling.
- Updated existing tests to validate the integration of the new histogram sampling mechanism.

* Enhance CompactHistogram and HopScalingModule: add per-hop distribution functionality, improve time handling for unit tests, and refine test setup for deterministic behavior

* CompactHistogram: add mesh size estimation, improve entry replacement logic, and update logging for per-hop distribution

* Refactor HopScalingModule and CompactHistogram integration

- Removed the suggestedHopFromCompactHistogram function to streamline hop suggestion logic.
- Updated HopScalingModule to directly utilize CompactHistogram's internal methods for hop suggestions and sampling.
- Enhanced logging in HopScalingModule to provide detailed histogram statistics.
- Modified test cases to ensure comprehensive coverage of new histogram behaviors and sampling logic.
- Improved node ID distribution in tests to better exercise sampling mechanisms.
- Ensured that filtering denominators are held for 12 hours before dropping, enhancing stability in sampling.

* Refactor CompactHistogram to support 13-hour activity tracking and introduce politeness regimes

- Updated the bitfield structure to accommodate 13-hour seen tracking.
- Changed the logic in rollHour() to analyze activity over the last 0-2 hours vs. 1-3 hours for politeness factor calculation.
- Introduced three politeness levels: GENEROUS, DEFAULT, and STRICT based on recent activity ratios.
- Adjusted filtering and sampling logic to reflect the new 13-hour tracking period.
- Updated unit tests to validate new behavior and ensure proper functionality of politeness regimes.

* Enhance CompactHistogram and HopScalingModule for improved sampling and decision-making

- Introduced a session-specific hash seed in CompactHistogram to reduce bias in node ID sampling.
- Updated sampling logic to use hashed node IDs instead of raw IDs for filtering and entry management.
- Added histogram rollover tracking in HopScalingModule to ensure proper decision-making after initial data collection.
- Adjusted logging to reflect the active state of the histogram and its comparison with NodeDB advisory hops.
- Enhanced unit tests to validate new sampling logic and memory layout changes.

* Expose hashNodeId for testing in CompactHistogram

* Add mesh trend statistics to CompactHistogram for enhanced activity tracking

* Implement histogram state persistence in CompactHistogram with save and load functions

* Refactor CompactHistogram to improve entry management and enhance rollHour logging

* feat: add HopScalingModule for adaptive hop limit recommendations

Introduces HopScalingModule, a sampled hop-distance histogram that
recommends the minimum hop limit needed to reach ~40 nodes, and
automatically reducing the hops as the mesh grows.

Key design:
- 512-byte packed histogram (128 × 4-byte Record entries) embedded
   in a new HopScalingModule.
- Each Record: 16-bit node hash, 3-bit hop distance, 13-bit seen bitmap
- Sampling filter: only nodes where (hash & (denom-1)) == 0 are kept;
  denominator doubles on overflow and halves when utilisation is low
- Hourly rollHour(): tallies per-hop counts, walks scaled buckets to
  find the minimum hop satisfying TARGET_AFFECTED_NODES (40), applies a
  politeness extension based on recent/older activity ratio, shifts all
  seen bitmaps, and persists state to /prefs/hopScalingState.bin
- Hop recommendation gated by bootstrap (requires >=1 rollHour before
  overriding HOP_MAX)
- NodeDB calls samplePacketForHistogram() on every non-MQTT rx packet
- Module also estimates total mesh size and logs useful information about
  mesh characteristics.

Changes:
- src/modules/HopScalingModule.h/.cpp: new module
- src/mesh/NodeDB.cpp: wire up samplePacketForHistogram
- src/mesh/Router.cpp: consume getLastRequiredHop()
- test/test_hop_scaling/: 12-test suite covering all mesh topologies and
  anticipated operational requirements

* test: increase run iterations in sparse to dense transition test

* feat: refactor HopScalingModule to use RUNS_PER_HOUR constant and improve logging

* feat: enhance HopScalingModule with filtering denominator management and add tests for state transitions

* refactor: remove CompactHistogram module and related files

* address copilot review comments

* Tweak: packet sampling only lora

* ove role-based hop floor logic and related definitions into the module - keep it in one place.

* Refactor MockNodeDB to use nodeInfoLiteSetBit for MQTT flag setting

* Refactor hop scaling parameters and logic to integer maths and put default values in defaults.h. Small flash size reduction, no functional impact.

* Update unit test preprocessor directives to PIO_UNIT_TESTING for consistency

* refactor: improve test output organization and clarity in hop scaling tests

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Tom
2026-06-04 05:59:49 -05:00
committed by GitHub
co-authored by GitHub Copilot
parent ef51c7ec11
commit de345939af
12 changed files with 1672 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/README.md
+15 -8
View File
@@ -200,7 +200,7 @@ class MockNodeDB : public NodeDB
node.num = num;
node.has_hops_away = hasHops;
node.hops_away = hopsAway;
node.via_mqtt = viaMqtt;
nodeInfoLiteSetBit(&node, NODEINFO_BITFIELD_VIA_MQTT_MASK, viaMqtt);
node.last_heard = getTime() - ageSecs;
testNodes.push_back(node);
meshNodes = &testNodes;
@@ -358,6 +358,7 @@ PlatformIO defines `PIO_UNIT_TESTING` during `pio test` builds. Several producti
- [ ] Set `nodeDB = mockNodeDB`
- [ ] Delete persisted state files (`FSCom.remove(...)`)
- [ ] Reset file-scope mutable globals
- [ ] Reset mock clock to a safe base value (e.g. `mockTime = ONE_HOUR_MS`) — prevents unsigned subtraction underflow in time-dependent logic
- [ ] Disable randomness/jitter flags
- [ ] In `tearDown`: null the global singleton pointer, restore flags
@@ -375,15 +376,21 @@ A well-structured test suite follows this pattern:
| Suite | Module Under Test |
| ---------------------------- | ----------------------------- |
| `test_admin_radio` | Admin + LoRa region config |
| `test_atak` | ATAK integration |
| `test_crypto` | CryptoEngine |
| `test_mqtt` | MQTT integration |
| `test_radio` | Radio interface |
| `test_default` | Default configuration helpers |
| `test_hop_scaling` | Hop scaling algorithm |
| `test_http_content_handler` | HTTP handling |
| `test_mac_from_string` | MAC address parsing |
| `test_mesh_module` | Module framework |
| `test_meshpacket_serializer` | Packet serialization |
| `test_transmit_history` | Retransmission tracking |
| `test_atak` | ATAK integration |
| `test_default` | Default configuration helpers |
| `test_http_content_handler` | HTTP handling |
| `test_mqtt` | MQTT integration |
| `test_packet_history` | Packet history tracking |
| `test_position_precision` | Position precision helpers |
| `test_radio` | Radio interface |
| `test_serial` | Serial communication |
| `test_hop_scaling` | Hop scaling algorithm |
| `test_traffic_management` | Traffic management |
| `test_transmit_history` | Retransmission tracking |
| `test_type_conversions` | NodeDB v25 type conversions |
| `test_utf8` | UTF-8 utilities |
test/test_hop_scaling/test_main.cpp
+738
View File
@@ -0,0 +1,738 @@
#include "MeshTypes.h"
#include "TestUtil.h"
#include <unity.h>
#if HAS_VARIABLE_HOPS
#include "FSCommon.h"
#include "gps/RTC.h"
#include "mesh/NodeDB.h"
#include "modules/HopScalingModule.h"
#include <cstdio>
#include <cstring>
#include <memory>
// Unity only shows TEST_MESSAGE output. printf goes to stdout which the runner swallows.
#define MSG_BUF_LEN 200
#define TEST_MSG_FMT(fmt, ...) \
do { \
char _buf[MSG_BUF_LEN]; \
snprintf(_buf, sizeof(_buf), fmt, __VA_ARGS__); \
TEST_MESSAGE(_buf); \
} while (0)
static constexpr NodeNum kLocalNode = 0x11111111;
// Shared mock clock — drives HopScalingModule::nowMs()
static uint32_t &mockTime = HopScalingModule::s_testNowMs;
static constexpr uint32_t ONE_HOUR_MS = 3600UL * 1000UL;
// ---------------------------------------------------------------------------
// MockNodeDB — not used for hop decisions any more, kept for completeness
// ---------------------------------------------------------------------------
class MockNodeDB : public NodeDB
{
public:
void clearTestNodes()
{
testNodes.clear();
numMeshNodes = 0;
}
void addTestNode(NodeNum num, uint8_t hopsAway, bool hasHops, uint32_t ageSecs, bool viaMqtt = false)
{
meshtastic_NodeInfoLite node = meshtastic_NodeInfoLite_init_zero;
node.num = num;
node.has_hops_away = hasHops;
node.hops_away = hopsAway;
nodeInfoLiteSetBit(&node, NODEINFO_BITFIELD_VIA_MQTT_MASK, viaMqtt);
node.last_heard = getTime() - ageSecs;
testNodes.push_back(node);
meshNodes = &testNodes;
numMeshNodes = testNodes.size();
}
std::vector<meshtastic_NodeInfoLite> testNodes;
};
// ---------------------------------------------------------------------------
// Test shim — expose protected/private members for direct invocation
// ---------------------------------------------------------------------------
class HopScalingTestShim : public HopScalingModule
{
public:
using HopScalingModule::runOnce;
using HopScalingModule::samplePacketForHistogram;
using HopScalingModule::getLastRequiredHop;
// Test-only helpers (require UNIT_TEST friend access)
void rollHourTest() { rollHour(); }
void setHistogramDenominator(uint8_t d) { setSamplingDenominator(d); }
/// Directly set denominator state, bypassing any scale-up/down logic.
/// Used by tests that need a specific pre-condition without triggering trim.
void forceFilterDenomState(uint8_t samp, uint8_t filt, uint8_t holdRolls)
{
samplingDenominator = samp;
filteringDenominator = filt;
filteringDenomHoldRollsRemaining = holdRolls;
}
uint8_t getFilteringDenomHoldRollsRemaining() const { return filteringDenomHoldRollsRemaining; }
/// Insert an entry with an explicit hash, bypassing the sampling filter.
/// Used to fill the histogram to a known state without depending on hashNodeId distribution.
void forceInsertEntry(uint16_t hash, uint8_t hops)
{
if (count < CAPACITY) {
entries[count].nodeHash = hash;
entries[count].hops_away = hops;
entries[count].seenHoursAgo = 1u;
count++;
}
}
// Size introspection for test_memory_layout
static constexpr size_t sizeofSelf() { return sizeof(HopScalingModule); }
};
static MockNodeDB *mockNodeDB = nullptr;
// Create deterministic IDs that produce a broad spread of 16-bit hashes.
// HopScalingModule admission uses passesFilter(hashNodeId(nodeId), denom), NOT a raw nodeId
// modulo check — do not assume (nodeId & (denom-1)) == 0 determines whether a node is admitted.
static uint32_t makeDistributedNodeId(uint32_t baseId, uint32_t ordinal, uint32_t salt = 0)
{
return baseId + salt + (ordinal * 33u);
}
// ---------------------------------------------------------------------------
// Helpers — mesh topology builders
// ---------------------------------------------------------------------------
// Helper: add N nodes at a given hop with ages spread across a time range.
static void addNodesAtHop(uint32_t baseId, uint8_t hop, uint32_t count, uint32_t ageSecs, uint32_t stride = 10)
{
for (uint32_t i = 0; i < count; i++) {
const uint32_t nodeId = makeDistributedNodeId(baseId, i, static_cast<uint32_t>(hop) << 8);
mockNodeDB->addTestNode(nodeId, hop, true, ageSecs + i * stride);
}
}
// Feed sampled traffic into the histogram.
// Advances mock clock by one hour per roll and calls rollHour() so each roll produces data.
static void injectSampleTraffic(HopScalingTestShim &shim, uint32_t baseId, const uint16_t hopDist[HOP_MAX + 1],
uint8_t numRolls = 16)
{
shim.setHistogramDenominator(HopScalingModule::DENOM_MIN);
for (uint8_t roll = 0; roll < numRolls; ++roll) {
mockTime += ONE_HOUR_MS;
uint16_t ordinal = 0;
for (uint8_t hop = 0; hop <= HOP_MAX; ++hop) {
for (uint16_t n = 0; n < hopDist[hop]; ++n) {
const uint32_t nodeId = makeDistributedNodeId(baseId, ordinal);
shim.samplePacketForHistogram(nodeId, hop);
++ordinal;
}
}
shim.rollHourTest();
}
}
static void assertCompactHistogramActive(HopScalingTestShim &shim)
{
TEST_ASSERT_GREATER_THAN_UINT8(0, shim.getCompactHistogramEntryCount());
TEST_ASSERT_TRUE(shim.getCompactHistogramAllSampleCount() > 0);
}
// ---------------------------------------------------------------------------
// Topology builders
// ---------------------------------------------------------------------------
// Scenario A: Dense local mesh — 110 nodes, heavy at hops 0–2.
static void buildDenseLocalMesh()
{
mockNodeDB->clearTestNodes();
addNodesAtHop(0x1000, 0, 25, 120);
addNodesAtHop(0x2000, 1, 30, 300);
addNodesAtHop(0x3000, 2, 15, 600);
addNodesAtHop(0x4000, 3, 5, 1200);
addNodesAtHop(0x5000, 4, 10, 1800);
addNodesAtHop(0x6000, 5, 15, 2400);
addNodesAtHop(0x7000, 6, 10, 3000);
}
// Scenario B: Spread sparse mesh — 76 nodes across hops 0–7.
static void buildSpreadSparseMesh()
{
mockNodeDB->clearTestNodes();
addNodesAtHop(0x1000, 0, 5, 120);
addNodesAtHop(0x2000, 1, 8, 300);
addNodesAtHop(0x3000, 2, 12, 600);
addNodesAtHop(0x4000, 3, 15, 900);
addNodesAtHop(0x5000, 4, 10, 1200);
addNodesAtHop(0x6000, 5, 6, 1800);
addNodesAtHop(0x7000, 6, 10, 3000);
addNodesAtHop(0x8000, 7, 10, 3600);
}
// Scenario C: Deep linear chain — 22 thin nodes, never reaches 40.
static void buildDeepLinearChain()
{
mockNodeDB->clearTestNodes();
addNodesAtHop(0x1000, 0, 2, 120);
addNodesAtHop(0x2000, 1, 3, 300);
addNodesAtHop(0x3000, 2, 3, 600);
addNodesAtHop(0x4000, 3, 4, 900);
addNodesAtHop(0x5000, 4, 3, 1200);
addNodesAtHop(0x6000, 5, 2, 1800);
addNodesAtHop(0x7000, 6, 2, 2400);
addNodesAtHop(0x8000, 7, 3, 3600);
}
// Scenario D: Router cluster — 71 nodes, 45 at hop 2.
static void buildRouterCluster()
{
mockNodeDB->clearTestNodes();
addNodesAtHop(0x1000, 0, 3, 120);
addNodesAtHop(0x2000, 1, 5, 300);
addNodesAtHop(0x3000, 2, 45, 600);
addNodesAtHop(0x4000, 3, 8, 1200);
addNodesAtHop(0x5000, 4, 3, 1200);
addNodesAtHop(0x6000, 5, 2, 1800);
addNodesAtHop(0x7000, 6, 2, 2400);
addNodesAtHop(0x8000, 7, 3, 3600);
}
// Scenario E: Megamesh — 199 nodes (DB near capacity).
static void buildMegamesh()
{
mockNodeDB->clearTestNodes();
addNodesAtHop(0x01000, 0, 30, 120);
addNodesAtHop(0x02000, 1, 40, 300);
addNodesAtHop(0x03000, 2, 35, 600);
addNodesAtHop(0x04000, 3, 30, 900);
addNodesAtHop(0x05000, 4, 20, 1200);
addNodesAtHop(0x06000, 5, 15, 1800);
addNodesAtHop(0x07000, 6, 14, 2400);
addNodesAtHop(0x08000, 7, 15, 3600);
}
// ---------------------------------------------------------------------------
// Tests — Topology-driven hop reduction scenarios
// ---------------------------------------------------------------------------
void test_dense_local_telemetry()
{
TEST_MESSAGE("=== Dense local mesh: telemetry broadcast ===");
TEST_MESSAGE("Topology: 110 nodes with 25/30/15 nodes at hops 0/1/2 and a thinner tail to hop 6.");
TEST_MESSAGE("Expectation: cumulative reaches 55 nodes by hop 1, result stays tightly constrained.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
buildDenseLocalMesh();
const uint16_t distA[HOP_MAX + 1] = {25, 30, 15, 5, 10, 15, 10, 0};
injectSampleTraffic(*shim, 0x91000000, distA);
shim->runOnce();
TEST_MSG_FMT("Dense local: hop=%u", shim->getLastRequiredHop());
TEST_ASSERT_TRUE(shim->getLastRequiredHop() <= 3);
TEST_ASSERT_TRUE(shim->getLastRequiredHop() >= 1);
assertCompactHistogramActive(*shim);
hopScalingModule = nullptr;
}
void test_spread_sparse_position()
{
TEST_MESSAGE("=== Spread sparse mesh: position broadcast ===");
TEST_MESSAGE("Topology: 76 nodes spread across all hops, reaching 40 nodes only when hop 3 is included.");
TEST_MESSAGE("Expectation: hop settles in the 3-5 range.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
buildSpreadSparseMesh();
const uint16_t distB[HOP_MAX + 1] = {5, 8, 12, 15, 10, 6, 10, 10};
injectSampleTraffic(*shim, 0x92000000, distB);
shim->runOnce();
TEST_MSG_FMT("Spread sparse: hop=%u", shim->getLastRequiredHop());
TEST_ASSERT_TRUE(shim->getLastRequiredHop() >= 3);
TEST_ASSERT_TRUE(shim->getLastRequiredHop() <= 5);
assertCompactHistogramActive(*shim);
hopScalingModule = nullptr;
}
void test_deep_chain_position()
{
TEST_MESSAGE("=== Deep linear chain: position broadcast ===");
TEST_MESSAGE("Topology: 22 nodes spread thinly across hops 0-7, never reaching the 40-node floor.");
TEST_MESSAGE("Expectation: module must keep HOP_MAX.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
buildDeepLinearChain();
const uint16_t distC[HOP_MAX + 1] = {2, 3, 3, 4, 3, 2, 2, 3};
injectSampleTraffic(*shim, 0x93000000, distC);
shim->runOnce();
TEST_MSG_FMT("Deep chain: hop=%u", shim->getLastRequiredHop());
TEST_ASSERT_EQUAL_UINT8(HOP_MAX, shim->getLastRequiredHop());
assertCompactHistogramActive(*shim);
hopScalingModule = nullptr;
}
void test_router_cluster_telemetry()
{
TEST_MESSAGE("=== Router cluster: telemetry broadcast ===");
TEST_MESSAGE("Topology: 71 nodes with a concentrated 45-node cluster at hop 2.");
TEST_MESSAGE("Expectation: result stays in the 2-4 range.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
buildRouterCluster();
const uint16_t distD[HOP_MAX + 1] = {3, 5, 45, 8, 3, 2, 2, 3};
injectSampleTraffic(*shim, 0x94000000, distD);
shim->runOnce();
TEST_MSG_FMT("Router cluster: hop=%u", shim->getLastRequiredHop());
TEST_ASSERT_TRUE(shim->getLastRequiredHop() >= 2);
TEST_ASSERT_TRUE(shim->getLastRequiredHop() <= 4);
assertCompactHistogramActive(*shim);
hopScalingModule = nullptr;
}
void test_megamesh_eviction_scaling()
{
TEST_MESSAGE("=== Megamesh with eviction scaling ===");
TEST_MESSAGE("Topology: NodeDB at capacity (199 nodes), ~2000-node mesh with sustained eviction pressure.");
TEST_MESSAGE("Expectation: sustained evictions tracked in rolling average, hop stays well below HOP_MAX.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
buildMegamesh();
const uint16_t distE[HOP_MAX + 1] = {301, 402, 352, 301, 201, 151, 141, 151};
injectSampleTraffic(*shim, 0x9B000000, distE);
shim->runOnce();
uint8_t hopBefore = shim->getLastRequiredHop();
TEST_MSG_FMT("Megamesh initial: hop=%u", hopBefore);
for (int hour = 0; hour < 3; hour++) {
mockTime += ONE_HOUR_MS;
{
const uint16_t megaDist[HOP_MAX + 1] = {301, 402, 352, 301, 201, 151, 141, 151};
uint16_t ordinal = 0;
for (uint8_t hop = 0; hop <= HOP_MAX; ++hop) {
for (uint16_t n = 0; n < megaDist[hop]; ++n) {
const uint32_t nodeId = makeDistributedNodeId(0x9C000000u, ordinal, static_cast<uint32_t>(hour) * 0x10000u);
shim->samplePacketForHistogram(nodeId, hop);
++ordinal;
}
}
}
for (int run = 0; run < 7; run++)
shim->runOnce();
TEST_MSG_FMT("Megamesh hour %d: hop=%u", hour + 1, shim->getLastRequiredHop());
}
TEST_MESSAGE("Assertion: hop stays well below HOP_MAX on a large-distribution mesh.");
TEST_ASSERT_TRUE(shim->getLastRequiredHop() <= 3);
assertCompactHistogramActive(*shim);
hopScalingModule = nullptr;
}
void test_sparse_to_dense_transition()
{
TEST_MESSAGE("=== Sparse-to-dense transition ===");
TEST_MESSAGE("Topology change: start with a 22-node deep chain, then inject 50 new neighbors at hops 0-1.");
TEST_MESSAGE("Expectation: hop drops sharply once the local neighborhood becomes dense.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
buildDeepLinearChain();
const uint16_t distC2[HOP_MAX + 1] = {2, 3, 3, 4, 3, 2, 2, 3};
injectSampleTraffic(*shim, 0x95000000, distC2);
shim->runOnce();
uint8_t hopSparse = shim->getLastRequiredHop();
TEST_MSG_FMT("Phase 1 sparse: hop=%u (expect %u)", hopSparse, HOP_MAX);
TEST_ASSERT_EQUAL_UINT8(HOP_MAX, hopSparse);
addNodesAtHop(0xA000, 0, 25, 120);
addNodesAtHop(0xB000, 1, 25, 300);
for (uint32_t i = 0; i < 25; ++i)
shim->samplePacketForHistogram(makeDistributedNodeId(0xA000, i, static_cast<uint32_t>(0) << 8), 0);
for (uint32_t i = 0; i < 25; ++i)
shim->samplePacketForHistogram(makeDistributedNodeId(0xB000, i, static_cast<uint32_t>(1) << 8), 1);
for (int run = 0; run < HopScalingModule::RUNS_PER_HOUR; run++)
shim->runOnce();
uint8_t hopDense = shim->getLastRequiredHop();
TEST_MSG_FMT("Phase 2 dense: hop=%u (expect <= 3)", hopDense);
TEST_ASSERT_TRUE(hopDense < hopSparse);
TEST_ASSERT_TRUE(hopDense <= 3);
assertCompactHistogramActive(*shim);
hopScalingModule = nullptr;
}
void test_state_persistence()
{
TEST_MESSAGE("=== State persistence across restart ===");
TEST_MESSAGE("Expectation: histogram entries survive instance teardown and reload.");
{
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
const uint16_t dist[HOP_MAX + 1] = {5, 8, 12, 10, 5, 3, 2, 1};
injectSampleTraffic(*shim, 0x9D000000, dist, 2);
TEST_MSG_FMT("Phase 1: entries=%u hop=%u", shim->getEntryCount(), shim->getLastRequiredHop());
TEST_ASSERT_GREATER_THAN_UINT8(0, shim->getEntryCount());
hopScalingModule = nullptr;
}
{
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
shim->runOnce();
TEST_MSG_FMT("Phase 2 restored: entries=%u hop=%u", shim->getEntryCount(), shim->getLastRequiredHop());
TEST_ASSERT_GREATER_THAN_UINT8(0, shim->getEntryCount());
hopScalingModule = nullptr;
}
}
void test_hourly_roll()
{
TEST_MESSAGE("=== Hourly roll cycle ===");
TEST_MESSAGE("Expectation: histogram accumulates data and provides valid hop recommendation after multiple rolls.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
buildSpreadSparseMesh();
shim->setHistogramDenominator(HopScalingModule::DENOM_MIN);
for (uint32_t i = 1; i <= 30; i++) {
const uint32_t nodeId = makeDistributedNodeId(0x97000000, i, 0xAAu);
shim->samplePacketForHistogram(nodeId, static_cast<uint8_t>(i % (HOP_MAX + 1)));
}
for (int run = 0; run < 13; run++) {
int32_t interval = shim->runOnce();
TEST_ASSERT_GREATER_THAN(0, interval);
}
TEST_MSG_FMT("Hourly roll: hop=%u entries=%u", shim->getLastRequiredHop(), shim->getEntryCount());
assertCompactHistogramActive(*shim);
hopScalingModule = nullptr;
}
void test_intermediate_status()
{
TEST_MESSAGE("=== Intermediate status (no recomputation) ===");
TEST_MESSAGE("Expectation: runs between hourly updates leave hop unchanged.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
buildRouterCluster();
const uint16_t distD[HOP_MAX + 1] = {3, 5, 45, 8, 3, 2, 2, 3};
injectSampleTraffic(*shim, 0x98000000, distD);
shim->runOnce();
uint8_t hopAfterInitial = shim->getLastRequiredHop();
TEST_MSG_FMT("Initial: hop=%u", hopAfterInitial);
for (int run = 0; run < 3; run++) {
shim->runOnce();
TEST_ASSERT_EQUAL_UINT8(hopAfterInitial, shim->getLastRequiredHop());
}
TEST_MSG_FMT("After 3 intermediate runs: hop=%u (unchanged)", shim->getLastRequiredHop());
hopScalingModule = nullptr;
}
void test_startup_blank_state()
{
TEST_MESSAGE("=== Startup with blank state ===");
TEST_MESSAGE("Expectation: fresh instance starts with zeroed rolling averages and a valid hop result.");
#ifdef FSCom
FSCom.remove("/prefs/hopScalingState.bin");
#endif
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
buildDeepLinearChain();
int32_t interval = shim->runOnce();
TEST_ASSERT_GREATER_THAN(0, interval);
TEST_ASSERT_TRUE(shim->getLastRequiredHop() <= HOP_MAX);
TEST_MSG_FMT("Startup blank: hop=%u", shim->getLastRequiredHop());
hopScalingModule = nullptr;
}
// ---------------------------------------------------------------------------
// Tests — Denominator state machine
// ---------------------------------------------------------------------------
void test_denominator_rises_on_overflow()
{
TEST_MESSAGE("=== samplingDenominator doubles when histogram overflows ===");
TEST_MESSAGE("Fill to > FILL_HIGH_PCT with forceInsertEntry, then trigger via samplePacketForHistogram.");
TEST_MESSAGE("Expectation: samp/filt both double to 2, hold set to FILTER_DENOM_HOLD_ROLLS.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
// Insert 103 entries with hashes 1..103 (all distinct, no sampling-filter skew).
// 103 / 128 = 80.4% fill, which meets FILL_HIGH_PCT=80.
// Odd hashes (1,3,...,103) will be evicted when denom doubles to 2; even ones survive.
static constexpr uint8_t FILL_COUNT = 103u;
for (uint8_t i = 1; i <= FILL_COUNT; i++)
shim->forceInsertEntry(i, 2u);
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::DENOM_MIN, shim->getSamplingDenominator());
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::DENOM_MIN, shim->getFilteringDenominator());
TEST_ASSERT_EQUAL_UINT8(0u, shim->getFilteringDenomHoldRollsRemaining());
TEST_ASSERT_EQUAL_UINT8(FILL_COUNT, shim->getEntryCount());
// A new node passes the denom=1 admission gate; fill ≥ 80% triggers trimIfNeeded → doubling.
shim->samplePacketForHistogram(0xB0000000u, 1u);
TEST_MSG_FMT("After scale-up: samp=1/%u filt=1/%u holdRolls=%u entries=%u", shim->getSamplingDenominator(),
shim->getFilteringDenominator(), shim->getFilteringDenomHoldRollsRemaining(), shim->getEntryCount());
TEST_ASSERT_EQUAL_UINT8(2u, shim->getSamplingDenominator());
TEST_ASSERT_EQUAL_UINT8(2u, shim->getFilteringDenominator());
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::FILTER_DENOM_HOLD_ROLLS, shim->getFilteringDenomHoldRollsRemaining());
// After evicting entries with (hash & 1) != 0, roughly half the entries remain.
TEST_ASSERT_LESS_THAN_UINT8(FILL_COUNT, shim->getEntryCount());
hopScalingModule = nullptr;
}
void test_filtering_denom_hold_counts_down()
{
TEST_MESSAGE("=== filteringDenominator held while hold counter > 0 ===");
TEST_MESSAGE("Force filt=4 samp=1 hold=3; verify no step for 2 rolls, then step fires on roll 3.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
// samp=DENOM_MIN so scale-down in step 4 can't go lower; hold=3 for a short, fast test.
shim->forceFilterDenomState(HopScalingModule::DENOM_MIN, 4u, 3u);
shim->rollHourTest(); // hold 3→2, no step
TEST_ASSERT_EQUAL_UINT8(4u, shim->getFilteringDenominator());
TEST_ASSERT_EQUAL_UINT8(2u, shim->getFilteringDenomHoldRollsRemaining());
shim->rollHourTest(); // hold 2→1, no step
TEST_ASSERT_EQUAL_UINT8(4u, shim->getFilteringDenominator());
TEST_ASSERT_EQUAL_UINT8(1u, shim->getFilteringDenomHoldRollsRemaining());
// Roll 3: hold 1→0, step fires — filteringDenominator halves to max(2, samp=1) = 2.
shim->rollHourTest();
TEST_MSG_FMT("After hold expires: filt=1/%u samp=1/%u holdRolls=%u", shim->getFilteringDenominator(),
shim->getSamplingDenominator(), shim->getFilteringDenomHoldRollsRemaining());
TEST_ASSERT_EQUAL_UINT8(2u, shim->getFilteringDenominator());
TEST_ASSERT_EQUAL_UINT8(0u, shim->getFilteringDenomHoldRollsRemaining());
hopScalingModule = nullptr;
}
void test_filtering_denom_steps_down_gradually()
{
TEST_MESSAGE("=== filteringDenominator descends one halving per rollHour() after hold expires ===");
TEST_MESSAGE("Force filt=8 samp=1 hold=1; expect 8→4→2→1 over 3 rolls, then stable.");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
shim->forceFilterDenomState(HopScalingModule::DENOM_MIN, 8u, 1u);
shim->rollHourTest(); // hold 1→0, step: 8/2=4 > 1, filt=4
TEST_ASSERT_EQUAL_UINT8(4u, shim->getFilteringDenominator());
shim->rollHourTest(); // hold=0 (no decrement), step: 4/2=2 > 1, filt=2
TEST_ASSERT_EQUAL_UINT8(2u, shim->getFilteringDenominator());
shim->rollHourTest(); // step: 2/2=1, not > samp=1, filt=samp=1 — converged
TEST_ASSERT_EQUAL_UINT8(1u, shim->getFilteringDenominator());
shim->rollHourTest(); // filt==samp, outer if is false — no further change
TEST_ASSERT_EQUAL_UINT8(1u, shim->getFilteringDenominator());
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::DENOM_MIN, shim->getSamplingDenominator());
hopScalingModule = nullptr;
}
void test_full_at_denom_max_drops_entry()
{
TEST_MESSAGE("=== Full histogram at DENOM_MAX drops new entries ===");
TEST_MESSAGE("Fill CAPACITY entries, force samp=DENOM_MAX, sample admissible node.");
TEST_MESSAGE("Expectation: entry count stays at CAPACITY (LOG_WARN fires; visible in test output).");
auto shim = std::unique_ptr<HopScalingTestShim>(new HopScalingTestShim());
hopScalingModule = shim.get();
shim->setHashSeed(0); // deterministic hash for admissible-ID search
shim->forceFilterDenomState(HopScalingModule::DENOM_MAX, HopScalingModule::DENOM_MAX, 0u);
// Fill with odd hashes 1,3,5,...,(2*CAPACITY-1). None are multiples of 128, so none
// collide with the admissible node's hash (which must be a multiple of 128).
for (uint16_t i = 0; i < HopScalingModule::CAPACITY; i++)
shim->forceInsertEntry(static_cast<uint16_t>(2u * i + 1u), 1u);
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::CAPACITY, shim->getEntryCount());
// Find a node ID whose hash passes DENOM_MAX, i.e. (hash & 127) == 0.
uint32_t admissibleId = 0;
for (uint32_t id = 1u; id < 0x10000u; id++) {
if ((shim->hashNodeIdPublic(id) & (HopScalingModule::DENOM_MAX - 1u)) == 0u) {
admissibleId = id;
break;
}
}
TEST_ASSERT_NOT_EQUAL(0u, admissibleId); // sanity: the hash space is dense enough to find one quickly
shim->samplePacketForHistogram(admissibleId, 3u);
TEST_MSG_FMT("After drop attempt: entries=%u CAPACITY=%u admissibleId=0x%08x hash=0x%04x", shim->getEntryCount(),
static_cast<unsigned>(HopScalingModule::CAPACITY), admissibleId,
static_cast<unsigned>(shim->hashNodeIdPublic(admissibleId)));
TEST_ASSERT_EQUAL_UINT8(HopScalingModule::CAPACITY, shim->getEntryCount());
hopScalingModule = nullptr;
}
void test_scenario_summary_output()
{
TEST_MESSAGE("=== Scenario summary ===");
TEST_MESSAGE("Scenario | Nodes | Distribution | Hop | Why");
TEST_MESSAGE("A: Dense local | 110 | 25/30/15/5/10/15/10 h0-6 | 1-2 | 55 nodes at h1 >> 40");
TEST_MESSAGE("B: Spread | 76 | 5/8/12/15/10/6/10/10 h0-7 | 3-4 | Need h3 to reach 40");
TEST_MESSAGE("C: Deep chain | 22 | 2/3/3/4/3/2/2/3 h0-7 | 7 | Never reaches 40");
TEST_MESSAGE("D: Router | 71 | 3/5/45/8/3/2/2/3 h0-7 | 2-3 | 45-node hop-2 cluster");
TEST_MESSAGE("E: Megamesh | 199 | 30/40/35/30/20/15/14/15 h0-7 | 0-1 | Dense low-hop histogram");
TEST_MESSAGE("F: Transition | 22->72 | Chain -> dense local | 7-><=3 | Adapts to new neighbors");
TEST_MESSAGE("G: Persistence | -- | -- | -- | Eviction avg survives reboot");
TEST_MESSAGE("");
TEST_MESSAGE("=== Denominator state machine summary ===");
TEST_MESSAGE("Test | Pre-condition | Expectation");
TEST_MESSAGE("H: Rises on overflow | 103 entries forced, denom=1 | samp/filt→2, holdRolls=13");
TEST_MESSAGE(
"I: Hold counts down | filt=4 samp=1 hold=3 | no step for 2 rolls, step on roll 3: filt→2");
TEST_MESSAGE("J: Steps down gradually | filt=8 samp=1 hold=1 | 8→4→2→1 over 3 rolls, stable on 4th");
TEST_MESSAGE("K: Full at DENOM_MAX drops entry | 128 entries, samp=filt=128 | count stays 128, LOG_WARN emitted");
}
static void test_memory_layout()
{
TEST_MSG_FMT("%-35s %6s %s", "Type", "bytes", "Notes");
TEST_MSG_FMT("%-35s %6zu %s", "Record", sizeof(Record), "nodeHash:16 + hops:3 + seen:13 (32-bit packed)");
TEST_MSG_FMT("%-35s %6zu %s", "HopScalingModule::PerHopCounts", sizeof(HopScalingModule::PerHopCounts),
"perHop[8](16) + total(2)");
TEST_MSG_FMT("%-35s %6zu %s", "HopScalingModule (instance)", HopScalingTestShim::sizeofSelf(),
"entries[128](512) + denom state + cached results + OSThread overhead");
TEST_PASS();
}
// ---------------------------------------------------------------------------
// Unity setup / teardown / main
// ---------------------------------------------------------------------------
void setUp(void)
{
if (!mockNodeDB)
mockNodeDB = new MockNodeDB();
mockNodeDB->clearTestNodes();
config = meshtastic_LocalConfig_init_zero;
moduleConfig = meshtastic_LocalModuleConfig_init_zero;
myNodeInfo.my_node_num = kLocalNode;
nodeDB = mockNodeDB;
#ifdef FSCom
FSCom.remove("/prefs/hopScalingState.bin");
#endif
// Reset mock clock to a known base (1 hour in so subtraction never underflows)
mockTime = ONE_HOUR_MS;
}
void tearDown(void)
{
hopScalingModule = nullptr;
}
void setup()
{
initializeTestEnvironment();
nodeDB = mockNodeDB;
UNITY_BEGIN();
printf("\n=== Topology-driven hop reduction ===\n");
RUN_TEST(test_dense_local_telemetry);
RUN_TEST(test_spread_sparse_position);
RUN_TEST(test_deep_chain_position);
RUN_TEST(test_router_cluster_telemetry);
RUN_TEST(test_megamesh_eviction_scaling);
RUN_TEST(test_sparse_to_dense_transition);
printf("\n=== Lifecycle ===\n");
RUN_TEST(test_state_persistence);
RUN_TEST(test_hourly_roll);
RUN_TEST(test_intermediate_status);
RUN_TEST(test_startup_blank_state);
printf("\n=== Denominator state machine ===\n");
RUN_TEST(test_denominator_rises_on_overflow);
RUN_TEST(test_filtering_denom_hold_counts_down);
RUN_TEST(test_filtering_denom_steps_down_gradually);
RUN_TEST(test_full_at_denom_max_drops_entry);
printf("\n=== Summary ===\n");
RUN_TEST(test_memory_layout);
RUN_TEST(test_scenario_summary_output);
exit(UNITY_END());
}
void loop() {}
#else // !HAS_VARIABLE_HOPS
void setUp(void) {}
void tearDown(void) {}
void setup()
{
initializeTestEnvironment();
UNITY_BEGIN();
exit(UNITY_END());
}
void loop() {}
#endif