* asdf * Implement SphereOfInfluenceModule for traffic management and eviction tracking * Implement Sphere of Influence module for dynamic hop limit adjustment and role-based floor * Update SAMPLING_DENOMINATOR to improve mesh size estimation accuracy * Add debug logging for scale factor estimation and per-hop node counts in SphereOfInfluenceModule * Enable variable hop limits and role-based hop floors in Sphere of Influence module * Respond to copilot review * Disable variable hop limits and role-based hop floors in Sphere of Influence module * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Implement adaptive sampling for unique node ID tracking in Sphere of Influence module * Add state persistence for Sphere of Influence module * Enhance Sphere of Influence module with state management and adaptive sampling adjustments * Refactor hop scaling functionality: remove SphereOfInfluenceModule and introduce HopScalingModule - Deleted SphereOfInfluenceModule.h, consolidating its responsibilities into the new HopScalingModule. - Added HopScalingModule.h and HopScalingModule.cpp to manage hop scaling logic, including eviction tracking and sampling-based mesh size estimation. - Implemented methods for recording evictions and packet senders, estimating scale factors, and computing required hops based on node activity. - Introduced state persistence for hop scaling parameters to maintain continuity across reboots. - Enhanced thread safety and modularity by utilizing concurrency features. * Guard out STM32. Sowwy. * Refactor HopScalingModule: enhance sampling logic and improve state management * Add unit tests for HopScalingModule: implement mock database and various test scenarios * Refactor test output in HopScalingModule tests: replace printf with TEST_MESSAGE for better integration with Unity * Refactor HopScalingModule logging: replace lastStatusMode with descriptive mode names for improved readability * Refactor test_main.cpp: change NodeNum variable to static and improve comments for clarity * Remove unnecessary delay in setup function for improved test performance * Add missing include for MeshTypes in test_main.cpp * Refactor HopScalingModule tests: enhance mesh topology scenarios and improve test clarity * Update HopScalingModule tests: flesh out node scenarios and improve clarity for dense and sparse mesh cases * Fix politeness factor calculations in HopScalingModule and update related test scenarios for clarity. Remove outdated design doc. * Enhance HopScalingModule: add sampled estimate for scaling decisions and refactor initial run state management * Add sample traffic injection for HopScaling tests to enhance sampledEst visibility * Enhance HopScalingModule: adjust windowFraction calculation for early triggers and improve test output formatting * Enhance HopScalingModule: add jitter functionality to sampling denominator and update tests for consistent behavior * Enhance HopScalingModule: implement adaptive sampling denominator adjustment and add reset functionality for tests * Enhance HopScalingTestShim: add test-only clock and window helpers, update injectSampleTraffic for adaptive sampling, and improve scenario summary output * Enhance HopScalingModule: add detailed documentation for functions, improve clarity of jitter and sampling logic, and reset functionality in tests * Enhance HopScalingModule: add evictionEstimate parameter to estimateScaleFactor and update related logging for improved mesh size estimation * Enhance HopScalingModule: adjust effective rolls calculation for improved accuracy, add eviction estimate logic, responding to all copilot review points * Implement CompactHistogram for parallel hop scaling sampling - Added CompactHistogram class to track node hop distances with bitwise sampling. - Integrated CompactHistogram into HopScalingModule for independent packet sampling. - Updated NodeDB to feed both the hop scaling module and the new histogram sampler. - Enhanced HopScalingModule with methods to sample packets for the histogram and retrieve hop distribution statistics. - Implemented tests for CompactHistogram functionality, including sampling, window rolling, and adaptive denominator scaling. - Updated existing tests to validate the integration of the new histogram sampling mechanism. * Enhance CompactHistogram and HopScalingModule: add per-hop distribution functionality, improve time handling for unit tests, and refine test setup for deterministic behavior * CompactHistogram: add mesh size estimation, improve entry replacement logic, and update logging for per-hop distribution * Refactor HopScalingModule and CompactHistogram integration - Removed the suggestedHopFromCompactHistogram function to streamline hop suggestion logic. - Updated HopScalingModule to directly utilize CompactHistogram's internal methods for hop suggestions and sampling. - Enhanced logging in HopScalingModule to provide detailed histogram statistics. - Modified test cases to ensure comprehensive coverage of new histogram behaviors and sampling logic. - Improved node ID distribution in tests to better exercise sampling mechanisms. - Ensured that filtering denominators are held for 12 hours before dropping, enhancing stability in sampling. * Refactor CompactHistogram to support 13-hour activity tracking and introduce politeness regimes - Updated the bitfield structure to accommodate 13-hour seen tracking. - Changed the logic in rollHour() to analyze activity over the last 0-2 hours vs. 1-3 hours for politeness factor calculation. - Introduced three politeness levels: GENEROUS, DEFAULT, and STRICT based on recent activity ratios. - Adjusted filtering and sampling logic to reflect the new 13-hour tracking period. - Updated unit tests to validate new behavior and ensure proper functionality of politeness regimes. * Enhance CompactHistogram and HopScalingModule for improved sampling and decision-making - Introduced a session-specific hash seed in CompactHistogram to reduce bias in node ID sampling. - Updated sampling logic to use hashed node IDs instead of raw IDs for filtering and entry management. - Added histogram rollover tracking in HopScalingModule to ensure proper decision-making after initial data collection. - Adjusted logging to reflect the active state of the histogram and its comparison with NodeDB advisory hops. - Enhanced unit tests to validate new sampling logic and memory layout changes. * Expose hashNodeId for testing in CompactHistogram * Add mesh trend statistics to CompactHistogram for enhanced activity tracking * Implement histogram state persistence in CompactHistogram with save and load functions * Refactor CompactHistogram to improve entry management and enhance rollHour logging * feat: add HopScalingModule for adaptive hop limit recommendations Introduces HopScalingModule, a sampled hop-distance histogram that recommends the minimum hop limit needed to reach ~40 nodes, and automatically reducing the hops as the mesh grows. Key design: - 512-byte packed histogram (128 × 4-byte Record entries) embedded in a new HopScalingModule. - Each Record: 16-bit node hash, 3-bit hop distance, 13-bit seen bitmap - Sampling filter: only nodes where (hash & (denom-1)) == 0 are kept; denominator doubles on overflow and halves when utilisation is low - Hourly rollHour(): tallies per-hop counts, walks scaled buckets to find the minimum hop satisfying TARGET_AFFECTED_NODES (40), applies a politeness extension based on recent/older activity ratio, shifts all seen bitmaps, and persists state to /prefs/hopScalingState.bin - Hop recommendation gated by bootstrap (requires >=1 rollHour before overriding HOP_MAX) - NodeDB calls samplePacketForHistogram() on every non-MQTT rx packet - Module also estimates total mesh size and logs useful information about mesh characteristics. Changes: - src/modules/HopScalingModule.h/.cpp: new module - src/mesh/NodeDB.cpp: wire up samplePacketForHistogram - src/mesh/Router.cpp: consume getLastRequiredHop() - test/test_hop_scaling/: 12-test suite covering all mesh topologies and anticipated operational requirements * test: increase run iterations in sparse to dense transition test * feat: refactor HopScalingModule to use RUNS_PER_HOUR constant and improve logging * feat: enhance HopScalingModule with filtering denominator management and add tests for state transitions * refactor: remove CompactHistogram module and related files * address copilot review comments * Tweak: packet sampling only lora * ove role-based hop floor logic and related definitions into the module - keep it in one place. * Refactor MockNodeDB to use nodeInfoLiteSetBit for MQTT flag setting * Refactor hop scaling parameters and logic to integer maths and put default values in defaults.h. Small flash size reduction, no functional impact. * Update unit test preprocessor directives to PIO_UNIT_TESTING for consistency * refactor: improve test output organization and clarity in hop scaling tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
122 lines
6.0 KiB
C++
122 lines
6.0 KiB
C++
#pragma once
|
||
#include <MeshRadio.h>
|
||
#include <NodeDB.h>
|
||
#include <RadioInterface.h>
|
||
#include <cmath>
|
||
#include <cstdint>
|
||
#include <meshUtils.h>
|
||
#define ONE_DAY 24 * 60 * 60
|
||
#define ONE_MINUTE_MS 60 * 1000
|
||
#define THIRTY_SECONDS_MS 30 * 1000
|
||
#define TWO_SECONDS_MS 2 * 1000
|
||
#define FIVE_SECONDS_MS 5 * 1000
|
||
#define TEN_SECONDS_MS 10 * 1000
|
||
#define MAX_INTERVAL INT32_MAX // FIXME: INT32_MAX to avoid overflow issues with Apple clients but should be UINT32_MAX
|
||
|
||
#define min_default_telemetry_interval_secs IF_ROUTER(ONE_DAY / 2, 30 * 60)
|
||
#define default_gps_update_interval IF_ROUTER(ONE_DAY, 2 * 60)
|
||
#define default_telemetry_broadcast_interval_secs IF_ROUTER(ONE_DAY / 2, 60 * 60)
|
||
#define default_broadcast_interval_secs IF_ROUTER(ONE_DAY / 2, 60 * 60)
|
||
#define default_broadcast_smart_minimum_interval_secs 5 * 60
|
||
#define min_default_broadcast_interval_secs IF_ROUTER(ONE_DAY / 2, 60 * 60)
|
||
#define min_default_broadcast_smart_minimum_interval_secs 5 * 60
|
||
#define default_wait_bluetooth_secs IF_ROUTER(1, 60)
|
||
#define default_sds_secs IF_ROUTER(ONE_DAY, UINT32_MAX) // Default to forever super deep sleep
|
||
#define default_ls_secs IF_ROUTER(ONE_DAY, 5 * 60)
|
||
#define default_min_wake_secs 10
|
||
#define default_screen_on_secs IF_ROUTER(1, 60 * 10)
|
||
#define default_node_info_broadcast_secs 3 * 60 * 60
|
||
#define default_neighbor_info_broadcast_secs 6 * 60 * 60
|
||
#define min_node_info_broadcast_secs 60 * 60 // No regular broadcasts of more than once an hour
|
||
#define min_neighbor_info_broadcast_secs 4 * 60 * 60
|
||
#define default_map_publish_interval_secs 60 * 60
|
||
|
||
enum class TrafficType { POSITION, TELEMETRY };
|
||
|
||
// Traffic management defaults
|
||
#define default_traffic_mgmt_position_precision_bits 24 // ~10m grid cells
|
||
#define default_traffic_mgmt_position_min_interval_secs (ONE_DAY / 2) // 12 hours between identical positions
|
||
|
||
// Hop scaling defaults
|
||
#define default_hop_scaling_min_target_nodes 40 // walk threshold: first hop reaching this cumulative count
|
||
#define default_hop_scaling_max_target_nodes 80 // generous extension ceiling (2 × min)
|
||
#define default_hop_scaling_min_target_nodes_floor 5 // minimum allowed min_target_nodes
|
||
#define default_hop_scaling_max_target_nodes_ceiling 512 // maximum allowed max_target_nodes
|
||
|
||
#ifdef USERPREFS_RINGTONE_NAG_SECS
|
||
#define default_ringtone_nag_secs USERPREFS_RINGTONE_NAG_SECS
|
||
#else
|
||
#define default_ringtone_nag_secs 15
|
||
#endif
|
||
#define default_network_ipv6_enabled false
|
||
|
||
#define default_mqtt_address "mqtt.meshtastic.org"
|
||
#define default_mqtt_username "meshdev"
|
||
#define default_mqtt_password "large4cats"
|
||
#define default_mqtt_root "msh"
|
||
#define default_mqtt_encryption_enabled true
|
||
#define default_mqtt_tls_enabled false
|
||
|
||
#define IF_ROUTER(routerVal, normalVal) \
|
||
((config.device.role == meshtastic_Config_DeviceConfig_Role_ROUTER || \
|
||
config.device.role == meshtastic_Config_DeviceConfig_Role_ROUTER_LATE) \
|
||
? (routerVal) \
|
||
: (normalVal))
|
||
|
||
class Default
|
||
{
|
||
public:
|
||
static uint32_t getConfiguredOrDefaultMs(uint32_t configuredInterval);
|
||
static uint32_t getConfiguredOrDefaultMs(uint32_t configuredInterval, uint32_t defaultInterval);
|
||
static uint32_t getConfiguredOrDefault(uint32_t configured, uint32_t defaultValue);
|
||
// Note: numOnlineNodes uses uint32_t to match the public API and allow flexibility,
|
||
// even though internal node counts use uint16_t (max 65535 nodes)
|
||
static uint32_t getConfiguredOrDefaultMsScaled(uint32_t configured, uint32_t defaultValue, uint32_t numOnlineNodes);
|
||
static uint32_t getConfiguredOrDefaultMsScaled(uint32_t configured, uint32_t defaultValue, uint32_t numOnlineNodes,
|
||
TrafficType type);
|
||
static uint8_t getConfiguredOrDefaultHopLimit(uint8_t configured);
|
||
static uint32_t getConfiguredOrMinimumValue(uint32_t configured, uint32_t minValue);
|
||
|
||
private:
|
||
// Note: Kept as uint32_t to match the public API parameter type
|
||
static float congestionScalingCoefficient(uint32_t numOnlineNodes)
|
||
{
|
||
if (numOnlineNodes <= 40) {
|
||
return 1.0;
|
||
} else {
|
||
// Resolve SF and BW from preset or manual config
|
||
// When use_preset is true, config.lora.spread_factor and bandwidth may be 0
|
||
// because applyModemConfig() sets them on RadioInterface, not on config.lora
|
||
float bwKHz;
|
||
uint8_t sf;
|
||
uint8_t cr;
|
||
if (config.lora.use_preset) {
|
||
modemPresetToParams(config.lora.modem_preset, false, bwKHz, sf, cr);
|
||
} else {
|
||
sf = config.lora.spread_factor;
|
||
bwKHz = bwCodeToKHz(config.lora.bandwidth);
|
||
}
|
||
|
||
// Guard against invalid values
|
||
sf = clampSpreadFactor(sf);
|
||
bwKHz = clampBandwidthKHz(bwKHz);
|
||
|
||
// throttlingFactor = 2^SF / (BW_in_kHz * scaling_divisor)
|
||
// With scaling_divisor=100:
|
||
// In SF11 and BW=250khz (longfast), this gives 0.08192 rather than the original 0.075
|
||
// In SF10 and BW=250khz (mediumslow), this gives 0.04096 rather than the original 0.04
|
||
// In SF9 and BW=250khz (mediumfast), this gives 0.02048 rather than the original 0.02
|
||
// In SF7 and BW=250khz (shortfast), this gives 0.00512 rather than the original 0.01
|
||
float throttlingFactor = static_cast<float>(pow_of_2(sf)) / (bwKHz * 100.0f);
|
||
|
||
#if USERPREFS_EVENT_MODE
|
||
// If we are in event mode, scale down the throttling factor by 4
|
||
throttlingFactor = static_cast<float>(pow_of_2(sf)) / (bwKHz * 25.0f);
|
||
#endif
|
||
|
||
// Scaling up traffic based on number of nodes over 40
|
||
int nodesOverForty = (numOnlineNodes - 40);
|
||
return 1.0 + (nodesOverForty * throttlingFactor); // Each number of online node scales by throttle factor
|
||
}
|
||
}
|
||
}; |