597f6767b5e0c37758444071477d70c74657eecc
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  
|
21e3a583bd |
Yaml check for Meshtasticd (#11224)
* feat(portduino): add `meshtasticd --check` config validator Users hand-writing files in /etc/meshtasticd/config.d/ get no feedback when a key is misplaced, misspelled or duplicated: meshtasticd silently ignores what it does not read, so a broken config looks identical to a working one. Add a --check mode that loads the configuration exactly as startup does, then reports what it found and exits: - Duplicate keys, via the yaml-cpp Parser/EventHandler stream. The Node API cannot see them because the map is already collapsed by the time it exists, and yaml-cpp keeps the FIRST occurrence, so a later override is discarded. - Unknown or misnested keys, against a schema mirroring what loadConfig() reads, with a hint naming the section a stray key actually belongs to. - rfswitch_table validation: unrecognised pins, mode rows whose length does not match the pin list, values that are not HIGH/LOW, and unknown modes. - Cross-file overlap: every .yaml in the config directory merges into one portduino_config, so the file loaded LAST wins, the opposite of the within-file rule. Those files are read in filesystem order, not alphabetical. - A warning when more than one file defines a Lora section: spidev, spiSpeed, gpiochip, DIO2_AS_RF_SWITCH, DIO3_TCXO_VOLTAGE and USB_PID/VID/Serialnum are assigned unconditionally with a default every time one is seen, so any of them not repeated in the last file loaded is silently reset. - The resolved gpiochip/line for each pin, since a line that exists on the wrong chip is claimed successfully and then silently does nothing. Exits non-zero when errors were found so it can also gate CI over bin/config.d/**, keeping one implementation rather than a second schema. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(portduino): flag pins that resolve to -1 in --check A pin key whose value will not convert to a number falls back to RADIOLIB_NC (-1) while still being marked enabled, and initGPIOPin() then trips an assertion inside LinuxGPIOPin rather than failing cleanly. YAML indentation makes this easy to hit by accident: a stray line under "CS: 8" folds into the value as a multi-line scalar, so the file parses, the daemon crashes with a stack trace from a library file, and --check reported "Configuration looks good" while printing "pin -1" two lines above. Report it as an error naming the likely cause instead. Also correct a comment claiming unparseable config.d files are skipped silently. They are not: loadConfig() prints "*** Exception ..." with the line and column. It is the discarded return value, not the diagnostic, that makes the file's absence from the merged config easy to miss. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(portduino): cover `meshtasticd --check` with fixtures and a fuzz suite Adds the tests the config validator was missing, and the checks and fixes that writing them turned up. The theme throughout is configuration that the YAML parser accepts but that does not mean what it looks like it means. Tests ----- bin/test-config-check.sh - 57 assertions driving a built meshtasticd against test/fixtures/portduino-config (50 fixtures plus two config.d trees). A shell test rather than a Unity suite because both behaviours under test are properties of the process: --check is judged by its exit status and printed report, and the "a normal run rejects a bad config" path ends in exit() inside portduinoSetup(), neither of which is reachable from a suite that links one translation unit. Every fixture carries a comment header naming its planted fault and the expected finding, so it can be read on its own. Coverage: * a clean config for each of the ten radio module families (RF95, sx1262, sx1268, LLCC68, sx1280, lr1110, lr1120, lr1121, sim, auto), asserted both findings-free and resolving to that module, so a silent fallback to sim cannot pass * LR11xx rfswitch tables: unrecognised pins, rows longer and shorter than the pin count, levels that are not exactly HIGH, a missing pins list, more than five pins, a scalar table, unknown MODE_ keys, a MODE_ row stranded one level out, and a legal partial table * the PA gain table in both accepted shapes, entries outside the uint16 range it is stored in, and more than the 22 points that are kept * values of the wrong type, split by consequence: the two settings read with no fallback stop meshtasticd starting, everything else is silently replaced by its default * out-of-range and unit mistakes: TCXO voltage written in millivolts, ports outside their usable range, an over-long StatusMessage * MAC sources: both keys set at once, a malformed address, an interface that does not exist * structural faults: duplicate keys, non-mapping and unknown sections, a key left at the top level, a sequence at the document root, an empty file, unreadable pins, unparseable YAML * cross-file behaviour over a config.d directory, including the switch tables that do not override each other * five configs run WITHOUT --check, each of which must still be refused, so check mode cannot quietly make the normal path permissive test/test_fuzz_config - adversarial fuzzing of the checker itself, the "the tool meant to diagnose your config crashes on it" failure mode. Scope is deliberately narrow: yaml-cpp does the parsing and is fuzzed upstream, so what is exercised here is our code above the parse, above all the duplicate-key detector, which is the one hand-rolled piece and walks the raw parser event stream with its own stack. Groups: the checked-in fixtures as a seed corpus, 3000 byte mutations of them (flips, truncation, insertion, splicing, deletion), and structural torture (nesting to 4096 in flow and block style, duplicate keys at depth, anchors, aliases and merge keys, 64KB keys, 256KB scalars, multi-document files). A fourth group of random bytes is present but disabled behind FUZZ_CONFIG_RANDOM_BYTES: it was half the runtime for the least return, since uniform noise is rejected on the first token. The contract is crash-freedom and termination under AddressSanitizer, not any particular finding. CI runs the shell test in the existing native simulator job; the fuzz suite is picked up by the existing ^test_fuzz_ area rule. native-suite-count 40 -> 41. The fixtures are exempt from trunk in .trunk/trunk.yaml, since prettier rejects the duplicate keys and bad indentation that are the point of them. Checker fixes found while writing the tests ------------------------------------------- --check reported a clean exit 0 on configs meshtasticd then refuses to boot, the worst failure a diagnostic tool can have. Four hard exits inside loadConfig() killed the report before it printed: an unparseable file, an unknown Lora.Module, MACAddress and MACAddressSource both set, and HUB75 on a build without it. All are now reported as findings, and all are still refused on a normal run. New validation: Lora.Module against the accepted spellings, which are matched exactly and inconsistently cased, with a suggestion when only case differs; a per-key value type table covering ~85 keys, tested by asking yaml-cpp to perform the same conversion loadConfig() will so it cannot drift; the PA gain table; DIO3_TCXO_VOLTAGE, which is in volts and multiplied by 1000, so the millivolt value everything else uses silently asks for 1800V; APIPort and Webserver.Port ranges; MaxNodes; StatusMessage truncation; MAC address and source; and an unreadable ConfigDirectory. Also fixes a crash: a ConfigDirectory that cannot be read threw an uncaught filesystem_error from directory_iterator and aborted meshtasticd with SIGABRT, taking --check down with it. It now fails cleanly. Two smaller ones: cppcheck's uselessCallsSubstr on the ancestor walk, which was failing every check job; and the duplicate-key detector's stack pop, which was unguarded and relied on yaml-cpp emitting balanced events. Switch tables are the one place "the file loaded last wins" is false. The loader only ever writes HIGH and never writes LOW back, so a HIGH from an earlier file survives a later file that clears it and the radio drives the OR of every table loaded. Confirmed with --output-yaml. Reported as an error for now; the loader itself is left alone, as that changes RF behaviour. * fix(portduino): report CH341 pins as adapter indexes, not gpiochip lines --check printed "Resolved GPIO lines (what meshtasticd will try to claim)" for every config, listing a gpiochip and line for each Lora pin and advising they be confirmed against gpiodetect and gpioinfo. For spidev: ch341 every part of that is false. portduinoSetup() skips initGPIOPin() for every Lora pin when spidev is ch341 and hands the raw numbers to Ch341Hal, so nothing is claimed from a gpiochip -- and on Windows and macOS, where a USB adapter is the only way to attach a radio, there is no gpiochip, gpiodetect or gpioinfo to check against in the first place. The checker had no ch341 coverage at all: not one fixture used it, so the whole USB-SPI path went unexercised. The summary now splits on the transport. A ch341 device gets its pins listed as adapter indexes with the gpiod advice dropped, and a gpiochip or line mapping written alongside it is reported: those are read, stored, and never used. Also: "RF switch table: not set" read as a gap on an SX126x, where there is nothing to set. setRfSwitchTable() is only ever called for an LR11xx, so absence is now "not needed for this module" everywhere else, and "not resolved yet" for auto, which has no module to judge against. Fixtures: usb-ch341.yaml (clean, the meshstick shape) and ch341-gpiochip.yaml. CI fix ------ test-native was RED on "config.d overrides are reported", which wanted 2 warnings and got 1. The fixture's two config.d files name different modules, so which one wins -- and whether the LR11xx-without-a-switch-table warning fires -- depends on the order the filesystem returns them in. That is the very thing the fixture exists to demonstrate, so the count is no longer asserted; the report's own order caveat is asserted instead. Review fixes ------------ The unreadable-ConfigDirectory diagnostic was the one new print in PortduinoGlue.cpp not gated behind !configCheck, so it landed ahead of the report header and broke the clean output the rest of the change is careful to keep. Docs: rfswitch-valid.yaml carries seven modes, not eight, and empty-file.yaml is comments-only rather than zero bytes. * style(portduino): trim --check comment blocks and reconcile suite count Condense the multi-paragraph comment blocks in the --check validator to the one-to-two-line convention, and bump test/native-suite-count to 42 for the test_fuzz_config suite added here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
8e104a909b |
fix(admin): persist TAK module config (team color / member role) (#11216)
Setting TAK team/role ACKed and rebooted but stored nothing: handleSetModuleConfig had no tak case, saveToDisk never set has_tak, and handleGetModuleConfig had no TAK_CONFIG case. Add all three, plus a native suite sweeping every ModuleConfig submessage through set -> save -> load -> get and a TAK value-fidelity suite. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
TomandGitHub
|
3becaf2d95 | emdashes begone (#10847) | ||
 nomdetom
|
90f01c2b95 | docs: update test instructions to prefer bin/run-tests.sh; fix suite count to 19; add Copilot interface caveat | ||
|
TomandGitHub
|
ab882c5619 |
EU regions merge (#10675)
* stronger together * validate 2.4ghz regions * less noise * you're right, and that shapens the analysis significantly * sassy rejoinder |
||
|
de345939af |
Automatic variable hop limits based on mesh activity and size estimation (#10176)
* asdf * Implement SphereOfInfluenceModule for traffic management and eviction tracking * Implement Sphere of Influence module for dynamic hop limit adjustment and role-based floor * Update SAMPLING_DENOMINATOR to improve mesh size estimation accuracy * Add debug logging for scale factor estimation and per-hop node counts in SphereOfInfluenceModule * Enable variable hop limits and role-based hop floors in Sphere of Influence module * Respond to copilot review * Disable variable hop limits and role-based hop floors in Sphere of Influence module * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Implement adaptive sampling for unique node ID tracking in Sphere of Influence module * Add state persistence for Sphere of Influence module * Enhance Sphere of Influence module with state management and adaptive sampling adjustments * Refactor hop scaling functionality: remove SphereOfInfluenceModule and introduce HopScalingModule - Deleted SphereOfInfluenceModule.h, consolidating its responsibilities into the new HopScalingModule. - Added HopScalingModule.h and HopScalingModule.cpp to manage hop scaling logic, including eviction tracking and sampling-based mesh size estimation. - Implemented methods for recording evictions and packet senders, estimating scale factors, and computing required hops based on node activity. - Introduced state persistence for hop scaling parameters to maintain continuity across reboots. - Enhanced thread safety and modularity by utilizing concurrency features. * Guard out STM32. Sowwy. * Refactor HopScalingModule: enhance sampling logic and improve state management * Add unit tests for HopScalingModule: implement mock database and various test scenarios * Refactor test output in HopScalingModule tests: replace printf with TEST_MESSAGE for better integration with Unity * Refactor HopScalingModule logging: replace lastStatusMode with descriptive mode names for improved readability * Refactor test_main.cpp: change NodeNum variable to static and improve comments for clarity * Remove unnecessary delay in setup function for improved test performance * Add missing include for MeshTypes in test_main.cpp * Refactor HopScalingModule tests: enhance mesh topology scenarios and improve test clarity * Update HopScalingModule tests: flesh out node scenarios and improve clarity for dense and sparse mesh cases * Fix politeness factor calculations in HopScalingModule and update related test scenarios for clarity. Remove outdated design doc. * Enhance HopScalingModule: add sampled estimate for scaling decisions and refactor initial run state management * Add sample traffic injection for HopScaling tests to enhance sampledEst visibility * Enhance HopScalingModule: adjust windowFraction calculation for early triggers and improve test output formatting * Enhance HopScalingModule: add jitter functionality to sampling denominator and update tests for consistent behavior * Enhance HopScalingModule: implement adaptive sampling denominator adjustment and add reset functionality for tests * Enhance HopScalingTestShim: add test-only clock and window helpers, update injectSampleTraffic for adaptive sampling, and improve scenario summary output * Enhance HopScalingModule: add detailed documentation for functions, improve clarity of jitter and sampling logic, and reset functionality in tests * Enhance HopScalingModule: add evictionEstimate parameter to estimateScaleFactor and update related logging for improved mesh size estimation * Enhance HopScalingModule: adjust effective rolls calculation for improved accuracy, add eviction estimate logic, responding to all copilot review points * Implement CompactHistogram for parallel hop scaling sampling - Added CompactHistogram class to track node hop distances with bitwise sampling. - Integrated CompactHistogram into HopScalingModule for independent packet sampling. - Updated NodeDB to feed both the hop scaling module and the new histogram sampler. - Enhanced HopScalingModule with methods to sample packets for the histogram and retrieve hop distribution statistics. - Implemented tests for CompactHistogram functionality, including sampling, window rolling, and adaptive denominator scaling. - Updated existing tests to validate the integration of the new histogram sampling mechanism. * Enhance CompactHistogram and HopScalingModule: add per-hop distribution functionality, improve time handling for unit tests, and refine test setup for deterministic behavior * CompactHistogram: add mesh size estimation, improve entry replacement logic, and update logging for per-hop distribution * Refactor HopScalingModule and CompactHistogram integration - Removed the suggestedHopFromCompactHistogram function to streamline hop suggestion logic. - Updated HopScalingModule to directly utilize CompactHistogram's internal methods for hop suggestions and sampling. - Enhanced logging in HopScalingModule to provide detailed histogram statistics. - Modified test cases to ensure comprehensive coverage of new histogram behaviors and sampling logic. - Improved node ID distribution in tests to better exercise sampling mechanisms. - Ensured that filtering denominators are held for 12 hours before dropping, enhancing stability in sampling. * Refactor CompactHistogram to support 13-hour activity tracking and introduce politeness regimes - Updated the bitfield structure to accommodate 13-hour seen tracking. - Changed the logic in rollHour() to analyze activity over the last 0-2 hours vs. 1-3 hours for politeness factor calculation. - Introduced three politeness levels: GENEROUS, DEFAULT, and STRICT based on recent activity ratios. - Adjusted filtering and sampling logic to reflect the new 13-hour tracking period. - Updated unit tests to validate new behavior and ensure proper functionality of politeness regimes. * Enhance CompactHistogram and HopScalingModule for improved sampling and decision-making - Introduced a session-specific hash seed in CompactHistogram to reduce bias in node ID sampling. - Updated sampling logic to use hashed node IDs instead of raw IDs for filtering and entry management. - Added histogram rollover tracking in HopScalingModule to ensure proper decision-making after initial data collection. - Adjusted logging to reflect the active state of the histogram and its comparison with NodeDB advisory hops. - Enhanced unit tests to validate new sampling logic and memory layout changes. * Expose hashNodeId for testing in CompactHistogram * Add mesh trend statistics to CompactHistogram for enhanced activity tracking * Implement histogram state persistence in CompactHistogram with save and load functions * Refactor CompactHistogram to improve entry management and enhance rollHour logging * feat: add HopScalingModule for adaptive hop limit recommendations Introduces HopScalingModule, a sampled hop-distance histogram that recommends the minimum hop limit needed to reach ~40 nodes, and automatically reducing the hops as the mesh grows. Key design: - 512-byte packed histogram (128 × 4-byte Record entries) embedded in a new HopScalingModule. - Each Record: 16-bit node hash, 3-bit hop distance, 13-bit seen bitmap - Sampling filter: only nodes where (hash & (denom-1)) == 0 are kept; denominator doubles on overflow and halves when utilisation is low - Hourly rollHour(): tallies per-hop counts, walks scaled buckets to find the minimum hop satisfying TARGET_AFFECTED_NODES (40), applies a politeness extension based on recent/older activity ratio, shifts all seen bitmaps, and persists state to /prefs/hopScalingState.bin - Hop recommendation gated by bootstrap (requires >=1 rollHour before overriding HOP_MAX) - NodeDB calls samplePacketForHistogram() on every non-MQTT rx packet - Module also estimates total mesh size and logs useful information about mesh characteristics. Changes: - src/modules/HopScalingModule.h/.cpp: new module - src/mesh/NodeDB.cpp: wire up samplePacketForHistogram - src/mesh/Router.cpp: consume getLastRequiredHop() - test/test_hop_scaling/: 12-test suite covering all mesh topologies and anticipated operational requirements * test: increase run iterations in sparse to dense transition test * feat: refactor HopScalingModule to use RUNS_PER_HOUR constant and improve logging * feat: enhance HopScalingModule with filtering denominator management and add tests for state transitions * refactor: remove CompactHistogram module and related files * address copilot review comments * Tweak: packet sampling only lora * ove role-based hop floor logic and related definitions into the module - keep it in one place. * Refactor MockNodeDB to use nodeInfoLiteSetBit for MQTT flag setting * Refactor hop scaling parameters and logic to integer maths and put default values in defaults.h. Small flash size reduction, no functional impact. * Update unit test preprocessor directives to PIO_UNIT_TESTING for consistency * refactor: improve test output organization and clarity in hop scaling tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
nomdetom
|
2d6f2ba1a4 | docs: enhance README with verbose test output instructions and usage examples | ||
|
nomdetom
|
5d55353939 | Some fixes and tidies for testing both online and in unit_tests | ||
|
f396200d38 |
Add authoring guide for native unit tests in README.md (#10201)
* Add authoring guide for native unit tests in README.md * Enhance documentation for agent tooling and native unit tests in README and related files --------- Co-authored-by: Ben Meadors <benmmeadors@gmail.com> |