Ben MeadorsandGitHub
24c1dccf00
* CI: track RAM (.data+.bss) in size reports and gate on per-env budgets
The 2.8.0 nRF52840 heap regression (99% heap in field reports) shipped
invisibly because CI only tracked flash. On nRF52840 the heap arena is the
linker gap after .bss, so every byte of static .data+.bss growth shrinks the
usable heap 1:1 - RAM needs the same guardrails flash already has.
What's added:
- bin/platformio-custom.py emits ram_bytes (.data + .bss from the ELF, via
the toolchain size tool) into each .mt.json manifest. Heap/stack
placeholder sections are deliberately excluded.
- bin/collect_sizes.py records {flash_bytes, ram_bytes} per env;
bin/size_report.py grows RAM and RAM-delta columns in the PR size report.
Older artifacts without ram_bytes (and legacy int-schema baselines)
degrade to "n/a" instead of crashing.
- bin/ram_budgets.json: per-env RAM/flash budgets, enforced only for envs
listed there. Seeded with rak4631: ram 113,000 (current 110,948 + ~2 KB
slack), flash 786,000 (current 765,192 + ~20 KB; the app region is
0x27000..0xEA000 = 798,720 and the image must stay clear of the
warm-store ring guard in extra_scripts/nrf52_warm_region.py).
- New size-budget-gate CI job runs size_report.py --enforce-budgets and
fails the build on violation; the informational firmware-size-report job
now also renders budget usage into the PR comment.
- src/main.cpp: opt-in boot heap watermark (-DMESHTASTIC_HEAP_WATERMARK_CHECK)
logs LOG_ERROR when less than 20% of the heap is free at the end of
setup(). Off by default; skipped on platforms without heap accounting.
How budgets are raised: deliberately, never automatically. If a change needs
more headroom, bump the env's limit in bin/ram_budgets.json in the same PR
and justify the increase in the PR description.
Verified: python3 bin/test_size_scripts.py (23/23 pass, including ram_bytes
parsing, n/a fallback, and over/under/missing-env budget-gate cases).
* Address review: fail the budget gate closed, fix RAM section matching
- size-budget-gate workflow: drop continue-on-error on the manifest
download and the empty-dir fallback, so the job fails when the data it
gates on cannot be fetched.
- size_report.py --enforce-budgets now fails closed on every
missing-data path instead of trivially passing: empty collected sizes,
a budgeted env that was not built, or a manifest without the budgeted
metric. Report-only mode keeps rendering those as n/a.
- load_budgets() rejects zero/negative/non-integer budgets with a clear
error (a typo'd budget could previously crash budget_markdown with
ZeroDivisionError or silently skip the check); the percentage render
keeps a defensive guard for direct callers.
- compute_ram_bytes(): count RISC-V small-data sections (.sdata/.sbss)
and exclude ESP-IDF .rtc.* sections, which live outside the
heap-competing SRAM.
- Trim the heap-watermark comment in main.cpp to two lines.
bin/test_size_scripts.py: 27/27 - the two fail-open assertions are
flipped to fail-closed, with new report-only counterparts plus cases for
empty sizes under enforcement and malformed budgets.
67 lines
2.3 KiB
Python
67 lines
2.3 KiB
Python
#!/usr/bin/env python3
|
|
|
|
"""Collect firmware binary sizes from manifest (.mt.json) files into a single report.
|
|
|
|
Output schema (consumed by bin/size_report.py):
|
|
{"<env>": {"flash_bytes": <int>, "ram_bytes": <int>}}
|
|
|
|
flash_bytes is the size of the main firmware image (.bin). ram_bytes is the
|
|
static RAM footprint (.data + .bss) emitted into the manifest by
|
|
bin/platformio-custom.py; it is omitted for manifests that predate it, and
|
|
size_report.py renders those as "n/a".
|
|
"""
|
|
|
|
import json
|
|
import os
|
|
import sys
|
|
|
|
|
|
def collect_sizes(manifest_dir):
|
|
"""Scan manifest_dir for .mt.json files and return {board: sizes_dict}."""
|
|
sizes = {}
|
|
for fname in sorted(os.listdir(manifest_dir)):
|
|
if not fname.endswith(".mt.json"):
|
|
continue
|
|
path = os.path.join(manifest_dir, fname)
|
|
with open(path) as f:
|
|
data = json.load(f)
|
|
board = data.get("platformioTarget", fname.replace(".mt.json", ""))
|
|
# Find the main firmware .bin size (largest .bin, excluding OTA/littlefs/bleota)
|
|
bin_size = None
|
|
for entry in data.get("files", []):
|
|
name = entry.get("name", "")
|
|
if name.startswith("firmware-") and name.endswith(".bin"):
|
|
bin_size = entry["bytes"]
|
|
break
|
|
# Fallback: any .bin that isn't ota/littlefs/bleota
|
|
if bin_size is None:
|
|
for entry in data.get("files", []):
|
|
name = entry.get("name", "")
|
|
if name.endswith(".bin") and not any(
|
|
x in name for x in ["littlefs", "bleota", "ota"]
|
|
):
|
|
bin_size = entry["bytes"]
|
|
break
|
|
if bin_size is not None:
|
|
entry = {"flash_bytes": bin_size}
|
|
ram_bytes = data.get("ram_bytes")
|
|
if isinstance(ram_bytes, int) and not isinstance(ram_bytes, bool):
|
|
entry["ram_bytes"] = ram_bytes
|
|
sizes[board] = entry
|
|
return sizes
|
|
|
|
|
|
if __name__ == "__main__":
|
|
if len(sys.argv) != 3:
|
|
print(f"Usage: {sys.argv[0]} <manifest_dir> <output.json>", file=sys.stderr)
|
|
sys.exit(1)
|
|
|
|
manifest_dir = sys.argv[1]
|
|
output_path = sys.argv[2]
|
|
|
|
sizes = collect_sizes(manifest_dir)
|
|
with open(output_path, "w") as f:
|
|
json.dump(sizes, f, indent=2, sort_keys=True)
|
|
|
|
print(f"Collected sizes for {len(sizes)} targets -> {output_path}")
|