CI: track RAM (.data+.bss) in size reports and gate on per-env budgets (#10899)
* CI: track RAM (.data+.bss) in size reports and gate on per-env budgets
The 2.8.0 nRF52840 heap regression (99% heap in field reports) shipped
invisibly because CI only tracked flash. On nRF52840 the heap arena is the
linker gap after .bss, so every byte of static .data+.bss growth shrinks the
usable heap 1:1 - RAM needs the same guardrails flash already has.
What's added:
- bin/platformio-custom.py emits ram_bytes (.data + .bss from the ELF, via
the toolchain size tool) into each .mt.json manifest. Heap/stack
placeholder sections are deliberately excluded.
- bin/collect_sizes.py records {flash_bytes, ram_bytes} per env;
bin/size_report.py grows RAM and RAM-delta columns in the PR size report.
Older artifacts without ram_bytes (and legacy int-schema baselines)
degrade to "n/a" instead of crashing.
- bin/ram_budgets.json: per-env RAM/flash budgets, enforced only for envs
listed there. Seeded with rak4631: ram 113,000 (current 110,948 + ~2 KB
slack), flash 786,000 (current 765,192 + ~20 KB; the app region is
0x27000..0xEA000 = 798,720 and the image must stay clear of the
warm-store ring guard in extra_scripts/nrf52_warm_region.py).
- New size-budget-gate CI job runs size_report.py --enforce-budgets and
fails the build on violation; the informational firmware-size-report job
now also renders budget usage into the PR comment.
- src/main.cpp: opt-in boot heap watermark (-DMESHTASTIC_HEAP_WATERMARK_CHECK)
logs LOG_ERROR when less than 20% of the heap is free at the end of
setup(). Off by default; skipped on platforms without heap accounting.
How budgets are raised: deliberately, never automatically. If a change needs
more headroom, bump the env's limit in bin/ram_budgets.json in the same PR
and justify the increase in the PR description.
Verified: python3 bin/test_size_scripts.py (23/23 pass, including ram_bytes
parsing, n/a fallback, and over/under/missing-env budget-gate cases).
* Address review: fail the budget gate closed, fix RAM section matching
- size-budget-gate workflow: drop continue-on-error on the manifest
download and the empty-dir fallback, so the job fails when the data it
gates on cannot be fetched.
- size_report.py --enforce-budgets now fails closed on every
missing-data path instead of trivially passing: empty collected sizes,
a budgeted env that was not built, or a manifest without the budgeted
metric. Report-only mode keeps rendering those as n/a.
- load_budgets() rejects zero/negative/non-integer budgets with a clear
error (a typo'd budget could previously crash budget_markdown with
ZeroDivisionError or silently skip the check); the percentage render
keeps a defensive guard for direct callers.
- compute_ram_bytes(): count RISC-V small-data sections (.sdata/.sbss)
and exclude ESP-IDF .rtc.* sections, which live outside the
heap-competing SRAM.
- Trim the heap-watermark comment in main.cpp to two lines.
bin/test_size_scripts.py: 27/27 - the two fail-open assertions are
flipped to fail-closed, with new report-only counterparts plus cases for
empty sizes under enforcement and malformed budgets.
This commit is contained in:
@@ -45,6 +45,49 @@ def infer_architecture(board_cfg):
|
||||
return "stm32"
|
||||
return None
|
||||
|
||||
def compute_ram_bytes(env):
|
||||
"""Static RAM usage (.data + .bss) of the ELF, via the toolchain size tool.
|
||||
|
||||
Deliberately excludes heap/stack placeholder sections (e.g. the nRF52 .heap
|
||||
section): on nRF52840 the heap arena is the linker gap after .bss, so static
|
||||
RAM growth shrinks the usable heap 1:1 - which is exactly why we track it.
|
||||
Returns None when the value cannot be determined; the manifest then simply
|
||||
omits ram_bytes and downstream size reports show "n/a".
|
||||
"""
|
||||
elf = env.File(env.subst("$BUILD_DIR/${PROGNAME}.elf"))
|
||||
if not elf.exists():
|
||||
return None
|
||||
size_tool = env.subst("$SIZETOOL") or "size"
|
||||
try:
|
||||
output = subprocess.check_output(
|
||||
[size_tool, "-A", elf.get_abspath()],
|
||||
env=env["ENV"],
|
||||
universal_newlines=True,
|
||||
)
|
||||
except Exception as exc:
|
||||
print(f"mtjson: skipping ram_bytes ({size_tool} failed: {exc})")
|
||||
return None
|
||||
ram = 0
|
||||
found = False
|
||||
for line in output.splitlines():
|
||||
parts = line.split()
|
||||
if len(parts) < 2:
|
||||
continue
|
||||
name = parts[0]
|
||||
# Main-SRAM static sections: .data/.bss, platform-prefixed variants (e.g.
|
||||
# ESP32 .dram0.data/.dram0.bss), and RISC-V small-data .sdata/.sbss.
|
||||
# ESP-IDF .rtc.* sections live outside the heap-competing SRAM; .heap and
|
||||
# .tdata never match.
|
||||
if name.startswith(".rtc"):
|
||||
continue
|
||||
if name in (".data", ".bss", ".sdata", ".sbss") or name.endswith(".data") or name.endswith(".bss"):
|
||||
try:
|
||||
ram += int(parts[1])
|
||||
found = True
|
||||
except ValueError:
|
||||
continue
|
||||
return ram if found else None
|
||||
|
||||
def manifest_gather(source, target, env):
|
||||
global manifest_ran
|
||||
if manifest_ran:
|
||||
@@ -98,9 +141,9 @@ def manifest_gather(source, target, env):
|
||||
d["part_name"] = partition_map[p]
|
||||
out.append(d)
|
||||
print(d)
|
||||
manifest_write(out, env)
|
||||
manifest_write(out, env, compute_ram_bytes(env))
|
||||
|
||||
def manifest_write(files, env):
|
||||
def manifest_write(files, env, ram_bytes=None):
|
||||
# Defensive: also skip manifest writing if we cannot determine architecture
|
||||
def get_project_option(name):
|
||||
try:
|
||||
@@ -137,6 +180,10 @@ def manifest_write(files, env):
|
||||
"has_mui": False,
|
||||
"has_inkhud": False,
|
||||
}
|
||||
# Static RAM footprint (.data + .bss); consumed by bin/collect_sizes.py for
|
||||
# the CI size report and the bin/ram_budgets.json budget gate.
|
||||
if ram_bytes is not None:
|
||||
manifest["ram_bytes"] = ram_bytes
|
||||
# Get partition table (generated in esp32_pre.py) if it exists
|
||||
if env.get("custom_mtjson_part"):
|
||||
# custom_mtjson_part is a JSON string, convert it back to a dict
|
||||
|
||||
Reference in New Issue
Block a user