kevin/meshtastic_firmware
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

CI: track RAM (.data+.bss) in size reports and gate on per-env budgets (#10899)

Browse Source 
* CI: track RAM (.data+.bss) in size reports and gate on per-env budgets

The 2.8.0 nRF52840 heap regression (99% heap in field reports) shipped
invisibly because CI only tracked flash. On nRF52840 the heap arena is the
linker gap after .bss, so every byte of static .data+.bss growth shrinks the
usable heap 1:1 - RAM needs the same guardrails flash already has.

What's added:

- bin/platformio-custom.py emits ram_bytes (.data + .bss from the ELF, via
  the toolchain size tool) into each .mt.json manifest. Heap/stack
  placeholder sections are deliberately excluded.
- bin/collect_sizes.py records {flash_bytes, ram_bytes} per env;
  bin/size_report.py grows RAM and RAM-delta columns in the PR size report.
  Older artifacts without ram_bytes (and legacy int-schema baselines)
  degrade to "n/a" instead of crashing.
- bin/ram_budgets.json: per-env RAM/flash budgets, enforced only for envs
  listed there. Seeded with rak4631: ram 113,000 (current 110,948 + ~2 KB
  slack), flash 786,000 (current 765,192 + ~20 KB; the app region is
  0x27000..0xEA000 = 798,720 and the image must stay clear of the
  warm-store ring guard in extra_scripts/nrf52_warm_region.py).
- New size-budget-gate CI job runs size_report.py --enforce-budgets and
  fails the build on violation; the informational firmware-size-report job
  now also renders budget usage into the PR comment.
- src/main.cpp: opt-in boot heap watermark (-DMESHTASTIC_HEAP_WATERMARK_CHECK)
  logs LOG_ERROR when less than 20% of the heap is free at the end of
  setup(). Off by default; skipped on platforms without heap accounting.

How budgets are raised: deliberately, never automatically. If a change needs
more headroom, bump the env's limit in bin/ram_budgets.json in the same PR
and justify the increase in the PR description.

Verified: python3 bin/test_size_scripts.py (23/23 pass, including ram_bytes
parsing, n/a fallback, and over/under/missing-env budget-gate cases).

* Address review: fail the budget gate closed, fix RAM section matching

- size-budget-gate workflow: drop continue-on-error on the manifest
  download and the empty-dir fallback, so the job fails when the data it
  gates on cannot be fetched.
- size_report.py --enforce-budgets now fails closed on every
  missing-data path instead of trivially passing: empty collected sizes,
  a budgeted env that was not built, or a manifest without the budgeted
  metric. Report-only mode keeps rendering those as n/a.
- load_budgets() rejects zero/negative/non-integer budgets with a clear
  error (a typo'd budget could previously crash budget_markdown with
  ZeroDivisionError or silently skip the check); the percentage render
  keeps a defensive guard for direct callers.
- compute_ram_bytes(): count RISC-V small-data sections (.sdata/.sbss)
  and exclude ESP-IDF .rtc.* sections, which live outside the
  heap-competing SRAM.
- Trim the heap-watermark comment in main.cpp to two lines.

bin/test_size_scripts.py: 27/27 - the two fail-open assertions are
flipped to fail-closed, with new report-only counterparts plus cases for
empty sizes under enforcement and malformed budgets.
This commit is contained in:
Ben Meadors
2026-07-06 14:52:22 -05:00
committed by GitHub
co-authored by GitHub
parent 12b2d973a4
commit 24c1dccf00
7 changed files with 769 additions and 88 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/main_matrix.yml
+29 -1
View File
@@ -337,7 +337,7 @@ jobs:
if: github.event_name == 'pull_request'
id: report
run: |
ARGS="./current-sizes.json"
ARGS="./current-sizes.json --budgets bin/ram_budgets.json"
if [ -f ./develop-sizes.json ]; then
ARGS="$ARGS --baseline develop:./develop-sizes.json"
fi
@@ -375,6 +375,34 @@ jobs:
./pr-number.txt
retention-days: 5
# RAM/flash guardrails: fails CI when an env listed in bin/ram_budgets.json
# exceeds its static RAM (.data+.bss) or flash budget. Kept separate from
# firmware-size-report, which is informational and continue-on-error.
size-budget-gate:
permissions:
contents: read
actions: read
runs-on: ubuntu-latest
needs: [build]
steps:
- uses: actions/checkout@v7
# No continue-on-error / empty-dir fallback: the gate must fail closed when
# the data it enforces on cannot be fetched (size_report.py additionally
# fails on missing budgeted envs under --enforce-budgets).
- name: Download current manifests
uses: actions/download-artifact@v8
with:
path: ./manifests/
pattern: manifest-*
merge-multiple: true
- name: Collect current firmware sizes
run: python3 bin/collect_sizes.py ./manifests/ ./current-sizes.json
- name: Enforce RAM/flash budgets
run: python3 bin/size_report.py ./current-sizes.json --budgets bin/ram_budgets.json --enforce-budgets
release-artifacts:
permissions: # Needed for 'gh release upload'.
contents: write
bin/collect_sizes.py
+16 -3
View File
@@ -1,6 +1,15 @@
#!/usr/bin/env python3
"""Collect firmware binary sizes from manifest (.mt.json) files into a single report."""
"""Collect firmware binary sizes from manifest (.mt.json) files into a single report.
Output schema (consumed by bin/size_report.py):
{"<env>": {"flash_bytes": <int>, "ram_bytes": <int>}}
flash_bytes is the size of the main firmware image (.bin). ram_bytes is the
static RAM footprint (.data + .bss) emitted into the manifest by
bin/platformio-custom.py; it is omitted for manifests that predate it, and
size_report.py renders those as "n/a".
"""
import json
import os
@@ -8,7 +17,7 @@ import sys
def collect_sizes(manifest_dir):
"""Scan manifest_dir for .mt.json files and return {board: size_bytes} dict."""
"""Scan manifest_dir for .mt.json files and return {board: sizes_dict}."""
sizes = {}
for fname in sorted(os.listdir(manifest_dir)):
if not fname.endswith(".mt.json"):
@@ -34,7 +43,11 @@ def collect_sizes(manifest_dir):
bin_size = entry["bytes"]
break
if bin_size is not None:
sizes[board] = bin_size
entry = {"flash_bytes": bin_size}
ram_bytes = data.get("ram_bytes")
if isinstance(ram_bytes, int) and not isinstance(ram_bytes, bool):
entry["ram_bytes"] = ram_bytes
sizes[board] = entry
return sizes
bin/platformio-custom.py
+49 -2
View File
@@ -45,6 +45,49 @@ def infer_architecture(board_cfg):
return "stm32"
return None
def compute_ram_bytes(env):
"""Static RAM usage (.data + .bss) of the ELF, via the toolchain size tool.
Deliberately excludes heap/stack placeholder sections (e.g. the nRF52 .heap
section): on nRF52840 the heap arena is the linker gap after .bss, so static
RAM growth shrinks the usable heap 1:1 - which is exactly why we track it.
Returns None when the value cannot be determined; the manifest then simply
omits ram_bytes and downstream size reports show "n/a".
"""
elf = env.File(env.subst("$BUILD_DIR/${PROGNAME}.elf"))
if not elf.exists():
return None
size_tool = env.subst("$SIZETOOL") or "size"
try:
output = subprocess.check_output(
[size_tool, "-A", elf.get_abspath()],
env=env["ENV"],
universal_newlines=True,
)
except Exception as exc:
print(f"mtjson: skipping ram_bytes ({size_tool} failed: {exc})")
return None
ram = 0
found = False
for line in output.splitlines():
parts = line.split()
if len(parts) < 2:
continue
name = parts[0]
# Main-SRAM static sections: .data/.bss, platform-prefixed variants (e.g.
# ESP32 .dram0.data/.dram0.bss), and RISC-V small-data .sdata/.sbss.
# ESP-IDF .rtc.* sections live outside the heap-competing SRAM; .heap and
# .tdata never match.
if name.startswith(".rtc"):
continue
if name in (".data", ".bss", ".sdata", ".sbss") or name.endswith(".data") or name.endswith(".bss"):
try:
ram += int(parts[1])
found = True
except ValueError:
continue
return ram if found else None
def manifest_gather(source, target, env):
global manifest_ran
if manifest_ran:
@@ -98,9 +141,9 @@ def manifest_gather(source, target, env):
d["part_name"] = partition_map[p]
out.append(d)
print(d)
manifest_write(out, env)
manifest_write(out, env, compute_ram_bytes(env))
def manifest_write(files, env):
def manifest_write(files, env, ram_bytes=None):
# Defensive: also skip manifest writing if we cannot determine architecture
def get_project_option(name):
try:
@@ -137,6 +180,10 @@ def manifest_write(files, env):
"has_mui": False,
"has_inkhud": False,
}
# Static RAM footprint (.data + .bss); consumed by bin/collect_sizes.py for
# the CI size report and the bin/ram_budgets.json budget gate.
if ram_bytes is not None:
manifest["ram_bytes"] = ram_bytes
# Get partition table (generated in esp32_pre.py) if it exists
if env.get("custom_mtjson_part"):
# custom_mtjson_part is a JSON string, convert it back to a dict
bin/ram_budgets.json
+24
View File
@@ -0,0 +1,24 @@
{
"_comment": [
"Per-environment static-size budgets, enforced by the size-budget-gate CI job",
"via: bin/size_report.py <sizes.json> --budgets bin/ram_budgets.json --enforce-budgets",
"Only environments listed here are gated, and only for the metrics they list.",
"",
"ram_bytes = static RAM (.data + .bss from the ELF, emitted into the .mt.json",
"manifest by bin/platformio-custom.py). On nRF52840 the heap arena is the linker",
"gap after .bss, so every byte of static RAM growth shrinks the usable heap 1:1;",
"that is how the 2.8.0 heap regression shipped without CI noticing.",
"",
"flash_bytes = size of the main firmware image (.bin). The rak4631 app region is",
"0x27000..0xEA000 = 798,720 bytes, and the image must also stay clear of the",
"warm-store record-ring guard (extra_scripts/nrf52_warm_region.py).",
"",
"Budgets are raised DELIBERATELY, never automatically: if your change needs more",
"headroom, bump the limit here in the same PR and justify the increase in the PR",
"description."
],
"rak4631": {
"ram_bytes": 113000,
"flash_bytes": 786000
}
}
bin/size_report.py
+257 -45
View File
@@ -4,6 +4,7 @@
Usage:
size_report.py <new_sizes.json> [--baseline <label>:<old_sizes.json>]...
[--budgets <budgets.json>] [--enforce-budgets]
Examples:
# Compare PR against develop and master baselines
@@ -12,18 +13,65 @@ Examples:
# Single baseline comparison
size_report.py pr.json --baseline develop:develop.json
# No baselines — shows sizes with blank delta columns
# No baselines - shows sizes with blank delta columns
size_report.py pr.json
# Render budget usage in the report (informational)
size_report.py pr.json --budgets bin/ram_budgets.json
# Enforce budgets: exit 1 when any env in the budgets file exceeds a limit
size_report.py pr.json --budgets bin/ram_budgets.json --enforce-budgets
Sizes JSON schema (produced by bin/collect_sizes.py):
{"<env>": {"flash_bytes": <int>, "ram_bytes": <int>}}
The legacy schema {"<env>": <flash_bytes>} (older baseline artifacts) is still
accepted; missing ram_bytes renders as "n/a".
Budgets JSON schema (bin/ram_budgets.json):
{"<env>": {"ram_bytes": <budget>, "flash_bytes": <budget>}}
Keys starting with "_" are comments. Only envs present in the budgets file are
gated, and only for the metrics they list. ram_bytes is the static RAM
footprint (.data + .bss): on nRF52840 the heap arena is just the linker gap
after .bss, so every byte of static growth shrinks the usable heap 1:1 - which
is how the 2.8.0 heap regression shipped without CI noticing. Budgets are
raised deliberately (edit the JSON in the PR that needs the headroom and
justify it), never automatically.
"""
import argparse
import json
import sys
RAM_LABEL = "RAM (.data+.bss)"
FLASH_LABEL = "flash"
def load_sizes(path):
with open(path) as f:
return json.load(f)
return normalize_sizes(json.load(f))
def normalize_sizes(raw):
"""Normalize a sizes dict to {board: {"flash_bytes": int, "ram_bytes": int}}.
Accepts both the current schema (dict values) and the legacy schema where
each value was the flash size as a plain int. Unknown/malformed entries are
dropped rather than crashing on old artifacts.
"""
sizes = {}
for board, val in raw.items():
if isinstance(val, bool):
continue
if isinstance(val, int):
sizes[board] = {"flash_bytes": val}
elif isinstance(val, dict):
entry = {}
for key in ("flash_bytes", "ram_bytes"):
if isinstance(val.get(key), int) and not isinstance(val[key], bool):
entry[key] = val[key]
if entry:
sizes[board] = entry
return sizes
def format_delta(n):
@@ -34,44 +82,46 @@ def format_delta(n):
return f"{sign}{n:,}"
def compute_deltas(entry, baselines, key):
"""Per-baseline delta of entry[key], or None when either side is missing."""
deltas = []
current = entry.get(key)
for _, old_sizes in baselines:
old = (
old_sizes.get(entry["_board"], {}).get(key) if current is not None else None
)
deltas.append(current - old if old is not None else None)
return deltas
def generate_markdown(new_sizes, baselines, top_n=5):
"""Generate a single table with current size and delta columns per baseline.
"""Generate a single table with flash/RAM columns and deltas per baseline.
baselines: list of (label, old_sizes_dict), may be empty
"""
labels = [label for label, _ in baselines]
# Build rows: (board, current_size, [(delta, abs_delta) per baseline])
# Build rows: (board, entry, flash_deltas, ram_deltas, max_abs_delta)
rows = []
for board in sorted(new_sizes):
current = new_sizes[board]
deltas = []
for _, old_sizes in baselines:
old = old_sizes.get(board)
if old is not None:
d = current - old
deltas.append((d, abs(d)))
else:
deltas.append((None, 0))
# Sort key: max abs delta across baselines (biggest changes first)
max_abs = max((ad for _, ad in deltas), default=0)
rows.append((board, current, deltas, max_abs))
entry = dict(new_sizes[board])
entry["_board"] = board
flash_deltas = compute_deltas(entry, baselines, "flash_bytes")
ram_deltas = compute_deltas(entry, baselines, "ram_bytes")
max_abs = max(
(abs(d) for d in flash_deltas + ram_deltas if d is not None), default=0
)
rows.append((board, entry, flash_deltas, ram_deltas, max_abs))
rows.sort(key=lambda r: r[3], reverse=True)
rows.sort(key=lambda r: r[4], reverse=True)
# Summary line
# Summary line (flash-based, matching the historical summary)
sections = []
summary_parts = [f"{len(new_sizes)} targets"]
for i, (label, old_sizes) in enumerate(baselines):
increases = sum(
1 for _, _, deltas, _ in rows if deltas[i][0] is not None and deltas[i][0] > 0
)
decreases = sum(
1 for _, _, deltas, _ in rows if deltas[i][0] is not None and deltas[i][0] < 0
)
net = sum(
deltas[i][0] for _, _, deltas, _ in rows if deltas[i][0] is not None
)
for i, (label, _) in enumerate(baselines):
increases = sum(1 for _, _, fd, _, _ in rows if fd[i] is not None and fd[i] > 0)
decreases = sum(1 for _, _, fd, _, _ in rows if fd[i] is not None and fd[i] < 0)
net = sum(fd[i] for _, _, fd, _, _ in rows if fd[i] is not None)
parts = []
if increases:
parts.append(f"{increases} increased")
@@ -89,30 +139,47 @@ def generate_markdown(new_sizes, baselines, top_n=5):
sections.append(f"**{' | '.join(summary_parts)}**\n")
# Table header
header = "| Target | Size |"
separator = "|--------|-----:|"
header = "| Target | Flash |"
separator = "|--------|------:|"
for label in labels:
header += f" vs `{label}` |"
separator += "----------:|"
header += " RAM |"
separator += "----:|"
for label in labels:
header += f" RAM vs `{label}` |"
separator += "----------:|"
sections.append(header)
sections.append(separator)
def format_row(board, current, deltas):
row = f"| `{board}` | {current:,} |"
for d, _ in deltas:
if d is None:
row += " |"
elif d == 0:
row += " 0 |"
else:
icon = "📈" if d > 0 else "📉"
row += f" {icon} {format_delta(d)} |"
def format_delta_cell(d, missing=""):
if d is None:
return f" {missing} |".replace(" ", " ")
if d == 0:
return " 0 |"
icon = "📈" if d > 0 else "📉"
return f" {icon} {format_delta(d)} |"
def format_row(board, entry, flash_deltas, ram_deltas):
flash = entry.get("flash_bytes")
ram = entry.get("ram_bytes")
row = (
f"| `{board}` | {flash:,} |"
if flash is not None
else f"| `{board}` | n/a |"
)
for d in flash_deltas:
row += format_delta_cell(d)
row += f" {ram:,} |" if ram is not None else " n/a |"
for d in ram_deltas:
# RAM data may be missing on one side (older artifacts): show n/a
row += format_delta_cell(d, missing="n/a")
return row
# Top N rows always visible
top = rows[:top_n]
for board, current, deltas, _ in top:
sections.append(format_row(board, current, deltas))
for board, entry, flash_deltas, ram_deltas, _ in top:
sections.append(format_row(board, entry, flash_deltas, ram_deltas))
# Remaining rows in expandable section
rest = rows[top_n:]
@@ -123,14 +190,112 @@ def generate_markdown(new_sizes, baselines, top_n=5):
)
sections.append(header)
sections.append(separator)
for board, current, deltas, _ in rest:
sections.append(format_row(board, current, deltas))
for board, entry, flash_deltas, ram_deltas, _ in rest:
sections.append(format_row(board, entry, flash_deltas, ram_deltas))
sections.append("\n</details>")
sections.append("")
return "\n".join(sections)
def load_budgets(path):
"""Load bin/ram_budgets.json, skipping "_comment"-style keys.
Fails loudly on malformed entries: a typo'd budget (zero, negative, or
non-integer) must not silently weaken or crash the gate.
"""
with open(path) as f:
raw = json.load(f)
budgets = {
env: limits
for env, limits in raw.items()
if not env.startswith("_") and isinstance(limits, dict)
}
for env, limits in budgets.items():
for key in ("ram_bytes", "flash_bytes"):
if key not in limits:
continue
budget = limits[key]
if not isinstance(budget, int) or isinstance(budget, bool) or budget <= 0:
print(
f"Error: invalid budget in {path}: {env}.{key} = {budget!r} "
"(must be a positive integer)",
file=sys.stderr,
)
sys.exit(1)
return budgets
def check_budgets(new_sizes, budgets, fail_on_missing=False):
"""Compare measured sizes against per-env budgets.
With fail_on_missing=False a budgeted env (or metric) absent from the
measured sizes is reported as "n/a" but does not fail - right for the
informational report. The enforcing gate passes fail_on_missing=True so
missing data fails closed instead of trivially passing.
Returns (violations, rows): violations is a list of human-readable failure
strings; rows is a list of (env, metric_label, measured, budget, over)
tuples for reporting, with measured=None when the data is unavailable.
"""
violations = []
rows = []
for env in sorted(budgets):
measured = new_sizes.get(env)
for key, label in (("ram_bytes", RAM_LABEL), ("flash_bytes", FLASH_LABEL)):
budget = budgets[env].get(key)
if not isinstance(budget, int) or isinstance(budget, bool):
continue
value = measured.get(key) if measured is not None else None
if value is None:
rows.append((env, label, None, budget, False))
if fail_on_missing:
reason = (
"env was not built in this run"
if measured is None
else "manifest is missing this metric"
)
violations.append(
f"{env} {label}: no measurement to check against the "
f"budget ({reason}); refusing to pass the gate blind"
)
continue
over = value > budget
rows.append((env, label, value, budget, over))
if over:
violations.append(
f"{env} {label}: {value:,} bytes exceeds the budget of "
f"{budget:,} bytes (over by {value - budget:,})"
)
return violations, rows
def budget_markdown(rows):
"""Render budget usage as a markdown section for the PR comment."""
if not rows:
return ""
lines = [
"### Size budgets",
"",
"| Env | Metric | Measured | Budget | Used |",
"|-----|--------|---------:|-------:|-----:|",
]
for env, label, value, budget, over in rows:
if value is None:
lines.append(f"| `{env}` | {label} | n/a | {budget:,} | n/a |")
continue
# load_budgets() rejects non-positive budgets; guard anyway for direct callers
pct = 100.0 * value / budget if budget > 0 else float("inf")
status = f"{pct:.1f}%" + (" ❌ **OVER BUDGET**" if over else "")
lines.append(f"| `{env}` | {label} | {value:,} | {budget:,} | {status} |")
lines.append("")
lines.append(
"*Budgets live in `bin/ram_budgets.json` and are raised deliberately in "
"the PR that needs the headroom - never automatically.*"
)
lines.append("")
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(description="Compare firmware size reports")
parser.add_argument("new_sizes", help="Path to new sizes JSON")
@@ -147,12 +312,34 @@ def main():
default=5,
help="Number of top changes to show before collapsing (default: 5)",
)
parser.add_argument(
"--budgets",
metavar="PATH",
help="Path to per-env budgets JSON (e.g. bin/ram_budgets.json)",
)
parser.add_argument(
"--enforce-budgets",
action="store_true",
help="Exit 1 when any env exceeds its budget (requires --budgets)",
)
args = parser.parse_args()
if args.enforce_budgets and not args.budgets:
print("Error: --enforce-budgets requires --budgets", file=sys.stderr)
sys.exit(1)
new_sizes = load_sizes(args.new_sizes)
# Silence output when no targets were built — repo maintainer choice
# Silence output when no targets were built - repo maintainer choice.
# Under enforcement that would be a silent pass, so fail closed instead.
if not new_sizes:
if args.enforce_budgets:
print(
f"Error: --enforce-budgets requested but no sizes were found in "
f"{args.new_sizes}; refusing to pass the budget gate blind",
file=sys.stderr,
)
sys.exit(1)
return
baselines = []
@@ -164,8 +351,33 @@ def main():
baselines.append((label, load_sizes(path)))
md = generate_markdown(new_sizes, baselines, top_n=args.top)
violations = []
if args.budgets:
budgets = load_budgets(args.budgets)
violations, rows = check_budgets(
new_sizes, budgets, fail_on_missing=args.enforce_budgets
)
budget_md = budget_markdown(rows)
if budget_md:
md += "\n" + budget_md
print(md)
if violations and args.enforce_budgets:
print("\nRAM/flash budget check FAILED:", file=sys.stderr)
for v in violations:
print(f" - {v}", file=sys.stderr)
print(
"\nOn nRF52840 the heap arena is the linker gap after .bss, so every\n"
"byte of static RAM growth shrinks the usable heap 1:1. Budgets are\n"
"raised deliberately, never automatically: if this growth is intended\n"
"and reviewed, raise the env's limit in bin/ram_budgets.json in this\n"
"same PR and justify the increase in the PR description.",
file=sys.stderr,
)
sys.exit(1)
if __name__ == "__main__":
main()
bin/test_size_scripts.py
+376 -37
View File
@@ -11,16 +11,19 @@ import tempfile
SCRIPTS_DIR = os.path.join(os.path.dirname(__file__), "..", "bin")
def make_manifest(target, firmware_bytes, extra_files=None):
def make_manifest(target, firmware_bytes, extra_files=None, ram_bytes=None):
"""Create a minimal .mt.json manifest dict."""
files = [{"name": f"firmware-{target}-2.6.0.bin", "bytes": firmware_bytes}]
if extra_files:
files.extend(extra_files)
return {
manifest = {
"platformioTarget": target,
"version": "2.6.0.test",
"files": files,
}
if ram_bytes is not None:
manifest["ram_bytes"] = ram_bytes
return manifest
def write_manifests(tmpdir, manifests):
@@ -31,6 +34,13 @@ def write_manifests(tmpdir, manifests):
json.dump(data, f)
def write_json(tmpdir, name, data):
path = os.path.join(tmpdir, name)
with open(path, "w") as f:
json.dump(data, f)
return path
def run_script(script, args):
"""Run a Python script and return (returncode, stdout, stderr)."""
result = subprocess.run(
@@ -58,7 +68,45 @@ def test_collect_sizes_basic():
with open(outfile) as f:
sizes = json.load(f)
assert sizes == {"heltec-v3": 1048576, "rak4631": 524288, "tbeam": 786432}
assert sizes == {
"heltec-v3": {"flash_bytes": 1048576},
"rak4631": {"flash_bytes": 524288},
"tbeam": {"flash_bytes": 786432},
}
def test_collect_sizes_ram_bytes():
"""collect_sizes carries ram_bytes through from the manifest."""
with tempfile.TemporaryDirectory() as tmpdir:
outfile = os.path.join(tmpdir, "sizes.json")
manifests = {
"rak4631": make_manifest("rak4631", 765192, ram_bytes=110948),
"heltec-v3": make_manifest("heltec-v3", 1048576), # no ram_bytes
}
write_manifests(tmpdir, manifests)
rc, stdout, stderr = run_script("collect_sizes.py", [tmpdir, outfile])
assert rc == 0, f"collect_sizes failed: {stderr}"
with open(outfile) as f:
sizes = json.load(f)
assert sizes["rak4631"] == {"flash_bytes": 765192, "ram_bytes": 110948}
# Manifest without ram_bytes: key absent, not zero / not crashing
assert sizes["heltec-v3"] == {"flash_bytes": 1048576}
def test_collect_sizes_non_int_ram_bytes_ignored():
"""collect_sizes drops malformed (non-integer) ram_bytes values."""
with tempfile.TemporaryDirectory() as tmpdir:
outfile = os.path.join(tmpdir, "sizes.json")
manifests = {"rak4631": make_manifest("rak4631", 765192, ram_bytes="110948")}
write_manifests(tmpdir, manifests)
rc, stdout, stderr = run_script("collect_sizes.py", [tmpdir, outfile])
assert rc == 0, f"collect_sizes failed: {stderr}"
with open(outfile) as f:
sizes = json.load(f)
assert sizes == {"rak4631": {"flash_bytes": 765192}}
def test_collect_sizes_fallback_bin():
@@ -82,7 +130,7 @@ def test_collect_sizes_fallback_bin():
with open(outfile) as f:
sizes = json.load(f)
assert sizes == {"custom-board": 500000}
assert sizes == {"custom-board": {"flash_bytes": 500000}}
def test_collect_sizes_skips_ota_littlefs():
@@ -128,11 +176,11 @@ def test_collect_sizes_ignores_non_mt_json():
def test_size_report_no_baseline():
"""size_report with no baselines shows sizes only."""
"""size_report with no baselines shows sizes only (legacy int schema)."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = os.path.join(tmpdir, "new.json")
with open(sizes_file, "w") as f:
json.dump({"heltec-v3": 1000000, "rak4631": 500000}, f)
sizes_file = write_json(
tmpdir, "new.json", {"heltec-v3": 1000000, "rak4631": 500000}
)
rc, stdout, stderr = run_script("size_report.py", [sizes_file])
assert rc == 0, f"size_report failed: {stderr}"
@@ -140,17 +188,23 @@ def test_size_report_no_baseline():
assert "no baseline available yet" in stdout
assert "`heltec-v3`" in stdout
assert "`rak4631`" in stdout
# Legacy schema has no RAM data: column renders n/a, never crashes
assert "n/a" in stdout
def test_size_report_with_baseline():
"""size_report shows deltas against a baseline."""
"""size_report shows deltas against a baseline (legacy int schema)."""
with tempfile.TemporaryDirectory() as tmpdir:
new_file = os.path.join(tmpdir, "new.json")
old_file = os.path.join(tmpdir, "old.json")
with open(new_file, "w") as f:
json.dump({"heltec-v3": 1050000, "rak4631": 500000, "tbeam": 800000}, f)
with open(old_file, "w") as f:
json.dump({"heltec-v3": 1000000, "rak4631": 500000, "tbeam": 810000}, f)
new_file = write_json(
tmpdir,
"new.json",
{"heltec-v3": 1050000, "rak4631": 500000, "tbeam": 800000},
)
old_file = write_json(
tmpdir,
"old.json",
{"heltec-v3": 1000000, "rak4631": 500000, "tbeam": 810000},
)
rc, stdout, stderr = run_script(
"size_report.py", [new_file, "--baseline", f"develop:{old_file}"]
@@ -167,22 +221,84 @@ def test_size_report_with_baseline():
assert "vs `develop`" in stdout
def test_size_report_ram_columns():
"""size_report renders RAM and RAM-delta columns from the new schema."""
with tempfile.TemporaryDirectory() as tmpdir:
new_file = write_json(
tmpdir,
"new.json",
{"rak4631": {"flash_bytes": 765192, "ram_bytes": 112000}},
)
old_file = write_json(
tmpdir,
"old.json",
{"rak4631": {"flash_bytes": 765192, "ram_bytes": 110948}},
)
rc, stdout, stderr = run_script(
"size_report.py", [new_file, "--baseline", f"develop:{old_file}"]
)
assert rc == 0, f"size_report failed: {stderr}"
assert "RAM vs `develop`" in stdout
assert "112,000" in stdout
# RAM grew by 1052 bytes
assert "+1,052" in stdout
def test_size_report_ram_na_for_legacy_baseline():
"""RAM delta shows n/a when the baseline predates ram_bytes (legacy ints)."""
with tempfile.TemporaryDirectory() as tmpdir:
new_file = write_json(
tmpdir,
"new.json",
{"rak4631": {"flash_bytes": 765192, "ram_bytes": 110948}},
)
old_file = write_json(tmpdir, "old.json", {"rak4631": 760000})
rc, stdout, stderr = run_script(
"size_report.py", [new_file, "--baseline", f"develop:{old_file}"]
)
assert rc == 0, f"size_report failed: {stderr}"
# Current RAM shown, delta degrades to n/a
assert "110,948" in stdout
assert "n/a" in stdout
# Flash delta still computed
assert "📈" in stdout
def test_size_report_ram_na_for_current_without_ram():
"""RAM column shows n/a when the current manifest lacks ram_bytes."""
with tempfile.TemporaryDirectory() as tmpdir:
new_file = write_json(tmpdir, "new.json", {"board-a": {"flash_bytes": 100000}})
old_file = write_json(
tmpdir,
"old.json",
{"board-a": {"flash_bytes": 100000, "ram_bytes": 50000}},
)
rc, stdout, stderr = run_script(
"size_report.py", [new_file, "--baseline", f"develop:{old_file}"]
)
assert rc == 0, f"size_report failed: {stderr}"
assert "n/a" in stdout
def test_size_report_multiple_baselines():
"""size_report handles multiple baselines."""
with tempfile.TemporaryDirectory() as tmpdir:
new_file = os.path.join(tmpdir, "new.json")
dev_file = os.path.join(tmpdir, "develop.json")
master_file = os.path.join(tmpdir, "master.json")
with open(new_file, "w") as f:
json.dump({"board-a": 100000}, f)
with open(dev_file, "w") as f:
json.dump({"board-a": 95000}, f)
with open(master_file, "w") as f:
json.dump({"board-a": 90000}, f)
new_file = write_json(tmpdir, "new.json", {"board-a": 100000})
dev_file = write_json(tmpdir, "develop.json", {"board-a": 95000})
master_file = write_json(tmpdir, "master.json", {"board-a": 90000})
rc, stdout, stderr = run_script(
"size_report.py",
[new_file, "--baseline", f"develop:{dev_file}", "--baseline", f"master:{master_file}"],
[
new_file,
"--baseline",
f"develop:{dev_file}",
"--baseline",
f"master:{master_file}",
],
)
assert rc == 0, f"size_report failed: {stderr}"
assert "vs `develop`" in stdout
@@ -192,12 +308,10 @@ def test_size_report_multiple_baselines():
def test_size_report_new_target_no_baseline_entry():
"""size_report handles targets not present in baseline (new boards)."""
with tempfile.TemporaryDirectory() as tmpdir:
new_file = os.path.join(tmpdir, "new.json")
old_file = os.path.join(tmpdir, "old.json")
with open(new_file, "w") as f:
json.dump({"new-board": 300000, "existing": 500000}, f)
with open(old_file, "w") as f:
json.dump({"existing": 500000}, f)
new_file = write_json(
tmpdir, "new.json", {"new-board": 300000, "existing": 500000}
)
old_file = write_json(tmpdir, "old.json", {"existing": 500000})
rc, stdout, stderr = run_script(
"size_report.py", [new_file, "--baseline", f"develop:{old_file}"]
@@ -210,9 +324,9 @@ def test_size_report_new_target_no_baseline_entry():
def test_size_report_all_unchanged():
"""size_report shows 'no changes' when all sizes match."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = os.path.join(tmpdir, "sizes.json")
with open(sizes_file, "w") as f:
json.dump({"board-a": 100000, "board-b": 200000}, f)
sizes_file = write_json(
tmpdir, "sizes.json", {"board-a": 100000, "board-b": 200000}
)
rc, stdout, stderr = run_script(
"size_report.py", [sizes_file, "--baseline", f"develop:{sizes_file}"]
@@ -221,6 +335,233 @@ def test_size_report_all_unchanged():
assert "no changes" in stdout
def test_budget_gate_under_budget():
"""Budget gate passes and reports usage when all envs are within budget."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(
tmpdir,
"sizes.json",
{"rak4631": {"flash_bytes": 765192, "ram_bytes": 110948}},
)
budgets_file = write_json(
tmpdir,
"budgets.json",
{
"_comment": "budgets are raised deliberately",
"rak4631": {"ram_bytes": 113000, "flash_bytes": 786000},
},
)
rc, stdout, stderr = run_script(
"size_report.py",
[sizes_file, "--budgets", budgets_file, "--enforce-budgets"],
)
assert rc == 0, f"expected pass, got rc={rc}: {stderr}"
assert "Size budgets" in stdout
assert "113,000" in stdout
assert "OVER BUDGET" not in stdout
def test_budget_gate_over_ram_budget():
"""Budget gate fails with a clear message when RAM exceeds the budget."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(
tmpdir,
"sizes.json",
{"rak4631": {"flash_bytes": 765192, "ram_bytes": 118000}},
)
budgets_file = write_json(
tmpdir,
"budgets.json",
{"rak4631": {"ram_bytes": 113000, "flash_bytes": 786000}},
)
rc, stdout, stderr = run_script(
"size_report.py",
[sizes_file, "--budgets", budgets_file, "--enforce-budgets"],
)
assert rc == 1, f"expected failure, got rc={rc}"
# Message names the env, measured value, budget, and how to raise it
assert "rak4631" in stderr
assert "118,000" in stderr
assert "113,000" in stderr
assert "ram_budgets.json" in stderr
assert "OVER BUDGET" in stdout
def test_budget_gate_over_flash_budget():
"""Budget gate fails when flash exceeds the budget."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(
tmpdir,
"sizes.json",
{"rak4631": {"flash_bytes": 790000, "ram_bytes": 110948}},
)
budgets_file = write_json(
tmpdir,
"budgets.json",
{"rak4631": {"ram_bytes": 113000, "flash_bytes": 786000}},
)
rc, stdout, stderr = run_script(
"size_report.py",
[sizes_file, "--budgets", budgets_file, "--enforce-budgets"],
)
assert rc == 1, f"expected failure, got rc={rc}"
assert "rak4631" in stderr
assert "790,000" in stderr
assert "786,000" in stderr
def test_budget_gate_env_not_built_fails_closed():
"""Under --enforce-budgets, a budgeted env missing from the sizes fails."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(
tmpdir, "sizes.json", {"heltec-v3": {"flash_bytes": 1000000}}
)
budgets_file = write_json(
tmpdir,
"budgets.json",
{"rak4631": {"ram_bytes": 113000, "flash_bytes": 786000}},
)
rc, stdout, stderr = run_script(
"size_report.py",
[sizes_file, "--budgets", budgets_file, "--enforce-budgets"],
)
assert rc == 1, f"expected fail-closed for unbuilt env, got rc={rc}"
assert "not built" in stderr
def test_budget_report_env_not_built_shows_na():
"""Without enforcement, a budgeted env missing from the sizes is just n/a."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(
tmpdir, "sizes.json", {"heltec-v3": {"flash_bytes": 1000000}}
)
budgets_file = write_json(
tmpdir,
"budgets.json",
{"rak4631": {"ram_bytes": 113000, "flash_bytes": 786000}},
)
rc, stdout, stderr = run_script(
"size_report.py", [sizes_file, "--budgets", budgets_file]
)
assert rc == 0, f"expected report-only pass, got rc={rc}: {stderr}"
assert "n/a" in stdout
def test_budget_gate_missing_ram_metric_fails_closed():
"""Under --enforce-budgets, a budgeted env without ram_bytes fails."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(
tmpdir, "sizes.json", {"rak4631": {"flash_bytes": 765192}}
)
budgets_file = write_json(
tmpdir,
"budgets.json",
{"rak4631": {"ram_bytes": 113000, "flash_bytes": 786000}},
)
rc, stdout, stderr = run_script(
"size_report.py",
[sizes_file, "--budgets", budgets_file, "--enforce-budgets"],
)
assert rc == 1, f"expected fail-closed for missing metric, got rc={rc}"
assert "missing this metric" in stderr
def test_budget_report_missing_ram_metric_shows_na():
"""Without enforcement, a missing metric reports n/a and does not fail."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(
tmpdir, "sizes.json", {"rak4631": {"flash_bytes": 765192}}
)
budgets_file = write_json(
tmpdir,
"budgets.json",
{"rak4631": {"ram_bytes": 113000, "flash_bytes": 786000}},
)
rc, stdout, stderr = run_script(
"size_report.py", [sizes_file, "--budgets", budgets_file]
)
assert rc == 0, f"expected report-only pass, got rc={rc}: {stderr}"
assert "n/a" in stdout
def test_budget_gate_empty_sizes_fails_closed():
"""--enforce-budgets with an empty sizes file must fail, not silently pass."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(tmpdir, "sizes.json", {})
budgets_file = write_json(
tmpdir, "budgets.json", {"rak4631": {"ram_bytes": 113000}}
)
rc, stdout, stderr = run_script(
"size_report.py",
[sizes_file, "--budgets", budgets_file, "--enforce-budgets"],
)
assert rc == 1, f"expected fail-closed for empty sizes, got rc={rc}"
assert "no sizes" in stderr
# Report-only mode keeps the quiet no-op behavior
rc, stdout, stderr = run_script("size_report.py", [sizes_file])
assert rc == 0
def test_budget_invalid_budget_rejected():
"""Zero/negative/non-int budgets fail loudly instead of crashing or passing."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(
tmpdir, "sizes.json", {"rak4631": {"ram_bytes": 110948}}
)
for bad in (0, -5, "113000", True):
budgets_file = write_json(
tmpdir, "budgets.json", {"rak4631": {"ram_bytes": bad}}
)
rc, stdout, stderr = run_script(
"size_report.py",
[sizes_file, "--budgets", budgets_file, "--enforce-budgets"],
)
assert rc == 1, f"expected rejection of budget {bad!r}, got rc={rc}"
assert "positive integer" in stderr
def test_budget_render_without_enforce():
"""--budgets alone renders the table but never fails, even over budget."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(
tmpdir,
"sizes.json",
{"rak4631": {"flash_bytes": 765192, "ram_bytes": 999999}},
)
budgets_file = write_json(
tmpdir,
"budgets.json",
{"rak4631": {"ram_bytes": 113000, "flash_bytes": 786000}},
)
rc, stdout, stderr = run_script(
"size_report.py", [sizes_file, "--budgets", budgets_file]
)
assert rc == 0, f"render-only budgets must not fail: {stderr}"
assert "OVER BUDGET" in stdout
def test_budget_enforce_requires_budgets():
"""--enforce-budgets without --budgets is an argument error."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = write_json(tmpdir, "sizes.json", {"x": 1})
rc, stdout, stderr = run_script(
"size_report.py", [sizes_file, "--enforce-budgets"]
)
assert rc == 1
assert "--budgets" in stderr
def test_collect_sizes_bad_args():
"""collect_sizes exits with error on wrong arg count."""
rc, stdout, stderr = run_script("collect_sizes.py", [])
@@ -231,9 +572,7 @@ def test_collect_sizes_bad_args():
def test_size_report_bad_baseline_format():
"""size_report exits with error on malformed --baseline."""
with tempfile.TemporaryDirectory() as tmpdir:
sizes_file = os.path.join(tmpdir, "sizes.json")
with open(sizes_file, "w") as f:
json.dump({"x": 1}, f)
sizes_file = write_json(tmpdir, "sizes.json", {"x": 1})
rc, stdout, stderr = run_script(
"size_report.py", [sizes_file, "--baseline", "no-colon-here"]
src/main.cpp
+18
View File
@@ -42,6 +42,9 @@
#include "mesh/generated/meshtastic/config.pb.h"
#include "meshUtils.h"
#include "modules/Modules.h"
#ifdef MESHTASTIC_HEAP_WATERMARK_CHECK
#include "memGet.h"
#endif
#include "sleep.h"
#include "target_specific.h"
#include <memory>
@@ -1157,6 +1160,21 @@ void setup()
// We manually run this to update the NodeStatus
nodeDB->notifyObservers(true);
#ifdef MESHTASTIC_HEAP_WATERMARK_CHECK
// Opt-in CI guardrail: on nRF52840 static RAM growth eats the heap arena 1:1,
// so flag loudly when less than 20% of the heap is free at the end of setup().
{
uint32_t heapTotal = memGet.getHeapSize();
// Platforms without heap accounting report UINT32_MAX (or 0); skip those
if (heapTotal != 0 && heapTotal != UINT32_MAX) {
uint32_t heapFree = memGet.getFreeHeap();
if (heapFree < heapTotal / 5) {
LOG_ERROR("Boot heap watermark: only %u of %u bytes free (<20%%)", heapFree, heapTotal);
}
}
}
#endif
}
#endif