Add USB camera and uhubctl support for new test suite. Also included some bug fixes (#10204)
* Add USB camera and uhubctl support for new test suite. Also added some bug fixes * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Refactor test messages for clarity and consistency in regex tests --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Ben Meadors
co-authored by
GitHub
Copilot Autofix powered by AI
GitHub
Copilot Autofix powered by AI
parent
6b15571e14
commit
de23e5199d
@@ -26,7 +26,12 @@ This prompt assumes the meshtastic MCP server is registered with your VS Code Co
|
||||
- `get_config(section="lora", port=<p>)` → region, preset, channel_num, tx_power, hop_limit
|
||||
- If anything looks off (can't connect, `num_nodes` wrong, missing `firmware_version`), open a short firmware-log window: `serial_open(port=<p>, env=<inferred>)`, wait 3 seconds, `serial_read(session_id, max_lines=100)`, `serial_close(session_id)`. Infer env from VID (0x239a → `rak4631`, 0x303a/0x10c4 → `heltec-v3`) unless an `MESHTASTIC_MCP_ENV_<ROLE>` env var overrides it.
|
||||
|
||||
4. **Render per-device report** as a compact block:
|
||||
4. **Hub health** (call once, not per-device): `uhubctl_list()` — enumerates every USB hub the host sees. Cross-reference each Meshtastic device's VID to find which hub + port it's on. Flag in the report if:
|
||||
- No hub advertises `ppps=true` → `tests/recovery/` can't run; hard-recovery via `uhubctl_cycle` isn't available.
|
||||
- A Meshtastic device is on a non-PPPS hub → note it; moving to a PPPS hub unlocks auto-recovery.
|
||||
- `uhubctl_list` raises `ConfigError: uhubctl not found` → report as "uhubctl not installed"; don't treat as a device fault.
|
||||
|
||||
5. **Render per-device report** as a compact block:
|
||||
|
||||
```text
|
||||
[nrf52 @ /dev/cu.usbmodem1101] fw=2.7.23.bce2825, hw=RAK4631
|
||||
@@ -35,20 +40,22 @@ This prompt assumes the meshtastic MCP server is registered with your VS Code Co
|
||||
tx_power : 30 dBm, hop_limit=3
|
||||
peers : 1 (esp32s3 0x433c2428, pubkey ✓, SNR 6.0 / RSSI -24 dBm)
|
||||
primary ch : McpTest
|
||||
hub : 1-1.3 port 2 (PPPS, uhubctl-controllable)
|
||||
firmware : no panics in last 3s
|
||||
```
|
||||
|
||||
Flag abnormalities inline with `⚠︎ <short reason>` — missing pubkey on a known peer, region UNSET, mismatched channel name, etc.
|
||||
Flag abnormalities inline with `⚠︎ <short reason>` — missing pubkey on a known peer, region UNSET, mismatched channel name, device on non-PPPS hub, etc.
|
||||
|
||||
5. **Cross-device correlation** (when >1 device selected):
|
||||
6. **Cross-device correlation** (when >1 device selected):
|
||||
- Do both see each other in `nodesByNum`?
|
||||
- Do `region`, `channel_num`, `modem_preset` match across devices?
|
||||
- Do the primary channel names match? (Different name → different PSK → no decode.)
|
||||
|
||||
6. **Suggest next steps only for recognizable failure modes**, never speculatively:
|
||||
7. **Suggest next steps only for recognizable failure modes**, never speculatively:
|
||||
- Stale PKI one-way → "`/mcp-test tests/mesh/test_direct_with_ack.py` — the test's retry+nodeinfo-ping heals this."
|
||||
- Region mismatch → "re-bake one side via `./mcp-server/run-tests.sh --force-bake`."
|
||||
- Device unreachable → refer operator to the touch_1200bps + CP2102-wedged-driver notes in `run-tests.sh`.
|
||||
- Device unreachable, DFU reachable → `touch_1200bps(port=...)` + `pio_flash`. If not even DFU responds and the device is on a PPPS hub, escalate to `uhubctl_cycle(role=..., confirm=True)`.
|
||||
- CP2102-wedged-driver on macOS → see `run-tests.sh` notes.
|
||||
|
||||
## Hard constraints
|
||||
|
||||
|
||||
@@ -46,7 +46,8 @@ Equivalent of `.claude/commands/repro.md`. Use when the operator says "that one
|
||||
- **LoRa airtime collision** — pass rate improves with fewer concurrent transmitters. Suggest a `time.sleep` gap or retry bump in the test body.
|
||||
- **PKI key staleness** — first attempt fails, subsequent ones pass; existing retry-loop pattern in `test_direct_with_ack.py` is the fix.
|
||||
- **NodeInfo cooldown** — `Skip send NodeInfo since we sent it <600s ago` in fail-only logs; needs a `broadcast_nodeinfo_ping()` warmup.
|
||||
- **Hardware-specific** — one direction consistently fails, firmware versions differ, CP2102 driver wedged, etc.
|
||||
- **Hardware-specific** — one direction consistently fails, firmware versions differ, CP2102 driver wedged, etc. For a device wedged past `touch_1200bps`, recommend `uhubctl_cycle(role=..., confirm=True)` to hard-power-cycle its hub port (requires `uhubctl` installed).
|
||||
- **Device went dark mid-run** — fails from some iteration onward and never recovers; firmware log stops arriving. Almost always a Guru crash with frozen CDC. Recommend `uhubctl_cycle` before the next iteration; escalate to replug if that also fails.
|
||||
- **Unknown** — say so. Don't invent a root cause.
|
||||
|
||||
7. **Report back** with:
|
||||
|
||||
@@ -21,19 +21,25 @@ Equivalent of the Claude Code `/test` slash command in `.claude/commands/test.md
|
||||
|
||||
2. **Read the pre-flight header** (first few lines of wrapper output). The `detected hub :` line lists role → port → env mappings. If it reads `(none)`, the wrapper narrowed to `tests/unit` only — call that out explicitly so the operator knows hardware tiers were skipped.
|
||||
|
||||
3. **On pass**: one-line summary like `N passed, M skipped in <duration>`. Don't enumerate test names. DO mention any non-placeholder SKIPs (things like "role not present on hub") because they indicate missing hardware or setup issues.
|
||||
3. **On pass**: one-line summary like `N passed, M skipped in <duration>`. Don't enumerate test names. DO mention any non-placeholder SKIPs and name the cause:
|
||||
- `"role not present on hub"` → device unplugged; operator should reconnect.
|
||||
- `"firmware not baked with USERPREFS_UI_TEST_LOG"` → tests/ui skipped; the UI-log compile macro isn't in the baked firmware. Suggest `--force-bake`.
|
||||
- `"uhubctl not installed"` → tests/recovery + `test_peer_offline_recovery` skipped. Suggest `brew install uhubctl` / `apt install uhubctl`.
|
||||
- `"no PPPS-capable hubs detected"` → tests/recovery skipped because the attached hub doesn't support per-port power switching; won't run on that setup.
|
||||
- `"opencv-python-headless is not installed"` → tests/ui auto-deselected by `run-tests.sh`. Suggest `pip install -e 'mcp-server/.[ui]'`.
|
||||
|
||||
4. **On failure**: open `mcp-server/tests/report.html` (pytest-html output, self-contained) and extract the `Meshtastic debug` section for each failed test. That section includes a firmware log stream (last 200 lines) and device state dump. For each failure, summarise:
|
||||
- test name
|
||||
- one-line assertion message
|
||||
- the specific firmware log lines that explain why (look for `PKI_UNKNOWN_PUBKEY`, `Skip send NodeInfo`, `Error=`, `Guru Meditation`, `assertion failed`, `No suitable channel`)
|
||||
- for UI-tier failures also check `mcp-server/tests/ui_captures/<session>/<test>/transcript.md` (per-step frame + OCR)
|
||||
|
||||
5. **Classify each failure** as one of:
|
||||
- **Transient flake** — LoRa collision, first-attempt NAK with self-heal pattern, timing-sensitive assertion. Suggest `/mcp-repro <test-id>` to confirm.
|
||||
- **Environmental** — device unreachable, port busy, CP2102 driver wedged on macOS. Suggest specific recovery (USB replug, `touch_1200bps`, `git status userPrefs.jsonc`).
|
||||
- **Environmental** — device unreachable, port busy, CP2102 driver wedged on macOS. Suggest recovery in escalation order: (a) replug USB, (b) `touch_1200bps` + `pio_flash` for nRF52 DFU, (c) `uhubctl_cycle(role=..., confirm=True)` for a device wedged past DFU (needs `uhubctl` installed; `baked_single` does this once automatically when available). Also check `git status userPrefs.jsonc`.
|
||||
- **Regression** — same assertion fails repeatedly on re-runs, firmware log shows novel errors. Identify the firmware module likely responsible.
|
||||
|
||||
6. **Do NOT run destructive recovery automatically**. If a failure looks like it needs a reflash, factory*reset, or replug — \_describe the steps* and let the operator decide. Never burn airtime or flash cycles without approval.
|
||||
6. **Do NOT run destructive recovery automatically**. If a failure looks like it needs a reflash, factory*reset, `uhubctl_cycle`, or replug — \_describe the steps* and let the operator decide. Never burn airtime or flash cycles without approval.
|
||||
|
||||
## Arguments convention
|
||||
|
||||
|
||||
Reference in New Issue
Block a user