F1 dashboard validation and FP backtesting phase 2
Continue F1 predictor validation after MC-4824. Requested by Elmar: "Carry on with testing and validating and updating the dashboard and back testing." Scope: 1. Backfill FP t...
StateDoneNext ActionClosedOwnerLuciRuntimeClosedAge4d ago
Ticket is done; runtime is closed.·profile claude_opus_1m_medium
Description
MC-4841
Continue F1 predictor validation after MC-4824.
Requested by Elmar: "Carry on with testing and validating and updating the dashboard and back testing."
Scope:
1. Backfill FP telemetry one completed race at a time, respecting FastF1 limits. Start with Miami 2026, then Japan/China/Australia only after the previous race is validated.
2. Compare FP analysis against actual race outcomes and race telemetry where available:
- weighted FP pace vs actual race pace / finishing result
- ideal lap vs qualifying/race front group
- corner/load/aero/PU signals where FastF1 deep telemetry exists
- identify systematic misses and whether weights should change.
3. Improve dashboard/backtest UX if gaps are found:
- show validation/backtest results, not just raw FP telemetry charts
- make missing/degraded/race-mismatched data explicit
- ensure charts remain useful on OpenF1 fallback races.
4. Add regression tests for any dashboard/API/backtest changes, following TDD.
5. Commit and push source/docs/data changes that are intentionally part of the repo. Do not commit transient screenshots/caches unless they are canonical validation artifacts.
Acceptance criteria:
- At least Miami 2026 validation/backtest report produced.
- Focused dashboard/API/scoring tests pass.
- Any code/dashboard changes are committed and pushed.
- Elmar gets a clear recommendation: keep/change FP weights, and which race to validate next.
Expected check-in: after Miami validation pass today.
Activity
done
INTERACTIVE
Luci is working...
Details —
Done
· High
· Luci
▼
SState
Done
Closed
PPeople
TTiming / Details▼
api (human)
Mission Control
4d ago
4d ago
Advanced / Operator evidence
RRouting owner
ROperator console
Ticket is done; runtime is closed.MC is visibility-only. Hermes Luci launches and gates work outside MC, then mirrors evidence/status here.
WWorkflow
Start Dev Review + QA ▾
Select phases to include:
Agents
Review Gates
Decision
WAT routing: choose an agent, review gate, or decision. Buttons use the live runtime when one is attached.
[visibility-only] Queued message recorded for Hermes Luci. MC did not claim the ticket or spawn a runtime.
api4d ago
Progress update for MC-4841:
Implemented and pushed FP-vs-actual validation in the F1 dashboard/API.
Commits:
- 5c9d3a2 Add FP telemetry actual-result validation
- e8f0a13 docs: add FP validation phase 2 report
Verification rerun in primary checkout after pulling worker changes:
- node --check dashboard/server.js: pass
- node tests/test_dashboard_fp_api.js: pass
- PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=. python3 -m pytest -q tests/test_fastf1_cache_bypass.py tests/test_openf1_fp_summary.py tests/test_next_race_fp_quality.py tests/test_scoring_correctness.py: 28 passed
Local API smoke from dashboard server:
- japan-2026 validation: available=true, top10_hits=8, exact_positions=5, winner_hit=true
- miami-2026 validation: available=true, top10_hits=8, exact_positions=2, winner_hit=false
- monaco-2026 validation: available=true, top10_hits=4, exact_positions=0, winner_hit=false
- canada-2026 validation: available=false, reason=actual result not found
Report:
- docs/fp-validation-phase2.md
Recommendation recorded: do not change global FP weights yet. Use FP weighted pace as a top-ten membership signal; consider circuit-sensitive weighting after China/Australia/Japan richer backfills. Next safe backfill target: China 2026, one race only.
api4d ago
Mobile visual gate evidence for MC-4841:
APPROVED mobile verdict: FP Telemetry page is usable and not visibly broken after the new validation UI.
Live dashboard smoke:
- Server: node dashboard/server.js from current primary checkout.
- Page: F1 dashboard root, selected Miami GP, opened FP Telemetry.
- Screenshot: /home/lucienne/.hermes/cache/screenshots/browser_screenshot_e4567ef7ff834fdf89afdfff185e84e4.png
Observed evidence:
- FP Telemetry page renders validation cards: 8/10 FP top-10 hits, 80% hit rate, 2 exact positions, winner hit = No.
- Weighted FP Top 10 vs actual table renders rows without overlap.
- Actual Top 10 renders alongside validation data.
- Warnings render explicitly for missing Sprint laps.
- Missing-data chart sections show explicit "No data available" style fallback instead of blank/broken widgets.
- No console-blocking browser/CAPTCHA state observed.
Caveat: layout is still compressed desktop/tablet style rather than a full mobile-first redesign; however it passes the current mobile visual gate because the validation content is readable and not broken.
api4d ago
Mobile visual gate evidence for MC-4841:
APPROVED mobile verdict: pass.
Live 375px screenshot captured from the current primary checkout dashboard server:
- Screenshot path: /tmp/mc-4841-visual/f1-dashboard-375.png
- Screenshot dimensions verified by file command: 375 x 812 PNG
- Capture command used live URL: http://127.0.0.1:3002/
Additional interactive browser smoke on the same live dashboard selected Miami GP and opened FP Telemetry. Evidence observed:
- Validation cards rendered: 8/10 FP top-10 hits, 80% hit rate, 2 exact positions, winner hit = No.
- Weighted FP Top 10 vs actual table rendered without overlap.
- Actual Top 10 rendered alongside validation data.
- Warnings rendered explicitly for missing Sprint laps.
- Missing-data chart sections showed explicit fallback text rather than blank/broken widgets.
APPROVED mobile: no obvious broken UI, unreadable overlap, or missing validation content was observed.
api4d ago
Continuation update for MC-4841:
Completed China 2026 one-race FP validation/backfill and pushed to origin/master.
Commits now on origin/master:
- eef5462 docs: backfill China FP validation
- e59a9d9 docs: add FP validation phase 2 report
Important data-source note:
- OpenF1 was attempted for china-2026 only, but OpenF1 did not expose a 2026 China meeting.
- To avoid querying more races or doing a season-wide FastF1 run, the China backfill uses local cached China research and sprint-aware weighting as a validation proxy, not raw FastF1/OpenF1 telemetry.
China validation via live local API:
- /api/fp-analysis/china-2026 validation.available=true
- top10_hits=7/10
- top10_hit_rate=0.70
- exact_positions=0
- winner_hit=false
- podium_set_hit=false
Cross-race validation from current dashboard API:
- China: 7/10 hits, 0 exact, winner=false
- Miami: 8/10 hits, 2 exact, winner=false
- Japan: 8/10 hits, 5 exact, winner=true
- Monaco: 4/10 hits, 0 exact, winner=false
Verification in isolated origin/master worktree:
- node --check dashboard/server.js: pass
- node tests/test_dashboard_fp_api.js: pass
- focused pytest suite: 28 passed
Recommendation unchanged: do not increase global FP weighting yet. Current evidence supports FP as a strong top-10 inclusion signal, but exact-order/winner value varies sharply by circuit/session type. Next safe target: Australia 2026, one race only, preferably raw data if available.
api4d ago
Continuation update for MC-4841:
Completed Australia 2026 one-race FP validation/backfill and pushed to origin/master.
Commit:
- 1f8da04 docs: validate Australia FP analysis
Data source:
- Real OpenF1 practice data for australia-2026 only.
- Meeting: Australian Grand Prix, meeting_key=1279.
- Sessions fetched: Practice 1, Practice 2, Practice 3.
- No FastF1 run and no season-wide job.
Australia validation via live local API:
- /api/fp-analysis/australia-2026 validation.available=true
- top10_hits=7/10
- top10_hit_rate=0.70
- exact_positions=4
- winner_hit=true
- podium_set_hit=false
Updated cross-race validation from current dashboard API:
- Australia: 7/10 hits, 4 exact, winner=true
- China: 7/10 hits, 0 exact, winner=false
- Miami: 8/10 hits, 2 exact, winner=false
- Japan: 8/10 hits, 5 exact, winner=true
- Monaco: 4/10 hits, 0 exact, winner=false
Verification in isolated origin/master worktree:
- node --check dashboard/server.js: pass
- node tests/test_dashboard_fp_api.js: pass
- focused pytest suite: 28 passed
Recommendation: still no global FP-weight change. Australia and Japan support FP as useful on normal circuits; Miami/China support top-10 membership use on sprint/fallback races; Monaco remains circuit-specific caution. Next safe target: rebuild Japan 2026 with the current richer FP-analysis code, one race only, if we want richer ideal/corner/load fields.
luci-board-manager4d ago
[visibility-only controller] Routed this Luci-owned technical ticket to internal Hermes Kanban card t_d4b14a21 on board mc-internal (assignee=codexbuilder), running in isolated worktree `/home/lucienne/workspace/state/kanban-worktrees/mc-4841-f1-validation-phase2` (verified worker pid 2396114 cwd matches). MC did not launch/claim/send/harvest any runtime. Ticket is parked while Kanban produces the Miami 2026 FP validation/backtest handoff.
luci-board-manager4d ago
[visibility-only controller gate] Closed from verified artifact/test evidence. Verified pushed commits in f1-predictor: code commit 5c9d3a2 is on origin/master; report commit e8f0a13 is pushed on origin/kanban/mc-4768-openf1-401-fallback. Verified report exists at docs/fp-validation-phase2.md with Miami 2026 validation (8/10 top-10 hits, 2 exact positions, winner miss) and recommendation not to change global FP weights yet; next backfill order China, Australia, then richer Japan rebuild. Re-ran focused checks in /home/lucienne/workspace/projects/f1-predictor: node --check dashboard/server.js; node tests/test_dashboard_fp_api.js; PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=. python3 -m pytest -q tests/test_fastf1_cache_bypass.py tests/test_openf1_fp_summary.py tests/test_next_race_fp_quality.py tests/test_scoring_correctness.py -> 28 passed. Prior ticket comments include mobile screenshot/visual gate evidence for the dashboard validation UI. MC did not launch/claim/send/harvest a runtime.