{"definition_raw":"---\nid: data-freshness-check\ntitle: Data freshness watchdog \u2014 check input feeds, alert on stale\nschedule: \"0 * * * *\"\ntimeout: 120\nretry: false\nenabled: true\nnotify_on: failure\nruntime_profile: direct_no_llm\nrun_as: shell\ncommand: \"cd /home/lucienne/workspace && python3 scripts/data_freshness_check.py\"\ntags: [monitoring, watchdog, freshness, mc-2936]\n---\n\n**OVERRIDES runtime profile:** pure stdlib Python, no LLM calls \u2014 `direct_no_llm`. Hourly check of canonical input feeds (CEO dashboard, Findash, Exco, ACSA, IATA fuel, FlightAware, booking-curve, wiki pages, ccgram/MC services, MC DB).\n\nWrites `~/workspace/state/data_freshness.json`. MC `/api/v1/health/data-freshness` exposes it; dashboard `/` renders 'Data feeds' grid. Telegram fires on first stale event per feed per 24h with owner unit (LaunchAgent / systemd / scheduler task) named in body. 3+ stale at once \u2192 escalate priority `critical`, tag Lucienne. Silent on green.\n\nTrigger: MC-2936. Recurring class of bug \u2014 pipeline silently stops, nobody notices for days (Exco MD pipeline 10-day silent failure 2026-05-07).\n","id":"data-freshness-check","last_run":{"duration_s":4.360936,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/413828.log","output":"data-freshness: 9/21 stale\n","started_at":"2026-06-13T06:00:02.740582+02:00","status":"completed"},"next_run":"2026-06-13 07:00","next_run_iso":"2026-06-13T07:00:00+02:00","runs":[{"duration_s":4.360936,"finished_at":"2026-06-13T06:00:07.103484+02:00","id":413828,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/413828.log","output":"data-freshness: 9/21 stale\n","started_at":"2026-06-13T06:00:02.740582+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":1.55685,"finished_at":"2026-06-13T05:00:09.632262+02:00","id":413659,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/413659.log","output":"data-freshness: 9/21 stale\n","started_at":"2026-06-13T05:00:08.072903+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":1.622259,"finished_at":"2026-06-13T04:00:05.789364+02:00","id":413482,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/413482.log","output":"data-freshness: 9/21 stale\n","started_at":"2026-06-13T04:00:04.164233+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":4.09596,"finished_at":"2026-06-13T03:00:06.805758+02:00","id":413302,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/413302.log","output":"data-freshness: 9/21 stale\n","started_at":"2026-06-13T03:00:02.706528+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":4.297111,"finished_at":"2026-06-13T02:00:21.037081+02:00","id":413128,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/413128.log","output":"data-freshness: 9/21 stale\n","started_at":"2026-06-13T02:00:16.737316+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":4.856171,"finished_at":"2026-06-13T01:00:07.717679+02:00","id":412953,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/412953.log","output":"data-freshness: 9/21 stale\n","started_at":"2026-06-13T01:00:02.859069+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":1.608866,"finished_at":"2026-06-13T00:00:04.499327+02:00","id":412775,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/412775.log","output":"data-freshness: 7/21 stale\n","started_at":"2026-06-13T00:00:02.887634+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":4.642706,"finished_at":"2026-06-12T23:00:07.374080+02:00","id":412601,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/412601.log","output":"data-freshness: 7/21 stale\n","started_at":"2026-06-12T23:00:02.728695+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":1.730579,"finished_at":"2026-06-12T22:00:04.533470+02:00","id":412427,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/412427.log","output":"data-freshness: 7/21 stale\n","started_at":"2026-06-12T22:00:02.801131+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":1.80155,"finished_at":"2026-06-12T21:00:04.424780+02:00","id":412254,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/412254.log","output":"data-freshness: 7/21 stale\n","started_at":"2026-06-12T21:00:02.620364+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":0.436052,"finished_at":"2026-06-12T20:00:03.102035+02:00","id":412078,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/412078.log","output":"data-freshness: 7/20 stale\n","started_at":"2026-06-12T20:00:02.663455+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":0.16553,"finished_at":"2026-06-12T19:00:02.712222+02:00","id":411910,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/411910.log","output":"data-freshness: 7/20 stale\n","started_at":"2026-06-12T19:00:02.543991+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":0.406023,"finished_at":"2026-06-12T18:00:02.952188+02:00","id":411735,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/411735.log","output":"data-freshness: 7/20 stale\n","started_at":"2026-06-12T18:00:02.543487+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":0.11969,"finished_at":"2026-06-12T17:00:02.565425+02:00","id":411562,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/411562.log","output":"data-freshness: 7/20 stale\n","started_at":"2026-06-12T17:00:02.444341+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":0.162851,"finished_at":"2026-06-12T16:00:02.954464+02:00","id":411388,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/411388.log","output":"data-freshness: 7/20 stale\n","started_at":"2026-06-12T16:00:02.789391+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":0.156495,"finished_at":"2026-06-12T15:00:02.983190+02:00","id":411215,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/411215.log","output":"data-freshness: 7/20 stale\n","started_at":"2026-06-12T15:00:02.825007+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":0.15479,"finished_at":"2026-06-12T14:00:02.950947+02:00","id":411042,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/411042.log","output":"data-freshness: 7/20 stale\n","started_at":"2026-06-12T14:00:02.793690+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":0.122321,"finished_at":"2026-06-12T13:00:02.647442+02:00","id":410868,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/410868.log","output":"data-freshness: 7/20 stale\n","started_at":"2026-06-12T13:00:02.523016+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":0.15582,"finished_at":"2026-06-12T12:00:02.895254+02:00","id":410706,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/410706.log","output":"data-freshness: 7/20 stale\n","started_at":"2026-06-12T12:00:02.734545+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"},{"duration_s":0.169175,"finished_at":"2026-06-12T11:00:02.461875+02:00","id":410532,"log_path":"/home/lucienne/workspace/logs/task-runs/data-freshness-check/410532.log","output":"data-freshness: 7/20 stale\n","started_at":"2026-06-12T11:00:02.290165+02:00","status":"completed","task_id":"data-freshness-check","task_name":"Data freshness watchdog \u2014 check input feeds, alert on stale"}],"runs_limit":20,"schedule":"0 * * * *","schedule_label":{"description":"Every hour at :00","is_custom":false,"label":"Hourly","sort":2,"sort_time":""},"stats":{"avg_duration":0.42813364000000004,"completed":175,"failed":0,"timeout":0,"total":175},"task":{"_description":"**OVERRIDES runtime profile:** pure stdlib Python, no LLM calls \u2014 `direct_no_llm`. Hourly check of canonical input feeds (CEO dashboard, Findash, Exco, ACSA, IATA fuel, FlightAware, booking-curve, wiki pages, ccgram/MC services, MC DB).\n\nWrites `~/workspace/state/data_freshness.json`. MC `/api/v1/health/data-freshness` exposes it; dashboard `/` renders 'Data feeds' grid. Telegram fires on first stale event per feed per 24h with owner unit (LaunchAgent / systemd / scheduler task) named in body. 3+ stale at once \u2192 escalate priority `critical`, tag Lucienne. Silent on green.\n\nTrigger: MC-2936. Recurring class of bug \u2014 pipeline silently stops, nobody notices for days (Exco MD pipeline 10-day silent failure 2026-05-07).","_file":"data-freshness-check.md","_path":"/home/lucienne/workspace/tasks/data-freshness-check.md","command":"cd /home/lucienne/workspace && python3 scripts/data_freshness_check.py","enabled":true,"id":"data-freshness-check","notify_on":"failure","retry":false,"run_as":"shell","runtime_profile":"direct_no_llm","schedule":"0 * * * *","tags":["monitoring","watchdog","freshness","mc-2936"],"timeout":120,"title":"Data freshness watchdog \u2014 check input feeds, alert on stale"}}