You're offline — showing cached data

Tailscale UDP/DERP health watch

tailscale-watch
2026-06-13 06:09:20 SAST
← Back to Tasks

OVERRIDES runtime profile: uses direct_python because this network health watch runs deterministic Tailscale checks and does not invoke Claude/Codex or any LLM API.

Half-hourly health check on Tailscale's NAT-traversal layer.

Background: 2026-05-07 SAST — Tailscale on Luci has broken UDP NAT discovery (netcheck reports UDP: false, IPv4: no addr found, no DERP latency). Phone-to-server traffic relays via DERP-jnb with 241ms-2.1s RTT, making the mobile PWA feel unusable. Workaround in place: phone uses public IP http://204.168.188.33:3001 direct. SSH and admin paths still work fine via DERP since lower-frequency traffic tolerates the latency.

Full diagnosis: ~/workspace/reports/tailscale-degraded-2026-05-07.md

Behaviour of this watcher: - Run tailscale netcheck and tailscale ping elmars-s26-ultra - Classify state: healthy | degraded | broken - Persist a rolling history to ~/workspace/state/tailscale_health.json - Alert (force=True, bypasses quiet hours): - Recovery: when state flips from degraded → healthy - Warn: degraded continuously for ≥24h - Escalate: degraded continuously for ≥7d, recommend Cloudflare Tunnel - Each alert rate-limited to once per 24h per key

Parent MC ticket: MC-2963.

Every 30 minutes*/30 * * * *
Yes
shell
60s
No
in 21m (2026-06-13 06:30)

Task Definition

tasks/tailscale-watch.md
shell
set
No
No
python3 /home/lucienne/workspace/scripts/tailscale_watch.py

Edit Schedule

Result:
349
Runs (7d)
349
Completed
0
Failed
0
Timeout
100%
Success (7d)
7s
Avg Duration
infrastructure tailscale monitoring

Run History (50)

StartedFinishedStatusDurationOutputLog
8m ago 8m ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=878.67
View log
39m ago 39m ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=878.16
View log
1h ago 1h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=877.73
View log
2h ago 2h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=877.16
View log
2h ago 2h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=876.73
View log
3h ago 3h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=876.18
View log
3h ago 3h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=875.67
View log
4h ago 4h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=875.16
View log
4h ago 4h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=874.7
View log
5h ago 5h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=874.17
View log
5h ago 5h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=873.67
View log
6h ago 6h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=873.16
View log
6h ago 6h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=872.67
View log
7h ago 7h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=872.16
View log
7h ago 7h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=871.69
View log
8h ago 8h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=871.16
View log
8h ago 8h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=870.67
View log
9h ago 9h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=870.16
View log
9h ago 9h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=869.66
View log
10h ago 10h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=869.16
View log
10h ago 10h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=868.66
View log
11h ago 11h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=868.16
View log
11h ago 11h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=867.72
View log
12h ago 12h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=867.17
View log
12h ago 12h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=866.68
View log
13h ago 13h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=866.17
View log
13h ago 13h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=865.71
View log
14h ago 14h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=865.17
View log
14h ago 14h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=864.69
View log
15h ago 15h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=864.16
View log
15h ago 15h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=863.66
View log
16h ago 16h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=863.17
View log
16h ago 16h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=862.67
View log
17h ago 17h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=862.17
View log
17h ago 17h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=861.67
View log
18h ago 18h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=861.16
View log
18h ago 18h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=860.81
View log
19h ago 19h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=860.16
View log
19h ago 19h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=859.66
View log
20h ago 20h ago completed 10s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=None hours_degraded=859.16
View log
20h ago 20h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=858.66
View log
21h ago 21h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=858.16
View log
21h ago 21h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=857.66
View log
22h ago 22h ago completed 8s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=857.16
View log
22h ago 22h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=856.71
View log
23h ago 23h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=856.16
View log
23h ago 23h ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=855.66
View log
24h ago 24h ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=855.16
View log
1d ago 1d ago completed 6s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=854.68
View log
1d ago 1d ago completed 7s
tailscale-watch: state=degraded udp=False ipv4=None ping_via=derp hours_degraded=854.16
View log

Comments (0)

No comments yet — add instructions for Luci
Help