project image
Errorplanewatch 2024–2025
Flight fare monitoring platform. Durable job scheduler, multi worker scraping, and alerts that only fire once per fare.
- 99.9% API uptime on the pilot host
- Docker Compose deployment behind Nginx
- Playwright scraping with typed failure handling
PythonFlaskSQLAlchemyPlaywrightDockerNginx
click for details↻
What I built
- Flask API for creating, listing, canceling and soft deleting monitored routes, with a capacity check before a route is accepted.
- Scheduler that atomically claims each route for exactly one worker, so two processes never scrape the same search.
- Lease heartbeats and a claim token on every job, so a worker that dies mid scrape gets picked up again instead of stalling.
- Honest failure handling: a CAPTCHA or throttle pauses the job instead of rescheduling it, and an all failed run is recorded as failed with backoff.
- Scraping layer isolated behind a lazy Playwright import, with typed errors for blocked, throttled, timeout and unexpected table layouts.
- Alert dedup keyed on route plus fare identity, with money normalized to cents so the same fare never notifies twice.
- Telegram notifications, an admin console, and a Docker Compose deployment behind a hardened Nginx reverse proxy.