Local ↔ VPS Sync
The Windows PC scrapes locally (headed Chrome + C:\Python312\python.exe), then pushes
to this VPS. Some scrapers run on-box (Predictz via FlareSolverr).
Authoritative detail lives in the
local-vps-upload-trackingskill. This note is the
VPS-side view.
Windows Task Scheduler jobs (local)
| Job | Script | Schedule | Cycle |
|---|---|---|---|
| Snooker scrape | C:\SNOOKER\scrape.bat |
04:00 | 24h |
| Snooker push | C:\SNOOKER\push_to_vps.bat |
05:30 | 24h |
| Badminton sync | C:\BADMINTON\sync-all.bat |
05:30 | 24h |
| GHY greyhound | C:\GHY\run_full_scheduled.bat |
06:00 | 24h |
| Badminton push | C:\BADMINTON\push_to_vps.bat |
06:30 | 24h |
| HorseScraper | C:\HORSESCRAPER\run_scheduled.bat |
07:00 | 24h |
| HorseScraper2 RaceFields | C:\HORSESCRAPER2\run_scheduled.bat |
07:30 | 24h |
| Tennis LTA sync | C:\TENNIS\sync-daily.bat |
09:00 + logon | 24h |
| SPORTSTLM local-feeds push | C:\local-feeds\run_push.bat |
23:30 + 6h | 6h |
Upload mechanisms
- scp → docker cp (direct): snooker
push_to_vps.bat=scp db → VPS:/tmp/x.new
thenssh "docker stop X && docker cp … && docker start X". - git auto-commit (indirect): some scrapers commit locally; VPS pulls
(greyhound/racing). - Headless 6-hourly push:
C:\local-feeds\push_feeds.py(snooker-only; Predictz
runs on VPS/FlareSolverr) — the only job that reliably survives. - Remote:
git@github.com:investconfig-spec/sportstlm-local-feeds.git.
Git remotes (auto-commit, push_to_vps.bat)
Each module's push now runs git add → commit → push after shipping, so code is versioned every cycle.
| Module | Remote |
|---|---|
| snooker | investconfig-spec/snooker |
| badminton | investconfig-spec/badminton ⚠️ re-pointed 2026-11-03 from oscarlaaaa/badminton-api (was 403/denied); repo re-inited, clean .gitignore (now correctly excludes data/*.db + *.log, includes Dockerfile) |
| tennis | investconfig-spec/tennis |
| ufc | investconfig-spec/ufc |
VPS-side Predictz ingest
/pz/api/predictz/ingest* → opsd :9600, key-gated (off-box fetch because
Cloudflare blocks this IP range). FlareSolverr on :8191.
Failure patterns (Windows Last Result decode)
| Code | Meaning |
|---|---|
0x80070490 (-2147020576) |
ERROR_NOT_FOUND → machine asleep during 04:00–07:30; headed Chrome needs an awake logged-in desktop |
0xC0000409 (-1073740791) |
crash — read the job's .log |
snooker.org 403/401 |
rate-limit → backoff / switch source |
0 |
success |
Scrape handshake (VPS-side freshness monitor)
/root/.hermes/scripts/scrape_handshake.py — runs 24/7 on the VPS Hermes gateway (hourly),
checks each module's source-DB mtime on the VPS volume. A fresh mtime = the local scrape
landed (ACK); a stale mtime (>28h) = the scrape missed (desktop off / failed) → Telegram alert.
Two modes:
- (default) watchdog — silent when all fresh; prints stale/missing only (cron alert).
- --all — echoes every pipeline's ✅/❌ status. Each push_to_vps.bat calls this after
shipping (ssh … scrape_handshake.py --all), so every upload self-verifies + reports the
other 13 pipelines too (post-upload echo).
Watches: snooker snooker_archive.db, badminton badminton.db, tennis lta.db,
ufc ufcstats.db, racing index.json (DETAILS neural) + rc-index.json (RC race-fields),
plus 8 VPS self-scrape sources (statarea/oddsmath/predictz/news/debrief/previews/sportsmole/youtube).
No-overlap rule
Each job lives in exactly ONE place — local Task Scheduler or VPS (systemd timer / Hermes
cron), never both. The desktop-only jobs (headed-Chrome scrapes) stay on Windows; the
monitoring + reminders (watchdogs, appointments) live on the VPS gateway so they survive the
desktop being off.
Deploy note
The VPS tree can be ahead of local —
scpDOWN before deploying local changes
(RSSSPORTS). Source of truth is split: app source lives local, but VPS edits and
on-box scripts (Sports Mole, ops collectors) live only on the VPS.