A fleet of isolated agents

A panel and a fleet of agents: every account gets its own server with its own address, and an agent there holds the connection to the platform and carries out the panel’s commands. The panel hands out work, collects state and shows all of it in one window.

Panel and agent live on different machines — and the protocol admits it

The protocol description sits in two identical files, one in the panel and one in the agent. This is not a forgotten duplicate: the components are deployed on different servers and updated independently, and a shared package would mean that deploying the panel breaks agents the new code has not reached yet.

That is why every authentication message carries a protocol version number: the two sides learn whether they are compatible immediately, instead of discovering it on the first command one of them cannot parse.

Traits are derived from the account key, not drawn at random

The browser parameters — user agent, connection signature profile, language — are picked deterministically: a hash of the account key modulo the list length, with a separate salt per parameter. The same agent looks the same every time.

The obvious alternative, a fresh random set on every start, would give the automation away faster than having none: a real person does not change browser and system language between visits. The headroom is computed in the code too: 30 × 20 × 12 ≈ 7,200 combinations, roughly a 2.7% chance of a collision across twenty accounts.

The supervisor restarts only what actually crashed

The agent’s background threads have their own supervisor. Its rule is written down in the code together with the reason: a restart happens only on an exception caught inside the thread itself — and on nothing else.

Before that an external watchdog judged liveness by a proxy signal, and one day it killed healthy agents because the signal turned out to be coarse. The restart budget is hard — five within three hundred seconds; once spent, the thread stops for good and someone has to look at it, instead of spinning forever.

Two loops are deliberately not supervised: the platform listener and the reconnect loop. Both have recovery logic of their own, and a second supervisor would race against it.

An agent works out on its own that it no longer exists

If the panel answers “unknown token” twenty times in a row — with a growing pause that is 15 to 25 minutes — the agent concludes the account was deleted in the database while the update had not reached the server yet. It wipes its own credentials, marks itself retired and exits.

Without it, an account deleted in the panel would keep knocking and keep working with live credentials on a server everyone had forgotten about.

The first pause is short — because of a race with the panel’s database

A freshly deployed agent manages to ask for authentication before the panel has finished writing it into the database. The retry used to be thirty seconds, which turned a start into minutes of downtime for no reason. Now the pause begins at three seconds and doubles: the race resolves in one or two retries, while a rare genuine failure still backs off properly.

Python · asyncio · WebSocket · SQLite (WAL) · systemd · Caddy