Watching in-game storefronts

The dashboard continuously watches the in-game shops of two dozen titles and shows in one window what appeared, what was pulled and what went up in price. Every item keeps a history, and changes go out to subscribers. The hard part is that almost no game hands its storefront over willingly: each has its own way of obtaining it, and all of them hide behind one interface.

One contract hides methods that have nothing in common

Every scanner inherits a single abstract class: a mandatory game name and scan method, optional cleanup and availability checks. A wrapper turns any exception into a result with a “failed” status.

The ways of getting a storefront are incomparable: an open web shop here, data from the client’s cache there, traffic analysis elsewhere, screen recognition in another. Without a shared contract every source would leak into scheduling, the loop and the mailing, and adding a game would mean edits in five places. The wrapper also guarantees that one scanner failing does not bring the whole cycle down.

An item is not removed on its first disappearance

Every record carries a miss counter: a removal is only recorded after three consecutive successful scans without the item. A return resets the counter.

The obvious “not in the new scan, so removed” chases its own tail. Sources periodically return an incomplete list for no visible reason, and a naive comparison produces ghost pairs: the item is marked removed, then “appears” as the same item on the next scan — and both events reach subscribers as real.

Bulk removal is blocked when a scan looks like a failure

Before marking anything, the share of vanished items is computed. An empty answer while active items are known, a disappearance of more than half, and a smaller share for unstable sources — all removals are skipped with a log entry.

An expired session, a network failure or a shop under maintenance produce an empty answer. Without the safeguard one such scan “removes” the entire shop, sends subscribers a hundred removals, and the next successful scan sends the same hundred as new arrivals. The threshold separates “the shop really did empty” — rare — from “the scan failed”, and when in doubt the system stays quiet rather than lying.

Ghosts are cancelled by matching on name and price

Before sending, the “added” and “removed” lists are intersected on name and price, and matches are dropped from both.

Some games rotate an item’s internal identifier between scans while the item stays the same. By identifier that reads as “old removed, new added” — two false notifications about one unchanged item. Matching on what a person actually sees removes the pair entirely.

Only paid changes reach the chat

Before sending, every change is checked twice: is this real money, and is this a human-readable name rather than an internal code. The very first scan into an empty history sends nothing at all.

The client cares about purchases for real money, while shops are full of in-game currency rewards and service records. Without the filter the first run alone would send the whole catalogue as hundreds of lines, and after that the chat would fill with internal codes leaking out of a cache.

Scan progress is streamed rather than returned in one answer

Heavy scanners emit a stream of events: a step, another step, done with a summary.

A single scan takes from seconds to minutes — starting the game, waiting for it to load, parsing. A plain request-response would hang the tab for that time with no sign of life, and any intermediate timeout would break it off as an error. A stream shows that work is happening and where it stalled, if it stalled.

Light and heavy scanners are separated by their pauses

Fast requests go first with a short pause between them, then the heavy ones that need a device, with a pause an order of magnitude longer and another after a full round.

The heavy ones compete for one physical device: running them back to back guarantees conflicts. The light ones should not suffer that queue, so they go first.

The “new” badge lives until a real change, not until the next scan

The data holds two timestamps: when it was last refreshed and when something last changed. The badge in the interface is tied to the second one.

Tie it to the scan and it disappears after a single cycle: someone visiting an hour later sees no changes at all. A badge should mark an event, not the fact of polling.

When a session is refreshed, the cleanup order is inverted

When a stored session expires, the system refreshes it on its own. The shutdown order matters: first the device’s network settings are carefully restored, with a check that they were, and only then the helper process is terminated. Plus a flag that a refresh is already running.

The obvious “job done, kill everything” leaves the device in a state where its requests hit an already dead helper: every subsequent command hangs, and it takes hands to fix. And without the flag two parallel refreshes of one session would overwrite each other.

Python · FastAPI · httpx · protobuf · Telegram Bot API