Google's Review Bots Ate My Remote Commands: A Lesson in Device Scoping from an Internal-Test App

I added a remote-command feature to a personal Android app: drop a command into a server queue, and the app picks it up and executes it on its next sync. One day, commands started showing "completed" on the server — while nothing happened on my phone. The culprit, improbably, was Google: the Play Store's automated review devices were launching my app and swallowing the commands first. A story about how even a single-user app never has just one user.

1. Background: Why a Personal App Goes Through the Store

This app is not for sale. It exists for exactly one phone — mine. It still ships through the Play Store's internal testing track for one reason: modern devices effectively block sideloading (direct APK installs) by security policy. Even one-person apps ride official distribution now. That choice brought an unexpected roommate.

2. The Incident: Completed, but Never Executed

The remote-command design is humble. Insert a command row into the server DB (say, "move these three records to trash"); the app fetches and executes it on the next sync and reports back. One day I queued a command, synced my phone — nothing. The server log was stranger still:

command #12: status=completed, result ok
(yet the data on my phone was untouched)

The command had clearly been delivered to and executed by someone. Someone who wasn't my phone.

3. Root Cause: Review Bots Are Users Too

Digging through sync receipts revealed unfamiliar device identifiers — three or four devices besides my phone, periodically launching the app and syncing with the server. They were the Play Store's automated review and compatibility-test devices. The store runs uploaded apps on a fleet of virtual and physical devices; my app, when launched, dutifully syncs first. That sync polls the command queue — and since the queue was first-come-first-served, the review bots drained my commands before my phone ever arrived.

The bots' devices contain none of my data, so the commands "succeeded" against nothing, and the server stamped them complete. A perfect silent failure.

4. The Fix: No Message Without an Addressee

• Mandatory device scoping. Every command now requires a target device ID. The command-enqueue tool defaults to my real device's ID, and broadcast-to-all was made deliberately awkward. Review bots still poll — and find nothing addressed to them.
• Making the device fleet visible. A later update records each device's app version and last sync time server-side. I can still watch a few review-bot devices faithfully syncing on the latest version — now as harmless spectators that can't touch the queue.
• An accidental feature. That device table also answers "did my phone get the update?" from the server, without touching the phone. The cleanup became a capability.

5. Lessons

• Even a single-user app has many users. The moment it's on a store, review bots, compatibility rigs, and someday-your-next-phone all become launchers of your app. Your server talks to all of them.
• Never default a command queue to broadcast. A message without an explicit addressee will eventually be taken by the wrong one. Safe defaults are what keep mistakes from becoming incidents.
• "Completed" is not "completed as intended." A design where a command targeting nothing quietly succeeds breeds silent failures. A completion stamp means something only when the result records which device, how many rows affected.
• Bot traffic pollutes data. Had the bots' syncs fed usage analytics, the analytics would have lied; since they fed a command queue, the commands did. Server-side metrics for store-distributed apps are fiction until bot devices are filtered out.

FAQ

Q. Can't you stop the review bots from launching the app?

No. Automated launch testing is part of the store's process. When traffic can't be blocked, the answer is to design the server so that traffic is handled "normally but harmlessly."

Q. Wouldn't authentication have prevented this?

Authentication was already there — the bots run a legitimately installed build, so they pass with the same token. The problem was never identity but addressing. "Any authenticated device" and "that exact device" are different concepts.

Q. Is there no distribution path other than internal testing?

There are others — manufacturer developer-mode workarounds, enterprise MDM distribution — but at personal scale, the internal testing track has the least friction. Cohabiting with review bots is the rent, and this post is the fine print of that lease.