The source of a released app was not in git: A solo developer's server backup hole full inspection log

There's an app being distributed to actual users on the app store. There are also operational scripts that cron runs daily. But one day, I checked and found that a significant number of them were outside the version control system (git). This meant they would be lost if the server disk died. This article records how I discovered this fact, why the assumption "it's okay because there are automatic commits" was wrong, and the process of inspecting all projects and putting them into a backup system in just one day.

1. Discovery: What I learned when trying to commit

The trigger was ordinary. I had added several features to the financial analysis system over a week, and the final step was a simple task: "commit changes and back them up." But when I opened `git status`, something was off. All the files I had created this week were piled up in an *untracked* state. And there were 36 of them.

This server has a hook that automatically creates WIP commits whenever a task is finished. So I believed that "commits were happening automatically." Indeed, commit logs were accumulating daily. The problem was that these automatic commits **only included modified files and did not touch newly created (untracked) files**.

# Automatic commit log looked fine
WIP(analyzers): Improve analysis module
WIP(targets): Update briefing
...

# But in git status
?? analyzers/새기능_A.py      ← Operational cron running daily
?? analyzers/새기능_B.py      ← Dashboard importing
?? tests/ (16 files)
?? docs/plans/ (7 documents)

In other words, all the new modules created over several weeks — including operational code running daily via cron — existed only on the local disk. I had been reassured by seeing commit records accumulate, but the most important new code had never actually entered the repository.

2. Full Inspection: There wasn't just one hole

If one is like this, other projects must be checked. I inspected all projects on the server, and the results were divided into three stages.

2-1. Repositories with new untracked files

The same pattern appeared in the AI assistant project, in addition to the financial analysis system. What was particularly painful was that **4 types of operational scripts run daily by cron** were untracked. Things like signal collectors, market guards, and error message dictionaries — files that, if the server crashed, the cron would remain but the scripts would disappear, forcing recovery to start with "what did this cron run?".

2-2. Repositories without a remote repository

One app project had good git management, but it was local-only without a remote. No matter how clean the commit history, it was a history that would disappear with the disk.

2-3. Projects without git itself

This was the most severe case. **Two Android apps actually being distributed via the App Store's internal test track weren't even git repositories.** These apps had been distributed with version codes up to 14, but their source only existed in the server directory. The build artifact (AAB) was uploaded to the store, creating the illusion that "the app is alive," but the source history was effectively zero copies.

3. Solution: Half a day of cleanup

The cleanup itself was a simple task. The order was important.

3-1. Check secrets first

Rushing to `git add -A` can lead to an accident. Android projects had signing keystores (`.jks`), key passwords (`keystore.properties`), and service account keys (JSON) for automatic store uploads living together in the directory. Before committing, I first created a `.gitignore`, searched for secret patterns in the staged file list to confirm 0 matches, and then committed.

# Pre-commit verification — staging list must not contain secrets
git diff --cached --name-only | grep -iE "jks|keystore|service-account|local.properties"
# (No output = safe)

3-2. Create and connect private remote repositories

I created private repositories for the three projects and pushed their entire history. One pitfall: when creating a repository via CLI, the remote sometimes attaches with an SSH address. If the server is configured only for token authentication instead of SSH keys, the first push will fail. Changing to an HTTPS address and connecting a credential helper solves this.

3-3. Leave "why" in the commit message

The message for this batch commit stated the reason: "Automatic WIP commit missed new files, starting tracking of previously untracked operational code." This is because months later, I would ask, "Why did 36 files come in all at once at this point?"

4. Lessons Learned

4-1. Automation doesn't tell you what it *doesn't* do

The automatic commit hook didn't lie. It said it would commit modified files, and it did exactly that. The problem was that I extrapolated "commits are happening" to "everything is being committed." Wherever there is automation, it is crucial to periodically check **outside the boundaries of that automation**. In this case, the inspection command was a single line: `git status`. No one had run that single line for weeks.

4-2. "Deployed" and "backed up" are completely different statements

The sense of security from having an app on the store obscured the absence of source backup. The existence of an artifact has nothing to do with the safety of the source.

4-3. Do not batch commit next to secrets

Creating a key leakage incident while trying to fix a backup hole is a worse trade. The sequence of "gitignore first, verify with staging scan, then commit" takes a few minutes, while an accident in the reverse order can take days for key revocation and re-issuance.

4-4. Inspections happen incidentally

This hole wasn't discovered because I decided to conduct a backup audit; it was a byproduct of a routine "commit this" task. The previous 4-day silent outage of the automatic publishing pipeline was also accidentally discovered during API key cleanup. While planning regular audits is good, **not overlooking subtle anomalies encountered during daily tasks** proved to be a more practical audit system for solo operations.

FAQ

Q. Can't automatic commits also include untracked files?

It's possible but not recommended. New files are prone to mixing in secrets, large artifacts, and temporary files, and automatically doing `add -A` immediately opens up the accident path described in lesson 3 above. It's safer for a person to decide on new file tracking after checking gitignore, and instead, monitoring like "notify if the number of untracked files exceeds N" is a realistic compromise.

Q. Is a private repository necessary for a solo project?

From a backup perspective, it's almost essential. It's not about collaboration features, but about whether there's a copy of the history physically separated from the server. Cloud servers can disappear more easily than you think — due to billing issues, region outages, or accidental instance deletion.

Q. Where do you commit, and what do you exclude?

The criteria applied this time: operational code, tests, design documents, and cron scripts are all tracked. Runtime database files, logs, user conversation records, screenshots, signing keys, and credentials are all excluded. Especially for conversation records, the principle was to not put them in the repository because they are "data, not code."