Expense-Trackers
Add guarded production-to-local database refresh tooling
Summary
Implemented backend/scripts/db_refresh.py and make db-refresh to copy the complete pennywise database unchanged into an existing local PostgreSQL 18 Docker container. The CLI required explicit replacement confirmation, rejected remote Docker destinations and incompatible versions, validated the production archive, backed up local data, restored transactionally into staging, and checked for public tables before replacing only pennywise; containers, volumes, and other databases were left untouched. Failed staging restores retained recovery artifacts without dropping the existing database. Switched the default dump method to SSH plus pg_dump inside the production container after TCP password authentication failed; retained optional TCP mode. Installed and verified PostgreSQL 18.6 client tools for that mode. Removed make db-up and updated root setup documentation rather than creating another database server. Moved private deployment defaults into ignored local JSON configuration, provided a dummy example, isolated tests from real configuration, and excluded password files, backups, and Python caches from Git. Committed six files as 567e499 on local branch chore/local-db-refresh without publishing. Copied only .gitignore to main and amended its commit to c08e742, chore(gitignore): ignore local database refresh settings and backups. Final working tree was clean; nothing was pushed. Verification evidence showed 13 unit tests passed, formatting checks passed, a read-only live production schema dump succeeded and was validated locally, and a scan of 329 tracked/nonignored candidate files plus 326 index entries found no known private values; private settings/password files were untracked. No successful full live refresh or local data replacement was demonstrated. Graph refresh attempts were blocked by the existing smaller-graph overwrite safeguard.
Decisions
- Used a local Python CLI orchestrating PostgreSQL native tools rather than an application endpoint.
- Reused the running PostgreSQL 18 database container after discovering repository Compose settings did not describe the actual local server; reverted the proposed container/volume upgrade.
- Copied application data unchanged as requested, without copying the separate authentication database; local authentication must use the corresponding production user UUID.
- Kept private connection settings outside committed code and tests, with CLI options overriding ignored local configuration.
Learnings
- Inspect running infrastructure before relying on repository Compose defaults to plan database upgrades.
- The failed TCP authentication persisted without prompt/password-file handling; SSH access to the production container’s Unix socket provided a verified alternative without changing production authentication.