Skip to content

feat(docker): backup OpenSPP filestore (#66) #304

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Tarekchehahde wants to merge 3 commits into
base: 19.0
Choose a base branch
from
Open

feat(docker): backup OpenSPP filestore (#66) #304

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
047f06f
feat(docker): add OpenSPP filestore backup (#66)
Jul 11, 2026
d7a29a1
fix(docker): address filestore backup review feedback
Tarekchehahde Sep 19, 2026
cc4f547
style(docker): wrap README backup bullets for pre-commit
Tarekchehahde Sep 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docker/.env.production.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,15 @@ FLUENTD_LOG_PATH=./fluentd/log
# Default: daily at 2am
BACKUP_SCHEDULE=0 2 * * *

# Filestore backup (opt-in). Full copies grow as KEEP_* × filestore size —
# suitable for small/mid on-prem; large deployments prefer object storage + snapshots.
BACKUP_FILESTORE=false
BACKUP_FILESTORE_COMPRESS=false
# FILESTORE_SRC=/odoo_data/filestore/openspp
# BACKUP_FILESTORE_KEEP_DAYS=7
# BACKUP_FILESTORE_KEEP_WEEKS=4
# BACKUP_FILESTORE_KEEP_MONTHS=6

# =============================================================================
# ANTIVIRUS (enable with COMPOSE_PROFILES=https,clamav)
# =============================================================================
Expand Down
34 changes: 29 additions & 5 deletions docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,19 +215,43 @@ DATABASE_URL=postgres://user:password@hostname:5432/openspp?sslmode=require
The production stack includes automated PostgreSQL backups:

- **Schedule:** Daily at 2am (configurable via `BACKUP_SCHEDULE`)
- **Retention:** 7 daily, 4 weekly, 6 monthly
- **Retention:** 7 daily, 4 weekly, 6 monthly (DB dumps via `BACKUP_KEEP_*`)
- **Location:** `backup_data` Docker volume

To restore a backup:
- **Filestore:** Opt-in via `BACKUP_FILESTORE=true`. When enabled, attachments under
`FILESTORE_SRC` (default `/odoo_data/filestore/<database>`) are archived alongside the
dump as `*_filestore_*.tar` (or `.tar.gz` if `BACKUP_FILESTORE_COMPRESS=true`).
Filestore retention uses `BACKUP_FILESTORE_KEEP_*` (defaults match `BACKUP_KEEP_*`).
Plan disk as roughly `KEEP_* × filestore size` — nightly full copies do not dedupe.
- **Consistency:** Dump then filestore is crash-consistent, not a true point-in-time
pair. A genuinely consistent restore needs a volume snapshot or a brief Odoo stop.
- **Data protection:** `backup_data` holds beneficiary documents in the clear (same
class of exposure as the DB dump). Encrypt the volume at rest in production.

To restore a backup (pair dump and filestore files that share the same `TIMESTAMP`):

```bash
# List backups
docker compose -f docker/docker-compose.production.yml exec backup ls -la /backups
docker compose -f docker/docker-compose.production.yml exec backup ls -la /backups/daily

# Restore (stop services first)
docker compose -f docker/docker-compose.production.yml stop odoo queue-worker

# 1) Database (pg_dump -Fc custom format; filenames use underscores)
docker compose -f docker/docker-compose.production.yml exec db \
pg_restore -U odoo -d openspp /backups/daily/openspp-YYYYMMDD-HHMMSS.sql.gz
pg_restore -U odoo -d openspp --clean --if-exists \
/backups/daily/openspp_YYYYMMDD_HHMMSS.dump

# 2) Filestore (same timestamp as the dump; adjust extension if compressed)
docker compose -f docker/docker-compose.production.yml run --rm --no-deps \
-v odoo_data:/odoo_data \
-v backup_data:/backups:ro \
backup sh -c '
rm -rf /odoo_data/filestore/openspp
mkdir -p /odoo_data/filestore
tar -xf /backups/daily/openspp_filestore_YYYYMMDD_HHMMSS.tar -C /odoo_data/filestore
chown -R 101:101 /odoo_data/filestore/openspp
'

docker compose -f docker/docker-compose.production.yml start odoo queue-worker
```

Expand Down
62 changes: 62 additions & 0 deletions docker/backup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,13 @@
# Environment variables:
# PGHOST, PGPORT, PGUSER, PGPASSWORD, PGDATABASE (standard PostgreSQL vars)
# BACKUP_DIR (default: /backups)
# BACKUP_FILESTORE (default: false) — set true to archive Odoo filestore
# FILESTORE_SRC (default: /odoo_data/filestore/$PGDATABASE)
# BACKUP_FILESTORE_COMPRESS (default: false) — gzip filestore archives
# BACKUP_KEEP_DAYS (default: 7)
# BACKUP_KEEP_WEEKS (default: 4)
# BACKUP_KEEP_MONTHS (default: 6)
# BACKUP_FILESTORE_KEEP_DAYS / _WEEKS / _MONTHS (default: same as BACKUP_KEEP_*)

set -e

Expand All @@ -22,6 +26,11 @@ BACKUP_DIR="${BACKUP_DIR:-/backups}"
BACKUP_KEEP_DAYS="${BACKUP_KEEP_DAYS:-7}"
BACKUP_KEEP_WEEKS="${BACKUP_KEEP_WEEKS:-4}"
BACKUP_KEEP_MONTHS="${BACKUP_KEEP_MONTHS:-6}"
BACKUP_FILESTORE="${BACKUP_FILESTORE:-false}"
BACKUP_FILESTORE_COMPRESS="${BACKUP_FILESTORE_COMPRESS:-false}"
BACKUP_FILESTORE_KEEP_DAYS="${BACKUP_FILESTORE_KEEP_DAYS:-${BACKUP_KEEP_DAYS}}"
BACKUP_FILESTORE_KEEP_WEEKS="${BACKUP_FILESTORE_KEEP_WEEKS:-${BACKUP_KEEP_WEEKS}}"
BACKUP_FILESTORE_KEEP_MONTHS="${BACKUP_FILESTORE_KEEP_MONTHS:-${BACKUP_KEEP_MONTHS}}"

# Directories
DAILY_DIR="${BACKUP_DIR}/daily"
Expand All @@ -31,6 +40,10 @@ MONTHLY_DIR="${BACKUP_DIR}/monthly"
# Create directories
mkdir -p "${DAILY_DIR}" "${WEEKLY_DIR}" "${MONTHLY_DIR}"

# Skip overlapping runs (filestore archives can outlast the cron interval)
exec 9>"${BACKUP_DIR}/.backup.lock"
flock -n 9 || { echo "[$(date -Iseconds)] Backup already running; skipping"; exit 0; }

# Timestamp
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
DATE=$(date +%Y%m%d)
Expand All @@ -52,15 +65,58 @@ ln -sf "${BACKUP_FILE}" "${DAILY_DIR}/${PGDATABASE:-openspp}_latest.dump"

echo "[$(date -Iseconds)] Daily backup complete: ${BACKUP_FILE}"

# Filestore backup (attachments, documents) — opt-in.
# Dump runs before the archive on purpose: an attachment written between dump and
# tar leaves an orphan file that Odoo's filestore GC reaps; the reverse order
# produces a DB row whose file was never captured (FileNotFoundError on restore).
# The pair is crash-consistent, not a true point-in-time snapshot.
FILESTORE_SRC="${FILESTORE_SRC:-/odoo_data/filestore/${PGDATABASE:-openspp}}"

@kneckinator kneckinator Sep 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No lock around the run.

backup.sh has no flock, so if a run outlasts the cron interval the next one starts on top of it. That was survivable when the script only ran pg_dump; once a multi-hour filestore tar is in the mix on a daily schedule, overlapping runs become realistic — and two concurrent tars writing the same .part name would corrupt each other.

flock is in busybox, so this is cheap:

exec 9>"${BACKUP_DIR}/.backup.lock"
flock -n 9 || { echo "[$(date -Iseconds)] Backup already running; skipping"; exit 0; }

Strictly a pre-existing gap that this change amplifies — fine to split into its own PR if you'd rather keep this one tight, but please don't drop it.

FILESTORE_SRC="${FILESTORE_SRC%/}"
if [ "${BACKUP_FILESTORE_COMPRESS}" = "true" ]; then
FILESTORE_EXT="tar.gz"
FILESTORE_TAR_FLAGS="-czf"
else
FILESTORE_EXT="tar"
FILESTORE_TAR_FLAGS="-cf"
fi
FILESTORE_BACKUP_FILE="${PGDATABASE:-openspp}_filestore_${TIMESTAMP}.${FILESTORE_EXT}"
FILESTORE_BACKUP_CREATED=0

if [ "${BACKUP_FILESTORE}" != "true" ]; then
echo "[$(date -Iseconds)] Filestore backup disabled (BACKUP_FILESTORE=${BACKUP_FILESTORE})"
elif [ ! -d "${FILESTORE_SRC}" ]; then
echo "[$(date -Iseconds)] Filestore not found at ${FILESTORE_SRC}; skipping filestore backup"
else
echo "[$(date -Iseconds)] Starting filestore backup from ${FILESTORE_SRC}..."
# tar exits 1 when Odoo's filestore GC unlinks a file mid-archive. Running it
# as an `if` condition keeps `set -e` from skipping the retention pass below.
if tar ${FILESTORE_TAR_FLAGS} "${DAILY_DIR}/${FILESTORE_BACKUP_FILE}.part" \
-C "$(dirname "${FILESTORE_SRC}")" "$(basename "${FILESTORE_SRC}")"; then
mv "${DAILY_DIR}/${FILESTORE_BACKUP_FILE}.part" "${DAILY_DIR}/${FILESTORE_BACKUP_FILE}"
ln -sf "${FILESTORE_BACKUP_FILE}" "${DAILY_DIR}/${PGDATABASE:-openspp}_filestore_latest.${FILESTORE_EXT}"
FILESTORE_BACKUP_CREATED=1
echo "[$(date -Iseconds)] Filestore backup complete: ${FILESTORE_BACKUP_FILE}"
else
rm -f "${DAILY_DIR}/${FILESTORE_BACKUP_FILE}.part"
echo "[$(date -Iseconds)] WARNING: filestore backup failed; database dump kept"
fi
fi

# Weekly backup (Sunday)
if [ "${DAY_OF_WEEK}" = "7" ]; then
cp "${DAILY_DIR}/${BACKUP_FILE}" "${WEEKLY_DIR}/"
if [ "${FILESTORE_BACKUP_CREATED}" = "1" ] && [ -f "${DAILY_DIR}/${FILESTORE_BACKUP_FILE}" ]; then
cp "${DAILY_DIR}/${FILESTORE_BACKUP_FILE}" "${WEEKLY_DIR}/"
fi
echo "[$(date -Iseconds)] Weekly backup saved"
fi

# Monthly backup (1st of month)
if [ "${DAY_OF_MONTH}" = "01" ]; then
cp "${DAILY_DIR}/${BACKUP_FILE}" "${MONTHLY_DIR}/"
if [ "${FILESTORE_BACKUP_CREATED}" = "1" ] && [ -f "${DAILY_DIR}/${FILESTORE_BACKUP_FILE}" ]; then
cp "${DAILY_DIR}/${FILESTORE_BACKUP_FILE}" "${MONTHLY_DIR}/"
fi
echo "[$(date -Iseconds)] Monthly backup saved"
fi

Expand All @@ -76,6 +132,12 @@ find "${WEEKLY_DIR}" -name "*.dump" -type f -mtime +$((BACKUP_KEEP_WEEKS * 7)) -
# Remove monthly backups older than BACKUP_KEEP_MONTHS months (approximate: 30 days per month)
find "${MONTHLY_DIR}" -name "*.dump" -type f -mtime +$((BACKUP_KEEP_MONTHS * 30)) -delete 2>/dev/null || true

# Filestore retention stays outside the BACKUP_FILESTORE guard so archives age out
# even after the feature is turned back off.
find "${DAILY_DIR}" -name "*_filestore_*.tar*" -type f -mtime +${BACKUP_FILESTORE_KEEP_DAYS} -delete 2>/dev/null || true
find "${WEEKLY_DIR}" -name "*_filestore_*.tar*" -type f -mtime +$((BACKUP_FILESTORE_KEEP_WEEKS * 7)) -delete 2>/dev/null || true
find "${MONTHLY_DIR}" -name "*_filestore_*.tar*" -type f -mtime +$((BACKUP_FILESTORE_KEEP_MONTHS * 30)) -delete 2>/dev/null || true

# Report disk usage
echo "[$(date -Iseconds)] Backup sizes:"
du -sh "${DAILY_DIR}" "${WEEKLY_DIR}" "${MONTHLY_DIR}" 2>/dev/null || true
Expand Down
8 changes: 8 additions & 0 deletions docker/docker-compose.nginx.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -423,10 +423,18 @@ services:
BACKUP_KEEP_DAYS: ${BACKUP_KEEP_DAYS:-7}
BACKUP_KEEP_WEEKS: ${BACKUP_KEEP_WEEKS:-4}
BACKUP_KEEP_MONTHS: ${BACKUP_KEEP_MONTHS:-6}
# Filestore backup (opt-in; see docker/README.md for sizing)
BACKUP_FILESTORE: ${BACKUP_FILESTORE:-false}
BACKUP_FILESTORE_COMPRESS: ${BACKUP_FILESTORE_COMPRESS:-false}
FILESTORE_SRC: ${FILESTORE_SRC:-/odoo_data/filestore/openspp}
BACKUP_FILESTORE_KEEP_DAYS: ${BACKUP_FILESTORE_KEEP_DAYS:-${BACKUP_KEEP_DAYS:-7}}
BACKUP_FILESTORE_KEEP_WEEKS: ${BACKUP_FILESTORE_KEEP_WEEKS:-${BACKUP_KEEP_WEEKS:-4}}
BACKUP_FILESTORE_KEEP_MONTHS: ${BACKUP_FILESTORE_KEEP_MONTHS:-${BACKUP_KEEP_MONTHS:-6}}
volumes:
- ./backup.sh:/backup.sh:ro,z
- ./backup-entrypoint.sh:/backup-entrypoint.sh:ro,z
- backup_data:/backups:rw,z
- odoo_data:/odoo_data:ro,z

@kneckinator kneckinator Sep 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing here has been executed yet, including by CI.

Both test-plan checkboxes are unticked and the note on #66 says the runtime test needs a production stack — so as far as I can tell this code has never run. CI hasn't covered the gap either: the head repo is Tarekchehahde/OpenSPP2, and workflows on fork PRs need maintainer approval, so not even pre-commit has looked at it. That part is on us, not you — I'll get the workflow run approved. mergeable_state is clean and docker/ has drifted by only two unrelated commits since your branch point, so there's nothing to rebase.

Worth exercising this stack specifically rather than the Traefik one, because it's the more constrained of the two: read_only: true, cap_drop: [ALL], and a cpus: "0.5" limit. I did confirm the two things most likely to bite — root plus DAC_OVERRIDE can read the filestore, and tar writing into /backups is unaffected by read_only — but the end-to-end run is still worth doing once by hand, with BACKUP_FILESTORE=true, before this merges.

networks:
- openspp-prod
restart: unless-stopped
Expand Down
8 changes: 8 additions & 0 deletions docker/docker-compose.production.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -329,10 +329,18 @@ services:
BACKUP_KEEP_DAYS: ${BACKUP_KEEP_DAYS:-7}
BACKUP_KEEP_WEEKS: ${BACKUP_KEEP_WEEKS:-4}
BACKUP_KEEP_MONTHS: ${BACKUP_KEEP_MONTHS:-6}
# Filestore backup (opt-in; see docker/README.md for sizing)
BACKUP_FILESTORE: ${BACKUP_FILESTORE:-false}
BACKUP_FILESTORE_COMPRESS: ${BACKUP_FILESTORE_COMPRESS:-false}
FILESTORE_SRC: ${FILESTORE_SRC:-/odoo_data/filestore/openspp}
BACKUP_FILESTORE_KEEP_DAYS: ${BACKUP_FILESTORE_KEEP_DAYS:-${BACKUP_KEEP_DAYS:-7}}
BACKUP_FILESTORE_KEEP_WEEKS: ${BACKUP_FILESTORE_KEEP_WEEKS:-${BACKUP_KEEP_WEEKS:-4}}
BACKUP_FILESTORE_KEEP_MONTHS: ${BACKUP_FILESTORE_KEEP_MONTHS:-${BACKUP_KEEP_MONTHS:-6}}
volumes:
- ./backup.sh:/backup.sh:ro
- ./backup-entrypoint.sh:/backup-entrypoint.sh:ro
- backup_data:/backups
- odoo_data:/odoo_data:ro

@kneckinator kneckinator Sep 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new settings aren't reachable from configuration.

FILESTORE_SRC is documented in the script header (line 14) but isn't wired anywhere, so there's no supported way to override it short of editing the compose file. The same will apply to BACKUP_FILESTORE and the BACKUP_FILESTORE_KEEP_* knobs. Please add all five to the backup service environment: block in both compose files, e.g.:

      # Filestore backup (opt-in; see docker/README.md for sizing)
      BACKUP_FILESTORE: ${BACKUP_FILESTORE:-false}
      BACKUP_FILESTORE_KEEP_DAYS: ${BACKUP_FILESTORE_KEEP_DAYS:-${BACKUP_KEEP_DAYS:-7}}
      BACKUP_FILESTORE_KEEP_WEEKS: ${BACKUP_FILESTORE_KEEP_WEEKS:-${BACKUP_KEEP_WEEKS:-4}}
      BACKUP_FILESTORE_KEEP_MONTHS: ${BACKUP_FILESTORE_KEEP_MONTHS:-${BACKUP_KEEP_MONTHS:-6}}

and give them a block in docker/.env.production.example under the existing BACKUPS heading (around line 141), which currently documents only BACKUP_SCHEDULE.

One trap to avoid: backup-entrypoint.sh writes /etc/profile.d/pg_env.sh, which looks like the place to add these, but it isn't. busybox crond execs jobs as children of the daemon, so they inherit the container environment directly and never source that file — it's effectively dead code for the cron path. The compose environment: block is what actually reaches backup.sh.

networks:
- openspp-prod
restart: always
Expand Down
Loading