Offsite Backups with Restic, Borg and S3 Storage

Imagine getting this message: “Our VPS died, and I think the backups are on the same server.” It’s a classic trap: we all mean to set up offsite backups, but it’s surprisingly easy to postpone. And then one quiet Tuesday becomes “why is everything broken?” Tuesday.

This guide compares two offsite backup workflows: Restic directly to S3-compatible storage, and a Borg SSH repository with a separate passive copy in object storage. It covers repository setup, encryption, retention and recovery conditions. The command examples are checked against Restic 0.18 and Borg 1.2.8 documentation; use compatible maintained versions and verify the commands against the installed release.

Choose a Separate Failure Domain

Consider database corruption after a plugin update, followed by failure of the disk array holding local snapshots. An offsite Restic repository in an S3-compatible bucket could provide a recovery copy, provided a usable backup predates the corruption. Retention and restore checks matter as much as having the bucket.

Here’s the thing: offsite storage reduces the blast radius when it is in a separate failure domain. Ransomware can still delete remote backups if compromised credentials permit it; offsite placement alone is not deletion protection. And if that “somewhere else” is object storage via an S3-compatible endpoint, you gain an object-storage interface whose durability, capacity and access features still depend on the provider and configuration. There’s also the cost angle. With a smart retention strategy—keeping dailies, weeklies, and monthlies—you’re not paying for a hoarder’s attic full of stale snapshots.

When I say “S3-compatible,” I mean anything that speaks the S3 API: major cloud providers, specialty object storage vendors, and even self-hosted solutions. The beauty of this is you can pick what fits your budget and geography, and Restic can use supported S3 endpoints, while Borg needs the intermediate repository and replication workflow described below. Check the provider’s API compatibility and required features rather than assuming every S3-compatible service behaves identically.

Restic and Borg: How They Think (and Why That Matters)

Restic and Borg are separate deduplicating backup tools. Restic encrypts repository data; Borg supports encryption, including the mode selected below. Restic calls retained backups snapshots, while Borg calls them archives. Both require a usable source: a file backup is not an atomic snapshot of a changing filesystem or a transactionally consistent live database.

Restic supports S3 directly. The Borg 1.x workflow here uses an SSH repository followed by a separate, consistent copy to S3. Borg’s documentation recommends independently initialized repositories for independent redundancy; a byte-for-byte mirror shares the original repository ID. Treat that mirror as a passive disaster-recovery copy, not another independently writable Borg repository. Mounting S3 through FUSE does not establish the locking and filesystem semantics Borg requires.

Compare the number of maintained components and the complete recovery path. Restic can read the S3 repository directly. Recovering from the Borg object-storage copy first requires downloading a consistent repository onto a suitable filesystem, then validating it and extracting data. Include that additional transfer and temporary capacity in the recovery budget.

Configure Restic for S3-Compatible Storage

Prepare a dedicated repository for the single web01 backup set used below. Create the bucket, check endpoint compatibility and permissions, and choose a strong repository password. Use controlled secret injection, a restricted password file or a supported secret command for automation. Environment-variable placeholders illustrate configuration; they are not a reason to put real credentials into interactive history or source control.

Environment variables that keep your setup tidy

export RESTIC_REPOSITORY="s3:https://s3.example.com/my-backups"
export RESTIC_PASSWORD="use-a-long-unique-passphrase-here"
export AWS_ACCESS_KEY_ID="YOURACCESSKEY"
export AWS_SECRET_ACCESS_KEY="YOURSECRETKEY"
export AWS_DEFAULT_REGION="us-east-1"

The values above are placeholders: keep real secrets out of shell history and source control, and retain the repository password securely outside the backed-up server. The repository path format is the little nuance to get right. Restic supports S3 natively, so you can point to an S3 endpoint over HTTPS with the bucket name at the end. Use the provider’s actual region instead of assuming us-east-1; temporary AWS credentials also need AWS_SESSION_TOKEN. Ensure the chosen bucket and prefix are dedicated to this repository. Initialize it only for first setup, after validating the destination:

restic init

It will ask for the password (or read it from the environment variable). Restic encrypts file contents and backup metadata before upload, hiding filenames and contents from the provider, though object sizes and access patterns remain visible.

Your first backup: keep it targeted, tag it for sanity

restic backup \
  --tag server:web01 \
  --tag type:daily \
  /etc /var/www /var/lib/mysql-dumps

Before either backup command, produce completed, consistent database dumps in /var/lib/mysql-dumps using a method appropriate to the database engine. Confirm the dump job succeeded and publish completed files without letting the backup read a half-written dump. Neither Restic nor Borg creates those dumps. Verify that the account can read every required path; an unreadable file or partial backup needs investigation. Tags identify the source and role but do not make a database copy consistent.

Inspect the Recorded Snapshots

restic snapshots

Check the snapshot’s host, paths, timestamp and recorded result. The example repository contains one host’s backup set. In a shared repository, use the appropriate filters and inspect the selected snapshot before restoration; a latest snapshot may belong to a different source.

Retention that doesn’t hoard your budget

The example policy retains one snapshot from each of the last seven days, five weeks and twelve months that contain snapshots, using each period’s most recent snapshot. Rules can retain overlapping selections, and gaps in backup history are not filled. Restic 0.18 groups by host and paths by default and applies retention to each group. Preview the actual retained and removed snapshots before deletion:

restic forget --dry-run \
  --keep-daily 7 \
  --keep-weekly 5 \
  --keep-monthly 12

# After reviewing the preview and confirming a usable current backup:
restic forget \
  --keep-daily 7 \
  --keep-weekly 5 \
  --keep-monthly 12 \
  --prune

The second command removes snapshot references and prunes data no longer referenced by retained snapshots. It is a deletion step, so keep the preview and the execution scope aligned and prevent intervening jobs from changing the expected selection. Pruning can take time and locks the repository; schedule it so backup jobs can complete. A new snapshot alone does not prove that a partial or corrupted source is safe to replace older history.

Restore to an Isolated Test Location

# list what's inside the latest snapshot
restic ls latest

# restore a folder to a temp directory
restic restore latest --include /var/www --target /tmp/restore-test

The restore example writes the included tree beneath /tmp/restore-test, such as /tmp/restore-test/var/www. Use a new empty destination and inspect latest before running it; pin the reviewed snapshot ID for a real recovery. Verify content, ownership, permissions and required extended attributes. For application recovery, restore the database consistently and test startup and dependencies. A successful file extraction does not prove a complete service recovery.

Borg with an S3 Twist: SSH First, Then Replicate

This workflow uses Borg 1.x, at least 1.2, with compatible client and SSH-target installations; it is not a Borg 2 recipe. The server must have the intended repository path, sufficient storage and a restricted SSH account. Verify the SSH host key and required command permissions before initialization. Back up to that repository, then create a passive S3 mirror while all writers are stopped or from a consistent filesystem snapshot. A Borg lock held only during one command does not protect a later external copy; coordination must cover the entire copy interval.

Initialize a Borg repository with encryption

export BORG_REPO="ssh://backup@backup-vps.example.com:22/home/backup/borg-repos/web01"
export BORG_PASSPHRASE="use-a-long-unique-passphrase-here"

borg init --encryption=repokey-blake2 "$BORG_REPO"

Replace the example hostname and passphrase through your controlled configuration process. The absolute remote repository path is /home/backup/borg-repos/web01; initialize a new repository only once. repokey-blake2 stores the passphrase-protected key in the repository configuration. Keep a separate protected key export and the passphrase for recovery. Encryption does not stop a credentialed attacker deleting the repository, and changing the passphrase does not rotate the underlying data-encryption key.

Create a Named Archive with Compression

borg create --stats --compression zstd,10 \
    "$BORG_REPO"::"{hostname}-{now:%Y-%m-%d-%H%M%S}" \
    /etc /var/www /var/lib/mysql-dumps

Ensure the archive name is unique; a timestamp does not prevent two starts in the same second. zstd,10 is an illustrative compression level, not a recommended default. Compare CPU time, size and backup duration for the installed build and representative data. Deduplication can reduce stored data, but scanning and changed content still consume resources. Confirm all required files and completed dumps were included before retention runs.

Prune old archives (same idea as Restic)

borg prune -v --list --dry-run "$BORG_REPO" \
  --keep-daily 7 --keep-weekly 5 --keep-monthly 12

# After reviewing the preview and confirming a usable current archive:
borg prune -v --list "$BORG_REPO" \
  --keep-daily 7 --keep-weekly 5 --keep-monthly 12

This is a dedicated web01 repository. Borg prune otherwise considers every archive in the repository; use an appropriate –glob-archives filter when a repository contains distinct backup sets. Daily, weekly and monthly rules select archives from periods with backups, not every backup within those periods. Unlike simply deleting archive references, a separate compact command is needed to reclaim repository disk space in this Borg version.

Replicate the repository into S3-compatible object storage

After a successful reviewed prune, compact from the same client that already has BORG_REPO configured. Then switch explicitly to the repository VPS for the external copy. Keep the repository quiescent throughout that copy, or copy a consistent filesystem snapshot. A directory sync is not atomic: interruption can leave a mixed or incomplete destination and sync can propagate deletions. Retain an independent recoverable previous generation before replacing a mirror.

# On the backup client, after successful prune:
borg compact "$BORG_REPO"

After that command finishes, the following preview runs on the repository VPS as an account permitted to read the entire repository. The rclone remote named s3 must already be configured for the correct provider, credentials and bucket. BORG_REPO_LOCAL is the local filesystem path on this VPS, not the client’s SSH URL.

# On the repository VPS; all repository writers remain stopped:
BORG_REPO_LOCAL="/home/backup/borg-repos/web01"
rclone sync --dry-run "$BORG_REPO_LOCAL" s3:my-bucket/web01

The preview copies nothing. Review additions and deletions against the intended dedicated prefix; only then run the same sync without –dry-run, keeping the source unchanged throughout. Check completion and retain the prior generation until the new mirror has been validated. Borg compact itself does not require the encryption key; copying repository bytes does not require placing the decryption secret on the storage VPS.

Record backup, maintenance and mirror outcomes separately. A failed copy may leave the live SSH repository usable while the S3 destination is incomplete; do not report offsite success until the mirror is validated. Rehearse downloading a whole consistent repository into an isolated recovery environment, checking it and extracting selected data. Never independently update both same-ID copies, and do not bypass Borg’s cache/security warnings merely to accept an older mirror.

Versioning, Retention, and Encryption: The Real-World Balancing Act

Let’s translate buzzwords into something useful. Versioning, in the context of Restic and Borg, is simply the presence of multiple snapshots/archives over time. You don’t have one backup—you have a story of your data’s changes. That’s gold when you realize something went wrong three days ago, not five minutes ago. The more your changes matter, the tighter your retention around the present should be. Dailies save you from today’s typo, weeklies catch the slow-moving mistakes, and monthlies guard against the long-tail “oops we never noticed.”

But there’s a second kind of versioning: bucket-level versioning. Where supported, object versioning can preserve older versions after overwrites or ordinary deletes. It does not stop credentials with version-deletion permissions from permanently removing them. This isn’t the same thing as Restic/Borg snapshots; it’s a safety net under your safety net. It can increase storage usage, so it’s something you do mindfully. If you want to go even further, look at S3 Object Lock for immutability—think of it as “write once, then hands off.” When ransomware is part of your threat model, immutability can help, but confirm provider support and tool compatibility: Object Lock protects specific versions; ordinary delete markers and new versions can still be created. Depending on mode, permissions and API operations, retention can prevent permanent version deletion and change cleanup costs. Repository recovery requires a consistent set of object versions, not just a few undeleted objects.

Now, encryption. Restic encrypts repository contents client-side using keys protected by a repository password; the object-storage provider receives ciphertext. Borg encrypts when initialized in an encrypted mode. With repokey-blake2, the encrypted key is stored in the repository, not only on the client; keep an independent key export and its passphrase for recovery. Both approaches mean you control the crypto. If your compliance team wants belts and suspenders, you can layer in server-side encryption on the bucket as well, but that’s defense-in-depth rather than a replacement for client-side encryption. The crucial part is key hygiene. The password or key material that decrypts your repository is a crown jewel. Treat it like one.

Retention is where cost meets peace of mind. A simple starting policy is to keep a week of dailies, a month or so of weeklies, and a year of monthlies. For volatile systems, hourly snapshots retained for a rolling day may help. Set frequency and retention to match recovery requirements: returning to Tuesday at 11 AM requires a retained backup from that time or suitable application recovery logs.

Small choices that help in big ways

Use stable tags to identify backup roles. Calendar retention uses timestamps; daily or monthly tags are not required. Restic 0.18’s default forget grouping is host and paths, with tags available through explicit grouping or filtering. Keep backup and maintenance scopes documented, especially when hostnames or paths change. Stagger jobs where useful, and measure their actual contention rather than assuming a quiet-hour schedule removes it.

Automation, Monitoring, Restore Drills, and Practical Guardrails

Use the scheduler already supported by your operating environment, such as systemd timers or cron. A later start time does not prove an earlier job has finished. Gate maintenance on acceptable backup results and serialize repository changes with external copying. Preserve failure status, record completion and alert when no successful backup arrives within the expected interval. Scheduling is separate from verification.

What to monitor (and what to ignore)

Monitor the last successful backup’s age, unreadable or skipped files, data growth, retention outcome, repository checks and offsite-copy completion. Alert on missed or failed jobs and investigate unexpected changes. If needed, the guide to set up simple uptime and alerting with Prometheus, Grafana, and Uptime Kuma provides monitoring context. Test notification delivery as well as the job itself.

Restore drills that won’t wreck your day

Start with an isolated directory restore and verify content and metadata. For the Borg S3 mirror, include the repository download and consistency check before extracting an archive; borg extract writes into the current working directory, so use a new empty target. Extend drills to database recovery and application startup. Record transfer volume, duration and assumptions; a small-file drill cannot establish a full-service recovery time.

A quick word on performance and bandwidth

Backup throughput is a dance between CPU (for encryption and compression), disk I/O, and network bandwidth. If your server is pegged during working hours, schedule backups late at night. If egress costs matter, compare the charged routes before adding a staging VPS: off-hours replication does not inherently reduce transfer charges, and another hop can add cost. Restic and Borg both let you exclude noisy paths; cut out caches, temporary files, and anything that regenerates automatically. Exclude a path only after proving how it will be recreated; required logs, uploads and configuration may not be disposable. Measure the resulting savings.

Costs: better to be deliberate than surprised

Object storage is usually a great deal, but versioning and immutability can multiply storage if you’re not watching. Keep an eye on unintentional churn (like huge changed log files). For Restic on S3-compatible storage, bucket versioning can be your safety net, but don’t forget it has a price tag. For Borg replication, pruning and compacting before a consistent mirror can reduce the live repository size, but compaction may also increase transfer churn. Versioned or locked buckets can retain deleted objects, so local cleanup does not necessarily reduce billed storage.

Security: least privilege and split secrets

Limit storage credentials to the needed bucket and prefix and separate backup creation from privileged maintenance where supported. Restic’s normal S3 workflow needs read, write and list operations plus lock cleanup; pruning needs deletion. Simply denying all deletes can break normal operation, so an append-only policy must be compatible with the tool. Keep repository passwords and Borg key exports outside the source server’s failure domain. Evaluate versioning, Object Lock or MFA-dependent deletion against automation and recovery requirements. The Restic documentation explains the supported backend and repository behavior.

Reality checks and regular maintenance

Run structural checks and schedule full data verification separately. Restic’s default check does not read every data pack; restic check –read-data does and can incur significant transfer. Borg’s default repository check includes size/CRC checks, while borg check –verify-data additionally decrypts, decompresses and cryptographically verifies file data. These checks require time and, where applicable, keys and network access. Neither replaces an application restore drill; investigate errors before pruning or attempting repair.

Record the Complete Operating Sequence

A complete run records source consistency preparation, backup completion, retention selection and maintenance, then the offsite-copy result where applicable. Restic writes directly to its S3 repository. Borg writes to the SSH repository; after coordinated maintenance, an unchanged repository or consistent filesystem snapshot is mirrored to S3. Validate that all stages succeeded and that a recoverable previous generation remains available. The next run must not overlap a still-running copy.

Recovery planning includes storage access, decryption material, a clean target, compatible software and application dependencies. Check that an operator can obtain these without relying on the failed source machine. Replication and encryption address specific risks; neither makes an untested backup automatically recoverable.

Keep Configuration and Recovery Ownership Clear

First, name your buckets and repositories clearly—future you will thank past you at 3 AM. Second, document the backup paths, exclusions, and retention policy in a short README that lives next to your scripts. Include tested recovery steps, version requirements and the operator responsible for each stage. Third, don’t mix credentials across environments. Production should have its own keys and its own buckets so staging can’t rewrite your lifeboat by mistake. And finally, when in doubt, start simple. A single Restic repository and a basic retention policy beats a grand design you never finish.

For deeper background on Borg itself, Borg’s official docs are thorough without being overwhelming. They’ll walk you through edge cases like partial file restores, patterns for excludes, and repo maintenance tasks you’ll probably only need once in a blue moon—but when you need them, you’ll be glad they’re there.

Choose and Test One Complete Recovery Path

Choose direct Restic-to-S3 or the Borg 1.x SSH-plus-passive-mirror workflow according to the recovery steps your team can maintain. Keep client-side encryption and the required secrets accessible during an incident. Snapshot retention, bucket versioning and object retention serve different purposes; recovering a repository requires a consistent set of objects, not just rolling back an isolated file.

Begin with a small dataset, restore it into an isolated target, then expand coverage and establish a measured retention policy. Plan recurring directory and application drills based on recovery requirements; monthly testing is an example, not a universal rule. Monitor both failed runs and overdue success, and revisit the plan when source data, providers or software versions change.