The Calm Way to Secrets on a VPS: GitOps with sops + age, systemd Magic, and Rotation You Can Sleep On

So, Here’s the Secret About Secrets

Ever had that moment when you SSH into a VPS, peek at an old .env file, and instantly feel your shoulders tighten because you have no idea who last touched it, whether it leaked into a backup, or if the values even match production? I’ve been there. A few years back, I walked into a client’s environment where each server had its own shadow copy of secrets, half of them outdated, and nobody was brave enough to rotate anything. The team moved fast, the app shipped features, but the secrets—API keys, database passwords, webhook tokens—were like dusty skeletons in the closet.

That’s when I started leaning hard into GitOps and found a workflow that finally felt sane: sops + age for encryption in Git, with a tiny sprinkle of systemd for runtime decryption and safe reloads. It wasn’t magic, but it felt like it. No more guessing, no more strings stuffed into CI variables that silently rot, no more “please don’t cat that file on prod.” Just a friendly, predictable flow that lives alongside your code and deploys like any other artifact.

In this post, I’ll show you exactly how I manage secrets on a VPS with sops and age, how I wire that into systemd so services get the right environment at the right time, and how I rotate without breaking anything—or losing sleep. We’ll talk practical repo layout, first-boot bootstrapping, reloads that don’t cause downtime, and a rotation playbook that actually works. By the end, you’ll have a quiet confidence about secrets again.

Why GitOps + sops + age Feels Like a Breath of Fresh Air

Here’s the thing: people love Git because it tells the story of change. Secrets need that story too, but they also need to stay secret. sops gives us that sweet spot. It stores encrypted values right inside YAML or JSON, and it does it in a way that plays nicely with Git diffs, reviews, and branches. And the encryption part? That’s where age shines—simple, modern, and fast. The first time I replaced a gnarly GPG setup with age recipients, I felt like someone took training wheels off my workflow. Keys were just… readable, rotate-able, understandable.

What I love most is the flow: developers can open a pull request with updated secrets, reviewers can see exactly what changed (encrypted, yes, but key placement and structure still readable), and the server decrypts at runtime using its own private key. That last part is important—we never ship the secret in plain text over CI or stash it in some shared bucket that turns into a quiet liability. The VPS itself is the one with the key. The repository stays safe to clone. And if you lose the server? Rotate the recipient, reencrypt, redeploy, and move on.

For installs and docs, I like to keep the official resources handy. If you haven’t met these tools yet, take a quick skim of sops on GitHub and the lovely age project. And for the glue that keeps the runtime neat, the systemd documentation is worth bookmarking. We’ll lean on systemd just enough to keep decrypted secrets ephemeral and reloads predictable.

Bootstrapping the VPS: Keys, Rules, and a Calm First Commit

Generating and placing the age key

I keep decryption on the server. That means the VPS needs its own age keypair. On first boot, I’ll log in and create it:

sudo mkdir -p /etc/age
sudo age-keygen -o /etc/age/key.txt
sudo chmod 600 /etc/age/key.txt
sudo chown root:root /etc/age/key.txt

Age key files are just short and friendly. Open the file and you’ll see a public key line starting with age1. That’s the recipient you add to your repo so sops can encrypt for this VPS. I usually export the path for convenience:

export SOPS_AGE_KEY_FILE=/etc/age/key.txt

If you automate your VPS builds (and I think you should), you can bake this step into first-boot user-data or config management. I covered a similar first-boot approach in my write-up on how I bootstrap a VPS with cloud‑init and Ansible. The exact dance is up to you—what matters is the private key only lives on the node that needs to decrypt.

Teaching sops your intentions with .sops.yaml

In the Git repo, I add a .sops.yaml file that spells out who can decrypt which files. I like creation rules with clear patterns, so future-me doesn’t forget how secrets flow:

creation_rules:
  - path_regex: secrets/.*.yaml$
    encrypted_regex: '^(data|secrets)$'
    age: ["age1examplepublickeyfor-prod", "age1examplepublickeyfor-staging"]
  - path_regex: secrets/.*.env.sops$
    age: ["age1examplepublickeyfor-prod"]

You can mix and match patterns by environment. For example: staging files encrypt to the staging recipient, production to production. If you have multiple servers in a cluster, you can add multiple recipients. sops will encrypt the same value for each recipient, and any one of the matching private keys can decrypt. It’s like having multiple copies of the same safe combination, each sealed for a specific person.

Writing your first encrypted file

I usually start with a simple secrets/app.env.sops file. It can be a plain env file with key=value lines, or a tidy YAML blob—use what your services are happiest to consume. Let’s say it’s an env file:

DB_HOST=10.10.0.5
DB_USER=app
DB_PASS=please-rotate-me
API_TOKEN=super-secret

Now encrypt it:

sops --encrypt --in-place secrets/app.env.sops

And in Git, commit the encrypted file. It’ll look like a block of structured ciphertext. One of the best moments is when you realize that everything secret is now inside your repo, but still safe. No more DMing credentials. No more mystery variables.

Repo hygiene so you don’t shoot your foot

I always add basic protections:

– A strict .gitignore that rejects any decrypted paths like secrets/*.env or secrets/*.dec or temporary files in /run.
– Keep decrypted artifacts out of backups. If you write to /run, you’re already in good shape because it’s a tmpfs.
– Consider a pre-commit hook that barks if a plaintext .env slips in. Friendly barks save days.

Decryption at Runtime with systemd: Ephemeral and Predictable

Once your repo lands on the VPS, you want services to get fresh secrets at start and during rotation, without leaving decrypted files on disk. This is where systemd quietly shines. I like to decrypt into /run—it’s memory-backed and disappears on reboot. Then I point EnvironmentFile at that decrypted file.

Here’s a minimal service that does exactly that:

[Unit]
Description=My App
After=network.target

[Service]
Type=simple
EnvironmentFile=/run/myapp/app.env
ExecStartPre=/usr/bin/mkdir -p /run/myapp
ExecStartPre=/usr/bin/sops --decrypt 
  --output /run/myapp/app.env 
  /srv/myrepo/secrets/app.env.sops
ExecStart=/usr/local/bin/myapp
User=myapp
Group=myapp
# Hardening bits you can layer on as needed
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=full
ProtectHome=true

[Install]
WantedBy=multi-user.target

Two things I’ve learned: keep permissions tight on the decrypted file (either via chmod in an ExecStartPre step or by using a specific user for your service), and don’t log its contents by accident. Your logs should be helpful, not explosive.

If your app prefers YAML or JSON instead of env files, you can decrypt straight to that format and pass a different flag to your app. The trick is the same: decrypt to /run, then point your process at it. And if you need a short-lived process to read secrets and exit, consider sops exec-env to run a command with secrets injected temporarily—you’ll appreciate how clean it feels for one-off tasks.

One more layer: crash-safe decrypts

Every once in a while, with lots of restarts flying around, I’ll add a tiny lock so decrypts don’t race each other. It’s not strictly required, but it keeps journals less noisy:

ExecStartPre=/usr/bin/flock -n /run/myapp/.sops.lock 
  /usr/bin/sops --decrypt --output /run/myapp/app.env /srv/myrepo/secrets/app.env.sops

It’s a small courtesy to your future self during busy deploys.

Living the GitOps Life: Editing, Reviewing, and Deploying Secrets

The happy path looks like this: a teammate needs to update a token. They open the encrypted file with sops (which handles editing with your normal $EDITOR), change the value, save, and commit. The PR shows a clean structural diff where it matters, and reviewers know exactly what was changed and why.

On your CI, you do not decrypt. That’s the point. CI should lint, build, test, and ship artifacts—but secrets stay encrypted end-to-end until they reach the node that owns the key. Your deploy step brings the repo to the VPS (or pulls, depending on your strategy), and systemd does the rest when the service restarts. If you’re using configuration management like Ansible, the community.sops plugin can decrypt on the node too, as long as the age key is present there. It blends well with the same philosophy: decrypt where you consume.

In my experience, the biggest win isn’t even security—it’s clarity. You stop guessing where secrets live and which ones are current. It’s all in Git, with a story and a timeline you can trust.

Safe Rotation Without the Drama

Rotation used to terrify me. Then I started treating it like a deploy with a steady cadence. The safest way I’ve found looks like a short, two-step dance with sops and age recipients:

First, you add the new recipient (the new public key) to your .sops.yaml rules. Then re-encrypt affected files so they include both the old and the new recipients. Now both parties can decrypt. Deploy that. Confirm services still hum on all nodes. Only then, remove the old recipient and re-encrypt again. Deploy the removal. That’s it—two moves, no cliff.

In practice, it looks like this:

# Step 1: add the new recipient
# .sops.yaml: age: [old, new]
# Then re-encrypt
sops --encrypt --in-place secrets/app.env.sops  # or: sops updatekeys -r

# Deploy and verify decrypt works everywhere

# Step 2: remove the old recipient
# .sops.yaml: age: [new]
# Re-encrypt again
sops --encrypt --in-place secrets/app.env.sops  # or: sops updatekeys -r

# Deploy and verify again

When rotating individual secret values (like replacing DB credentials), I tend to duplicate the variable into a new key and let the app read from both for a short window. Think of it like changing the lock on your front door while leaving the old key working for a day. Once everything’s moved over, delete the old variable and push a final clean-up.

One more practical tip: if your app supports hot reloads for credentials, wire that into systemd so a rotation is a reload, not a restart. You can orchestrate a graceful handoff where no user ever notices.

Reloads That Respect Users: systemd.path + Oneshot Services

Rotations shouldn’t feel like pulling the plug. I like pairing a Path unit with a tiny oneshot service that re-decrypts and tells the app to reload. The flow is simple: whenever the encrypted file changes in Git and reaches the server, systemd sees the change, runs the decrypt, and triggers a reload. The main service never fully restarts unless you want it to.

Here’s a pattern that’s treated me well:

# /etc/systemd/system/myapp-reload.service
[Unit]
Description=Re-decrypt secrets and reload My App

[Service]
Type=oneshot
ExecStart=/usr/bin/sops --decrypt --output /run/myapp/app.env /srv/myrepo/secrets/app.env.sops
ExecStart=/bin/systemctl reload myapp.service

# /etc/systemd/system/myapp.path
[Unit]
Description=Watch encrypted secrets for My App

[Path]
PathChanged=/srv/myrepo/secrets/app.env.sops
Unit=myapp-reload.service

[Install]
WantedBy=multi-user.target

Enable both the main service and the path unit, and you’ve got a self-healing loop for secrets. It’s quietly elegant. When you merge a PR with a rotated token, the node decrypts and smoothly reloads without a blip.

Little Habits That Make This Rock-Solid

Keep decrypted material ephemeral

Always write decrypted files to /run. If you must write elsewhere, consider tmpfs mounts or cleanup hooks. And make sure systemd hardening options don’t block your app from reading the file—tight but thoughtful permissions are the game.

Audit who can decrypt

As your team grows, it’s tempting to add every machine’s recipient to every file. Resist. Keep .sops.yaml rules specific, and embrace per-environment or per-role files so a node only decrypts what it needs. You’ll thank yourself during incident reviews.

Practice rotation on staging

Do one dry run start-to-finish: add recipient, re-encrypt, deploy, remove recipient, re-encrypt, deploy. Practice the ritual so production is just muscle memory. While you’re at it, rotate a few secret values and confirm your reload path works just as expected.

Don’t leak in CI logs

Even if you never decrypt in CI, some build tools get chatty. Make sure no step tries to print decrypted content. If you have custom scripts, silence them on sensitive paths. Your future self, far from a pager, will appreciate the peace.

Know when to restart vs reload

Some apps read credentials only on startup. If that’s your world, pair decrypt + restart, but do it with grace—drain traffic, then restart, then un-drain. If your app supports live reloads, treat yourself to that smoother path and wire systemd accordingly.

A Real-World Story: The Day a Token Went Bad

One afternoon, an integration token got revoked on a client’s stack. It was the kind of moment that used to turn into a firefight: who has the token, who knows where it lives, how do we deploy it without breaking everything else? This time, the fix was almost boring.

We generated a new token, opened the encrypted file with sops, replaced the value, and pushed a PR. Another engineer reviewed it, we merged, the repo sync hit the VPS, and the systemd.path unit triggered our oneshot reload. The app kept running while it reloaded the config. The error counters flattened. No emergency call. No “who has permissions for that CI variable?” scramble. It felt odd in the best way—like discovering a new door to an old room that’s been there all along.

Edge Cases, Gotchas, and the Quiet Wisdom

There are always little corners to watch out for. If you run multiple apps on the same node, keep secrets separated by directory and permissions—don’t let one service read another’s decrypted files just because they’re neighbors. If you back up the box, double-check you’re not accidentally capturing /run. And if you mirror the repo to other places, make sure write access stays tight—encrypted or not, Git hygiene matters.

Another gentle reminder: if you ever need to move a server or rebuild it, that private key in /etc/age/key.txt is the crown jewel. Decide whether you’ll migrate it to the new node or rotate recipients and re-encrypt. Both approaches are fine—what matters is that you do it intentionally, with the same calm steps you practiced in staging.

Finally, let your team share in the ritual. A short readme in the repo that explains how to edit secrets with sops, how rotation works, and how services pick up changes, removes so much mystery. The fewer “secret keepers” you have, the safer your stack feels.

Putting It All Together (and Sleeping Better)

If we zoom out, the flow looks beautifully simple:

1) The VPS owns an age private key. Its public half lives in .sops.yaml as a recipient.
2) Secrets live in Git as encrypted files, edited with sops, reviewed like normal code.
3) Deploy brings the repo to the server. systemd decrypts to /run at start or on change, and your app reads a clean, ephemeral file.
4) Rotation is just two pull requests with a grace period between them—add recipient, then remove the old one. Or, for values, switch keys in app config and later prune the old ones.

That’s the calm path. You trade fear for a routine. And quietly, that routine raises your bar—fewer secrets drifting through chat, no plaintext accidents on disk, and a clear trail of who changed what and when.

If you’re starting fresh, try it on a smaller service first. Add a single encrypted env file, wire a systemd unit that decrypts to /run, and practice the reload dance. Once it clicks, roll it across the stack. And if you’re automating first boot, tuck the age key generation right into your provisioner so new nodes are born ready.

Hope this was helpful. If you want to layer this into a larger automation story, I’ve had a lot of fun pairing it with cloud-init and configuration management—the same calm approach extends nicely. Until next time, keep your secrets tidy and your reloads boring, because boring is quietly beautiful in production.