It started with a small, uncomfortable thought: I had secrets absolutely everywhere. Database passwords sitting in plain text inside Portainer stack definitions. API tokens copied into my own notes so I could pick work back up. A scratch folder quietly holding a CA bundle and half a dozen live credentials. Nothing was leaking, exactly, but the whole arrangement had the feel of a wizard leaving spellbooks open on every flat surface in the Unseen University. Sooner or later something reads the wrong page.

So I set out to give the homelab a proper secrets manager. What looked like an evening’s job turned into an estate-wide migration, a detour through a cluster-wide outage I did not know I had, a different secret-handling trick for almost every single application, and a licensing wall right at the finish. This is what actually happened.

The scattered hoard

The homelab runs a stack of services behind Traefik on docker1, a Proxmox VM, all managed through Portainer. NetBox, Authentik, Forgejo, Immich, the monitoring stack, a dashboard, a remote-desktop gateway. A survey of where secrets actually lived turned up a reassuring surprise and one clear offender.

The reassuring part: most stacks were already using Docker or file based secrets, thanks to earlier tidying. The offender: a scattering of plain text values, some inline in compose files, most tucked into Portainer’s per-stack environment variables where a ${VAR} reference hides a real password in Portainer’s own database. NetBox had a fistful. Traefik had a Cloudflare API token sitting inline. Forgejo, Immich and Authentik each kept their database and mail passwords there too.

Choosing the vault

The shortlist was Infisical, OpenBao (the open-source fork of HashiCorp Vault) and Vaultwarden. OpenBao is the most powerful and the most demanding to run. Vaultwarden is lovely for human passwords but poor at feeding secrets into running containers. Infisical sat in the middle: modern, Docker-native, a genuinely pleasant web interface, and a machine-identity model made for exactly this kind of automation. Infisical it was. Remember that choice, because it comes back to bite at the end.

Deploying it was straightforward: a small stack with its own Postgres and Redis, fronted by Traefik with an internally issued certificate. One lesson worth passing on, though. There are two image tags floating around, and the older one speaks an out-of-date API that the current command-line agent refuses to talk to. Pin the current image, not the legacy one, or you will spend a while wondering why a fresh install cannot talk to itself.

A detour through broken DNS

The new service came up healthy but its hostname would not resolve. Chasing that led somewhere unexpected. The little service that keeps internal DNS in step with Traefik had been quietly failing every sixty seconds, for who knows how long, with a Kerberos error: the password on its Active Directory service account had expired. Nothing new could get a DNS record. The estate had been running on records created before the account lapsed.

This is the homelab equivalent of discovering the reason the post never arrives is that the clacks tower two hills over has been on fire since Tuesday. A password reset on the domain controller (set to never expire this time), a fresh keytab, and a rotation of the Docker secret that carries it, and DNS started registering new services again. One small task, one latent disaster averted. On we go.

The Portainer trap

Here is the technical heart of the whole exercise, and the one thing I would tell anyone attempting this. My first plan was neat: an agent renders each stack’s secrets into a single env file, and the containers read it with Docker Compose’s env_file. It failed instantly with a baffling error about a path not being a directory.

The reason is subtle and important. Portainer’s compose engine reads env_file from inside its own container, not from the Docker host. A host path like /docker/netbox/secrets/netbox.env simply does not exist from where Portainer is looking. Bind mounts, by contrast, are resolved by the Docker daemon itself, so those work fine. The fix, then, is to never use env_file for this. Render one file per secret, bind-mount the directory into the container as /run/secrets, and let each application read its secrets from files. That single realisation shaped everything that followed.

One agent per stack

The pattern that emerged is tidy. Each stack gets a small Infisical agent sidecar. It authenticates with a machine identity (the one bootstrap credential, kept in root-owned files, not in the compose), pulls that stack’s secrets, and renders them as individual files into a directory that is bind-mounted into every service as /run/secrets. The applications then read their credentials from those files. No plain text in the compose, and rotating a secret becomes: change it once in Infisical, wait for the agent to rewrite the file, restart the container.

With the mechanism proven on NetBox (a five-service stack with a live database, migrated without losing a single request), the rest of the estate followed one careful stack at a time.

Every application hides its secrets differently

If there is a running joke in this project, it is that no two applications agree on how to read a secret from a file. Verifying each one in a throwaway container before touching production became a ritual. The tally:

  • PostgresPOSTGRES_PASSWORD_FILE, pointing at the file. Civilised.
  • Redis: no file support at all, so the start command reads it inline with --requirepass "$(cat /run/secrets/redis_password)".
  • Traefik (for the Cloudflare token): append _FILE to the environment variable name.
  • Immich: same idea, DB_PASSWORD_FILE.
  • Forgejo: its own scheme, FORGEJO__database__PASSWD__FILE, applied to its config on every start.
  • Authentik: a file:///path URI as the value of the environment variable itself.
  • Homepage: none of the above, because its dashboard widget key lives in a Docker label, and labels are static strings that cannot read files.

That last one deserves a note. The Forgejo dashboard widget’s API token sat in a container label, so the file trick was useless. The clean fix was to move the widget definition into the dashboard’s own configuration, where Homepage supports a {{HOMEPAGE_FILE_...}} placeholder that reads a file rendered by an agent. As a bonus, the widget now actually works: it had been silently inert because label-based discovery was never configured in the first place.

Handling the crown jewels

Authentik was left until last, deliberately, because it is the identity provider that fronts single sign-on for half the estate. Break it and the failure cascades: no logins to the photo library, the git server, the remote-desktop gateway. Its database and mail passwords were routine. Its SECRET_KEY was not.

That key signs sessions and, more dangerously, encrypts other secrets stored in the database. Change it by so much as a stray newline and every stored provider secret becomes undecryptable rubbish. So before deploying anything, I hashed the original value and the freshly rendered file and compared them: identical, to the byte. After the switch, the proof it had survived was watching Authentik’s embedded outpost happily authenticate to itself using a token derived from that very key. Everything came back healthy, single sign-on intact, not a session dropped.

The wall

With every secret migrated, one satisfying finishing touch remained: let me log into Infisical itself with single sign-on through Authentik, like everything else. I built the Authentik side end to end. Provider, application, group restriction so only the right admin group can get in, discovery endpoint live and verified. Then I asked Infisical to accept the configuration and it replied, politely:

Failed to create OIDC SSO configuration due to plan restriction. Upgrade plan to update SSO configuration.

And there it is. Single sign-on is a paid, enterprise-tier feature in Infisical. The self-hosted community edition I had just spent the day filling with secrets will not federate logins without a licence that is quoted, not listed, and clearly aimed at companies rather than one person and a rack of Discworld-named virtual machines.

It is a fair reminder to check the fine print before you commit. The irony is not lost on me: OpenBao, the heavier option I passed over precisely because it looked like more work, gives OIDC login away for free along with the rest of its feature set. For a single-admin homelab tool, though, paying an enterprise licence just to sign into the vault I already control is poor value. Local admin login stays, and if I ever want an SSO-style front door I can put Authentik forward-auth in front of it in Traefik, for nothing.

What I would keep

The secrets are where they should be now: one place, encrypted at rest, out of the compose files and out of my notes. The pattern (per-secret files, bind-mounted, one small agent per stack) is boring in the best way and ports to almost anything. And I have a fresh appreciation for two truths of the homelab. The tool that markets itself as the friendly one will still gate the feature you want behind a sales call. And chasing one small annoyance, a hostname that would not resolve, is how you find the fire two hills over.

The one secret that cannot live in the secrets manager, of course, is the key the secrets manager uses to encrypt everything else. That one goes in the password manager, written down, kept safe. Even in Ankh-Morpork, somebody has to hold the master key.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.