loadcredential, a trivial python package for using secrets

Context 

The default way to run services on Linux machines (ignoring the existence of Docker and consorts) is to use Systemd. Those services are described and confidured through service files looking like:

[Unit]
Description=Nix Daemon
Documentation=man:nix-daemon https://docs.lix.systems/manual/lix/stable
RequiresMountsFor=/nix/store
RequiresMountsFor=/nix/var
RequiresMountsFor=/nix/var/nix/db
ConditionPathIsReadWrite=/nix/var/nix/daemon-socket

[Service]
ExecStart=@/nix/store/31ddnl7rvfhi9fa4s33lxh7dzvjad9dp-lix-2.92.0-dev/bin/nix-daemon nix-daemon --daemon
KillMode=process
LimitNOFILE=1048576
TasksMax=1048576

[Install]
WantedBy=multi-user.target

Some services need to have access to secret values (e.g. login details to connect to a remote server, private keys, etc), one way to pass them is via the EnvironmentFile directive. As this can be cumbersome, systemd devised the Credentials mechanism to have a better management of secrets.

In short, this mechanism allows placing secrets in files under a special directory, visible only to the running service ($CREDENTIALS_DIRECTORY).

$CREDENTIALS_DIRECTORY/
 ├─ SECRET_A
 ├─ SECRET_B
 └─ SECRET_C

Then, by reading those files, the service can access those secrets.

However infinitely useful this mechanism is, most frameworks lack a way to use it, and it is necessary to reinvent the (simple) wheel for each service we want to develop.

The python package 

Henceforth, I wrote this simple wheel (get it ?) that can be reused in all python project: loadcredential.
It allows for reading systemd's Credentials, and falling back to environment variables for more flexibility.

The API 

The loadcredential api has a single endpoint:

from loadcredential import Credentials

creds = Credentials()

Its signature is the following:

class Credentials:
    encoding: str = "utf-8"
    env_fallback: bool = True
    env_prefix: str = ""

    def __getitem__(self, key: str) -> str:
        pass

    def get(self, key: str, default: T = None) -> str | T:
        pass

    def get_json(
        self, key: str, default: Any = None, fail_missing: bool = False
    ) -> Any:
        pass

Parameters 

The parameters of the instantiation are the following:

• encoding, the encoding of the secrets
• env_fallback, whether or not to look up the corresponding environment variables when no credential is found
• env_prefix, what string to prefix to environment variables when doing the lookup. E.g., when env_prefix=LD_, the non existence of the KEYFILE secret will induce the lookup of the LD_KEYFILE environment variable.

Methods 

• __getitem__(key), the most basic method looks for a secret, and fails when nothing is found.
• get(key, default=None), similar to dict's method of the same name, looks for a secret and returns it, or the default when nothing is found
• get_json(key, default=None, fail_missing=False), this allows for retreiving secrets containing json values, and deserializing them

Example 

My main use for this library is to write Django applications, where production secrets must be fetched at runtime. The previous method used was to put a file secrets.py next to the settings, which is cumbersome at best. This has been replaced by settings looking like:

"""
Django settings for the toto project.
"""

from pathlib import Path

from django.contrib.messages import constants as messages
from django.utils.translation import gettext_lazy as _
from loadcredential import Credentials

credentials = Credentials(env_prefix="TOTO_")

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent

# WARNING: keep the secret key used in production secret!
SECRET_KEY = credentials["SECRET_KEY"]

# WARNING: don't run with debug turned on in production!
DEBUG = credentials.get_json("DEBUG", False)

ALLOWED_HOSTS = credentials.get_json("ALLOWED_HOSTS", [])

ADMINS = credentials.get_json("ADMINS", [])

###
# E-Mail configuration

DEFAULT_FROM_EMAIL = credentials["FROM_EMAIL"]
EMAIL_HOST = credentials.get("EMAIL_HOST", "localhost")
EMAIL_HOST_PASSWORD = credentials.get("EMAIL_HOST_PASSWORD", "")
EMAIL_HOST_USER = credentials.get("EMAIL_HOST_USER", "")
EMAIL_PORT = credentials.get_json("EMAIL_PORT", 465)
EMAIL_USE_SSL = credentials.get("EMAIL_USE_SSL", False)
SERVER_EMAIL = credentials["SERVER_EMAIL"]