Mastering Environment Variables in Docker: Configuration vs. Secrets

Environment variables are convenient in Docker because they let the same image run in development, staging, and production with different settings. That convenience becomes risky when teams put passwords, signing keys, and API tokens in the same bucket as log levels and port numbers.

The clean mental model is simple: environment variables are fine for non-sensitive runtime configuration. Secrets should come from a secret store or mounted secret file, with limited access and a rotation plan.

Understanding Environment Variables for Configuration

Environment variables are a straightforward and widely adopted method for passing runtime configuration to applications, including those running in Docker containers. They allow you to modify an application's behavior without rebuilding the Docker image, making your containers more flexible and portable. This is ideal for non-sensitive, dynamic settings like application port numbers, debug flags, or third-party service URLs.

Methods for Passing Configuration Variables

Docker provides several ways to define and inject environment variables into your containers:

1. `ENV` Instruction in Dockerfile

The ENV instruction sets a default environment variable that will be available inside the container when it runs. This is suitable for variables that are unlikely to change or provide sensible defaults for your application.

FROM alpine:latest

ENV APP_PORT=8080
ENV DEBUG_MODE=false

COPY ./app /app
WORKDIR /app
CMD ["/app/start.sh"]

Tip: While ENV sets defaults, these can be overridden at runtime.

2. `-e` or `--env` Flag with `docker run`

When launching a single container, you can use the -e or --env flag to pass environment variables directly. This is common for ad-hoc testing or for providing specific settings that differ from the Dockerfile defaults.

docker run -d -p 80:8080 --name my_app_instance \
  -e APP_PORT=80 \
  -e DEBUG_MODE=true \
  my_app_image:latest

3. `env_file` in Docker Compose

For managing multiple environment variables, especially across several services defined in a docker-compose.yml file, the env_file option is very convenient. It allows you to load variables from one or more .env files, keeping your docker-compose.yml cleaner.

docker-compose.yml:

version: '3.8'
services:
  webapp:
    image: my_app_image:latest
    ports:
      - "80:8080"
    env_file:
      - ./config/app.env

./config/app.env:

APP_PORT=8080
DEBUG_MODE=false
API_ENDPOINT=https://api.example.com/v1

4. `environment` Key in Docker Compose

Alternatively, you can define environment variables directly within the environment section of a service in docker-compose.yml. This is often preferred for a small number of variables or for variables that are specific to a single service.

version: '3.8'
services:
  webapp:
    image: my_app_image:latest
    ports:
      - "80:8080"
    environment:
      APP_PORT: 8080
      DEBUG_MODE: false

The Pitfalls of Using Environment Variables for Secrets

While environment variables are excellent for configuration, they are fundamentally not secure for managing sensitive data (secrets) like database passwords, API keys, or private SSH keys. This is a critical security vulnerability that often gets overlooked, especially in development environments.

Why Environment Variables are Unsafe for Secrets:

Visibility via docker inspect: Anyone with access to the Docker host can easily view the environment variables of a running container using docker inspect <container_id>. This means your secrets are plainly visible in plain text.
```
# Example of exposing a secret (DO NOT DO THIS IN PRODUCTION)
docker run -d -e DB_PASSWORD=mysecretpassword --name insecure_app nginx:latest

# Anyone can see the password
docker inspect insecure_app | grep DB_PASSWORD
```
Process Snooping: Inside the container, other processes or users (if multiple users exist) might be able to read environment variables, especially if the application runs as root or has elevated privileges.
Logging and History: Environment variables can inadvertently end up in logs, CI/CD pipeline history, or shell history, leading to accidental exposure.
Image Layers: If you use ENV in a Dockerfile with a secret, that secret is baked into an image layer and remains there, even if you try to unset it in a later layer. This makes the secret retrievable from the image itself.
Accidental Sharing: .env files or docker-compose.yml files containing secrets are often committed to version control systems or shared inappropriately, leading to widespread exposure.

Warning: Treating sensitive information as regular environment variables is a common security misstep. Always assume environment variables are publicly visible on the host and within the container.

Securely Managing Secrets in Docker

To address the security shortcomings of environment variables for sensitive data, Docker provides dedicated secrets management capabilities, primarily through Docker Secrets (for Docker Swarm) and external tools like Docker Compose with secrets functionality (which can leverage Docker Swarm secrets or simply mount files).

Docker Secrets (Docker Swarm Mode)

Docker Secrets is a feature integrated with Docker Swarm mode that provides a secure way to transmit and store sensitive data for services. Secrets are:

Encrypted at rest in the Swarm manager's Raft logs.
Transmitted securely to authorized service tasks.
Mounted as in-memory files within the container's filesystem, typically at /run/secrets/<secret_name>, rather than exposed as environment variables.
Only accessible by services explicitly granted access.

How to Use Docker Secrets (Swarm Mode)

Initialize Swarm (if not already):

docker swarm init ```

Create a Secret: Secrets are created from a file or standard input.

echo "my_secure_db_password" | docker secret create db_password_secret - echo "SG.your_api_key_here" | docker secret create sendgrid_api_key - ```

Deploy a Service with the Secret: Services reference secrets by name. Docker mounts the secret into the container.

docker service create --name my-webapp
--secret db_password_secret
--secret sendgrid_api_key
my_app_image:latest ```

Accessing Secrets in the Container: Applications read the secret from the mounted file path.

In your Python application code (or similar for other languages)

with open('/run/secrets/db_password_secret', 'r') as f: db_password = f.read().strip()

with open('/run/secrets/sendgrid_api_key', 'r') as f: sendgrid_key = f.read().strip() ```

Docker Compose and Secrets (for single host or Swarm)

Docker Compose version 3.1+ introduced a secrets section, which allows you to define and reference secrets within your docker-compose.yml. When running in Swarm mode, Compose leverages Docker Swarm's native secrets. When running on a single host without Swarm mode, Compose still supports secrets by mounting files from the host into the container securely, though without the encryption at rest provided by Swarm.

Using `secrets` in `docker-compose.yml`

Define Secrets: You can define secrets either by referencing an external file or by making it an external secret (pre-created Swarm secret).

# docker-compose.yml
version: '3.8'

services:
  webapp:
    image: my_app_image:latest
    ports:
      - "80:8080"
    secrets:
      - db_password
      - sendgrid_api_key

secrets:
  db_password:
    file: ./secrets/db_password.txt # Path to a file on the host containing the password
  sendgrid_api_key:
    external: true                   # Refers to a pre-existing Docker Swarm secret named 'sendgrid_api_key'

Create Local Secret Files (if file is used):

mkdir secrets echo "my_local_db_password" > ./secrets/db_password.txt ```

Deploy with Compose: docker compose up -d will deploy your services, making secrets available at /run/secrets/<secret_name> inside the containers.
```
# Inside the container, the content of ./secrets/db_password.txt will be at:
# /run/secrets/db_password
```

Choosing the Right Tool: Configuration vs. Secrets

The decision of whether to use an environment variable for configuration or a dedicated secret management solution boils down to one primary question:

Is the data sensitive?

If Yes (sensitive data): Use Docker Secrets (with Swarm) or a similar secrets management system (e.g., Kubernetes Secrets, HashiCorp Vault). For single-host Compose setups, use the secrets section to mount files securely.
If No (non-sensitive configuration): Use environment variables (via ENV in Dockerfile, -e flag, env_file, or environment in Compose).

Best Practices for Both

For Configuration (Environment Variables):

Provide sensible defaults in your Dockerfile using ENV. This makes your images runnable out of the box and clearly documents expected variables.
Externalize configuration where possible. Use .env files with docker compose or external configuration services for larger deployments.
Document all configuration options and their expected values, perhaps in a README.md or application documentation.
Avoid hardcoding values that might change between environments (development, staging, production).

For Secrets (Docker Secrets and beyond):

Never commit secrets (e.g., .env files containing secrets, db_password.txt) to version control systems like Git.
Rotate secrets regularly. This minimizes the window of exposure if a secret is compromised.
Grant least privilege. Only give services access to the secrets they absolutely need.
Avoid logging secret values. Ensure your application and infrastructure logging do not print secret contents.
For large-scale, enterprise-grade deployments, consider dedicated secrets management solutions like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault, which offer more advanced features like auditing, dynamic secret generation, and integration with Identity and Access Management (IAM).

A Practical Rule for Deciding What Goes Where

Before adding a value to environment, ask what would happen if a teammate pasted it into a support ticket. If the answer is "nothing serious," it is probably configuration. If the answer is "we would rotate credentials," it is a secret.

Good environment variables:

APP_ENV=production
LOG_LEVEL=info
PUBLIC_BASE_URL=https://example.com
FEATURE_SIGNUP_ENABLED=false
REDIS_HOST=redis

Bad environment variables:

DATABASE_PASSWORD=...
STRIPE_SECRET_KEY=...
JWT_SIGNING_KEY=...
AWS_SECRET_ACCESS_KEY=...
PRIVATE_SSH_KEY=...

There are gray areas. A database hostname is usually configuration. A full database URL that includes username and password is a secret. A public analytics key may be safe for a browser application, while a private API token for the same vendor is not. When in doubt, treat the value as sensitive until you prove otherwise.

Compose `.env` Files Are Easy to Misunderstand

Docker Compose uses .env in two different ways that people often mix up.

First, Compose reads a project-level .env file for variable substitution inside compose.yml:

services:
  web:
    image: "${APP_IMAGE}"
    ports:
      - "${HOST_PORT}:8080"

Second, env_file passes variables into the container:

services:
  web:
    image: my-app
    env_file:
      - ./app.env

Those files may look similar, but they serve different purposes. The first helps Compose render the configuration. The second becomes runtime environment inside the container. Do not assume a value in the project .env automatically appears inside the container unless you explicitly pass it through.

For local development, a checked-in example file is helpful:

# .env.example
APP_ENV=development
LOG_LEVEL=debug
PUBLIC_BASE_URL=http://localhost:3000

Then keep the real .env out of Git:

.env
*.env.local
secrets/

The example file documents what the app expects without exposing private values.

Reading File-Based Secrets in an Application

Many applications already expect secrets in environment variables. Moving to file-based secrets is easier if you support both patterns for a while.

For example, a Node.js helper:

import fs from "node:fs";

function readSecret(name) {
  const filePath = process.env[`${name}_FILE`];
  if (filePath) {
    return fs.readFileSync(filePath, "utf8").trim();
  }
  return process.env[name];
}

const databasePassword = readSecret("DATABASE_PASSWORD");

Then your Compose file can point to a mounted secret file:

services:
  web:
    image: my-app
    environment:
      DATABASE_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt

This pattern works well because the application can still run in older environments while you move production toward file-mounted secrets. Many official images already support variables ending in _FILE for this reason.

Do Not Put Secrets in Build Arguments Either

Environment variables are not the only trap. Build arguments can leak too if you use them to fetch private packages or clone repositories:

ARG NPM_TOKEN
RUN npm config set //registry.npmjs.org/:_authToken=$NPM_TOKEN

Even if the final container does not show NPM_TOKEN, build history and intermediate layers may still expose more than you expect. With BuildKit, use secret mounts for build-time secrets:

# syntax=docker/dockerfile:1.7
FROM node:22-slim
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=secret,id=npm_token \
  NPM_TOKEN="$(cat /run/secrets/npm_token)" npm ci

Build it like this:

docker build \
  --secret id=npm_token,src=.npm-token \
  -t my-app .

That keeps the token out of the Dockerfile and avoids baking it into a normal layer. You still need to protect the local .npm-token file and CI secret storage.

Kubernetes, Cloud Secret Managers, and Docker

Docker Secrets are useful in Swarm, and Compose secrets are useful for local or single-host setups. In Kubernetes, you will normally use Kubernetes Secrets, an external secrets operator, or a cloud secret manager integration. On AWS, teams often use AWS Secrets Manager or Systems Manager Parameter Store. On Azure, Azure Key Vault is common. On Google Cloud, Secret Manager fills the same role.

The principle is the same across platforms:

Store sensitive values in a system designed for secrets.
Grant the runtime identity access only to the secrets it needs.
Mount or inject secrets at runtime.
Rotate secrets without rebuilding the image.
Keep secrets out of source control, image layers, logs, and dashboards.

Kubernetes Secrets are encoded by default, not automatically encrypted in every cluster configuration. Many managed clusters support encryption at rest, but verify the actual cluster settings instead of assuming. For high-risk credentials, use a cloud secret manager or a dedicated tool with audit logs and rotation support.

Rotation Is Part of the Design

A secret strategy that cannot rotate is unfinished. Ask these questions before production:

Can we change the database password without rebuilding the image?
Can two valid credentials overlap during a rollout?
Does the application reread secrets, or does it need a restart?
Where are old credentials logged, cached, or stored?
Who gets notified when a secret changes?

For databases, rotation often means creating a second credential, deploying the application with the new credential, verifying traffic, then revoking the old one. For API keys, it depends on the provider. Some services allow multiple active keys; others force a cutover. Design your deployment process around the least flexible dependency.

Clean Up Accidental Exposure

If a secret has already been committed to Git or baked into an image, deleting the line is not enough. Treat it as exposed.

The usual response is:

Revoke or rotate the credential.
Remove it from the current code or image.
Check CI logs, image registries, issue trackers, and chat messages for copies.
Rewrite Git history only if your organization is prepared to handle the coordination; rotation is still required.
Add scanning or pre-commit checks to reduce repeat mistakes.

Tools can help, but they do not replace habits. Keep secret files named clearly, ignore them in Git, and avoid printing configuration objects wholesale at startup.

The Working Pattern

Use environment variables for values that describe how the app should run in this environment: ports, log levels, feature flags, service hostnames, and non-sensitive URLs. Use secrets for values that prove identity or grant access: passwords, tokens, signing keys, private keys, and provider credentials.

The clean Docker image is the same across environments. Development, staging, and production change behavior at runtime. Configuration can travel as environment variables. Secrets should come from a secret store or mounted secret file with limited access. That separation keeps deployments flexible without turning every container inspection, log line, or image layer into a credential leak.