Understanding Systemd Dependencies: Preventing and Fixing Unit Conflicts

Systemd dependencies are easy to get half right. A unit file gets Requires=postgresql.service, the application still starts too early, and everyone wonders why systemd ignored the dependency. It did not ignore anything. Requires= and After= answer different questions.

That distinction is the heart of most systemd dependency problems. One directive controls whether another unit is pulled into the same transaction. A different directive controls ordering. Other directives link shutdown behavior, bind one unit's lifetime to another, or make two units mutually exclusive. Once you separate those ideas, unit conflicts become much less mysterious.

The Foundation: Systemd Unit Dependency Directives

Systemd uses specific directives within unit files (typically located in /etc/systemd/system/ or /lib/systemd/system/) to dictate when one unit should start, stop, or wait for another. Understanding these directives is the first step in managing dependencies correctly.

Core Dependency Directives

These directives control relationships between units. They do not, by themselves, always control startup order:

• Requires=:
  • Establishes a strong dependency. If the required unit fails to start, the current unit will also fail.
  • It does not imply PartOf=, and it does not automatically mean "start after this unit."
• Wants=:
  • A weak dependency. If the wanted unit fails, the current unit will still attempt to start. This is used for optional dependencies.
• BindsTo=:
  • Similar to Requires=, but stronger regarding stopping. If the bound unit stops (for any reason), the current unit is also stopped.
• PartOf=:
  • Indicates that the current unit is a subordinate part of another unit (e.g., a specific socket activation related to a main service). If the superior unit stops, the subordinate unit stops as well.
• Conflicts=:
  • Says two units should not be active at the same time. Starting one causes systemd to stop the other as part of the same transaction.

Core Startup Synchronization Directives

These directives dictate when the dependent unit should start relative to the required unit:

• After=:
  • Specifies that the current unit's start job is ordered after the listed unit's start job. It does not pull that unit in by itself.
• Before=:
  • Specifies that the current unit should start before the listed unit.

Best Practice: For typical service startup ordering, Wants= combined with After= is the most common and safest pattern. Requires= should be reserved for dependencies where failure of the dependency must cause the dependent service to fail.

Example: Defining Dependencies in a Service File

Consider a custom application service, myapp.service, that must communicate with a database managed by PostgreSQL (postgresql.service).

# /etc/systemd/system/myapp.service
[Unit]
Description=My Custom Application

# Ensure PostgreSQL is running before attempting to start me
Requires=postgresql.service
After=postgresql.service

[Service]
ExecStart=/usr/bin/myapp

[Install]
WantedBy=multi-user.target

That example is intentionally strict. If PostgreSQL cannot start, myapp.service should fail too. For a service that can run in degraded mode without the database, use Wants=postgresql.service with the same After=postgresql.service ordering. The application still asks systemd to start PostgreSQL first, but it is allowed to continue if PostgreSQL is unavailable.

There is no shame in choosing the weaker relationship when it matches reality. A metrics exporter, log shipper, or cache warmer may be useful when its dependency is present but should not block the main system from booting. Hard dependencies are best reserved for cases where starting without the other unit would be wrong or dangerous.

For networked applications, be more careful. network.target often means the basic networking stack is present, not that DHCP has completed or DNS is usable. If your application really needs configured networking at startup, use:

[Unit]
Wants=network-online.target
After=network-online.target

Then confirm your distribution has the matching wait-online service enabled, such as systemd-networkd-wait-online.service or NetworkManager's wait-online unit. Without that, network-online.target may not wait for what you think it waits for.

Diagnosing Dependency Problems

When a service fails to start, systemd usually provides enough information in the logs, but dependency chains can obscure the root cause. Here are essential tools and commands for troubleshooting.

1. Checking Unit Status and Logs

The fundamental starting point is checking the service status and reviewing its logs immediately after a failed start attempt.

# Check the overall status, which often mentions dependency failures
systemctl status myapp.service

# View detailed logs specifically related to the unit
journalctl -u myapp.service --since "5 minutes ago"

2. Analyzing the Dependency Tree

Systemd provides powerful visualization tools to see exactly what is waiting on what.

systemctl list-dependencies

This command shows the units that are required or wanted by the specified unit, traversing the entire dependency chain.

To see what myapp.service requires to start:

# Forward dependencies (what must start before me)
systemctl list-dependencies --after myapp.service

# Reverse dependencies (what depends on me)
systemctl list-dependencies --before myapp.service

Use --plain when the tree formatting gets in the way, and add --all when you need to see inactive units too:

systemctl list-dependencies --plain --all myapp.service

systemd-analyze dot

For larger dependency questions, generate a graph and inspect it visually:

systemd-analyze dot 'myapp.service' | dot -Tsvg > myapp-deps.svg

On a server without Graphviz installed, the text output is still useful because you can search for the unit names and see which edges systemd knows about. For boot timing issues, systemd-analyze critical-chain myapp.service is often easier to read than a full graph.

3. Detecting Conflicts and Ordering Issues

Dependency conflicts often manifest as services failing because they were started too early or stopping unexpectedly.

Circular Dependencies: This is the most dangerous conflict, where Unit A requires B, and Unit B requires A. Systemd attempts to resolve this but often results in one or both units remaining in a failed or activating state indefinitely.

To find potential issues spanning the entire system, you can search the logs for specific failure messages related to ordering:

journalctl -b | grep -E "failed|refused to start|dependency was not satisfied"

Also run systemd-analyze verify against custom units before assuming runtime behavior is the problem:

systemd-analyze verify /etc/systemd/system/myapp.service

It can flag ordering cycles, unknown directives, and invalid unit references early. It will not prove your design is correct, but it can save you from chasing a typo as if it were a complex dependency issue.

Fixing Common Dependency Issues

Once identified, dependency issues can be resolved by adjusting the directives in the relevant unit files.

Scenario 1: Service Starts Before Its Prerequisite is Ready

Symptom: Your application logs show database connection errors, but postgresql.service appears active in systemctl status.

Diagnosis: The service may be missing After=postgresql.service, or PostgreSQL may be active before the specific database, socket, credentials, or schema your application needs is ready. Systemd can order units, but it cannot automatically understand every application-level readiness condition.

Fix: Start with the simple unit relationship:

[Unit]
Requires=postgresql.service
After=postgresql.service

If the application still races the database, fix the readiness problem at the right layer. Some services support Type=notify and only report ready after initialization. Some applications need retry logic because dependencies can restart at any time, not just during boot. For a narrow local check, an ExecStartPre= command can be reasonable:

[Service]
ExecStartPre=/usr/bin/pg_isready -q -h 127.0.0.1 -p 5432
ExecStart=/usr/local/bin/myapp

Use that kind of precheck sparingly. If it grows into a shell script with sleeps and loops, the application probably needs proper retry behavior instead.

Avoid sleep 30 as a dependency fix. It may hide the race on a quiet development machine and fail again on slower storage, a busy VM, or a host waiting for DNS. A real ordering directive, readiness notification, socket activation, or application retry loop gives you a reason the service is ready instead of a hope that enough time has passed.

Scenario 2: Conflicting Start/Stop Order

Symptom: Stopping the system causes critical processes to hang or fail abruptly.

Diagnosis: This often indicates misuse of BindsTo= or a complex interaction between Before= and After= directives in sibling services.

Fix: Review services that are siblings (e.g., services started by the same target). Ensure that if Service A must run while Service B is running, you use BindsTo= or Requires=. If Service A must finish its tasks before Service B starts cleanup, verify the After= order is correct.

Remember that shutdown order is the reverse of startup order. If app.service has After=database.service, then on shutdown systemd stops app.service before database.service. That is usually what you want: the application stops accepting work before the database disappears. Many shutdown bugs come from missing startup ordering, not from a separate shutdown-only setting.

Scenario 3: Removing Unnecessary Dependencies

Symptom: System boot is slow because unnecessary services are being pulled into the startup chain.

Diagnosis: You might have used Requires= when only an optional connection was needed.

Fix: Change Requires= to Wants=. If the service doesn't absolutely need the dependency to function, Wants= allows the system to proceed even if the dependency fails or is masked.

# Before (Too Strict)
Requires=optional_logging.service

# After (Better)
Wants=optional_logging.service
After=optional_logging.service

This is especially useful for monitoring agents, metrics exporters, sidecar helpers, and optional local caches. If the main service can do useful work without them, Requires= turns a partial outage into a full outage. Use strict dependencies for things that are truly required: a local database for an application that cannot start without it, a mount point that holds the application's data, or a socket unit that is the only supported activation path.

Scenario 4: A Mount or Device Is Not Ready

Symptom: A service fails at boot with No such file or directory, but the path exists after you log in. This often happens with services that read from /mnt/data, /srv/app, removable disks, encrypted volumes, or network filesystems.

Diagnosis: The service is starting before the mount unit is active, or the mount is optional and failed without stopping the service.

Fix: Find the mount unit name:

systemd-escape -p --suffix=mount /mnt/data

For /mnt/data, that usually produces mnt-data.mount. Then order your service after it and require it if the service cannot run without the data:

[Unit]
Requires=mnt-data.mount
After=mnt-data.mount

If the mount comes from /etc/fstab, options such as nofail, x-systemd.automount, and _netdev affect boot behavior. Do not add a hard Requires= to an optional mount unless you really want that mount failure to block the service.

Scenario 5: A Target Pulls In Too Much

Symptom: Enabling a service seems to start unrelated units, or disabling one service does not stop it from appearing during boot.

Diagnosis: The service may be pulled in by a target through [Install] WantedBy=..., by another service's Wants=, or by a socket, timer, path, or mount unit. enable creates symlinks for boot-time activation; it is not the only way a unit can start.

Fix: Inspect both the unit and reverse dependencies:

systemctl cat myapp.service
systemctl list-dependencies --reverse myapp.service
systemctl list-timers --all | grep myapp
systemctl list-sockets --all | grep myapp

If a socket unit activates the service, disabling only myapp.service may not be enough. You may need to disable or mask myapp.socket too, depending on the desired behavior.

Applying Changes and Reloading

Whenever you modify a unit file, you must instruct systemd to reload its configuration before testing the changes.

# 1. Reload the systemd manager configuration
sudo systemctl daemon-reload

# 2. Restart the affected service
sudo systemctl restart myapp.service

# 3. Verify status
systemctl status myapp.service

A Small Mental Checklist

When a dependency problem feels tangled, reduce it to a few plain checks.

First, ask whether the other unit should be started at all. If yes, use Wants= or Requires=. If the current service can run without it, prefer Wants=. If failure of that unit means this service must fail, use Requires=.

Second, ask whether startup order matters. If yes, add After= or Before=. Do not expect dependency directives to handle ordering by themselves.

Third, ask whether lifetime coupling matters after startup. If one unit stopping should stop the other, look at BindsTo= or PartOf=. Do not use them casually; they can make routine restarts cascade farther than expected.

Finally, inspect what systemd actually loaded:

systemctl cat myapp.service
systemctl show myapp.service -p Wants -p Requires -p After -p Before -p Conflicts

That output is often more useful than the file you think you edited. Drop-ins, vendor units, generated units, sockets, timers, and targets can all change the final behavior.