Demystifying journalctl: Your Guide to Linux System Logs

The journalctl utility is the main command for viewing logs collected by systemd-journald. When a service fails, a boot hangs, or a kernel message scrolls past too quickly, journalctl is usually where you find the details.

Unlike plain text log files in /var/log, the journal stores structured entries with metadata such as unit name, priority, boot ID, and timestamp. That lets you filter logs precisely instead of opening several files and guessing.

Understanding the Systemd Journal

Before diving into commands, it helps to know what journalctl is managing. The systemd journal collects logs from the kernel, services (units), applications, and even the system startup process. These logs are typically stored in structured binary files, making them much faster to search and filter than traditional flat files.

Key Concepts:

Persistence: Logs can be configured to be stored permanently across reboots (persistent mode) or only kept in memory (volatile mode).
Structured Data: Logs contain metadata (like unit name, priority, boot ID) that journalctl leverages for filtering.

Essential Viewing Commands

The most common need is simply viewing the logs. Here are the foundational commands to start exploring your system's history.

1. Viewing All Logs

To see every entry collected by the journal since the system started recording logs, use the command without any arguments:

journalctl

By default, this output is paginated using a tool like less, allowing you to navigate up and down using arrow keys or spacebar.

2. Viewing Logs in Real-Time (Following)

Similar to using tail -f, you can monitor logs as they are being written to the journal. This is invaluable when starting or debugging a specific service:

journalctl -f

To exit the real-time view, press Ctrl+C.

3. Limiting Output by Number of Lines

If you only need the most recent entries, use the -n flag to specify the number of lines:

# Show the last 20 log entries
journalctl -n 20

4. Viewing Logs Since a Specific Time

Filtering by time is one of journalctl's strongest features. You can use relative or absolute timestamps.

Relative Time Examples:

Command	Description
`journalctl --since "1 hour ago"`	Logs from the last hour.
`journalctl --since "yesterday"`	Logs from the start of the previous calendar day.
`journalctl --since "2023-10-26 14:30:00"`	Logs after a specific date and time.

Tip: You can combine --since and --until to specify a precise range, for example: journalctl --since "2023-10-26" --until "2023-10-27 00:00:00".

Filtering Logs by Unit, Service, or Process

One of the most critical uses of journalctl is isolating messages related to a specific systemd unit (service).

Filtering by Service Name

Use the -u (or --unit) flag followed by the service name (e.g., sshd.service, nginx.service):

# View logs only for the Apache web server service
journalctl -u apache2.service

# Follow logs for the Docker service
journalctl -u docker.service -f

Filtering by Boot ID

Systemd tracks logs across different system boots. To see logs from a previous startup, first list the available boots:

journalctl --list-boots

This outputs an indexed list (e.g., -1, -2, 0 for the current boot). You can then use the index or the full Boot ID:

# Show logs from the immediately preceding boot
journalctl -b -1

# Show logs from a specific boot ID (use the long ID listed above)
journalctl -b a1b2c3d4e5f6...

Filtering by Kernel Messages

To view only messages originating from the kernel (similar to using dmesg):

journalctl -k
# Or, equivalently:
journalctl -b -k

Advanced Filtering and Output Control

Effective troubleshooting often requires combining filters and controlling the output format.

1. Filtering by Priority Level

Logs are assigned priority levels from 0 (emerg) through 7 (debug). You can use the -p or --priority flag to view messages at the selected severity and more severe levels:

Priority Level	Numeric Value
`err` or `error`	3
`warning`	4
`notice`	5
`info`	6

# Show all errors, critical, and emergency messages from the SSH service
journalctl -u sshd.service -p err

2. Searching for Specific Text

When standard filtering is not enough, you can pipe output to grep or use --grep on systems whose journalctl version supports it:

# Search all logs for lines containing the word 'failed'
journalctl | grep failed

# Or, use the --grep option for simpler filtering (if available/preferred)
journalctl --grep=failed

3. Changing Output Format

For scripting or moving logs to other tools, changing the output format is essential. The default is pretty (human-readable).

--output=json: Outputs entries as structured JSON objects.
--output=short: Similar to pretty but less colorful.
--output=export: Outputs raw journal entries for archival.

# Output recent logs as JSON
journalctl -n 5 --output=json

4. Combining Filters

Filters stack multiplicatively. To see all errors from Nginx recorded in the last 10 minutes:

journalctl -u nginx.service --since "10 minutes ago" -p err

Managing the Journal Size and Persistence

Over time, the journal can consume significant disk space, especially if configured for persistent logging. System administrators must manage its size.

Checking Current Usage

To see how much disk space the current journal files are occupying:

journalctl --disk-usage

Cleaning Up Old Logs

You can clear logs based on time or total size.

Cleanup by Time:

# Keep only logs from the last 7 days
journalctl --vacuum-time=7d

Cleanup by Size:

# Reduce total log size to a maximum of 500 Megabytes
journalctl --vacuum-size=500M

Warning: Be cautious when deleting logs, especially those from older boots (-b). Ensure critical diagnostic information is not removed prematurely.

Takeaway

For day-to-day troubleshooting, start with journalctl -u <service> -n 50 --no-pager, add -f when you need live output, and use --since or -b when the problem is tied to a time window or boot. Before deleting logs, check journalctl --disk-usage and keep enough history for rollback and incident review.