DevOps Knowledge Hub
DevOps Knowledge Hub

Step-by-Step Guide to Secure SSH Key-Based Authentication

Unlock superior remote access security with this comprehensive, step-by-step guide to SSH key-based authentication. Learn how to generate robust ED25519 key pairs, securely distribute your public key, and crucially, disable vulnerable password logins on your server. This tutorial provides actionable commands and essential best practices, including passphrase usage and key management, ensuring your SSH connections are both convenient and highly secure. Elevate your server's defense against brute-force attacks today.

36 views

Step-by-Step Guide to Secure SSH Key-Based Authentication

Secure Shell (SSH) is the backbone of remote system administration, enabling secure access to servers over an unsecured network. While password-based authentication is common, it's inherently vulnerable to brute-force attacks and credential stuffing. SSH key-based authentication offers a far more robust and secure alternative, leveraging cryptographic key pairs to verify identity without transmitting passwords.

This guide will walk you through the entire process of setting up secure SSH key-based authentication. You'll learn how to generate your SSH key pair, securely distribute your public key to a remote server, and crucially, harden your server's SSH configuration by disabling less secure password logins. By following these steps, you'll significantly enhance the security posture of your remote access, making your systems more resilient against common attack vectors.

Understanding SSH Key-Based Authentication

SSH key-based authentication relies on a pair of cryptographically linked keys: a private key and a public key.

  • Private Key: This key must remain secret and secured on your local machine. It's like a highly complex password that only you possess.
  • Public Key: This key can be freely shared and is placed on the remote server you wish to access. It's used by the server to verify your identity.

When you attempt to connect, the server uses your public key to challenge your local machine. Your local machine then uses its private key to respond to this challenge, proving your identity without ever sending the private key over the network. This method is not only more secure but also more convenient, as it eliminates the need to type a password for every connection once configured correctly.

Step 1: Generating Your SSH Key Pair

The ssh-keygen utility is used to create new SSH key pairs. It's recommended to use modern, strong algorithms like ED25519.

Choose a Key Type and Strength

While RSA keys are still widely used, ED25519 offers excellent security with shorter key lengths and faster operations. For new setups, ED25519 is generally preferred.

Generate the Key Pair

On your local machine (client), open your terminal and run the following command:

ssh-keygen -t ed25519 -C "[email protected]"
  • -t ed25519: Specifies the key type as ED25519.
  • -C "[email protected]": Adds a comment to the public key, often used for identification. Replace with your actual email or a descriptive label.

The command will prompt you for a location to save the key and an optional passphrase.

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/youruser/.ssh/id_ed25519): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/youruser/.ssh/id_ed25519
Your public key has been saved in /home/youruser/.ssh/id_ed25519.pub
The key fingerprint is: SHA256:...
The key's randomart image is: ...

When prompted, always set a strong passphrase for your private key. This passphrase encrypts your private key on your local machine, providing an additional layer of security. If your private key ever falls into the wrong hands, it will be useless without the passphrase. You can use ssh-agent to avoid typing the passphrase repeatedly (see Step 4).

Key File Locations

By default, ssh-keygen saves your private key to ~/.ssh/id_ed25519 and your public key to ~/.ssh/id_ed25519.pub.

Permissions for Your Private Key

It is crucial that your private key file has very strict permissions. Only the owner should be able to read it. ssh-keygen usually sets this correctly, but it's good to verify:

chmod 600 ~/.ssh/id_ed25519

Step 2: Distributing Your Public Key to the Server

Once you have generated your key pair, your public key needs to be copied to the remote server you wish to access. It should be placed in a file named authorized_keys within the ~/.ssh/ directory of your user on the remote server.

ssh-copy-id is the simplest and safest method. It logs into the remote server (using your password), creates the ~/.ssh directory if it doesn't exist, sets correct permissions, and appends your public key to ~/.ssh/authorized_keys.

ssh-copy-id user@your_server_ip

Replace user with your username on the remote server and your_server_ip with the server's IP address or hostname. You will be prompted for your password on the remote server.

Method 2: Manual Copying

If ssh-copy-id is not available, you can copy the public key manually.

  1. Copy the public key content from your local machine:
    bash cat ~/.ssh/id_ed25519.pub
    Copy the entire output to your clipboard (it starts with ssh-ed25519 ...).

  2. Log in to the remote server using your password:
    bash ssh user@your_server_ip

  3. Create the ~/.ssh directory if it doesn't exist and set permissions:
    bash mkdir -p ~/.ssh chmod 700 ~/.ssh

  4. Append your public key to authorized_keys:
    bash echo "PASTE_YOUR_PUBLIC_KEY_HERE" >> ~/.ssh/authorized_keys
    Make sure to replace PASTE_YOUR_PUBLIC_KEY_HERE with the actual content you copied. Using >> (append) is important to avoid overwriting existing keys if there are any.

  5. Set correct permissions for authorized_keys:
    bash chmod 600 ~/.ssh/authorized_keys

    • Warning: Incorrect permissions on ~/.ssh or ~/.ssh/authorized_keys will prevent key-based authentication from working.

Step 3: Test Your SSH Key-Based Authentication

Before proceeding to disable password authentication, it is absolutely critical to verify that key-based authentication works correctly. Log out of the remote server if you are still connected from the manual copying steps.

From your local machine, attempt to connect to the server without specifying a password:

ssh user@your_server_ip
  • If you set a passphrase for your private key, you will be prompted to enter it.
  • If the connection is successful without a password prompt (after the passphrase, if applicable), your key-based authentication is working. You should see the remote server's prompt.

Do NOT proceed to Step 4 if you cannot log in using your SSH key. Troubleshoot any issues before disabling password authentication, or you risk locking yourself out of the server.

Step 4: Enhancing Security - Disabling Password Authentication

Once you have confirmed that SSH key-based authentication is working, you can disable password-based logins on your server to significantly improve security. This prevents brute-force attacks against your password and ensures that only those with valid SSH keys can access the server.

  1. Log in to your remote server using your SSH key.
    bash ssh user@your_server_ip

  2. Edit the SSH daemon configuration file. This file is typically located at /etc/ssh/sshd_config.
    bash sudo nano /etc/ssh/sshd_config
    (You can use vi or your preferred text editor instead of nano.)

  3. Locate and modify the following directives:

    • Find PasswordAuthentication and change its value to no.
      #PasswordAuthentication yes PasswordAuthentication no
      (Uncomment the line if it's commented out with #)

    • Find ChallengeResponseAuthentication and change its value to no.
      #ChallengeResponseAuthentication yes ChallengeResponseAuthentication no

    • (Optional but recommended) Consider disabling root login directly via SSH if you plan to use sudo after logging in as a regular user.
      PermitRootLogin no

    • (Optional) Consider changing the default SSH port from 22 to a non-standard high port (e.g., 2222). This doesn't add security against targeted attacks but can reduce noise from automated port scanners.
      #Port 22 Port 2222
      If you change the port, remember to specify it with the -p flag when connecting (e.g., ssh -p 2222 user@your_server_ip).

  4. Save the changes and exit the text editor.

  5. Restart the SSH service to apply the new configuration. The command varies slightly depending on your operating system (e.g., Ubuntu/Debian vs. CentOS/RHEL).

    • Systemd-based systems (most modern Linux distributions):
      bash sudo systemctl restart sshd

    • Older SysVinit-based systems:
      bash sudo service ssh restart

  6. Crucially, open a new terminal window (do not close your current active SSH session!) and attempt to log in using your key. This tests the new configuration without locking yourself out if something went wrong.

    bash ssh user@your_server_ip
    If you changed the port:
    bash ssh -p 2222 user@your_server_ip

    If the connection is successful, you can now safely close your original SSH session.

    If the new login fails, immediately revert the changes in sshd_config in your original, still-active SSH session and restart the SSH service again, then re-evaluate.

Best Practices and Tips

  • Always use a strong passphrase for your private key. This is your last line of defense if your private key is compromised.
  • Protect your private key. Never share it, and ensure it's stored securely with strict file permissions (chmod 600 ~/.ssh/id_ed25519). Consider hardware security modules (HSMs) or YubiKeys for ultimate protection.
  • Use ssh-agent for convenience. ssh-agent allows you to load your private key(s) into memory and only type your passphrase once per session, even across multiple SSH connections. Add your key to the agent with ssh-add ~/.ssh/id_ed25519.
  • Regularly rotate your SSH keys. Periodically generate new key pairs and remove old public keys from your servers, especially if a team member leaves or a key's security is suspected to be compromised.
  • Limit PermitRootLogin to no or prohibit-password. It's generally better to log in as a regular user and use sudo for administrative tasks.
  • Configure a firewall. Ensure only necessary ports (like your SSH port) are open to the internet. Tools like ufw or firewalld can help.

Conclusion

By following this step-by-step guide, you have successfully set up and secured SSH key-based authentication for your remote server. You've generated a robust cryptographic key pair, distributed your public key, and most importantly, disabled less secure password-based logins. This significantly enhances the security of your server, making it more resistant to common hacking attempts.

Always remember to protect your private key and its passphrase vigilantly, as it is now the primary credential for accessing your server. Embrace these best practices to maintain a strong security posture for all your remote access needs.