Managing SSH Keys Safely: Create, Install, Rotate and Revoke

This article is a localized edition of the Japanese original. Read the Japanese original

SSH keys (public key authentication) let you log in to a server without sending a password. But if a private key leaks, every server that key can access is at risk. This article covers the operations you need for day-to-day key management: creating keys, storing them, installing them, checking them, rotating them and revoking them.

The commands assume OpenSSH (the client built into Linux, macOS and Windows 10/11). Server-side examples use Ubuntu, and differences on RHEL-family systems (AlmaLinux, Rocky Linux) are noted where they come up.

SSH key basics: public key and private key

File	Where it lives	How to handle it
Private key (e.g. ~/.ssh/id_ed25519)	Only on your own PC	Never give it to anyone. Never put it on a server
Public key (e.g. ~/.ssh/id_ed25519.pub)	~/.ssh/authorized_keys on the server you log in to	Safe for others to see

Only the public key goes on the server. Do not copy your private key to a server or send it by chat or email.

Step 1: Create a key

For a new key, choose Ed25519. The key is short and widely supported by current OpenSSH versions.

ssh-keygen -t ed25519 -C "taro@laptop-2026"

• -C is a comment. Putting the user name and device name in it makes it easy to tell later which device a key belongs to.
• When asked where to save it, press Enter to save it as ~/.ssh/id_ed25519. To keep separate keys per purpose, change the name, for example ~/.ssh/id_ed25519_work.
• Set a passphrase. Even if the private key file is stolen, it cannot be used without the passphrase. You can avoid typing it every time with ssh-agent, described below.

Use RSA only when an old server or device does not support Ed25519.

ssh-keygen -t rsa -b 4096 -C "taro@laptop-2026"

Step 2: Install the public key on the server

On Linux and macOS, ssh-copy-id appends your public key to ~/.ssh/authorized_keys on the server and sets the permissions too (you log in with your password at this point).

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@203.0.113.10

PowerShell on Windows does not have ssh-copy-id. Send the public key and append it like this:

type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh user@203.0.113.10 "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Once it is installed, check that you can log in with the key.

ssh -i ~/.ssh/id_ed25519 user@203.0.113.10

Step 3: Check file permissions

If permissions are too loose, your local ssh refuses to load the private key, and sshd on the server ignores authorized_keys (because sshd's StrictModes is enabled by default).

On your PC (Linux, macOS):

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

On the server:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Public key authentication also fails if the home directory itself is writable by the group or others (for example after chmod 777).

Moving the private key elsewhere and pointing a symbolic link at it only makes management more complicated; it does not make the key any safer. The basic approach is to leave it in ~/.ssh and protect it with permissions and a passphrase.

Step 4: Make keys easier to use with ssh-agent and ~/.ssh/config

Skip passphrase entry with ssh-agent

Once a key is added to ssh-agent, you do not need to re-enter the passphrase until you log out.

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
ssh-add -l    # list keys that have been added

On Windows, the ssh-agent service is disabled by default. Enable it once from an administrator PowerShell, then run ssh-add.

Get-Service ssh-agent | Set-Service -StartupType Automatic
Start-Service ssh-agent
ssh-add $env:USERPROFILE\.ssh\id_ed25519

Choose a key per host with ~/.ssh/config

When you have several keys, write the key to use for each host in ~/.ssh/config.

Host web01
    HostName 203.0.113.10
    User deploy
    IdentityFile ~/.ssh/id_ed25519_work
    IdentitiesOnly yes

Now ssh web01 is all you need to connect. With IdentitiesOnly yes, only the specified key is used. If ssh-agent holds many keys, the client may try them one after another until it reaches the server's limit on attempts (MaxAuthTries, default 6) and gets disconnected; this setting prevents that.

Step 5: Allow key-only login

After confirming that key login works, turn off password authentication on the server. Keep your current SSH session open and check from another terminal as you go. If you make a mistake in the settings, you can revert it from the session you kept open.

On Ubuntu 22.04 / 24.04, /etc/ssh/sshd_config reads /etc/ssh/sshd_config.d/*.conf at the top, and for any setting the first value read wins. Files are read in name order, so create yours with a name that sorts before any existing file (for example 50-cloud-init.conf, created in cloud environments).

sudoedit /etc/ssh/sshd_config.d/01-hardening.conf

PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin prohibit-password

PermitRootLogin prohibit-password allows root to log in with keys only. To forbid root login entirely, set it to no (first confirm that you can log in with a key as a regular user who can use sudo).

Check the syntax and the values that actually take effect before restarting.

sudo sshd -t
sudo sshd -T | grep -Ei 'passwordauthentication|kbdinteractive|permitrootlogin'
sudo systemctl restart ssh     # sshd on RHEL-family systems

From another terminal, confirm that a connection without a key is refused.

ssh -o PubkeyAuthentication=no user@203.0.113.10
# If you see Permission denied (publickey). password login is no longer possible

Step 6: Audit the installed keys

Which keys are installed

You can list the fingerprints and comments of the keys in the server's authorized_keys.

ssh-keygen -lf ~/.ssh/authorized_keys

To show the fingerprint of a key on your PC:

ssh-keygen -lf ~/.ssh/id_ed25519.pub

Which key was used to log in

Login records are kept in the server's logs, not in authorized_keys. Each successful login records the fingerprint of the key used.

# Ubuntu
sudo grep 'Accepted publickey' /var/log/auth.log | tail -n 20
# To read from the systemd journal (the service is ssh on Ubuntu, sshd on RHEL-family systems)
sudo journalctl -u ssh --since today | grep 'Accepted'

Match the SHA256:... part of Accepted publickey for deploy from 198.51.100.7 port 51234 ssh2: ED25519 SHA256:... against the list above to see which key (which person and which device) was used.

Step 7: Rotate or revoke keys

Change only the passphrase

You can change the passphrase while keeping the same key.

ssh-keygen -p -f ~/.ssh/id_ed25519

Rotate a key

1. Create a new key (Step 1)
2. Install the new public key on the server (Step 2)
3. Confirm that you can log in with the new key
4. Delete the old key's line from ~/.ssh/authorized_keys on the server

When a device is lost, someone leaves, or a key leaks

Delete the matching line from authorized_keys on every server that key can access. Find it by the comment at the end of the key line (the string you set with -C) or by its fingerprint.

grep -n 'taro@laptop-2026' ~/.ssh/authorized_keys

There is no way to "revoke" a key from the private key side, so deleting it on the server is how you revoke it. Keeping a list of which key is installed on which server lets you do this without missing any.

Step 8: Limit what each key can do (optional)

Adding options at the start of a line in authorized_keys restricts what that key can be used for.

from="198.51.100.0/24" ssh-ed25519 AAAAC3Nza... taro@laptop-2026
restrict,command="/usr/local/bin/backup.sh" ssh-ed25519 AAAAC3Nza... backup@nas

• from= accepts this key only from the specified source addresses.
• restrict disables port forwarding, terminal allocation and similar features in one go, and command= fixes the command that runs at login. This suits keys for automated jobs such as backups.

Key points for managing keys

• One key per person, and a separate key per device (with a shared key, you cannot revoke access for just one person)
• Protect private keys with a passphrase
• Put the user and device name in each key's comment, and keep a list of the servers each key is installed on
• Delete keys you no longer need from authorized_keys
• Turn off password authentication while keeping an existing session open

For SSH server settings in general, see also SSH security settings.

Questions about WordPress? The publisher of this site is based in Japan and replies in Japanese. Contact (Japanese)