AWS EC2 SSH Connection and Key Pair Management: Connect, Fix, Recover

This article is a localized edition of the Japanese original. Read the Japanese original

The basic way to connect to EC2 over SSH is to use the private key of the key pair you chose at launch, along with the username set for your AMI, and run ssh -i private-key username@public-IP. This article covers how to create and store a key pair, the usernames for each OS, simplifying connections with ~/.ssh/config, what to check for each error when you cannot connect, adding and swapping keys, and how to recover when you lose the private key.

The console screens and the supported key types can change, so the explanations here are a guide based on information as of October 2026.

Understand how key pairs work

An EC2 key pair is a pair of a public key and a private key. AWS stores only the public key, and when the instance first launches, it writes that public key to the OS user's ~/.ssh/authorized_keys. The private key exists only on your machine and is not kept on the AWS side.

Two things follow from this. First, if you lose the private key, you cannot download it again from AWS. Second, even if you change the key pair setting in the console after launch, the authorized_keys of an instance that is already running is not rewritten. When you want to change keys, you edit the file on the server directly.

Prepare a key pair

Create one in the console

Create it from "Key pairs" on the EC2 screen. You can choose RSA or ED25519 as the key type. For Linux instances, ED25519 is an easy choice because the keys are short and fast to process. However, ED25519 cannot be used with Windows instances, so choose RSA for those. For the file format, you can choose .pem, used by OpenSSH, or .ppk, used by PuTTY.

When you create it, the private key file is downloaded right then, only once. Also keeping this file in a password manager or encrypted storage reduces the risk of losing it to a PC failure.

Import a key you created locally

You can also register only the public key of a key you created on your own PC with AWS. The advantages are that the private key never travels over the network and that you can set a passphrase.

ssh-keygen -t ed25519 -f ~/.ssh/aws-wp -C "aws-wp"
aws ec2 import-key-pair \
  --region ap-northeast-1 \
  --key-name aws-wp \
  --public-key-material fileb://~/.ssh/aws-wp.pub

Key pairs are a per-Region resource. A key registered in the Tokyo Region does not appear on the instance creation screen in the Osaka Region. If you want to use it in another Region as well, import the same public key into that Region too.

Connect with SSH

Restrict the private key’s permissions

On macOS and Linux, unless the private key file is readable only by you, SSH warns "UNPROTECTED PRIVATE KEY FILE" and refuses to use the key.

chmod 400 ~/.ssh/aws-wp.pem

The same warning appears with OpenSSH on Windows. In that case, disable inheritance in the security settings in the file's properties, and leave read permission only for your own user.

Usernames by AMI

If you get the username wrong, the connection is refused even when the key is correct. The default usernames for typical AMIs are as follows.

• Amazon Linux: ec2-user
• Ubuntu: ubuntu
• Debian: admin
• Red Hat Enterprise Linux: ec2-user

For other AMIs, check the username given by the AMI's provider.

ssh -i ~/.ssh/aws-wp.pem ec2-user@203.0.113.10

The first time you connect, a message asks you to confirm the host key fingerprint. To verify strictly, compare it with the fingerprint output at boot time in the console's "Get system log", and then answer yes.

Simplify connections with ~/.ssh/config

Typing the options every time is tedious, so write them in the configuration file.

Host wp-prod
    HostName 203.0.113.10
    User ec2-user
    IdentityFile ~/.ssh/aws-wp.pem
    IdentitiesOnly yes

Now you can connect with just ssh wp-prod. IdentitiesOnly yes makes SSH use only the specified key. If you have many keys on your machine, SSH can reach the server's limit on attempts while it tries them in turn, and get disconnected before it tries the correct key; this setting prevents that.

What to check when you cannot connect

The location of the cause differs depending on the error message. Read the message first, then check in order of likelihood. Running with ssh -v shows in detail at which stage it stops.

Connection timed out (no response comes back)

Traffic is not reaching the server.

1. Security group: In the inbound rule for port 22, does the source include your current IP? If your IP changes at home or on the go, you may suddenly get timeouts even though you could get in before.
2. Public IP: When you stop and restart an instance, an automatically assigned public IP changes. Check that the IP you are connecting to matches what the console shows.
3. Route table and network ACL: Check that the route table of the public subnet has a route to the internet gateway, and that the network ACL is not denying port 22 or the return traffic.

Permission denied (publickey)

The connection reaches the server, but authentication fails.

1. Username: Is it the username that matches the AMI?
2. Key mix-up: Does the key pair name shown in the instance details match the private key you are using?
3. Server-side permissions: If you changed the owner or permissions of ~/.ssh or authorized_keys on the server, SSH stops reading the key. In this case, fix the files using the recovery methods described below.

Host key verification failed

The host key does not match the one recorded earlier, for example because a different instance was assigned the same IP. Delete the old record and reconnect only if you know you recreated the instance. If it appears and you have no idea why, check what you are connecting to.

ssh-keygen -R 203.0.113.10

Add or swap keys

When you want separate keys for each person, or want to swap keys, edit the server's authorized_keys directly. When swapping, following this order prevents you from locking yourself out.

1. Add the new public key as one line in ~/.ssh/authorized_keys
2. Without closing your current connection, confirm that you can connect with the new key from another terminal
3. Only after confirming, delete the line for the old public key

Make a copy of authorized_keys before editing. You can disable the key of someone who left or of a device you no longer use just by deleting its line. On servers used by several people, putting a name in the comment at the end of each public key makes it easier to tell whose key it is and to manage them.

Recover when you lose the private key

Losing the private key does not mean you have lost the contents of the instance. You can register a new public key again using the following methods.

• EC2 Instance Connect: A mechanism that sends a temporary public key to the instance through the AWS API so that you can connect with that key for a short time. It works on AMIs that have the supporting package installed (such as Amazon Linux 2023 and recent Ubuntu). When connecting from the console, you need to allow port 22 in the security group from the EC2 Instance Connect source.
• Session Manager: If the instance has the SSM agent and an IAM role with the required permissions, you can open a shell from the browser or the CLI without using SSH.
• Swap the volume: If the two methods above are not available, stop the instance, attach its root volume to another instance, edit authorized_keys, and put it back. Take a snapshot before you start.

Once you can connect, add the new public key to authorized_keys and delete the line for the old key. To prepare for situations like this, keeping Session Manager ready to use all the time lets you also move to an operation where the SSH port is closed.

Summary

• AWS keeps only the public key and writes it to authorized_keys at first launch. The private key cannot be downloaded again.
• For Linux, ED25519 is easy to work with. If you import a key you created locally, the private key never travels over the network.
• Make the private key readable only by you, and connect with the username that matches the AMI.
• A timeout is a network problem, and Permission denied is an authentication problem. Use the error to narrow it down.
• Swap keys in the order add, confirm, delete, and if you lose a key, recover with Instance Connect or Session Manager.

• How to manage SSH keys
• What to do when you cannot connect to SSH port 22

Questions about WordPress? The publisher of this site is based in Japan and replies in Japanese. Contact (Japanese)