Keyless SSH Login: Set It Up, Then Turn Passwords Off

Q: What permissions does authorized_keys need?

~/.ssh should be 700, ~/.ssh/authorized_keys should be 600, and the home directory itself must not be writable by group or others. SSH enforces this deliberately: a world-writable directory would let another user add their own key. ssh-copy-id sets all of this correctly, which is a good reason to use it rather than copying files by hand.

Keyless - or more precisely, passwordless - SSH login takes two commands: ssh-keygen -t ed25519 to create a key pair, then ssh-copy-id user@host to install the public half on the server. After that, ssh user@host logs you in without a prompt. The name is a small lie worth clearing up: it is not keyless at all, it is password-less. You are replacing a secret you type with a key pair you hold, which is both more convenient and dramatically more secure. But convenience is not the point, and if you stop after those two commands you have gained the convenience and none of the security - because passwords still work.

Every guide covers ssh-keygen and ssh-copy-id. The step that actually changes your security posture is the one after them, and it is the one that usually gets left as an exercise for the reader.

The two commands
The step everyone skips
Passphrases and the agent
Permissions are the usual reason it does not work
Managing keys past the first server
How this fits the rest of the stack
FAQ

The two commands

# 1. Generate a key pair. Ed25519 is the modern default.
ssh-keygen -t ed25519 -C "alice@laptop"

# 2. Copy the public key to the server.
ssh-copy-id [email protected]

# 3. Log in. No password.
ssh [email protected]

ssh-keygen writes two files to ~/.ssh/:

id_ed25519 - the private key. Never leaves your machine. Never gets committed, pasted, or emailed.
id_ed25519.pub - the public key. Safe to share, and the only half that goes on servers.

On key type: use ed25519. It is faster, the keys are shorter, and the security is better than RSA at any practical size. Use rsa -b 4096 only when talking to something too old to support ed25519, which is increasingly rare.

ssh-copy-id appends the public key to ~/.ssh/authorized_keys on the server and fixes the permissions. If it is not available, the manual equivalent is:

cat ~/.ssh/id_ed25519.pub | ssh alice@server \
  "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

The step everyone skips

You now have key-based login working. Passwords also still work, which means your server’s security is still exactly as strong as the weakest password on it. You have added convenience and changed nothing about your exposure.

The point of keys is to turn off passwords. In /etc/ssh/sshd_config:

PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin prohibit-password
ChallengeResponseAuthentication no

Then reload and - this is not optional - verify before you close your session:

sudo sshd -t              # validate the config BEFORE reloading
sudo systemctl reload sshd

# In a SECOND terminal, confirm you can still get in.
ssh [email protected]

Keep the original session open until the second one succeeds. A typo in sshd_config plus a closed session equals a server you cannot reach, and the recovery is a console session through your provider if you are lucky. sshd -t catches syntax errors before they lock you out; it does not catch a config that is valid and wrong, which is what the second terminal is for.

Why this matters so much: an internet-facing SSH port sees continuous automated password guessing, from the hour it comes up. Not targeted - just constant background scanning. With PasswordAuthentication no, all of it fails regardless of what your password is, because there is no password path to try.

Passphrases and the agent

ssh-keygen asks for a passphrase. People skip it because a passphrase seems to defeat the purpose of passwordless login. It does not, and here is why.

The passphrase encrypts the private key at rest. Without one, anyone who copies that file - a stolen laptop, a bad backup, a compromised sync folder - has your access to every server the key touches, immediately and silently. With one, they have an encrypted file.

The agent gives you both:

# Unlock once per session; the agent holds it in memory.
ssh-add ~/.ssh/id_ed25519

# What is currently loaded?
ssh-add -l

# Forget everything.
ssh-add -D

You type the passphrase once when you first use the key, and the agent handles every connection after that. The convenience is unchanged; the stolen-laptop scenario is completely different. Use a passphrase.

For CI and automation where no human can type a passphrase, use a dedicated key with no passphrase, scope it narrowly, and store it in the platform’s secret store - not in the repository. A key in git is a key that is public.

Permissions are the usual reason it does not work

SSH refuses to use keys if the permissions are loose, and it does so quietly - you just get a password prompt, which looks like the key was rejected for some mysterious reason.

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

The home directory itself matters too - it must not be group- or world-writable. This is the one people miss, because ~/.ssh looks correct while ~ is 775.

When it silently falls back to a password, get the actual reason from the client and the server:

# Client side - verbose.
ssh -vvv alice@server

# Server side - the real answer.
sudo tail -f /var/log/auth.log

The server log names the problem outright - Authentication refused: bad ownership or modes for directory /home/alice. Two minutes of reading beats an hour of guessing, and this specific message accounts for most of the failures.

Managing keys past the first server

Two servers is fine. Twenty is a system, and hand-copying keys stops scaling almost immediately.

Things that help:

~/.ssh/config - name your hosts once, stop retyping. This is the highest-value five minutes here.
One key per device, not one key per server. Revoking a lost laptop means removing one key everywhere, not auditing which of nine keys it held.
A record of which keys are authorized where. Otherwise removal is guesswork, and a key you cannot find is a key you cannot revoke.
Certificate-based auth if you outgrow this. Short-lived signed certificates rather than permanent keys.

# ~/.ssh/config
Host prod
    HostName 203.0.113.10
    User deploy
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Now it is ssh prod. IdentitiesOnly yes matters more than it looks: without it, the client offers every key it has, and a server with MaxAuthTries set low will disconnect you before it reaches the right one - a confusing failure with an obvious fix.

The honest observation: this is all undifferentiated work. Key distribution, rotation, and revocation across a fleet is a real job that makes no product better, which is why most teams eventually stop doing it by hand.

How this fits the rest of the stack

The security here is free; the ongoing management is not. Key rotation, access reviews, and the audit of who can reach what across a fleet is steady work that never ships a feature. If you are weighing what it is worth to have that handled, the RunxBuild hosting calculator puts the service, database, storage, and bandwidth side by side so the comparison is a number rather than a habit, and the RunxBuild dashboard is where access and deploys are visible together.

Useful related references:

FAQ

Is passwordless SSH secure?

More secure than passwords, provided you finish the job. A key pair is far stronger than any password a human will type, and it cannot be guessed by the automated scanning that hits every internet-facing SSH port continuously. But the security comes from setting PasswordAuthentication no afterwards. If passwords still work, you have added convenience and changed nothing about your exposure.

Should my SSH key have a passphrase?

Yes, for any key belonging to a human. The passphrase encrypts the private key at rest, so a stolen laptop or a leaked backup yields an encrypted file rather than instant access to every server the key touches. Use ssh-add so you type it once per session - the convenience is identical. For CI, use a dedicated passphrase-less key stored in the platform’s secret store and scoped narrowly.

Why is SSH still asking for a password after copying my key?

Almost always permissions. SSH silently refuses keys when the modes are too loose: ~/.ssh must be 700, authorized_keys 600, and your home directory must not be group- or world-writable - that last one is the commonly missed case. Run sudo tail -f /var/log/auth.log on the server while connecting; it names the exact reason rather than leaving you to guess.

What permissions does authorized_keys need?

~/.ssh should be 700, ~/.ssh/authorized_keys should be 600, and the home directory itself must not be writable by group or others. SSH enforces this deliberately: a world-writable directory would let another user add their own key. ssh-copy-id sets all of this correctly, which is a good reason to use it rather than copying files by hand.

What is the difference between ed25519 and RSA SSH keys?

Ed25519 keys are shorter, faster to verify, and offer better security than RSA at practical sizes. Use ssh-keygen -t ed25519 by default. RSA at 4096 bits remains fine and is worth keeping only for compatibility with older systems that predate ed25519 support, which is an increasingly small set. Avoid DSA and short RSA keys entirely.