Can't Open the SSH Pub File? Here's What's Going On

An SSH pub file you can’t open is usually a permissions issue, a missing file, or a wrong path. The right fix depends on which one. Five minutes with chmod 600 usually clears it.

The ‘Permission denied’ case
The ‘No such file’ case
The ‘Wrong path’ case
The ‘Wrong key’ case
The ‘ssh-copy-id is the easier path’
What usually breaks
The Windows-specific SSH issues
The debugging workflow
FAQ

The ‘Permission denied’ case

The most common error: ssh: Could not open authorized keys file /home/user/.ssh/authorized_keys: Permission denied. The fix:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub
chown -R user:user ~/.ssh

The required permissions: .ssh directory is 700, private keys are 600, public keys are 644. SSH refuses to use keys with too-permissive permissions.

The ‘No such file’ case

The second most common error: ssh: Could not open authorized keys file /home/user/.ssh/authorized_keys: No such file or directory. The fix:

mkdir -p ~/.ssh
touch ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

The file may not exist on a fresh server. The team that uses ssh-copy-id from the client handles this automatically; the team that copies the key manually has to create the file.

The ‘Wrong path’ case

The third case: the file exists but at a different path. SSH looks in ~/.ssh/authorized_keys by default. The team that puts the file elsewhere has to configure SSH:

# /etc/ssh/sshd_config
AuthorizedKeysFile /etc/ssh/authorized_keys/%u

Then reload sshd. The team that uses the default path avoids this issue.

The ‘Wrong key’ case

The fourth case: the pub key is in authorized_keys but SSH still asks for a password. The causes:

The key in authorized_keys is a different key than the one in ~/.ssh/id_rsa on the client. Generate a new key pair and update both sides.
The key has trailing whitespace. A trailing newline or space makes the key invalid. Use cat to view; the line should be one long string with no whitespace at the end.
The sshd config disables pubkey auth. PubkeyAuthentication no in sshd_config. Set to yes and reload.
The user’s home directory has wrong permissions. SSH refuses to use authorized_keys if the home directory is world-writable. chmod 755 /home/user.

The ‘ssh-copy-id is the easier path’

The right tool for copying a key to a server:

ssh-copy-id user@host

This prompts for the password once, then copies ~/.ssh/id_rsa.pub to the server’s ~/.ssh/authorized_keys with the right permissions. The team that uses this avoids all the manual fixes above.

What usually breaks

The four pitfalls:

Wrong permissions on .ssh. The team that runs chmod 777 ~/.ssh for “convenience” has SSH refuse to work.
SELinux or AppArmor blocking access. The team on a hardened system has SSH blocked by mandatory access control. Check the audit log.
The key has wrong line endings. A Windows-edited file has CRLF line endings; SSH expects LF. Convert with dos2unix.
The home directory is on a read-only mount. Some shared home directories are read-only; the team that needs authorized_keys has to use a different location.

The Windows-specific SSH issues

The team that uses SSH from Windows often has unique issues:

File path with spaces. %USERPROFILE%\.ssh\ becomes C:\Users\My Name\.ssh\ with a space. SSH handles this; some tools don’t. The fix: use forward slashes or quote the path.

CRLF line endings. Windows editors save with CRLF; SSH expects LF. The fix: use dos2unix or save with Notepad++ in Unix mode.

Permissions with icacls. Windows file permissions differ from Unix. icacls file /inheritance:r /grant:r "username:(R)" sets the right permissions. The team that uses cygwin or WSL has the standard Unix tools.

Different ~/.ssh location. Windows uses %USERPROFILE%\.ssh\ not ~/.ssh/ literally. The team that sets HOME to the wrong path has SSH look in the wrong place.

ssh-agent on Windows. Built-in to Windows 10+ OpenSSH. Start-Service ssh-agent and Set-Service -StartupType Automatic to start automatically.

The debugging workflow

The right way to debug SSH issues:

ssh -v user@host for verbose output. Shows authentication attempts, key loading, file permissions.
ssh -vvv user@host for maximum verbosity. Shows every step of the connection.
Check the server’s auth log. /var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL/CentOS) shows what the server thinks went wrong.
Test with a fresh key pair. Generate a new key, copy to the server, test. If that works, the old key was the issue.
Check known_hosts. If the server’s host key changed, SSH refuses to connect. Remove the old entry: ssh-keygen -R hostname.

The team that uses ssh -vvv first has the fastest debugging workflow. The team that guesses and pokes has the slowest.

FAQ

What permissions does authorized_keys need?

600 (or 644). The directory (.ssh) needs 700. SSH refuses to use the key if the permissions are too permissive.

Why does SSH say ‘Permission denied (publickey)’?

The pubkey is not in authorized_keys, or the permissions on authorized_keys or ~/.ssh are wrong, or the home directory permissions are wrong. Check each in order.

Can I have multiple pub keys in authorized_keys?

Yes, one per line. The team that has multiple team members adds one line per person.

Why is ssh-copy-id not working?

Either the password authentication is disabled (use a console instead), or the .ssh directory has wrong permissions, or the server is unreachable.

What does ‘Permission denied (publickey)’ mean?

The pubkey is not in authorized_keys, the permissions are wrong, or the home directory permissions are wrong. The team that checks each in order finds the issue.

Why is my private key rejected even though pubkey is in authorized_keys?

Most often permissions: the private key is world-readable, or the .ssh directory is world-writable. SSH refuses to use a key with insecure permissions.

Can I use the same SSH key on Windows and Linux?

Yes. The key is just a file. Copy the private key to the other machine’s .ssh directory with the right permissions. The team that uses the same key on multiple machines has the simplest key management.

If you are sizing the infrastructure for the kind of project this post covers, the RunxBuild hosting calculator is the right place to model the line items. The compute, the memory, the storage, the bandwidth, the database - each one is a separate number, and the team’s mental model for the platform is the sum of those numbers. The RunxBuild dashboard is where the team sees the actual usage in one place.

Useful related references: