How to Set Up SSH Keys on a VPS Without Locking Yourself Out

SSH keys make everyday VPS administration easier: no server password to type into every terminal, a separate credential for each administrator, and a clear way to revoke access when a device is lost. The important part is not just generating a key. It is changing the login policy without losing your only route into the machine.

This is a focused companion to our Linux VPS setup checklist. It covers a normal human administrator connecting to a Linux server, not an existing certificate authority, corporate single sign-on or multi-factor SSH deployment. If another team manages your authentication, agree changes with them first.

1. Prepare an account and a recovery route

Before changing authentication, sign in to your hosting panel and confirm you can open the VPS console. Confirm the credentials or recovery procedure needed to use it: seeing a console button is not the same as having a usable login. Keep one existing SSH session connected while you test a second.

Use a named non-root administrative account. If you already have one, keep using it. On a fresh Ubuntu or Debian server, an administrator can create one as follows; choose a unique username instead of admin if that account already exists:

# On the VPS, using an existing privileged account
sudo adduser admin
sudo usermod -aG sudo admin

When working from a root console, omit sudo. Set a strong local account password and store it securely: it may still be needed for sudo or console recovery even after SSH password login is disabled. Reconnect as the new user and verify sudo -v succeeds before proceeding.

Do not change the firewall, SSH port, username and authentication policy in one operation. If something fails, changing one layer at a time makes recovery much more straightforward.

2. Generate an SSH key on your computer

The private key stays on your device; the public key goes on the server. A key passphrase protects the private key at rest and is different from your VPS account password.

Linux or macOS terminal

# On your own computer
ssh-keygen -t ed25519 -f ~/.ssh/hyehost_admin -C "hyehost-admin"

Choose a strong passphrase when prompted. If that filename already exists, stop and choose another name instead of overwriting a working key. This creates hyehost_admin and hyehost_admin.pub. Ed25519 is suitable for ordinary modern OpenSSH environments; regulated cryptographic policies may require a different approved algorithm. Key generation options are documented in the ssh-keygen manual.

Windows PowerShell

Use the Windows OpenSSH client, not a Linux shell command pasted blindly into PowerShell. Check ssh -V first. If it is unavailable, install the OpenSSH Client optional feature through Windows settings.

# On your Windows computer, in PowerShell
ssh-keygen -t ed25519 -f "$env:USERPROFILE\.ssh\hyehost_admin" -C "hyehost-admin"

Keep both generated files in your user profile and protect the private file from other users. Do not upload it to the VPS, a support ticket or a Git repository. Microsoft's OpenSSH key-management documentation explains Windows key storage and agent handling.

3. Add the public key to the correct VPS user

On Linux, or a macOS installation with ssh-copy-id available, run this locally while the server still permits your existing login method:

ssh-copy-id -i ~/.ssh/hyehost_admin.pub admin@SERVER_IP

For a custom SSH port, add -p PORT to this command and all subsequent connection tests. On first connection, verify the server host-key fingerprint through a trusted route, such as its console, before accepting it. On the server, the Ed25519 host-key fingerprint can be displayed with sudo ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub. Compare the same key type.

Manual public-key installation, including Windows

Display the public key on your computer. In PowerShell:

Get-Content "$env:USERPROFILE\.ssh\hyehost_admin.pub"

Copy the complete single line beginning ssh-ed25519. In your existing server session, logged in as the target user, prepare the key file:

# On the VPS as admin, not as root
mkdir -p ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
nano ~/.ssh/authorized_keys

Append the public key on its own line and save. Preserve existing keys unless you intentionally mean to revoke their access. Do not paste the private file beginning with a private-key header. Files created as the target user should belong to that user; if they do not, investigate ownership before making broad recursive changes.

The server's home directory and key files must not be writable by unrelated users. Ubuntu's OpenSSH server guide covers public-key installation and permissions. If your server uses a non-default AuthorizedKeysFile, install the key at the configured location instead.

4. Prove that a new connection uses your key

Open a second local terminal. These options deliberately prevent fallback to a server password and avoid reusing an already authenticated multiplexed connection:

# Linux or macOS
ssh -i ~/.ssh/hyehost_admin -o IdentitiesOnly=yes -o PreferredAuthentications=publickey -o PasswordAuthentication=no -o KbdInteractiveAuthentication=no -o ControlPath=none admin@SERVER_IP

# Windows PowerShell
ssh -i "$env:USERPROFILE\.ssh\hyehost_admin" -o IdentitiesOnly=yes -o PreferredAuthentications=publickey -o PasswordAuthentication=no -o KbdInteractiveAuthentication=no -o ControlPath=none admin@SERVER_IP

A local key-passphrase prompt is expected if the key is encrypted. A server account-password prompt is not part of this test. After connecting, run whoami and sudo -v. You need both the intended identity and working administrative privileges, not merely a shell.

If it fails, leave your first session open and fix the key setup before changing server policy. The OpenSSH client manual documents identity selection and connection options.

5. Disable SSH passwords only after successful testing

This section deliberately changes who can log in. Do not apply it unchanged to a server using password-based automation or keyboard-interactive MFA. Migrate those users first. Save copies of the existing SSH configuration and any snippets you edit, with unique backup filenames.

Inspect /etc/ssh/sshd_config and its included files. On standard Ubuntu installations, snippets in /etc/ssh/sshd_config.d/ are included early. For most directives, the first obtained value wins: a file named 99-custom.conf does not necessarily override earlier settings. Cloud-image defaults and Match blocks also need attention.

Where that include is present and no earlier policy takes precedence, create a separate snippet using sudoedit /etc/ssh/sshd_config.d/00-local-key-only.conf. Check that this filename is unused first. Use:

PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no

This is a simple key-login baseline, not a replacement for an organisation's full authentication policy. PermitRootLogin no requires the non-root sudo account you just tested. Password and keyboard-interactive are separate mechanisms; changing only one may leave a password-like login route. Review the sshd_config reference for existing authentication requirements.

Validate before reloading

# On the VPS
sudo /usr/sbin/sshd -t
sudo /usr/sbin/sshd -T | grep -E '^(pubkeyauthentication|passwordauthentication|kbdinteractiveauthentication|permitrootlogin|authenticationmethods) '

Stop if the syntax check reports an error. The effective output should match the intended policy. Where there are conditional rules, evaluate the actual login context, replacing the example client and server addresses:

sudo /usr/sbin/sshd -T -C user=admin,addr=198.51.100.20,host=client.example,laddr=203.0.113.10,lport=22

The addresses above are documentation placeholders, not live destinations. Use the real client source address, server address, port and relevant hostname. The sshd manual explains syntax tests, effective configuration and connection-specific checks.

For the standard Ubuntu/Debian ssh.service, apply only after those checks succeed:

sudo systemctl reload ssh.service
sudo systemctl status ssh.service --no-pager

If your distribution uses a different unit or a managed SSH configuration, follow that service's reload procedure. Do not reboot or force a restart to hide a configuration error. Now repeat the fresh key-only login and sudo tests from section four. Also try a fresh connection with public-key authentication disabled; it should reject password access rather than offer a server password login.

6. Troubleshoot SSH login failures

Use the failing command with -vv added to show client diagnostics. On the server, review sudo journalctl -u ssh.service -n 80 --no-pager. Logs can reveal usernames, addresses and file paths; redact them before sharing publicly.

Symptom	Check first
Permission denied (publickey)	Username, selected private key, complete public key, ownership and server policy.
Too many authentication failures	Use the explicit key with IdentitiesOnly=yes rather than offering every agent identity.
Connection timed out	Address, route, firewall and whether you can reach the selected IPv4 or IPv6 endpoint.
Connection refused	Listening port and SSH service state through the console.
Host identification has changed	Verify the server's identity out of band before updating the saved host key.
Key works but sudo fails	Account privileges, local password and sudo policy; do not disable your remaining admin route.

After a legitimate reinstall, a host key may change, but so can the machine answering an address. Do not “fix” a warning with StrictHostKeyChecking=no. Verify first, then update only the affected saved entry. A timeout is not evidence that your key is wrong: the network connection has not yet reached authentication.

If the new policy blocks you, use the still-open administrative session or console to revert only your changed snippet or settings. Run the syntax check again, reload, and retest. If you cannot authenticate to the console, follow your provider's recovery procedure; do not rebuild a production server merely to regain SSH access without checking backups and recovery options.

7. Keep access manageable as the VPS grows

• Give people separate accounts and keys. Do not share one private key across a team.
• Keep a tested recovery route. A second protected device or recovery key is useful only if its access has been verified.
• Revoke lost-device keys. Remove the matching public key from every affected account and investigate possible use.
• Separate automation. Deployment and backup jobs should have narrowly scoped credentials, not your personal administrator identity.
• Maintain the operating system. Keys do not replace security updates, backups or monitoring.

A passphrase-protected key can be loaded into an SSH agent for convenience, but agent forwarding should not be enabled casually on untrusted hosts. For unattended jobs, choose a deliberate credential and restriction strategy rather than removing your personal key's passphrase to make a script work.

Using SSH keys with HYEHOST

A HYEHOST Cloud VPS gives you a Linux environment for Docker services, applications and bots where this administrative workflow applies. Choose the location and resources your workload needs; the key lives on your own device regardless of where the VM runs.

For multiple environments, VPS resource pools let you place VMs in Wolverhampton and Ashburn from one pool. Keep separate host entries and verify each machine's fingerprint rather than assuming identical usernames or addresses across deployments.

If you only need to run a bot and do not want to administer Linux yourself, compare Bot Hosting with a full VPS. This guide applies to the operating system on a VPS; it does not imply every managed hosting service exposes the same SSH controls.

Once access is secure, continue with Docker firewall checks, VPS monitoring and a backup strategy. Secure login is the starting point, not the entire security plan.

Final no-lockout checklist

1. Console or another independent recovery route tested.
2. Private key retained locally with a passphrase; only the public key uploaded.
3. Fresh key-only login and sudo confirmed for the non-root account.
4. Existing users, automation and MFA considered before changing policy.
5. Syntax and effective configuration checked, including relevant Match rules.
6. Service reloaded successfully and a new login tested again.
7. Key revocation and recovery details recorded somewhere accessible off-server.

Frequently asked questions

Can I use SSH keys from Windows?

Yes. With the Windows OpenSSH client installed, use ssh-keygen and ssh from PowerShell or Windows Terminal. Upload only the public .pub key to the Linux VPS; keep the private key on your computer.

Should I disable SSH passwords immediately?

No. First verify a fresh key-only connection to a non-root administrative account, confirm sudo works and check console recovery access. Validate the server configuration before applying password-login changes.

Why does SSH say Permission denied (publickey)?

Common causes include the wrong username, the wrong private key, an incomplete public key, incorrect ownership or permissions, and server policy. Compare verbose client output with the server SSH logs.

Does changing the SSH port replace SSH keys?

No. A different port can reduce routine scanning noise, but it does not replace secure authentication, software updates or access controls.

What if I lose my SSH private key?

Use a separately authorised recovery key or the provider console to install a replacement public key and revoke the lost key. You cannot recover a private key from its public key.