SSH Setup: A Comprehensive Guide

SSH (Secure Shell) is how you get a secure terminal on a remote Linux server — and it is also the transport underneath SCP and SFTP. This guide walks through the complete setup: installing the SSH server, connecting from a client, and then hardening the server with key-based authentication and password login disabled. Every step includes what the output should look like, so you know you are on the right track.

SSH client and server connection showing key exchange between a local machine and a remote server

How SSH Authentication Works

SSH encrypts everything in transit, so passwords and commands are never sent in plain text. You can authenticate in two ways:

Deploying the OpenSSH Server

The SSH server is the daemon (sshd) that accepts incoming connections. It is only needed on the machine you connect to — your laptop is the client.

Step 1: Install the OpenSSH Server

Refresh the package index, then install the server package:

sudo apt update
sudo apt install openssh-server

Step 2: Start and Enable the Service

enable makes the daemon start automatically on boot; start launches it now:

sudo systemctl enable ssh
sudo systemctl start ssh

Verify it is running — you should see active (running):

sudo systemctl status ssh

● ssh.service - OpenBSD Secure Shell server
     Loaded: loaded (/lib/systemd/system/ssh.service; enabled)
     Active: active (running) since Mon 2026-08-03 09:14:22 UTC

If your distribution's firewall is active, allow the port: sudo ufw allow ssh. For deeper control see iptables rules.

Connecting to the Server

From your local machine, connect with your username and the server's address:

ssh username@server-ip

The first time you connect, SSH asks you to verify the host key fingerprint and stores it in ~/.ssh/known_hosts. Type yes, then enter your password.

Custom Port (Optional)

To reduce automated port scans, some admins move SSH off port 22. Edit the server config:

sudo nano /etc/ssh/sshd_config

Find or add Port 22 and change it, e.g. Port 2222, then restart:

sudo systemctl restart ssh

Clients must then specify the port: ssh -p 2222 username@server-ip.

Transitioning to Key Authentication

Passwords can be guessed or brute-forced. Key pairs are effectively unguessable. The process has three steps: generate a key pair on your machine, copy the public key to the server, then disable password login.

Step 3: Generate a Key Pair

On your local machine, generate a key (a passphrase protects the private key at rest):

ssh-keygen -t rsa -b 4096

You will be asked where to save it (default ~/.ssh/id_rsa — press Enter) and for a passphrase. This creates two files: id_rsa (private — never share it) and id_rsa.pub (public — safe to copy anywhere).

Step 4: Copy the Public Key to the Server

ssh-copy-id username@server-ip

This appends your public key to ~/.ssh/authorized_keys on the server. You are asked for your password one last time. Afterwards, log in again — you should land in a shell without typing a password:

ssh username@server-ip
Welcome to Ubuntu 24.04.2 LTS (GNU/Linux 6.8.0-31-generic x86_64)

If ssh-copy-id is not available, do it manually: cat ~/.ssh/id_rsa.pub | ssh username@server-ip "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys".

Step 5: Enforce Password-less Security

Only disable passwords after you have confirmed key login works from a separate terminal window. In /etc/ssh/sshd_config:

PasswordAuthentication no

Then restart the daemon:

sudo systemctl restart ssh

Verifying the Configuration

ssh username@server-ip

You should connect directly using your key. If it asks for a password instead, the change did not take effect or your key is not in authorized_keys.

Common Problems

Problem 1: Connection refused

Error: ssh: connect to host ... port 22: Connection refused

Cause: sshd is not running, or the port is blocked by a firewall.

Solution: on the server check sudo systemctl status ssh and open the firewall port (sudo ufw allow ssh). Test reachability with nc -zv server-ip 22.

Problem 2: Connection timed out

Cause: the server is unreachable — wrong IP, network routing, or a firewall dropping packets (not just refusing them).

Solution: ping the host, then test the port with netcat; check iptables on the server.

Problem 3: Permission denied (publickey)

Cause: your key is not in ~/.ssh/authorized_keys on the server, or permissions are wrong.

Solution: permissions matter: on the server run chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys, then re-add the key with ssh-copy-id.

Problem 4: WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED

Cause: the server's host key changed (reinstall, IP reuse) — SSH protects you against impersonation.

Solution: if the change is legitimate, remove the stale entry: ssh-keygen -R server-ip, then reconnect.

Security Best Practices

Hardening the SSH Daemon (sshd_config)

Beyond the basics, /etc/ssh/sshd_config exposes settings that control how the daemon handles connections and which cryptographic algorithms it accepts. The following values tighten the default configuration:

# Disable legacy, weak key-exchange algorithms
KexAlgorithms -diffie-hellman-group1-sha1,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1

# Unauthenticated connection flood control: max 10 pending,
# reject 30% of new ones once the limit is reached, hard cap at 60
MaxStartups 10:30:60

# At most 10 sessions per connection
MaxSessions 10

# Disconnect a client after 4 failed authentication attempts
MaxAuthTries 4

# Client must authenticate within 60 seconds of connecting
LoginGraceTime 60

# Server sends a keep-alive every 300s and drops the client
# after 1 unanswered probe (about 5 minutes of true inactivity)
ClientAliveInterval 300
ClientAliveCountMax 1

What each one does: KexAlgorithms with a leading - removes the listed algorithms from the daemon's offer list, forcing strong key exchange. MaxStartups 10:30:60 is a rate-limit against connection floods: the first 10 unauthenticated connections are accepted, then 30% of new ones are rejected, and once 60 are pending the server drops new ones. MaxSessions, MaxAuthTries, and LoginGraceTime cap session multiplexing, password guessing, and slow handshakes respectively. ClientAliveInterval/ClientAliveCountMax control idle disconnects — with the values above, a client that misses one keep-alive is dropped after roughly five minutes of inactivity. For a longer idle window, raise both, e.g. ClientAliveInterval 600 with ClientAliveCountMax 3.

Apply any change by restarting the daemon: sudo systemctl restart sshd (on Debian/Ubuntu the service is named ssh). Always keep an existing session open while you test a new configuration — a broken sshd_config can lock you out of the server.

Conclusion

We installed the SSH server, started it as a service, generated a key pair, copied the public key to the server, and disabled password login. The result is a server you can manage securely from anywhere. The important part is understanding why: SSH encrypts everything in transit, and key authentication removes the weakest link — guessable passwords.

Related Articles

Looking to sharpen your defensive security skills and interactive shell mechanics? Sharpen your environment commands through interactive wargames on OverTheWire: Bandit Sandbox Lab Chambers.

A.M. Rinas

Written by A.M. Rinas

Full-Stack Software Engineer working on web applications, DevOps automation, and databases. More about me on the about page, or say hi through the contact page and LinkedIn.

← Back to all articles