{"id":1567,"date":"2025-11-09T15:42:31","date_gmt":"2025-11-09T12:42:31","guid":{"rendered":"https:\/\/www.dchost.com\/blog\/vps-ssh-hardening-without-the-drama-fido2-hardware-keys-ssh-ca-and-safe-key-rotation-step-by-step\/"},"modified":"2025-11-09T15:42:31","modified_gmt":"2025-11-09T12:42:31","slug":"vps-ssh-hardening-without-the-drama-fido2-hardware-keys-ssh-ca-and-safe-key-rotation-step-by-step","status":"publish","type":"post","link":"https:\/\/www.dchost.com\/blog\/en\/vps-ssh-hardening-without-the-drama-fido2-hardware-keys-ssh-ca-and-safe-key-rotation-step-by-step\/","title":{"rendered":"VPS SSH Hardening Without the Drama: FIDO2 Hardware Keys, SSH CA, and Safe Key Rotation Step-by-Step"},"content":{"rendered":"<div class=\"dchost-blog-content-wrapper\"><p>So there I was late one evening, watching a stream of failed SSH logins scroll by like a slot machine that never pays out, and it hit me: most of us treat SSH like a trusty old door lock. It works, we\u2019re used to it, and we only think about it when something rattles. Then a client called about a contractor who\u2019d left the company, their key was still trusted on dozens of servers, and the one person who knew how to clean it up was on vacation. You can imagine the stress. That\u2019s when I doubled down on a calmer way to run SSH: hardware-backed keys, short-lived certificates, and a rotation plan that doesn\u2019t make your heart race.<\/p>\n<p>In this guide, I\u2019ll walk you through the playbook I use today: FIDO2 hardware keys for real \u201csomething-you-have\u201d authentication, an SSH certificate authority (CA) to stop sprinkling raw keys across every server, and a safe, two-phase key rotation process. Think of it as moving from a pocketful of mismatched keys to a single, sensible access badge that can be updated or revoked from the couch. We\u2019ll keep it human, avoid the stiff jargon, and make sure each step builds on the last. And yes, we\u2019ll talk about the little hiccups I\u2019ve run into and how to dodge them.<\/p>\n<div id=\"toc_container\" class=\"toc_transparent no_bullets\"><p class=\"toc_title\">\u0130&ccedil;indekiler<\/p><ul class=\"toc_list\"><li><a href=\"#Why_Hardening_SSH_Feels_Scary_And_Why_It_Doesnt_Have_to_Be\"><span class=\"toc_number toc_depth_1\">1<\/span> Why Hardening SSH Feels Scary (And Why It Doesn\u2019t Have to Be)<\/a><\/li><li><a href=\"#FIDO2_Hardware_Keys_for_SSH_The_Calm_Physical_Factor\"><span class=\"toc_number toc_depth_1\">2<\/span> FIDO2 Hardware Keys for SSH: The Calm, Physical Factor<\/a><ul><li><a href=\"#What_Youll_Need\"><span class=\"toc_number toc_depth_2\">2.1<\/span> What You\u2019ll Need<\/a><\/li><li><a href=\"#Generating_a_Hardware-Backed_SSH_Key\"><span class=\"toc_number toc_depth_2\">2.2<\/span> Generating a Hardware-Backed SSH Key<\/a><\/li><li><a href=\"#Adding_Your_Public_Key_to_a_Server_The_Old_Way_Briefly\"><span class=\"toc_number toc_depth_2\">2.3<\/span> Adding Your Public Key to a Server (The Old Way, Briefly)<\/a><\/li><li><a href=\"#Locking_the_Front_Door_Tighter_sshd_config_Defaults\"><span class=\"toc_number toc_depth_2\">2.4<\/span> Locking the Front Door: Tighter sshd_config Defaults<\/a><\/li><\/ul><\/li><li><a href=\"#SSH_Certificates_CA_The_Grown-Up_Way_to_Trust_Keys\"><span class=\"toc_number toc_depth_1\">3<\/span> SSH Certificates (CA): The Grown-Up Way to Trust Keys<\/a><ul><li><a href=\"#Step_1_Create_the_User_CA\"><span class=\"toc_number toc_depth_2\">3.1<\/span> Step 1: Create the User CA<\/a><\/li><li><a href=\"#Step_2_Tell_Servers_to_Trust_the_CA\"><span class=\"toc_number toc_depth_2\">3.2<\/span> Step 2: Tell Servers to Trust the CA<\/a><\/li><li><a href=\"#Step_3_Sign_User_Keys_With_Principals_and_Short_TTLs\"><span class=\"toc_number toc_depth_2\">3.3<\/span> Step 3: Sign User Keys With Principals and Short TTLs<\/a><\/li><li><a href=\"#Step_4_Prune_authorized_keys_and_Enjoy_the_Quiet\"><span class=\"toc_number toc_depth_2\">3.4<\/span> Step 4: Prune authorized_keys and Enjoy the Quiet<\/a><\/li><li><a href=\"#Bonus_Host_Certificates_to_Kill_TOFU_Anxiety\"><span class=\"toc_number toc_depth_2\">3.5<\/span> Bonus: Host Certificates to Kill TOFU Anxiety<\/a><\/li><\/ul><\/li><li><a href=\"#Safe_Key_Rotation_A_No-Drama_Two-Phase_Plan\"><span class=\"toc_number toc_depth_1\">4<\/span> Safe Key Rotation: A No-Drama Two-Phase Plan<\/a><ul><li><a href=\"#Rotate_the_User_CA_or_Introduce_a_New_One_in_Two_Phases\"><span class=\"toc_number toc_depth_2\">4.1<\/span> Rotate the User CA (or Introduce a New One) in Two Phases<\/a><\/li><li><a href=\"#Rotate_User_Keys_with_Overlap\"><span class=\"toc_number toc_depth_2\">4.2<\/span> Rotate User Keys with Overlap<\/a><\/li><li><a href=\"#Rotate_Host_Keys_Without_Surprises\"><span class=\"toc_number toc_depth_2\">4.3<\/span> Rotate Host Keys Without Surprises<\/a><\/li><li><a href=\"#Break-Glass_Access_and_Rollback\"><span class=\"toc_number toc_depth_2\">4.4<\/span> Break-Glass Access and Rollback<\/a><\/li><\/ul><\/li><li><a href=\"#Day-to-Day_Short-Lived_Certs_Logs_and_Health_Checks\"><span class=\"toc_number toc_depth_1\">5<\/span> Day-to-Day: Short-Lived Certs, Logs, and Health Checks<\/a><\/li><li><a href=\"#Step-by-Step_A_Practical_Rollout_You_Can_Actually_Do\"><span class=\"toc_number toc_depth_1\">6<\/span> Step-by-Step: A Practical Rollout You Can Actually Do<\/a><ul><li><a href=\"#1_Prep_and_Baseline\"><span class=\"toc_number toc_depth_2\">6.1<\/span> 1) Prep and Baseline<\/a><\/li><li><a href=\"#2_Enroll_FIDO2_Keys_for_Your_Team\"><span class=\"toc_number toc_depth_2\">6.2<\/span> 2) Enroll FIDO2 Keys for Your Team<\/a><\/li><li><a href=\"#3_Introduce_the_User_CA\"><span class=\"toc_number toc_depth_2\">6.3<\/span> 3) Introduce the User CA<\/a><\/li><li><a href=\"#4_Add_Host_Certificates\"><span class=\"toc_number toc_depth_2\">6.4<\/span> 4) Add Host Certificates<\/a><\/li><li><a href=\"#5_Build_a_Rotation_Habit\"><span class=\"toc_number toc_depth_2\">6.5<\/span> 5) Build a Rotation Habit<\/a><\/li><li><a href=\"#6_Monitor_Alert_and_Practice_the_Break-Glass\"><span class=\"toc_number toc_depth_2\">6.6<\/span> 6) Monitor, Alert, and Practice the Break-Glass<\/a><\/li><\/ul><\/li><li><a href=\"#Little_Gotchas_and_Friendly_Fixes\"><span class=\"toc_number toc_depth_1\">7<\/span> Little Gotchas and Friendly Fixes<\/a><\/li><li><a href=\"#What_Good_Looks_Like_A_Day_in_the_Life\"><span class=\"toc_number toc_depth_1\">8<\/span> What Good Looks Like (A Day in the Life)<\/a><\/li><li><a href=\"#Common_Questions_I_Hear_Over_Coffee\"><span class=\"toc_number toc_depth_1\">9<\/span> Common Questions I Hear Over Coffee<\/a><\/li><li><a href=\"#Extra_Reading_and_Handy_References\"><span class=\"toc_number toc_depth_1\">10<\/span> Extra Reading and Handy References<\/a><\/li><li><a href=\"#Wrap-Up_Calm_Strong_and_Repeatable\"><span class=\"toc_number toc_depth_1\">11<\/span> Wrap-Up: Calm, Strong, and Repeatable<\/a><\/li><\/ul><\/div>\n<h2 id=\"section-1\"><span id=\"Why_Hardening_SSH_Feels_Scary_And_Why_It_Doesnt_Have_to_Be\">Why Hardening SSH Feels Scary (And Why It Doesn\u2019t Have to Be)<\/span><\/h2>\n<p>SSH is the front door to your <a href=\"https:\/\/www.dchost.com\/vps\">VPS<\/a> world. If you\u2019ve ever watched bots hammer your server or worried about a lost laptop with private keys on it, you already know the stakes. The good news is that modern SSH, especially with FIDO2 hardware keys and certificates, lets you lock things down without turning everyone\u2019s day into a scavenger hunt. The trick is setting the foundation so rotation and revocation are, frankly, boring. Boring is good.<\/p>\n<p>Here\u2019s the thing most people miss: raw public keys in <code>authorized_keys<\/code> work fine until you need to quickly revoke, expire, or audit at scale. Add a few dozen servers and a couple of human mistakes, and that simplicity turns into weekend work. FIDO2 keys help because they live on a hardware token, and SSH certificates help because you stop trusting raw keys and start trusting signed identities. It\u2019s like stamping a temporary pass on top of a key; the pass expires, the key without a stamp means nothing, and life moves on.<\/p>\n<h2 id=\"section-2\"><span id=\"FIDO2_Hardware_Keys_for_SSH_The_Calm_Physical_Factor\">FIDO2 Hardware Keys for SSH: The Calm, Physical Factor<\/span><\/h2>\n<p>I remember the first time I set up a FIDO2 key for SSH and realized I needed to touch it to log in. It felt oddly reassuring\u2014like turning a physical deadbolt. If someone copied my private key file, it wouldn\u2019t help them, because the signing happens on the hardware. Even better, the token can require a PIN and a touch, so you get that second factor vibe with zero SMS drama.<\/p>\n<p>Let\u2019s keep the overview friendly. FIDO2 is an open standard for secure authentication\u2014hardware keys that prove you are you without handing out secrets like candy. If you want to read more background later, the <a href=\"https:\/\/fidoalliance.org\/fido2\/\" rel=\"nofollow noopener\" target=\"_blank\">FIDO Alliance FIDO2 page<\/a> is a good starting point. OpenSSH has supported FIDO\/U2F keys since 8.2, and it calls them \u201csecurity key\u201d types. On most systems you\u2019ll use <code>ssh-keygen<\/code> with <code>-t ed25519-sk<\/code> or <code>-t ecdsa-sk<\/code>. The \u201c-sk\u201d stands for security key, the hardware thing you plug in or tap. The docs for this are surprisingly readable\u2014see the <a href=\"https:\/\/man.openbsd.org\/ssh-keygen.1\" rel=\"nofollow noopener\" target=\"_blank\">OpenSSH ssh-keygen manual<\/a> if you want every detail and option.<\/p>\n<h3><span id=\"What_Youll_Need\">What You\u2019ll Need<\/span><\/h3>\n<p>You\u2019ll want the following in place: a recent OpenSSH client (8.2 or later has FIDO support), the system libraries for FIDO (often <code>libfido2<\/code> and friends), and a hardware key that supports FIDO2 (YubiKey, SoloKey, Nitrokey, etc.). On macOS and modern Linux, it\u2019s usually plug-and-go after you install the libs. Windows works too (recent builds of OpenSSH and vendor tools are fine), though the setup flow can feel a bit different.<\/p>\n<h3><span id=\"Generating_a_Hardware-Backed_SSH_Key\">Generating a Hardware-Backed SSH Key<\/span><\/h3>\n<p>Warm up those fingers\u2014you\u2019re going to touch the key a few times. Here\u2019s a practical, safe default I use for most people:<\/p>\n<pre class=\"language-bash line-numbers\"><code class=\"language-bash\">ssh-keygen -t ed25519-sk -O resident -O verify-required -C &quot;alice@laptop (FIDO2)&quot;\n<\/code><\/pre>\n<p>What\u2019s going on here? The <code>-t ed25519-sk<\/code> chooses a hardware-backed Ed25519 key. The <code>-O resident<\/code> option stores the credential on the key so you can use it across machines (handy if you enroll the same token on multiple laptops later). The <code>-O verify-required<\/code> tells OpenSSH to require that physical touch during authentication. The comment is just to make your life easier when you look at it later.<\/p>\n<p>When prompted, set a PIN on the hardware key if you haven\u2019t already; it\u2019s your friend. You\u2019ll end up with a public key (something like <code>id_ed25519_sk.pub<\/code>) and the private side bound to the hardware. If you\u2019re curious about options, take a stroll through the ssh-keygen docs later. If you prefer ECDSA over Ed25519 for compatibility reasons, just swap <code>ed25519-sk<\/code> for <code>ecdsa-sk<\/code>.<\/p>\n<h3><span id=\"Adding_Your_Public_Key_to_a_Server_The_Old_Way_Briefly\">Adding Your Public Key to a Server (The Old Way, Briefly)<\/span><\/h3>\n<p>Let\u2019s do a quick sanity test before we get fancy with certificates. Copy your public key to a server\u2019s <code>authorized_keys<\/code>:<\/p>\n<pre class=\"language-bash line-numbers\"><code class=\"language-bash\">ssh-copy-id -i ~\/.ssh\/id_ed25519_sk.pub user@server\n<\/code><\/pre>\n<p>Or, if you prefer to do it by hand, append <code>id_ed25519_sk.pub<\/code> into <code>~\/.ssh\/authorized_keys<\/code> on the server. Try logging in. You should be asked to touch your hardware key. It feels a bit magical the first time.<\/p>\n<p>While we won\u2019t live in <code>authorized_keys<\/code> land for long, this step proves the hardware works. If you run into trouble, check logs on the server with <code>journalctl -u ssh<\/code> or <code>sudo tail -f \/var\/log\/auth.log<\/code>. Many times it\u2019s a permissions thing\u2014SSH is picky about ownership and <code>chmod 600<\/code> on <code>authorized_keys<\/code>.<\/p>\n<h3><span id=\"Locking_the_Front_Door_Tighter_sshd_config_Defaults\">Locking the Front Door: Tighter sshd_config Defaults<\/span><\/h3>\n<p>Since you\u2019re here for hardening, let\u2019s set a good baseline on the server. I like to make changes while keeping an existing session open, just in case. Consider these from experience:<\/p>\n<pre class=\"language-bash line-numbers\"><code class=\"language-bash\"># \/etc\/ssh\/sshd_config (snippets)\nProtocol 2\nPubkeyAuthentication yes\nPasswordAuthentication no\nKbdInteractiveAuthentication no\nChallengeResponseAuthentication no\nPermitRootLogin prohibit-password\nMaxAuthTries 3\nLoginGraceTime 20\nPubkeyAcceptedKeyTypes ssh-ed25519,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com\n<\/code><\/pre>\n<p>Reload with <code>sudo systemctl reload ssh<\/code> (or <code>sshd<\/code> depending on your OS). Test before you close your other session. I can\u2019t tell you how many times that last sentence saved me a trip to a data center.<\/p>\n<h2 id=\"section-3\"><span id=\"SSH_Certificates_CA_The_Grown-Up_Way_to_Trust_Keys\">SSH Certificates (CA): The Grown-Up Way to Trust Keys<\/span><\/h2>\n<p>Here\u2019s the big mindset shift: stop trusting raw keys and start trusting certificates. Instead of sprinkling public keys across every server, we create a simple SSH CA\u2014just a keypair that signs user keys\u2014and we tell servers to trust the CA. Users keep their hardware-backed keys, but those keys need a time-limited certificate stamp to log in. When the certificate expires, access quietly fades. If someone leaves the team, you don\u2019t have to hunt down which servers had their raw key; you just stop signing certificates for them or revoke their certs.<\/p>\n<p>Think of it like a short-lived gym pass. Your identity is your hardware key, but the certificate is the sticker that says you\u2019re allowed in this month. No sticker, no treadmill.<\/p>\n<h3><span id=\"Step_1_Create_the_User_CA\">Step 1: Create the User CA<\/span><\/h3>\n<p>On a secure machine (ideally offline or a well-guarded management host), create a user CA:<\/p>\n<pre class=\"language-bash line-numbers\"><code class=\"language-bash\">ssh-keygen -t ed25519 -f ~\/ssh-user-ca -C &quot;SSH User CA&quot;\n# Creates ssh-user-ca (private) and ssh-user-ca.pub (public)\n<\/code><\/pre>\n<p>Backup the private CA key carefully. It\u2019s the golden stamp. If it walks away, you\u2019ll be rotating everything a lot sooner than you planned. Some teams store it on an encrypted USB in a safe, with checks to sign certs via a controlled process.<\/p>\n<h3><span id=\"Step_2_Tell_Servers_to_Trust_the_CA\">Step 2: Tell Servers to Trust the CA<\/span><\/h3>\n<p>Copy the public part (<code>ssh-user-ca.pub<\/code>) to each server and configure sshd to trust it:<\/p>\n<pre class=\"language-bash line-numbers\"><code class=\"language-bash\"># \/etc\/ssh\/sshd_config\nTrustedUserCAKeys \/etc\/ssh\/ca\/trusted_user_ca.pub\n<\/code><\/pre>\n<p>Put the public CA key at <code>\/etc\/ssh\/ca\/trusted_user_ca.pub<\/code> with proper permissions. Reload sshd. Nothing changes yet for users, but now the server is ready to accept user certificates signed by your CA.<\/p>\n<h3><span id=\"Step_3_Sign_User_Keys_With_Principals_and_Short_TTLs\">Step 3: Sign User Keys With Principals and Short TTLs<\/span><\/h3>\n<p>Instead of adding raw keys to <code>authorized_keys<\/code>, we sign a user\u2019s public key with the CA and include one or more \u201cprincipals.\u201d Principals are like roles or identities. You might use someone\u2019s username plus a group, like <code>alice,ops<\/code>. You can also set source-address limits and command restrictions on the certificate, which means your policies travel with the cert, not the server config.<\/p>\n<pre class=\"language-bash line-numbers\"><code class=\"language-bash\"># Sign Alice's hardware-backed key for 1 week, with principals &quot;alice,ops&quot;\nssh-keygen -s ~\/ssh-user-ca -I alice-weekly -n alice,ops -V +1w -z 1001 \n  -O source-address=203.0.113.0\/24 ~\/alice\/id_ed25519_sk.pub\n\n# Result: ~\/alice\/id_ed25519_sk-cert.pub (the certificate)\n<\/code><\/pre>\n<p>The <code>-I<\/code> is a cert identity string; I use something human like <code>alice-weekly<\/code>. The <code>-n<\/code> sets principals. The <code>-V<\/code> sets validity\u2014one week is a nice place to start. The <code>-z<\/code> is a serial number; unique values make revocation and auditing friendlier. The <code>-O source-address<\/code> pins where the user can come from. Feel free to skip that option if your IPs are more fluid.<\/p>\n<p>On the client side, Alice uses <code>id_ed25519_sk<\/code> plus the new certificate <code>id_ed25519_sk-cert.pub<\/code> when connecting. OpenSSH glues them together automatically if they share the same base name and directory.<\/p>\n<h3><span id=\"Step_4_Prune_authorized_keys_and_Enjoy_the_Quiet\">Step 4: Prune authorized_keys and Enjoy the Quiet<\/span><\/h3>\n<p>Once your CA trust is live on all servers and users have certificates, you can remove raw keys from <code>authorized_keys<\/code>. I usually leave a break-glass key in there with a forced command and tight IP restriction, then store that key offline. It\u2019s an insurance policy for bad days. In normal life, you\u2019ll forget authorized_keys exists, because the trust moved to your CA.<\/p>\n<h3><span id=\"Bonus_Host_Certificates_to_Kill_TOFU_Anxiety\">Bonus: Host Certificates to Kill TOFU Anxiety<\/span><\/h3>\n<p>We\u2019ve talked about user certificates, but you can also sign host keys with a host CA. This replaces the awkward \u201cAre you sure you want to continue connecting (yes\/no\/[fingerprint])?\u201d moment with a clean assertion: your client trusts any host that presents a certificate signed by your host CA.<\/p>\n<pre class=\"language-bash line-numbers\"><code class=\"language-bash\"># Create a host CA\nssh-keygen -t ed25519 -f ~\/ssh-host-ca -C &quot;SSH Host CA&quot;\n\n# On each server, sign the server's host key (usually \/etc\/ssh\/ssh_host_ed25519_key.pub)\nssh-keygen -s ~\/ssh-host-ca -I host-example -h -n server.example.com -V +52w \n  \/etc\/ssh\/ssh_host_ed25519_key.pub\n\n# Add the cert to sshd_config on that server\nHostKey \/etc\/ssh\/ssh_host_ed25519_key\nHostCertificate \/etc\/ssh\/ssh_host_ed25519_key-cert.pub\n<\/code><\/pre>\n<p>On your clients, add a line to <code>~\/.ssh\/known_hosts<\/code> that says \u201cthis CA is trusted for these domains.\u201d It looks like this:<\/p>\n<pre class=\"language-bash line-numbers\"><code class=\"language-bash\">@cert-authority *.example.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA...host-ca-pub...\n<\/code><\/pre>\n<p>Now every server cert signed by your host CA within <code>*.example.com<\/code> Just Works. No TOFU (trust on first use) finger-crossing. It\u2019s amazing how much calmer your SSH sessions feel with this in place.<\/p>\n<h2 id=\"section-4\"><span id=\"Safe_Key_Rotation_A_No-Drama_Two-Phase_Plan\">Safe Key Rotation: A No-Drama Two-Phase Plan<\/span><\/h2>\n<p>Rotation is where people usually get nervous. I don\u2019t blame them. Years ago, I watched a team flip to new keys in one sweep and lock themselves out for half a day. The fix? Stagger the change. Always add before you remove. And give yourself overlap so you can roll back without calling someone at 3 a.m.<\/p>\n<h3><span id=\"Rotate_the_User_CA_or_Introduce_a_New_One_in_Two_Phases\">Rotate the User CA (or Introduce a New One) in Two Phases<\/span><\/h3>\n<p>Sometimes you need to rotate the CA itself\u2014say you\u2019re improving key hygiene, or you\u2019re retiring an old algorithm. The two-phase dance makes it painless:<\/p>\n<p>Phase A: Add. Deploy the <em>new<\/em> CA public key to every server alongside the old one. Your sshd_config ends up with two lines, or one file containing both keys:<\/p>\n<pre class=\"language-bash line-numbers\"><code class=\"language-bash\"># \/etc\/ssh\/sshd_config\nTrustedUserCAKeys \/etc\/ssh\/ca\/trusted_user_ca.pub\nTrustedUserCAKeys \/etc\/ssh\/ca\/trusted_user_ca_new.pub\n<\/code><\/pre>\n<p>Start issuing <em>new<\/em> user certificates from the new CA while still honoring the old. Keep cert validity short. People authenticate as usual, and most won\u2019t even notice the swap.<\/p>\n<p>Phase B: Remove. After everyone\u2019s certs are refreshed and you\u2019ve run for a comfort period, remove the old CA from the servers and stop issuing certs from it. If you need to cut earlier access, you can also publish a revocation list for cert serials you want to deny before expiry.<\/p>\n<h3><span id=\"Rotate_User_Keys_with_Overlap\">Rotate User Keys with Overlap<\/span><\/h3>\n<p>User key rotation is similar. The user generates a new hardware-backed key, you sign both the old and new keys with short-lived certs, and you give them an overlap period to migrate their workflows. If they forget to update a jump host or a CI runner, the old cert covers the gap. After the overlap window, stop issuing certs for the old key, and optionally revoke the old cert serials if you need an immediate cutoff.<\/p>\n<h3><span id=\"Rotate_Host_Keys_Without_Surprises\">Rotate Host Keys Without Surprises<\/span><\/h3>\n<p>Host keys are trickier because clients cache host identities. The nicest pattern is to sign both the old and new host keys with your host CA and deploy both certs on the server during the overlap window. Because clients trust the CA, they accept either cert and don\u2019t panic. When you\u2019re confident, remove the old host key and cert and reload sshd. Most people won\u2019t notice anything happened, which is exactly the outcome you want.<\/p>\n<h3><span id=\"Break-Glass_Access_and_Rollback\">Break-Glass Access and Rollback<\/span><\/h3>\n<p>Have a clearly labeled, heavily restricted break-glass key. Force a specific command (maybe a one-time rotation script), limit source addresses, and keep the private key offline. Note down the rollback steps: re-add the old CA public key, restore the old host cert, and reload sshd. It sounds obvious now, but written checklists save hours when adrenaline kicks in.<\/p>\n<h2 id=\"section-5\"><span id=\"Day-to-Day_Short-Lived_Certs_Logs_and_Health_Checks\">Day-to-Day: Short-Lived Certs, Logs, and Health Checks<\/span><\/h2>\n<p>In daily life, I like 24-hour or 1-week user certs. Short-lived certs are the quiet superpower. You don\u2019t need a mass revoke if a laptop is lost\u2014just stop issuing certs to that key. It feels almost too simple, and that\u2019s the beauty of it.<\/p>\n<p>Logging is your early warning system. Watch for certificate expiry denials, failed principal matches, or unexpected source addresses. If you want a calm way to centralize and visualize auth logs, I\u2019ve written about a setup that\u2019s lightweight and friendly. Take a look at <a href=\"https:\/\/www.dchost.com\/blog\/en\/merkezi-loglama-ve-gozlemlenebilirlik-vpste-loki-promtail-grafana-ile-sakin-kalan-bir-zihin\/\">Centralized Logging on a VPS with Loki, Promtail, and Grafana<\/a>\u2014it\u2019s a great companion for SSH hardening because you can alert on certificate errors and weird logins right away.<\/p>\n<p>On the client side, I\u2019ve found that using the URI-friendly name or a good comment helps people keep track of which key is which. Some folks prefer resident credentials on the token for portability; others keep per-machine keys without resident storage. Either path is fine\u2014just be consistent so rotation doesn\u2019t become a scavenger hunt.<\/p>\n<h2 id=\"section-6\"><span id=\"Step-by-Step_A_Practical_Rollout_You_Can_Actually_Do\">Step-by-Step: A Practical Rollout You Can Actually Do<\/span><\/h2>\n<p>Let\u2019s put it all together in a simple, repeatable plan. This is the playbook I use when a team asks for \u201cmore secure SSH\u201d but doesn\u2019t want downtime.<\/p>\n<h3><span id=\"1_Prep_and_Baseline\">1) Prep and Baseline<\/span><\/h3>\n<p>&#8211; Inventory your servers and who needs access. Keep it light\u2014usernames and the machines they touch are enough.<\/p>\n<p>&#8211; Update OpenSSH on clients and servers to support FIDO2. Install <code>libfido2<\/code> if needed. Make sure you can tap a key and see it light up.<\/p>\n<p>&#8211; Tighten <code>sshd_config<\/code> defaults as shown earlier. Test by keeping one existing session open while you reload.<\/p>\n<h3><span id=\"2_Enroll_FIDO2_Keys_for_Your_Team\">2) Enroll FIDO2 Keys for Your Team<\/span><\/h3>\n<p>&#8211; Have each person generate a hardware-backed key with <code>ssh-keygen -t ed25519-sk -O resident -O verify-required -C \"name (FIDO2)\"<\/code>.<\/p>\n<p>&#8211; Sanity test by adding the public key to a staging server\u2019s <code>authorized_keys<\/code> and logging in. Get that \u201ctouch to authenticate\u201d flow working comfortably.<\/p>\n<h3><span id=\"3_Introduce_the_User_CA\">3) Introduce the User CA<\/span><\/h3>\n<p>&#8211; Create a user CA and store it safely.<\/p>\n<p>&#8211; Deploy the CA\u2019s <em>public<\/em> key to all servers and set <code>TrustedUserCAKeys<\/code>.<\/p>\n<p>&#8211; Sign user keys with principals and a short validity, then remove their raw pubkeys from <code>authorized_keys<\/code> after a calm overlap period.<\/p>\n<h3><span id=\"4_Add_Host_Certificates\">4) Add Host Certificates<\/span><\/h3>\n<p>&#8211; Create a host CA, sign host keys on each server, and configure clients with a <code>@cert-authority<\/code> line in <code>known_hosts<\/code> for your domain. Hostname warnings melt away, and your team stops second-guessing TOFU prompts.<\/p>\n<h3><span id=\"5_Build_a_Rotation_Habit\">5) Build a Rotation Habit<\/span><\/h3>\n<p>&#8211; Put a calendar reminder to rotate user certs weekly or daily, depending on risk appetite. Short-lived certs are the best revoke you\u2019ll never do.<\/p>\n<p>&#8211; Document a two-phase CA rotation and a two-phase host key rotation. Practice in staging first. The muscle memory pays off.<\/p>\n<h3><span id=\"6_Monitor_Alert_and_Practice_the_Break-Glass\">6) Monitor, Alert, and Practice the Break-Glass<\/span><\/h3>\n<p>&#8211; Centralize logs and alert on certificate expiry denials and unusual source addresses.<\/p>\n<p>&#8211; Test that your break-glass key truly works with forced commands and tight IP fences. Then put it back where it belongs\u2014offline.<\/p>\n<h2 id=\"section-7\"><span id=\"Little_Gotchas_and_Friendly_Fixes\">Little Gotchas and Friendly Fixes<\/span><\/h2>\n<p>Hardware-backed SSH keys can surprise you the first week, mostly in delightful ways. Still, let me share a few speed bumps I\u2019ve hit and how to glide over them.<\/p>\n<p>&#8211; Agents and SK keys: Traditional SSH agents don\u2019t \u201chold\u201d hardware-backed keys the way they do file-backed keys. The signing happens on the token, so you\u2019ll touch the key when needed. This is good\u2014don\u2019t fight it. If you use an agent, it typically just helps with file-backed keys or cert selection.<\/p>\n<p>&#8211; Resident vs. non-resident credentials: Resident storage lets the key store credentials so you can discover them on a new machine. It\u2019s handy if you roam, but keep your token PIN safe. Non-resident is more classic\u2014you pair a local private key wrapper to the hardware token. Both are fine; pick one and be consistent.<\/p>\n<p>&#8211; Multiple tokens: I like issuing two tokens per person: a primary and a backup locked away. Sign both keys with the same principals, keep cert lifetimes short, and rotate calmly if one goes missing.<\/p>\n<p>&#8211; Windows and WSL: It works, but test your flow. Sometimes it\u2019s a matter of the right driver or ensuring your OpenSSH build has FIDO support. Yubico\u2019s developer docs are a helpful reference if you go that route; start at <a href=\"https:\/\/developers.yubico.com\/SSH\/\" rel=\"nofollow noopener\" target=\"_blank\">Yubico\u2019s SSH guides<\/a>.<\/p>\n<p>&#8211; Server-side policy: Don\u2019t cram every rule into <code>sshd_config<\/code>. Push source restrictions and command limits into the certificate options when possible. It\u2019s elegant because the policy travels with the identity, not the host.<\/p>\n<h2 id=\"section-8\"><span id=\"What_Good_Looks_Like_A_Day_in_the_Life\">What Good Looks Like (A Day in the Life)<\/span><\/h2>\n<p>Imagine a typical Monday. You plug in your FIDO2 key, tap to unlock the PIN, and SSH into a server. Your certificate expires in 24 hours, so on Tuesday morning your automation refreshes it with a small command or a friendly web portal that signs via the CA. Any laptop left in a taxi won\u2019t do much damage\u2014no cert, no access. If someone leaves the team, you stop issuing certificates for their principal. No frantic key hunts.<\/p>\n<p>Your servers trust the user CA and the host CA. Your team never sees TOFU warnings. You add a new server; it starts serving a host cert signed by the CA, and everyone sails in without touching their known_hosts file. Logs are clean and centralized. If a certificate fails due to expiry, you see the alert and fix it before someone gets stuck in a deploy.<\/p>\n<p>When it\u2019s time to rotate the CA, you don\u2019t sweat. You add the new CA to <code>TrustedUserCAKeys<\/code>, run both for a while, then remove the old one. You rotate host keys with an overlap, clients keep connecting, and your Tuesday stays boring. That\u2019s the goal: security that blends into the furniture.<\/p>\n<h2 id=\"section-9\"><span id=\"Common_Questions_I_Hear_Over_Coffee\">Common Questions I Hear Over Coffee<\/span><\/h2>\n<p>I\u2019ve been asked a dozen variations of these. If they\u2019re on your mind too, you\u2019re in good company.<\/p>\n<p><strong>What if I lose the hardware key?<\/strong> You\u2019ll be glad you kept cert lifetimes short. If you\u2019ve issued certs for one day, by tomorrow that key isn\u2019t getting in. Have a backup token enrolled, keep it safe, and you\u2019ll be back online without a scramble.<\/p>\n<p><strong>Is Ed25519-sk better than ECDSA-sk?<\/strong> Don\u2019t overthink it. I lean Ed25519 for modern defaults, but if compatibility nudges you toward ECDSA, that\u2019s fine. Your real win is \u201c-sk\u201d plus certificates and a rotation habit.<\/p>\n<p><strong>Can I still use jump hosts and tunnels?<\/strong> Absolutely. It all works the same. You\u2019ll just touch the hardware key when the cryptographic magic needs your sign-off. If you cache short-lived certs locally, the experience feels smooth.<\/p>\n<p><strong>Is all this extra work worth it?<\/strong> I\u2019ve found it saves time within two months. The first setup takes a day or two if you\u2019re methodical, and then rotations, revokes, and host additions become the kind of administrative chores you can do with coffee in hand.<\/p>\n<h2 id=\"section-10\"><span id=\"Extra_Reading_and_Handy_References\">Extra Reading and Handy References<\/span><\/h2>\n<p>If you want to dig deeper into the knobs and switches, the primary sources are excellent. The <a href=\"https:\/\/man.openbsd.org\/ssh-keygen.1\" rel=\"nofollow noopener\" target=\"_blank\">ssh-keygen manual<\/a> covers certificates, principals, validity windows, and every obscure option you might want someday. For FIDO2 context, the <a href=\"https:\/\/fidoalliance.org\/fido2\/\" rel=\"nofollow noopener\" target=\"_blank\">FIDO2 overview<\/a> helps explain why hardware-backed keys are so resilient. And if you\u2019re using a YubiKey, the <a href=\"https:\/\/developers.yubico.com\/SSH\/\" rel=\"nofollow noopener\" target=\"_blank\">Yubico SSH developer pages<\/a> have practical advice that maps neatly onto OpenSSH.<\/p>\n<h2 id=\"section-11\"><span id=\"Wrap-Up_Calm_Strong_and_Repeatable\">Wrap-Up: Calm, Strong, and Repeatable<\/span><\/h2>\n<p>If you take one thing from this, let it be this: SSH can be both simple and strong when you anchor it with hardware keys, short-lived certificates, and a rotation plan that errs on the side of boring. Start by getting a FIDO2 login working. Introduce a user CA and push trust to the servers. Add host certificates to kill the TOFU anxiety once and for all. Then make rotation a habit with a gentle two-phase rollout whenever you change something important.<\/p>\n<p>Your future self will thank you the first time you need to revoke access fast or add a new server at 5 p.m. on a Friday. You\u2019ll reach for a few well-documented steps instead of crossing your fingers. Hope this was helpful! If you want me to dive deeper into automating the CA workflow or building a self-serve certificate portal for your team, let me know\u2014I\u2019m happy to share what\u2019s worked for me.<\/p>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>So there I was late one evening, watching a stream of failed SSH logins scroll by like a slot machine that never pays out, and it hit me: most of us treat SSH like a trusty old door lock. It works, we\u2019re used to it, and we only think about it when something rattles. Then [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1568,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[26],"tags":[],"class_list":["post-1567","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-teknoloji"],"_links":{"self":[{"href":"https:\/\/www.dchost.com\/blog\/en\/wp-json\/wp\/v2\/posts\/1567","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dchost.com\/blog\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dchost.com\/blog\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dchost.com\/blog\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dchost.com\/blog\/en\/wp-json\/wp\/v2\/comments?post=1567"}],"version-history":[{"count":0,"href":"https:\/\/www.dchost.com\/blog\/en\/wp-json\/wp\/v2\/posts\/1567\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.dchost.com\/blog\/en\/wp-json\/wp\/v2\/media\/1568"}],"wp:attachment":[{"href":"https:\/\/www.dchost.com\/blog\/en\/wp-json\/wp\/v2\/media?parent=1567"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dchost.com\/blog\/en\/wp-json\/wp\/v2\/categories?post=1567"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dchost.com\/blog\/en\/wp-json\/wp\/v2\/tags?post=1567"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}