Daniel's follow-up to the sovereignty episode, and he's pushing on the part we barely touched. You've got encrypted data sitting in one place, never moving. You generated a key pair, that part's trivial, but now the key itself has to survive a dead drive without ever living on anything that touches a network. No password manager, no cloud sync. He wants the hardware token angle, and he wants it before we get anywhere near transmission.
The vault that never leaves the room. And his instinct is right that the threat model actually gets simpler, not harder. No key exchange, no forward secrecy, no intermediaries. You're not protecting a channel, you're protecting a single object. The whole problem collapses into two questions. How do I keep this key from being copied, and how do I keep it from being destroyed.
And those two pull in opposite directions. The thing that makes a key impossible to copy is the same thing that makes it one dropped drive away from vanishing.
And that tension is why the boring answer keeps winning. You want at least two encrypted backups in separate physical locations. That's the floor. Not a recommendation, the floor. And the primary key should live permanently offline, in a safe, while what you actually use day to day is a subkey that sits on hardware and can be replaced if it dies.
So the master key is the one you never touch, and the thing on the token is almost disposable.
Disposable in the sense that losing it is an inconvenience, not a catastrophe. You use the offline master to certify a new subkey, load it onto a fresh token, and you're back. But if you lose the master, the data's gone. That's the asymmetry.
Which is why Daniel's asking about hardware that's more specific than a YubiKey. And he's right to suspect there's a tier above it, but maybe not in the direction he thinks. It's not ruggedized bricks, it's HSMs.
Right. A YubiKey 5 is fifty to eighty dollars, does OpenPGP and FIDO2 and PIV, and the private key material cannot be copied back out. All the crypto happens on the device. That's already a serious piece of hardware. But the thing above it is a hardware security module, an HSM. Yubico makes one, the YubiHSM 2. It's a USB stick that is a real HSM, designed for protecting certificate authority root keys from being copied by attackers, malware, or your own insiders.
And the difference between a YubiKey and an HSM isn't just price. It's what the key is allowed to do.
The HSM is built around the idea that the key never leaves the sealed environment, ever. Key generation happens inside the box. Signing happens inside the box. If you want to move the key to another HSM, you don't export the key, you wrap it. There's a key-encryption key that lives on the device, and the actual private key gets encrypted under that before it ever leaves. So what you're backing up isn't really the key, it's a wrapped blob that only another device with the right key-encryption key can unwrap.
So even the backup is useless unless you already hold the thing that decrypts the backup. That's either brilliant or a great way to lock yourself out with extra steps.
It's brilliant if you think about what the attacker has to do. Stealing the backup file gets them ciphertext. Stealing the HSM gets them a device with a PIN and a tamper response. They need both, plus the PIN, plus physical access. And the SmartCard-HSM, which is the other big player here, calls this key-encryption key a DKEK, a device key encryption key. It's a 256-bit symmetric key that lives inside the card and wraps key material for backup and migration.
And the SmartCard-HSM comes in form factors that a YubiKey doesn't. Mini SIM, microSD, dual interface. That microSD one is interesting for an air-gapped machine that has an SD slot and no USB.
It holds 60 kilobytes of key material in the microSD version, 120 in the USB. Supports 4096-bit RSA, 521-bit ECC, 256-bit AES. And the whole point is that the private keys never leave in plaintext. To move a key between devices, or to keep an offline backup, the card encrypts it under that DKEK. So the backup exists, but it's only meaningful to a device in the same family.
Which brings up the thing nobody wants to say out loud. You've now trusted a vendor.
That's the HSM trust problem, and it's real. HSMs solve key exfiltration by making a sealed box that you cannot inspect. The key generation happens inside, and you have no way to verify what was generated. You're trusting the FIPS certification, which means trusting the vendor, which means trusting the vendor's supply chain, which means, if you're building a sovereign stack, you might be trusting a foreign company's sealed box. And there are really only two big players in the traditional HSM world, SafeNet and Thales, and that duopoly has its own questions hanging over it.
So the more secure the box, the more you're betting on the people who built the box. For a government doing data sovereignty, that's not a small bet.
And it's why the open-source options matter. Nitrokey, made in Germany, the Nitrokey 3 has a secure element certified to Common Criteria EAL 6 plus. The Nitrokey Storage adds a 4 gigabyte encrypted USB mass storage volume, with a hidden volume feature for plausible deniability. The firmware's open, the hardware's documented, you can actually look at what you're running.
EAL 6 plus is a serious certification. That's not a sticker.
It's the level where the evaluators assume the attacker has high attack potential, and they're looking at the formal design and the implementation. It's not unbreakable, nothing is, but it's the kind of assurance you want if the key is protecting something valuable. And the hidden volume thing is clever. If someone forces you to unlock the device, you unlock the decoy volume, and the real one stays invisible.
Though Daniel's scenario is a vault that isn't going anywhere. The key isn't being carried around. So the plausible deniability feature matters less than the fact that the volume itself is encrypted at rest on the token.
And that's the thing about the Nitrokey Storage specifically. It's not just a key manager, it's encrypted storage with the key material living in the secure element. So the key and the encrypted volume it protects are on the same physical device, but the key never enters the host machine's memory. The host just sends data to be encrypted, the device does it, the host gets ciphertext back.
Let's talk about the actual failure that Daniel's worried about. The drive dies. What's the recovery path?
Depends on what layer failed. If you're using LUKS for full disk encryption, and the drive dies, you restore from backup. That's straightforward. But there's a nastier failure that people don't think about until it happens to them. The LUKS header gets corrupted. The header is this small region at the start of the partition that holds the keyslots and the metadata. If that gets damaged, the passphrase is useless. The data's still there, but there's no way to unlock it.
And the cryptsetup FAQ basically says that's the most common support question they get. People who damaged the start of their LUKS partition, and there's nothing to be done.
The fix is so simple that it's almost insulting. You run a header backup command. It dumps the header and keyslot area to a file, maybe a few megabytes, and you store that file somewhere safe. Then if the header gets corrupted, you restore it from the backup, and as long as you have a passphrase that was valid at backup time, you can decrypt. Even if you changed the passphrase later, the backup's still good.
So the thing that saves you isn't a fancier key, it's a copy of the metadata that everyone forgets exists.
And this is the pattern that keeps showing up in high-security key management. The expensive hardware gets all the attention, but the actual resilience comes from boring, unglamorous backups. A header backup file on a USB stick in a drawer. A paper copy of the key in a safe. The drduh guide, which is the canonical community reference for this stuff, says the master key should be kept permanently offline in a physical safe. And it recommends a passphrase using only uppercase letters and numbers, specifically because that limited character set makes handwritten storage easier to write and read without ambiguity.
Handwritten. As in, the backup medium is paper.
Paper in a safe. And there's a tool called Paperkey that takes your secret key and exports just the secret portion, the part that's actually sensitive, in a format designed to be printed. The public key can be reconstructed from anywhere, it's public. The secret part is what you print, and you can encode it as a QR code so it's easy to scan back in.
Paper has no firmware, no supply chain, no vendor, no operating system. It doesn't phone home. It doesn't have a vulnerability disclosure process. It just sits there.
And that's the paradox Daniel's prompt is circling. He's asking for ruggedized hardware, and the most ruggedized backup medium in 2026 is still paper in a safe. Because every piece of hardware has firmware, and firmware has bugs, and firmware comes from a vendor. Paper has none of that.
Though paper burns, and floods, and fades. It's durable in the security sense, not the physical sense.
Which is why you have two copies in separate locations. The threat model isn't a single disaster, it's total loss. A fire in one building shouldn't take out both copies. And if you're really paranoid, you laminate the paper, or you use the metal plate thing that people do for cryptocurrency seed phrases. Stamp the words into steel, and now it survives a house fire.
The crypto people solved this years ago. Seed phrase, steel plate, safe deposit box. The whole point is that the backup is dumb. It doesn't compute anything, it just exists.
And there's a subtlety there that matters. Not every key needs to be backed up. A signing key, if you lose it, you generate a new one and move on. It's ephemeral. But an encryption key, if you lose it, the data is gone forever. So the backup strategy depends on what the key is for. Daniel's vault key is an encryption key. It needs the full treatment.
What about the PIN and access control side? He mentioned wanting access control on the device.
The YubiKey has a user PIN and an admin PIN. User PIN minimum six characters, admin minimum eight. Three wrong user PINs and the key blocks. Three wrong admin PINs and the device wipes itself. So the token has a built-in self-destruct for the admin path.
Three wrong admin tries and the whole thing zeroes. That's a lot of trust in whoever's typing.
That's the point. If someone steals the token and tries to brute force the admin PIN, they get three attempts before the key material is destroyed. And the user PIN is separate, so a normal user can sign and decrypt with their PIN, but they can't change the key's configuration. The admin PIN is for managing the device itself.
And this is where the subkey structure earns its keep. The master key, the Certify key, lives offline. The subkeys on the token have an expiration, typically two years. So even if the token is compromised, the damage is bounded. You revoke the subkey, generate a new one, and the master was never exposed.
The Certify key itself has no expiration, because it's the root of trust. It only gets used to sign new subkeys, and that happens on an air-gapped machine, ideally an ephemeral live system that boots from read-only media and has no network capability at all. The drduh guide ranks environments, and the top tier is an air-gapped system without network capabilities, preferably ARM-based, like a Raspberry Pi.
A Raspberry Pi as the most secure environment. That's going to upset some people.
The logic is sound. An ARM board has no Intel Management Engine, no AMD equivalent, no firmware that's a black box with a network stack baked in. It's simple enough that you can audit what's running. And for key generation, you boot a live image, generate the key, export what you need, and shut down. The machine never touches a network, the key never touches a hard drive.
And the drive the computer is backed by, in Daniel's scenario, is the thing that might fail. But if the key was generated on an ephemeral system, the key was never on that drive in the first place. It went straight to the token and the paper.
That's the workflow that makes the whole thing coherent. Generate on air-gapped hardware. Export the public key. Export the secret key via Paperkey to print. Load subkeys onto two hardware tokens. Store the master key offline. Store the paper in a safe. Store the header backup if you're using LUKS. And now the drive can fail, the token can fail, the paper can get lost, and you still have a path back.
The thing I keep noticing is how much of this is process, not product. Daniel asked about hardware devices, and the hardware matters, but the hardware is the easy part. The process is where people actually lose their data.
And the process has to be rehearsed. You can't wait until the drive dies to discover that your recovery procedure doesn't work. You do a dry run. You restore the header from backup on a test machine. You verify that the paper key actually reconstructs. You confirm that the second token can decrypt what the first token encrypted.
Which nobody does, because it's tedious, and the data's not gone yet, so why borrow trouble.
And then the drive dies, and they find out that the backup file was corrupted, or the paper was printed with a printer that caches documents in memory, or the second token was from a bad batch. The dry run is the difference between a backup and a hope.
Let's talk about the trust problem with HSMs more directly, because Daniel's context is sovereign networking stacks. A government building its own infrastructure has a specific problem with a sealed box from a foreign vendor.
The problem is that you can't verify what's inside. The key generation happens in the HSM, and the whole design philosophy is that the key can never leave. So you can't inspect the key. You can't prove that the vendor didn't put a backdoor in the firmware. You can't prove that the random number generator isn't biased in a way that makes the keys predictable. You're trusting the certification, which means trusting the certifying body, which means trusting the country that issued the certification.
For a sovereign stack, that's importing sovereignty from someone else.
And it's not a theoretical concern. The SafeNet and Thales duopoly has been discussed for years, with people pointing out that if a government wants to read your traffic, the easiest place to do it is at the HSM vendor. The HSM is the one place where the keys are generated and stored, and if the vendor cooperates, the whole thing's open.
The porcupine defense. The idea that the tamper-evident logging inside the HSM is so thorough that a government would rather not ask, because the cost of the request becoming known is too high.
That's the optimistic view. The pessimistic view is that the vendor just ships a firmware update that exfiltrates the keys, and nobody notices because the box is sealed and the firmware is signed. You'd need to be doing side-channel analysis on your own HSM to catch it, and almost nobody does that.
So the open-source options, Nitrokey, SmartCard-HSM, they're not just about price. They're about the ability to actually look at the thing.
SmartCard-HSM is interesting because it's a small German company, the firmware is documented, and the DKEK mechanism is described in enough detail that you can understand what's happening to your keys. You're still trusting the secure element hardware, which is a chip from a semiconductor vendor, so the supply chain problem doesn't fully go away. But it's a smaller trust surface than a sealed appliance from a defense contractor.
For a vault that never moves, the threat model is mostly about physical access anyway. Someone breaks into the room, finds the token, tries to brute force the PIN, fails, and the token wipes itself. The network attack surface is zero, because the key never touched the network.
That's the thing about air-gapped key management. The attacks that work against networked systems, the remote exploits, the supply chain attacks, the cloud provider subpoenas, they all disappear. What's left is physical theft, rubber hose cryptanalysis, and the failure modes of the hardware itself.
Rubber hose cryptanalysis being the one where someone hits you with a rubber hose until you type the PIN.
That's what the hidden volume on the Nitrokey Storage is for. You give them the decoy PIN, the decoy volume opens, and the real one stays hidden. Whether that holds up under sustained pressure is a different question, but it's a real feature for a real threat.
Daniel's also asking about the key generation itself. He says creating a key pair on Linux is trivial, and he's right. But there's a difference between generating a key and generating a key that you can trust.
The difference is entropy. On a normal Linux system, the kernel's random number generator is fine for most purposes. But if you're generating a key that protects something valuable, you want to be sure the entropy source is good, and you want to do it on a machine that hasn't been compromised. That's why the air-gapped live system is the standard recommendation. Fresh boot, no network, generate the key, export it, shut down.
The machine never knows what it generated, because it boots from read-only media and the key goes straight to the token and the paper. There's nothing left on the machine to find.
The drduh guide is very specific about this. The top tier is an air-gapped ARM board. The next tier down is a dedicated hardened Debian or OpenBSD machine. Below that, a VM. Below that, your daily OS. The point is that the key generation environment should be as clean as possible, because any compromise of that environment means the key is compromised before it's even created.
The cleanest environment is the one that doesn't exist anymore. Boot it, use it, shut it down. The key generation machine is a phantom.
Which is why the Raspberry Pi recommendation makes sense, even though it sounds like a joke. It's cheap enough to dedicate one to this purpose, it's simple enough to audit, and it has no built-in management engine that could be hiding something. You flash an SD card with a live image, generate the key, and pull the SD card. The machine is now blank again.
What about the actual encryption of the vault itself? We've been talking about key management, but the key has to encrypt something.
If it's a LUKS volume, the key is actually a passphrase that unlocks a keyslot, and the keyslot holds the actual encryption key. So the thing you're backing up is the passphrase, plus the header. If it's a file-level encryption like GPG, the key is the private key that decrypts the file. If it's a database, it might be an application-level key that's stored in a key management system. The hardware token is the thing that holds the key, but the key's relationship to the data depends on the encryption layer.
Daniel mentioned AES-256 specifically. That's the algorithm, but the algorithm is the least interesting part. The interesting part is where the key lives and how it's protected.
AES-256 is a symmetric cipher. The same key encrypts and decrypts. So if you're using AES-256 for a vault, the key is a symmetric key, not a key pair. Daniel said private key and public key pair, but for a vault that never moves, you don't need asymmetric crypto at all. You just need a symmetric key, or a passphrase that derives a symmetric key.
That's a good catch. The public-private pair is for when you need to exchange keys with someone else. If the data never leaves, symmetric is simpler and faster.
The hardware token story changes a bit. A YubiKey's PIV and OpenPGP features are built around asymmetric keys. For symmetric keys, you'd be using the token's challenge-response mode, or storing the symmetric key in the token's protected memory. The Nitrokey Storage's encrypted volume is actually a good fit, because it's doing symmetric encryption with the key in the secure element.
The tool depends on whether Daniel's vault is encrypted with a passphrase, a symmetric key, or an asymmetric key pair. And he said key pair, so maybe he's thinking of GPG-style encryption of files.
In that case, the YubiKey's OpenPGP applet is exactly the right tool. You generate the key pair on the air-gapped machine, move the subkeys to the YubiKey, and the private key never exists on the vault machine. You decrypt by having the YubiKey do the decryption on-device.
The backup is the master key, offline, on paper, in a safe. Plus a second YubiKey with the same subkeys loaded.
Two YubiKeys is the standard recommendation. One for daily use, one as backup, configured identically. If the daily one breaks or gets lost, you're not locked out. And the backup one lives in a safe place, not on your keychain.
The thing I want to pin down is the actual recovery procedure. Walk me through it. The drive dies, the vault is gone, you have a backup of the encrypted data and a backup of the key. What do you actually do?
You get a new machine. You restore the encrypted data from backup. You insert the backup YubiKey, or you reconstruct the master key from the paper backup on an air-gapped machine, and you decrypt. If you're using LUKS and the header was on the dead drive, you restore the header from the header backup file first, then unlock with the passphrase.
If the paper is the only copy of the master key, and the paper is in a safe, and the safe is in a building that burned down, you're done.
Which is why the two locations thing matters. The paper in one safe, the header backup in another, the second token in a third. You're spreading the single points of failure out so that no single physical event takes out everything.
This is where the process discipline comes in. You have to actually put the paper in the safe. You have to actually put the second token somewhere else. You have to actually make the header backup. The technology is easy, the follow-through is hard.
You have to document it. If you're the only person who knows where the safe is, and you get hit by a bus, the data's gone. So there's a recovery document, sealed, in a safe deposit box, that explains the procedure. And now you've got a whole other security problem, because that document is itself a key.
The recovery document is the master key with extra steps.
This is why real high-security environments have key ceremonies. Multiple people, each holding part of the recovery procedure, and no single person can reconstruct the whole thing. Shamir secret sharing, where the key is split into pieces and you need a threshold number of pieces to reconstruct it. I couldn't confirm specific government deployments, but the technique is standard in the HSM world.
The key ceremony is the thing where people in a room, with witnesses, generate a root key and split it into shares. It's theater, but it's theater with a purpose. The purpose is that no single person can compromise the key, and no single person can lose it either.
The theater is part of the security. The ceremony creates a record, the record creates accountability, and the accountability is what makes the whole system trustworthy. It's not just about the math.
Daniel's question, how is this done, has a layered answer. The hardware is a YubiKey or a Nitrokey or a SmartCard-HSM or a YubiHSM. The process is air-gapped generation, subkeys on tokens, master key offline, paper backup, header backup, two locations, dry run, documentation.
The surprising part is that the most secure component is the dumbest one. The paper. Because it has no attack surface.
Hilbert: Paper's not dumb.
Hilbert: I ran a print shop in Cleveland in eighty-four. We did backup jobs for a bank. They'd come in with these big magnetic tapes, and we'd print the contents onto green bar paper, box after box of it, and ship it to a limestone mine in Pennsylvania where they stored it. The mine was the backup. The paper was the backup of the backup.
Hilbert: The thing people get wrong is they think paper's fragile. Paper lasts. I've got a box in my flat with pay stubs from seventy-nine. They're fine. The ink's a little faded but you can read every number.
Hilbert: The problem with paper isn't the paper. It's the printer. That bank, they used a daisy wheel printer, because dot matrix ribbons smear and laser toner flakes off after twenty years. The daisy wheel actually stamped the characters into the paper. You could read it with your fingers.
Hilbert: If you're printing a key, you want the same thing. A printer that physically marks the paper, not one that lays toner on top. And you want to check it before you put it in the safe, because a bad print is the same as no print, just with more confidence.
The printer as the weak link. That's the thing nobody thinks about, and it's the thing that actually determines whether the paper backup works in ten years.
It connects to the dry run point. You don't just print the key and file it. You scan it back in, you verify the reconstruction works, and then you file it. The verification is the part that catches the bad print.
Hilbert: We did that. We'd print a test page, scan it back, compare it to the tape. Took forever. The bank paid by the hour so we didn't complain.
Hilbert: One more thing. The mine in Pennsylvania, it wasn't just for paper. They had a whole industry down there. Government records, corporate archives, film reels. The temperature stays the same all year. Fifty-five degrees, fifty percent humidity. Paper loves that.
Hilbert: If you're storing a key on paper for twenty years, a safe in your house is fine. But a hole in the ground with stable temperature, that's better. Just costs more.
The limestone mine as the ultimate air gap. No network, no firmware, no vendor, and the temperature never changes.
It's a reminder that physical security is part of key management. The best encryption in the world doesn't help if someone can walk into the room and take the token, or if the building floods and the paper dissolves.
Hilbert: The bank never lost a tape. The mine flooded once, in ninety-two, but the boxes were on pallets, up off the floor. Everything survived. They moved the whole operation after that, to a salt mine in Kansas. Salt mines are even drier.
Hilbert: Anyway, the point is the paper's not the problem. The problem is everything else.
The misconception, I think, is that the hardware token is the backup. People buy a YubiKey, load the key onto it, and think they're done. The token is not the backup. The token is the thing that can be lost, broken, or destroyed. The backup is the offline master key, the paper, the header file, in separate locations. The token is just the convenient front end.
The correction is that the master key is the backup. The token holds subkeys that can be revoked and replaced. The master key lives offline, in a safe, on paper, and it's the thing that makes the whole system recoverable. Lose the token, you're fine. Lose the master, you're done.
The one thing I'm left wondering is whether the sovereign stacks Daniel was asking about in the first episode are actually using any of this, or whether they're just buying HSMs from the usual vendors and hoping. The open-source path exists, but it takes real effort to do it right.
The effort is the point. A sovereign stack that outsources its key generation to a foreign HSM vendor isn't sovereign. The sovereignty is in the process, not the product. Paper in a safe, in your own country, under your own control. That's the thing that can't be subpoenaed from a cloud provider.
This has been My Weird Prompts. Thanks to our producer, Hilbert Flumingtop, for keeping the show running. If you want to send us a prompt like Daniel did, email us at show at my weird prompts dot com. We'll be back soon.
