You know, Herman, I was looking through some old bookmarks the other day, and it is honestly depressing how many of them lead to nowhere. Just a sea of four hundred four errors and parked domains. It feels like the digital equivalent of a library where the books just vanish off the shelves when you are not looking.
Herman Poppleberry at your service, and Corn, you are describing the exact heartbreak of the modern web. It is called link rot, and it is a much bigger deal than just losing a cool recipe or a funny video. Our housemate Daniel actually sent us a fascinating prompt about this. Studies have found that about twenty to thirty percent of links in citations just stop working after a few years.
That is a staggering number. If you are writing a history book or a legal brief and a third of your evidence just evaporates, your entire work loses its foundation. Daniel was asking about how we actually achieve digital permanence. He mentioned the InterPlanetary File System, or I-P-F-S, which sounds like something out of a science fiction novel, but apparently, it is a real attempt to fix this.
It is very real, and the name is not just for show, though we can get into the space travel implications in a bit. But you are right to be worried about the current state of things. We have built the greatest information repository in human history on top of a system that is fundamentally ephemeral. When you use a regular web link, you are using what we call location addressing. You are telling your browser, go to this specific server at this specific I-P address and look for a file with this specific name.
Right, and if that server goes down, or the owner moves the file to a different folder, the link breaks. The location is still there, maybe, but the content is gone. Or the location itself is gone because someone forgot to pay the domain registration fee. We saw a massive wave of this years ago when Google discontinued goo.gl shortened links in 2018. Millions of citations just turned into digital dust overnight.
Exactly. It is like trying to find a person by going to the last house they lived in. If they move, you are out of luck. And that is where I-P-F-S flips the script. Instead of location addressing, it uses content addressing. Instead of saying, go to this house to find the person, you are essentially shouting the person's name into a crowd, and anyone who knows them can point you in the right direction.
Okay, I like the analogy, but how does that actually work in practice? If I am looking for a specific P-D-F of a research paper, how does the network know what I am talking about if I am not giving it a U-R-L?
It uses something called a cryptographic hash. When you put a file onto I-P-F-S, the system looks at every single bit and byte in that file and runs it through an algorithm to produce a unique fingerprint, or a Content Identifier, usually called a C-I-D. If even a single comma changes in that document, the C-I-D changes completely. So, when you want to find that paper, you ask the network for that specific C-I-D. You are not asking for a location; you are asking for the content itself.
That is a fundamental shift. So if I have a copy of that paper on my computer, and you have a copy on yours, and a library in Sweden has a copy, the network can grab it from whoever is closest or fastest?
Precisely. It is a peer to peer network, similar to how BitTorrent works, but generalized for the whole web. This is why Daniel mentioned it being virtually indestructible. As long as at least one node on the network is hosting that C-I-D, the information is still accessible. It does not matter if the original author's website goes dark. The content lives on through the network.
I can see why Daniel is interested in this for book citations. If an author cites an I-P-F-S link, they are citing the permanent fingerprint of the data. But there is a catch, right? You mentioned earlier that as long as at least one node is hosting it, it stays alive. But what happens if everyone stops hosting it? Does it just vanish like a regular website?
That is the big hurdle. This is where we talk about pinning. In I-P-F-S, nodes usually keep a cache of the files they have recently accessed, but they will eventually clear that cache to make room for new stuff. If you want a file to stay on the network permanently, you have to pin it. That means you are telling your node, never delete this file.
So, for a book author, they would probably need to run their own node or pay a service to pin their citations to ensure they do not disappear. It is not exactly a set it and forget it solution for the average person yet.
Not yet, no. And that is why Daniel noted it is not very user friendly. For a while, the Brave browser had native IPFS support, but they deprecated it in 2022 because so few people were using it. Today, most people use the I-P-F-S Companion extension or access it through gateways, which are basically bridges between the regular web and the I-P-F-S network. But using a gateway brings back some of that centralization risk we were trying to avoid.
It is interesting that we are seeing this tension between convenience and permanence. We have become so used to the ease of clicking a link that the idea of managing our own data nodes feels like a huge burden. But we are paying for that convenience with the literal memory of our civilization. I remember back in episode two hundred thirty four, we talked about mesh networks in emergency shelters. It feels like I-P-F-S is the data layer version of that. It is about resilience through redundancy.
That is a great connection, Corn. In a mesh network, you do not rely on a central cell tower. In I-P-F-S, you do not rely on a central server. And the implications for things like censorship are huge. If a government wants to take down a specific website, they just go to the hosting provider or the domain registrar and shut it down. But if that website is on I-P-F-S, and thousands of people across the globe have pinned fragments of it, it is nearly impossible to kill. It is like trying to remove a drop of ink from the ocean.
That brings up a thorny issue, though. If it is indestructible and uncensorable, what happens with illegal or harmful content? On the regular web, there are mechanisms, however imperfect, to remove things. In a truly decentralized, content addressed world, does everything live forever, even the stuff we really wish would not?
That is the double edged sword of decentralization. Content moderation becomes a local choice rather than a global one. A node operator can choose not to host or serve certain C-I-Ds, but they cannot stop someone else from doing it. It puts the responsibility back on the individual and the community. But from a purely archival perspective, like what Daniel's author friend needs, that permanence is a feature, not a bug.
Let's talk about the name for a second. InterPlanetary. Is that just a marketing guy having a good day, or is there a technical reason for it?
Oh, I was hoping you would ask. It is actually a very technical reason related to the speed of light. Think about how the current web works. It is very chatty. Your browser and the server are constantly sending little messages back and forth. This works fine on Earth because the latency is low. But imagine you are on Mars and you are trying to access a website hosted on a server in Jerusalem.
The round trip time for light between Earth and Mars can be anywhere from four to twenty four minutes depending on where the planets are in their orbits.
Exactly. So if you are using H-T-T-P, which requires multiple round trips just to establish a connection and request a file, it would take forever to load a single page. It would be unusable. But with I-P-F-S, if someone else on Mars has already downloaded that file, your computer can just grab it from them. You do not have to go all the way back to Earth. The content is addressed by what it is, not where it is, so the network naturally optimizes for the nearest copy.
So it is literally built for a multi planetary civilization. That is actually brilliant. It solves the latency problem by turning every user into a potential server.
Precisely. And while we are not quite at the point of needing it for Mars yet, that same logic applies to local networks here on Earth. If we are in the same house in Jerusalem and I want to see a file you just looked at, my computer should not have to go to a data center in Virginia to get it. I-P-F-S makes the local network smarter.
So, why isn't it the standard yet? If it solves link rot, handles censorship better, and is ready for the space age, why are we still stuck with our fragile U-R-Ls?
A few reasons. First, the web we have is incredibly entrenched. Billions of dollars have been spent optimizing the current infrastructure. Second, I-P-F-S is still computationally expensive compared to just serving a file from a central disk. Generating those hashes and searching a global peer to peer network takes time and energy. And third, there is the incentive problem. Why would I spend my hard earned disk space and bandwidth hosting your book citations for free?
Right, the tragedy of the digital commons. If everyone just consumes and nobody hosts, the network dies.
Exactly. This is where things like Filecoin come in, which is a cryptocurrency built on top of I-P-F-S. It is designed to create a marketplace for storage. You can pay people in Filecoin to host your data, and they can earn it by providing storage to the network. It adds a layer of economic permanence to the technical permanence.
I can see how that would appeal to an institution like a library or a university. They could commit a certain amount of resources to ensuring their archives stay pinned. But for Daniel's friend, the author, it still feels like a high barrier to entry. I wonder if we will eventually see browsers just integrate this natively, so you do not even know you are using it.
We are moving in that direction, but as I mentioned with Brave, native support is actually a tough sell for browser companies right now. For this to really work, it has to be invisible. People do not want to manage nodes; they just want their links to work.
It reminds me of the early days of the internet, where you had to know all sorts of technical details just to send an email. Now it is just a button. I imagine digital permanence will follow a similar path. We will probably have services that act as the permanent backbone, where you pay a small fee or it is bundled with your hosting, and it automatically mirrors everything to I-P-F-S.
That is the dream. And there are projects like Perma dot c-c, which Daniel mentioned. They are doing great work, especially for legal and academic citations. They essentially take a snapshot of a page and host it on their own servers. It is a more centralized approach, which makes it easier to use right now, but it still relies on that one organization staying healthy.
Right, and if the organization behind Perma dot c-c goes under, we are right back to square one. It is a better band aid, but it is still a band aid. I-P-F-S feels more like a fundamental cure for the disease of ephemerality.
It really is. And it is not just about links. Think about the way we version software or the way we handle large datasets for scientific research. If you change one line of code or one data point, you have a new C-I-D. You can perfectly track the evolution of information without ever worrying that the previous version was overwritten or lost. It creates a verifiable history of data.
That is a huge point. In an era of deepfakes and misinformation, having a way to verify that a piece of data is exactly what it claims to be, because its C-I-D is a mathematical proof of its content, is incredibly powerful. You are not trusting the person who gave you the file; you are trusting the math.
Exactly. Truth by consensus and calculation, rather than by authority. It is a very different way of thinking about the world. But I should mention, it is not just I-P-F-S. There are other players in this space. Have you heard of Arweave?
I have heard the name, but I am not sure how it differs.
Arweave is another decentralized storage network, but they take a different approach to the incentive problem. Instead of a monthly fee like Filecoin, you pay a one time upfront cost to store data forever. They use an endowment model where the interest earned on your payment covers the cost of storage for hundreds of years as hard drive prices continue to drop.
That is an even bolder claim. Forever is a very long time in the world of technology.
It is! But it shows how much people are craving this. We are realizing that our digital lives are built on sand, and we are looking for ways to turn that sand into stone. Whether it is I-P-F-S or Arweave or something else entirely, the shift toward content addressing seems inevitable if we want our digital culture to survive us.
It is funny, we think of the digital world as being so advanced, but in some ways, a clay tablet from four thousand years ago is more reliable than a website from four years ago. We have traded durability for density and speed. Now we are trying to figure out how to get the durability back.
You nailed it. We are trying to build the modern equivalent of the Great Library of Alexandria, but one that cannot be burned down because it exists everywhere at once.
So, for someone like Daniel or his author friend, what is the practical takeaway here? Should they be jumping into I-P-F-S right now, or is it still too early?
I would say it is a both and situation. For critical stuff, use something like Perma dot c-c because it is reliable and easy for readers to use today. But if you are technically inclined, start experimenting with I-P-F-S. Pin your most important work. Support the network. The more people who use it, the more user friendly those tools will become. And honestly, just being aware of link rot is half the battle. If you know your citations are fragile, you can take steps to archive them in multiple places.
It also makes me think about how we personally handle our data. I have hard drives full of photos and documents, but who knows if those drives will even spin up in ten years? Maybe I should be looking into pinning my family history to a decentralized network.
It is a valid thought. We often think of the cloud as this magical, permanent place, but the cloud is just someone else's computer. And that someone else might go out of business or change their terms of service. Decentralized storage puts the power back in your hands, but it also puts the responsibility there too.
I think that is a recurring theme in a lot of the tech we discuss. We are moving away from the era of big, centralized gatekeepers and toward a more fragmented, but hopefully more resilient, landscape. It is messier, and it requires more from us, but the potential for a truly permanent human record is worth the effort.
I couldn't agree more. And you know, speaking of the human record, we have been doing this show for a while now, and I sometimes wonder where our own episodes will end up in fifty years.
Well, hopefully, they will be pinned on a thousand nodes across the solar system, so some bored colonist on a moon of Jupiter can listen to us talk about link rot.
That is the dream, Corn. That is the dream. But for now, we are still very much here on Earth, in Jerusalem, and we are so glad you all are listening.
Absolutely. And hey, if you find this kind of deep dive into the guts of the internet interesting, we would really appreciate it if you could leave us a review on your podcast app or over on Spotify. It genuinely helps other curious minds find the show.
It really does. We love seeing this community grow. And if you want to get in touch or see what else we have been up to, you can always find us at our website, myweirdprompts dot com. We have the full archive there, including some of those past episodes we mentioned.
Like episode one hundred twenty five where we first introduced ourselves. Hopefully, that link still works! If not, I guess we know what we need to do.
We will get right on pinning it. Thanks for the prompt, Daniel. It is always a pleasure to nerd out about the future of information with you, Corn.
Same here, Herman. This has been My Weird Prompts. We will catch you all in the next one.
Until next time! Stay curious, everyone.
