Daniel's been thinking about miniature clouds — not the weather kind, the kind where you take one physical machine and carve it up like a hyperscaler does with a warehouse full of racks. He's asking whether Proxmox is the right tool for that, or just the obvious one. What do S3-compatible storage and API-key management actually demand from a software stack? What hardware makes sense — and is a miniature cloud done well even a realistic goal, or are you just backing a business application with ill-fitting consumer tech?
"Ill-fitting consumer tech" — he's already spotted the trap.
He has. And the trap is that virtualization is easy, but cloud semantics are hard. A hypervisor gives you VMs. A cloud gives you self-service, identity, quotas, object storage, and the expectation that all of it works through an API.
So the question is — what actually separates a box with VMs from a cloud?
That's the episode. Let's start with the software. What's out there, and what does each piece actually do?
Proxmox is the obvious place to begin because it's what everyone reaches for first. It handles VMs and containers, it has software-defined storage baked in — Ceph and ZFS — and the web UI is genuinely good. For a single admin managing a few machines, it's basically everything you need.
And that's where the misconception lives. Proxmox alone is not a private cloud. It's a hypervisor with a nice coat of paint.
Right. The gap is in what Proxmox does not do natively. It does not provide S3-compatible object storage. It does not have a tenant-facing identity and access management layer. It has API tokens, but those tokens are for the admin — they're not designed for an end user who logs into a portal, generates their own credentials, and provisions resources against a quota.
So the API tokens exist, but they're the wrong kind of API tokens.
A Proxmox API token says "I am the administrator, let me create VMs." A cloud API key says "I am tenant number seven in project number three, I have a budget of four vCPUs and a hundred gigs of object storage, and I cannot see tenant number eight's resources." Those are different problems.
Let's make that concrete. If I'm tenant seven and I have a Proxmox API token, what can I actually do that I shouldn't be able to do?
You can list every VM on the cluster. You can see the hostnames, the resource allocations, the network configurations. You might not be able to delete tenant eight's VM — depending on how permissions are scoped — but you can absolutely see that it exists, what it's called, and how much RAM it has. In a cloud model, tenant eight's resources are invisible to you. The API simply doesn't return them. Proxmox doesn't have that concept — it has a flat namespace of VMs and a permission model designed for administrators who already have full visibility.
So it's not just that the tokens are different. It's that the entire mental model of who's asking the question is different.
The admin model says "I trust you, here are the keys to the building." The tenant model says "I don't trust you, here's a key that only opens door seven, and I'm not even going to tell you how many other doors there are."
And the S3 part — that's the one that trips people up, because S3 sounds like it should just be files on a disk.
It is not files on a disk. Object storage is a flat namespace of buckets and keys with no directory hierarchy, accessed over HTTP with a specific API. Your filesystem expects POSIX semantics — locks, permissions, directory listings. S3 expects you to PUT an object, GET an object, and list a bucket. They are completely different storage models, and a hypervisor's storage layer doesn't speak S3.
Can you give me an example of where that mismatch actually bites someone? Because on the surface, "I put a file somewhere and I get it back" sounds similar enough.
The classic one is renaming. In a filesystem, you rename a directory with thousands of files and it's an atomic metadata operation — it completes instantly. In S3, there are no directories and there is no rename. You have to copy every object to the new key prefix and then delete the originals. For a bucket with a million objects, that's a million API calls. Someone who treats MinIO like a mounted filesystem discovers this when their script that renames a folder has been running for forty-five minutes and their S3 bill — or their CPU — is on fire.
So you need a separate piece of software. What's the open-source answer?
MinIO. It's the de facto standard. MinIO is an S3-compatible object store that you run as its own service. It handles bucket creation, versioning, lifecycle policies, encryption — the whole S3 API surface. And it manages its own API keys, completely separate from whatever your hypervisor is doing.
Which means you now have two sets of credentials, two management UIs, two things to patch.
And we haven't even added the cloud management layer yet. That's the third piece.
Walk me through the options for that layer.
There are really two paths in open source. The first is OpenNebula. It's the lighter option — it can run on a single node, it has a web UI called Sunstone, and it provides the cloud semantics that Proxmox lacks. User and group management, role-based access control, API key generation for tenants, quotas, and a self-service portal where users can provision their own VMs without touching the hypervisor directly.
And it sits on top of Proxmox?
It can. OpenNebula has a Proxmox driver — it talks to the Proxmox API and manages VMs through it. So your stack becomes Proxmox for the actual compute, MinIO for object storage, and OpenNebula as the management overlay that ties them together with a single identity model and a single API.
That sounds almost elegant.
It does until you hit the seams. OpenNebula manages Proxmox VMs, but it doesn't manage MinIO natively. You can configure it to provision MinIO credentials alongside a VM, but that's integration work — it's not a unified control plane in the way AWS is. Each piece solves one problem well, and the seams between them are where the real work happens.
And those seams are exactly what a true cloud layer is supposed to hide.
That's the key insight. A hyperscaler's cloud feels like one thing because thousands of engineers spent years hiding the seams. When you build a miniature cloud, you are the engineer hiding the seams.
Let me push on that analogy. When I log into AWS and spin up an EC2 instance that writes to an S3 bucket, I don't think about the seams at all. What's actually happening behind that curtain that I'm not seeing?
You're not seeing the IAM policy evaluation that happens on every single API call. When your EC2 instance calls the S3 PutObject API, the request hits an endpoint that validates your instance's role, checks the policy attached to that role, verifies the bucket policy allows the action, checks that no explicit deny overrides it, and only then writes the object. That's multiple services coordinating in milliseconds. In your miniature cloud, your VM makes a request to MinIO with an API key, and MinIO checks its own policy store. Those two policy stores — OpenNebula's and MinIO's — don't talk to each other unless you write the bridge.
You mentioned two paths. What's the second one?
Apache CloudStack. It's heavier — more hyperscaler-like in its architecture. It supports S3 integration natively, it has a full IAM model, and it's designed for multi-tenant deployments from the start. The tradeoff is complexity. CloudStack expects a real deployment with management servers, database backends, and multiple hypervisor hosts. It's not something you spin up on a single box for a weekend project.
So CloudStack is for when you're serious enough to have a rack.
Or at least a small cluster. And that's where the hardware conversation starts.
Before we get to hardware — there's a third path, isn't there? The Kubernetes-adjacent one.
There is. You can build something cloud-like with Kubernetes — Rook to manage Ceph for object and block storage, Longhorn for block storage if you prefer something simpler, and then a layer of tooling on top for the self-service portal. The advantage is that Kubernetes already has strong identity and API primitives. The disadvantage is that you're now running Kubernetes, which is its own operational commitment.
So it's a different kind of complexity, not less complexity.
Right. And for the use case Daniel's describing — a miniature cloud on a small number of machines — Kubernetes is probably overkill. You'd spend more time managing the platform than using it.
I want to pause on that, because "you'd spend more time managing the platform than using it" is a claim that deserves a concrete example. What does that actually look like in practice?
Let's say you're running a three-node Kubernetes cluster for your miniature cloud. You've got Rook managing Ceph. Ceph has an issue — one OSD is flapping. You now have to debug that through the Rook operator, which abstracts the Ceph internals behind Kubernetes custom resources. Your Ceph knowledge has to translate through a Kubernetes lens. Meanwhile, the certificate for your ingress controller expired because cert-manager didn't renew it — something about a DNS challenge that failed. And Kubernetes itself just released a patch version that fixes a security vulnerability, so you need to drain each node and roll the upgrade. None of that is serving your actual workload. It's all platform maintenance.
And that's on top of the Proxmox-level maintenance you'd already be doing.
Kubernetes adds a layer, it doesn't replace one. If you're running Kubernetes on Proxmox VMs, you're now patching Proxmox, patching the VMs, patching Kubernetes, and patching everything inside Kubernetes.
Let's talk about what you'd actually run this on. Because the hardware question is where a lot of these projects go sideways.
The single-machine fallacy is the biggest one. You can run Proxmox on one box. You cannot run a cloud on one box.
Why not?
Because failure domains collapse to one point. If your management layer, your object storage, and your compute all live on the same machine, then a failed RAM stick takes down everything — your VMs, your S3 buckets, your API, your UI. A cloud abstracts hardware failure by spreading it across nodes. A single machine concentrates it.
And there's resource contention.
Massive. Ceph alone wants dedicated disks and a lot of memory for caching. MinIO wants its own storage and its own CPU for erasure coding calculations. Your VMs want whatever's left. On one machine, they're all fighting for the same bus, the same memory channels, the same everything.
Walk me through a real scenario. I've got a single beefy box — say, twenty-four cores, a hundred and twenty-eight gigs of RAM, a bunch of NVMe storage. I put Proxmox, Ceph, and MinIO on it. What actually goes wrong?
The first thing that goes wrong is Ceph's memory pressure. Ceph OSDs default to using about four gigs of RAM each. If you've got six OSDs, that's twenty-four gigs gone before you've launched a single VM. Then MinIO wants RAM for its erasure coding cache — another eight to sixteen gigs depending on your configuration. Proxmox itself needs a few gigs. You're now at roughly forty-five gigs of RAM consumed by infrastructure before any tenant workload exists. Your remaining eighty gigs sounds fine until you realize Ceph and MinIO are also competing for disk I/O. A Ceph scrub operation kicks off while a tenant is uploading a large object to MinIO, and suddenly both operations slow to a crawl because they're hitting the same NVMe bus.
And that's without any VMs running.
Without any VMs running. Now add ten VMs doing their own disk I/O. The single machine isn't just a failure domain problem — it's a performance collapse waiting to happen.
So what does "done well" actually look like?
Three nodes minimum. That's the magic number for Ceph to have proper redundancy — you can survive a node failure without data loss. Each node runs Proxmox, and Ceph is distributed across all three. MinIO can run on the same nodes or on a dedicated storage node. The management layer — OpenNebula or whatever you choose — runs on one of them or on a lightweight fourth node.
And the hardware itself?
The economics have flipped in the last couple of years. Ten-gigabit networking is dirt cheap now — you can get a used ten-gig switch for under two hundred dollars. Used enterprise servers are everywhere. Dell R730s, Supermicro boxes — these are decommissioned data center machines with eight-core Xeons and sixty-four to a hundred and twenty-eight gigs of RAM, and they sell for four to six hundred dollars each.
So for the price of a new MacBook Pro, you can have a three-node cluster.
Roughly. Three used R730s with sixty-four gigs of RAM each, a ten-gig switch, and either a fourth node with spinning disks or a dedicated NAS for bulk storage — you're looking at fifteen hundred to twenty-five hundred dollars total. The whole thing fits in a twelve-U rack.
And that's a real miniature cloud.
That's a real miniature cloud. Compute separated from storage, redundant networking, proper failure domains. You can pull a node for maintenance and nothing goes down.
But you mentioned a NAS — and I want to flag something here, because Daniel specifically brought up the NAS trap.
The NAS trap is real. A Synology or QNAP box is not a cloud component. It's a consumer appliance with a web UI. It serves files over SMB and NFS. It does not expose an S3-compatible API — or if it does, it's a bolt-on feature, not a native object store. It has no IAM model beyond local users and shared folders. And it absolutely does not participate in a multi-tenant cloud control plane.
So the Synology is fine as a backup target.
It's a great backup target. It's a terrible cloud storage node. If you want S3 semantics, you run MinIO on real server hardware with direct-attached disks. If you want block storage for VMs, you run Ceph or ZFS on the nodes themselves. The NAS is for file shares and backups — it's infrastructure, not part of the cloud.
I want to unpack why the Synology's S3 bolt-on isn't the same thing, because someone listening is thinking "but my Synology has a MinIO package."
It's the difference between a feature checkbox and a design commitment. The Synology MinIO package runs as a user-space application on top of a filesystem that was designed for SMB shares. Underneath, MinIO is writing objects to ext4 or Btrfs, which means every S3 PUT operation goes through a POSIX filesystem layer that doesn't understand object semantics. Erasure coding happens at the MinIO level, but the underlying disks are managed by Synology's volume manager, which is doing its own thing. You've got two storage stacks fighting for control. On a dedicated MinIO node with raw disks, MinIO talks directly to the drives and manages everything — bitrot detection, erasure coding, drive failure — without a filesystem layer in between.
So it's not that the Synology can't do it. It's that it's doing it with one hand tied behind its back.
And when something breaks, you're debugging through two abstraction layers instead of one. The Synology logs say the volume is healthy. MinIO says it's getting I/O errors. Somewhere in between, a translation is failing, and you have no visibility into it.
Let's put a concrete stack on the table. Proxmox on three nodes with Ceph for block storage. MinIO for S3. OpenNebula as the management overlay. What does the API key flow actually look like?
A user logs into the OpenNebula Sunstone portal with their credentials. They're in a specific group with a specific quota — say, four vCPUs, sixteen gigs of RAM, two hundred gigs of block storage, and fifty gigs of object storage. They click "create VM," pick a template, and OpenNebula talks to Proxmox through the driver and provisions it. For object storage, they generate an S3 API key through OpenNebula's interface, and OpenNebula — ideally — provisions a corresponding set of credentials in MinIO.
Ideally.
That's the seam I mentioned. Out of the box, OpenNebula doesn't automatically create MinIO users when you generate an S3 key. You're scripting that integration. You're writing the glue that says "when a user requests S3 credentials, call the MinIO admin API, create a user, attach a policy that limits them to their bucket, and return the keys."
And that glue is where most miniature clouds become abandonware.
It is. Because the glue works for a month, and then MinIO updates its API, or OpenNebula changes its credential model, and suddenly your integration is broken and you're debugging Python at eleven at night.
So the honest assessment is — for a single user or a small team, Proxmox plus MinIO is probably enough, and you don't need the management overlay at all.
If you're the only tenant, you don't need tenant isolation. You don't need quotas — you know what your hardware can do. You don't need a self-service portal — you're the admin. Proxmox gives you VMs, MinIO gives you S3, and you manage both directly. That's eighty percent of the value with twenty percent of the complexity.
The moment you add a second tenant who shouldn't see the first tenant's stuff, the complexity curve goes vertical.
Vertical. Now you need OpenNebula or CloudStack. Now you need IAM. Now you need metering, because tenant two is going to ask why tenant one gets more CPU. Now you need to patch five different pieces of software instead of two. The operational burden becomes the bottleneck.
And that's where most homelab clouds die. Not the hardware — the operations.
Patching, monitoring, backups, upgrade paths. Ceph upgrades are not trivial. Proxmox major version upgrades can break your storage configuration. MinIO has its own release cadence. OpenNebula has its own. Each one is a project you have to track, test, and deploy.
It's a part-time job.
It is a part-time job. And that's the knock-on effect nobody talks about in the "build your own cloud" blog posts. The hardware is the fun part. The software stack is the interesting part. The operations are the part that grinds you down.
I want to put a number on that. If someone asked you "how many hours a month does it take to maintain a three-node miniature cloud," what's your honest answer?
Once it's stable and you've automated your patching? Maybe four to six hours a month, averaged out. But that average hides the spikes. You'll go three months with nothing, and then a Ceph point release introduces a breaking change in the OSD configuration format, and you lose a Saturday. Or Proxmox 8.2 drops and your OpenNebula driver hasn't been updated to support it yet, so now you're holding Proxmox updates until the driver catches up, and you're tracking a GitHub issue thread for three weeks. It's not the steady-state maintenance that gets you. It's the incidents.
Which brings us to something I've been turning over. The problems Daniel's describing — multi-tenancy, quotas, API access, object storage — these are not new problems.
They're not.
And the open-source community has been solving them for decades under different names. I want to pull Hilbert in here, because he's been making noises back there. You have something to add?
Hilbert: I ran a web hosting company out of my garage in ninety-eight.
Of course you did.
Hilbert: Three towers. Commodity PCs. Patch panel and a lot of duct tape. Customers paid me twelve dollars a month for a hundred megs of disk and a cPanel login.
cPanel.
Hilbert: cPanel handled multi-tenancy. It handled quotas. It gave each customer a login and they couldn't see each other's files. It had an API — not a good one, but it had one. And we billed by the megabyte. What Daniel's describing — provision, isolate, meter — that's shared hosting with better branding.
You're saying "miniature cloud" is just "shared hosting" with S3 instead of FTP.
Hilbert: A thirty-year gap. The open-source community has spent three decades rebuilding the same abstractions with better names. Object storage instead of disk quotas. IAM instead of user groups. API keys instead of... well, API keys. We had those too. They were uglier.
What was your hardware?
Hilbert: One machine with a fifty-six-k modem uplink. Customers never noticed. The semantics were the same — they logged in, they got their resources, they were isolated from everyone else. The management layer was the product.
Wait. One machine? You just spent five minutes telling us a single machine isn't a cloud.
Hilbert: I didn't say it was a cloud. I said it was shared hosting. The difference is the management layer, not the hardware count. You can run real multi-tenancy on one box if the software enforces the boundaries. The problem isn't the box — it's whether your software stack actually isolates tenants or just kind of gestures at it.
CPanel actually did isolate tenants.
Hilbert: It had to. We were charging people money. If customer A could see customer B's database, we'd be out of business in a week.
The operational burden we've been describing — the patching, the monitoring, the upgrade paths — that's not new either.
Hilbert: It's the same burden every hosting provider has always carried. The difference is now the tools are open source and a hobbyist can afford them. In ninety-eight, cPanel cost money. The server cost money. The rack space cost money. Now the software is free and the used hardware is practically free. The burden that's left is the one that was never going away — someone has to do the work.
That reframes the whole thing. The question isn't "can I build a miniature cloud." It's "do I want to be a hosting provider."
Hilbert: That's the question.
Most people who start these projects don't realize that's what they're signing up for.
Hilbert: They find out around month four, when the first Ceph upgrade breaks something and they're on a forum at two in the morning.
How long did you run the hosting company?
Hilbert: Two years. Sold it to a guy in New Haven for six thousand dollars and a dot-matrix printer.
Was the printer part of the deal or...
Hilbert: He threw it in. I still have it. Works fine.
Of course you do.
Hilbert: The point is, the abstractions haven't changed. Provision, isolate, meter, bill. Whether you call it shared hosting or a miniature cloud, you're solving the same problems with shinier tools. The tools are better now — MinIO is good, Proxmox is good — but the seams are in the same places they've always been.
The seams are actually in exactly the same places. cPanel managed user accounts and quotas, but it didn't manage the underlying storage. If a disk filled up, cPanel couldn't magically provision more. It just threw an error and you got a phone call. That's the same seam between OpenNebula and MinIO — the management layer says "here's your quota," but if the underlying storage hits a limit, the error bubbles up through a layer that wasn't designed to handle it gracefully.
Hilbert: We handled it by checking disk usage every morning with a shell script and yelling at customers who were over their limit before the disk actually filled. That was our "integration layer." A cron job and a lot of awkward phone calls.
The glue hasn't changed either. It's just moved from shell scripts to Python.
Hilbert: The glue is eternal. The glue is the job.
That's a fair point to leave on. Let's pull the thread.
The open question I keep coming back to is whether the miniature cloud is a useful category, or if it's a solution in search of a problem for anyone who isn't a hobbyist or a very small team.
I think it depends on whether you need multi-tenancy. If you're a single user who wants VMs and S3 buckets on your own hardware, Proxmox and MinIO solve that beautifully. You don't need a cloud layer — you just need a hypervisor and an object store.
If you do need multi-tenancy?
Then you're a hosting provider, whether you call yourself one or not. And you should budget for the operational burden accordingly. It's not a weekend project. It's a part-time job that never ends.
The interesting thing to watch is how the gap keeps narrowing. Used enterprise hardware gets cheaper every year. The open-source cloud layers are maturing — OpenNebula 6.8 just dropped with better Proxmox integration. The question is whether the management layer or the hardware becomes the bottleneck first.
My money's on the management layer. Hardware is a solved problem at this scale. Three R730s and a ten-gig switch will run everything you need for years. The software integration — the glue between Proxmox, MinIO, and whatever management overlay you choose — that's where the time goes.
The glue is the part that doesn't have a vendor. Nobody sells "OpenNebula-to-MinIO credential synchronization as a service." You're writing that yourself, and you're maintaining it forever.
The next time someone says "I built a cloud," ask them about their IAM model. That's where the truth comes out.
If they look confused, they built a server. If they sigh and mention a Python script that keeps breaking, they built a cloud.
Thanks to Hilbert Flumingtop for producing, and for the dot-matrix printer lore we didn't know we needed.
This has been My Weird Prompts. If this scratched the itch and you've got a weird prompt of your own, send it in — show at my weird prompts dot com.
We'll be back soon.
See you then.
