Hypervisor Gotta say, I personally like a rather niche product. I love Apache Cloudstack.

Apache Cloudstack is actually meant for companies providing VMs and K8S clusters to other companies. However, I’ve set it up for myself in my lab accessible only over VPN.

What I like best about it is that it is meant to be deployed via Terraform and cloud init. Since I’m actively pushing myself into that area and seeking a role in DevOps, it fits me quite well.

Standing up a K8S cluster on it is incredibly easy. Basically it is all done with cloud init, though that process is quite automated. In fact, it took me 15m to stand up a 25 node cluster with 5 control nodes and 20 worker nodes.

Let’s compare it to other hypervisors though. Well, Cloudstack is meant to handle global operations. Typically, Cloudstack is split into regions, then into zones, then into pods, then into clusters, and finally into hosts. Let’s just say that it gets very very large if you need it to. Only it’s free. Basically, if you have your own hardware, it is more similar to Azure or AWS, then to VMWare. And none of that even costs any licensing.

Technically speaking, Cloudstack Management is capable of handling a number of different hypervisors if you would like it to. I believe that includes VMWare, KVM, Hyperv, Ovm, lxc, and XenServer. I think it is interesting because even if you choose to use another hypervisor that you prefer, it will still work. This is mostly meant as a transition to KVM, but should still work though I haven’t tested it.

I have however tested it with Ceph for storage and it does work. Perhaps doing that is slightly more annoying than with proxmox. But you can actually create a number of different types of storage if you wanted to take the cloud provider route, HDD vs SSD.

Overall, I like it because it works well for IaaS. I have 2000 vlans primed for use with its virtual networking. I have 1 host currently joined, but a second host in line for setup.

Here is the article I used to get it initially setup, though I will admit that I personally used a different vlan for the management ip and the public ip vlan. http://rohityadav.cloud/blog/cloudstack-kvm/

Favorite heavyweight Type 1 hypervisor: XCP-ng. It’s open source, runs on a ton of enterprise and consumer-grade hardware, has always been rock stable for me, even when forgetting to update it for like 6 months, still ran everything like a champ.

I need to try ProxMox, has some cool features. XCP-ng is pretty intuitive though, UI makes sense and is cleaner than Proxmox. The integration in Proxmox with the Incus project is pretty cool though, especially being able to run VMs and containers and manage them together. I’ve been thinking of trying that and seeing how it goes.

For containers, I just install Debian and run Docker on there. Stable, simple, nothing fancy. If I need something more up to date, I typically use Ubuntu Server.

A friend recommended me OpenSuse MicroOS, and it has been a great experience!
It’s a atomic OS designed to be just enough to run containers and it does it perfectly. It updates and reboots itself automatically so I never have to worry about it.
IMO, perfect for a home environment, just wish the documentation was better.

Anything that can run proxmox is running proxmox. Even if it’s a single OS running on it, it’s still running proxmox

Proxmox Virtual Environment (PVE, Hypervisor), my beloved. Especially in combination with Proxmox Backup Server (PBS).

My homelab would not exist without Proxmox VE, as I’m definitely not going to use Nutanix or VMWare. I love working with linux and Proxmox VE is literally debian with a modified kernel and a Management Webinterface on top.

I first learned about Proxmox VE in my company, while we still had VMWare for us and all of our customers. We gradually switched everyone over to Proxmox VE and now I’m using it at home too. Proxmox is an Austrian (my country) company, so I was double hyped about this software.

A few things I like most about Proxmox VE

• Ease of access to the correct part of the documentation you currently need (*)
• Open Source
• Company resides in my country (no US big tech walled garden)
• Linux / Debian based, so no learning new OS’s and toolchains
• Free version available
• Forum available and actually used

(*) What I mean by ease of access to the correct part of the documentation is: Whenever you’re in the WebUI and need to decide on some settings, there’s a button somewhere on the same page which is going to lead you directly to the portion of the documentation you need right now. I don’t know why this seems like such a great luxury, every software should have something like this.

Next steps

My “server” (some mini PC with spare parts I already had) is getting too weak for the workload I put it through, so I’m going to migrate to a better “server”. I already have a PC and most of the necessary parts, I just need some SSDs and an AMD CPU.

Even migrating from PVE (old) -> PVE (new) couldn’t be easier:

• PVE (old): create last backup to PBS, shut down PVE (old)
• PVE (new): add PBS, restore Backups
• ???
• profit

I think it’s great to have a series posting about personal achievements and troubles with selfhosting. There’s so much software out there, you always get to see someone doing something you didn’t even know could be done or using a software you didn’t realize even existed. Sharing is caring.

I have a nuc with Linux mint and host everything on docker containers. I expose any service I need through caddy.

OS: Unraid

It’s primarily NAS software, with a form of software raid functionality built in.
I like it mainly because it works well and the GUI makes is very easy to use and work with.

On top of that you can run VMs and docker containers, so it is very versatile as well.

I use it to host the following services on my network:

• Nextcloud
• Jellyfin
• CUPS

It costs a bit of money up-front, but for me it was well-worth the investment.

I have been using Proxmox VE with Docker running on the host not managed by Proxmox, and then Cockpit to manage NFS Shares with Home Assistant OS running in a VM. It’s been pretty rock solid. That was until I updated to Version 9 last night, it’s been a nightmare getting the docker socket to be available. I think Debian Trixie may have some sort of extra layers of protection, I haven’t investigated it too much, but my plan tomorrow and this week is to migrate everything to Debian 12 as that’s the tried and true OS for me and I know it’s quite stable with Cockpit, docker and so forth with KVM for my Home Assistant installation.

One other OS for consideration if you are wanting to check it out is XCP-NG which I played with and Home Assistant with that was blazing fast, but they don’t allow NFS shares to be created and using existing data on my drives was not possible, so I would’ve had to format them .

openSUSE MicroOS

I’ve only tried it out on a VPS, so I’m not completely sold on it yet, but I do think I’ll be switching to it eventually. I’m currently on Leap, but since almost everything is containerized, I’m not getting much benefit from the slow release cycle.

For your questions:

Why would you want to try it out? Did you try it out already? What worked great? What didn’t

The main appeal is unattended, atomic updates using bleeding edge packages. You keep your apps as separate from the base system as possible (containerized), and the base handles itself.

My main issue is with the toolbox utility, which runs a container to hold userland utilities for debugging stuff. So far, it has been buggy with the underprivileged user I configured, and I’d really rather not login as root. I’ve worked around it for now, but it leaves a lot to be desired.

Where are you stuck right now? What are your next steps?

Mostly figuring out how I want to handle my VPN (for exposing LAN services to the outside world) config. My options are:

• containerize, and configure iptables rules to route traffic properly
• install the needed tools to the base system and configure it on the host

The main sticking point is that I need HAProxy in front and route traffic to the given device, so the VPN and HAProxy need to talk. The easiest solution is to put both on the host, but that breaks the whole point of MicroOS. The ideal is to have both the VPN and HAProxy containerized, but I ran into some issues with podman.

Why do you think it is the best tool for this job? Is it aimed at beginners or veterans?

This is definitely a veteran system right now, but I think it’s ideal because it means I can completely automate system updates and not worry about my apps breaking. It also means I can automate setting up a new server (say, if I move to a different VPS) or even new OS since I only need to deploy my containers and don’t need anything special from the OS setup.

I’m also playing with Aeon on my laptop, but that’d going a lot less smoothly than MicroOS on the server.

I’ve been using Ubuntu server on my server for close to a decade now and it has been just rock solid.

I know Ubuntu gets (deserved) hate for things like snaps shenanigans, but the LTS is pretty great. Not having to worry about a full OS upgrade for up to 10 years (5 years standard, 10 years if you go Ubuntu pro (which is free for personal use)) is great.

A couple times I’ve considered switching my server to another distro, but honestly, I love how little I worry about the state of my server os.

Debian on the host and everything else in containers

Maybe crazy, but I’ve been running flatcar lately. Automatic OS updates are nice and I pretty much exclusively use most of my machines to run containers.

I used to really like esxi, but broadcom screwed us on that.

Hyper-v sucks to run and manage. It’s also pretty bloated.

Proxmox is pretty awesome if you want full VMs. I’m gonna move everything I have onto it eventually.

For ease of use, if you have Synology that can run containers, it’s okay.

I also like and tend to use unraid at my house, but that’s more because of my insane storage requirements and how I upgrade with dissimilar disks fairly frequently. (I’m just shy of 500tb and my server holds 38 disks.)

I use TrueNAS SCALE at home on my NAS and since they ditched kubernetes (and Truecharts, which was a happy little accident) it’s been great.

It’s free.

New hardware is incorporated into the kernel reasonably regularly IMO.

ZFS file system

Pretty easy to control with GUI exclusively

Docker is now very easy to use, images are community supported mostly but I’ve not had issues with Jellyfin, *arr, pihole, reverse proxy etc.

I’m interested in learning more about nixOS but until i get there, proxmox all day