I’m moving to a new machine soon and want to re-evaluate some security practices while I’m doing it. My current server is debian with all apps containerized in docker with root. I’d like to harden some stuff, especially vaultwarden but I’m concerned about transitioning to podman while using complex docker setups like nextcloud-aio. Do you have experience hardening your containers by switching? Is it worth it? How long is a piece of string?
Definitely Podman
Alternatively you could use docker in virtual machines
As long as you’re on Linux, podman is superior and will do all of the things you’re asking about. If you need to also support Windows or Mac, Docker is the only thing that will work (although people have told me Rancher isn’t bad now for a couple of years).
podman works on windows hosts, as long as you don’t need windows containers
I’m actually in the process of switching over from docker to podman. Its definitely a learning curve, ie. Setting up systems integration etc.
I do love it but its been a bit of a pain, for instance I’m still trying to figure out the errors I’m getting when trying to deploy my matrix and vikunja containers, I’m getting permission errors and can’t find dB errors. I k ow little about podman right now but I would definitely recommend it since it is open source and runs rootless by deafult.
Podman
- rootless by default
- daemonless
- integration with systemd, made even easier by
podman-generate-systemd - no third-party APT repository required, follows the same lifecycle as my LTS (Debian) distro
podmananddockercommand-line are 100% compatible for my use cases
podman-generate-systemdis outdated. The currently supported way to run podman containers using systemd services would be Quadlet files.https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html
Edit: I just saw that you use debian so idk if Quadlets are a thing with the podman version on debian.
Podman not because of security but because of quadlets (systemd integration). Makes setting up and managing container services a breeze.
I’m very much biased towards Podman, but from what I understand rootless Docker is a bit of an afterthought, while Podman has been developed from the ground up with rootless in mind. That should be reason enough.
The very few things Docker can do that Podman struggles a bit with are stuff that usually involves mounting the Docker socket in the container or other stupid things. Since you care about security, you wouldn’t do that anyway. Not to mention there’s also rootful Podman, when you need that level of access.
I’d recommend an RPM-based distro with Podman, the few times I’ve tried Podman on a deb distro, there’s always been something wonky. It’s been a while, though.
Podman actually run fine on Debian 12. Though the packaged version is a bit old. Does not support podman compose command. Though podman-compose works.
podman-compose is packaged in a separate
podman-composepackage in Debian 12 (did not try it though). The only thing missing (for me) in Debian 12 is quadlets support (requires podman 4.x, Debian 12 has 3.x)No its on 4.x I think.
Thanks. Last time I tried it was just after bookworm released, and on ARM, so it has probably got better
Does Podman work well when you have multiple rootless containers that you want to communicate securely in a least-privilege configuration (each container only has access to what it needs)? That is the one thing I couldn’t figure out how to do well with Podman.
Do you mean networking between them? There’s two ways of networking between containers. One of them is to create a custom network for a set of containers that you want to connect between each other. Then you can access other containers in that network using their name and port number like so
container_name:1234Note: DNS is disabled in the default network by default so you can’t access other containers by their name if using it. You need to create a new network for it to work.
Another way is to group them together with a pod. Then you can access other services in that same pod using localhost like so
localhost:1234Personally in my current setup I’m using both pods and seperate networks for each of them. The reason is I use traefik and I don’t want all of my containers in a single network along with traefik. So I just made a seperate network for each of my pods and give traefik access to that network. As an example here’s my komga setup:
I have komga and komf running in a single pod with a network called komga assigned to the pod. So now I can communicate between komga and komf using localhost. I also added traefik to the komga network so that I can reverse proxy my komga instance.
I’m thinking about an immutable OS with podman support first and foremost. Would you recommend Fedora CoreOS?
It’s a really solid combo, but if you’re not familiar with CoreOS I wouldn’t change both at once. Meaning migrate the services to Podman first, then switch the OS. I’ve meant to switch from Alma 9 to CoreOS a long time, but haven’t found the time.
I noticed you run Nextcloud AIO, just so you know, that’s one of those “mount the docker socket” monstrosities. I’d look into switching to the community NC image and separate containers managed yourself. AIO is easy, but if someone gets shell to the NC container, it’s basically giving root to your host.
Either way, you’re going to have trouble running AIO with Podman.
I’m running podman and podman-compose with no problem. And I’m happy. At first I was confused by the uid and gid mapping the containers have, but you’ll get used to it.
This are some notes I took, please don’t take all of it for the right choice.
Podman-Stuff
https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md
storage.conf
To use the fuse-overlay driver, the storage must be configured:
.config/containers/storage.conf[storage] driver = "overlay" runroot = "/run/user/1000" graphroot = "/home/<user>/.local/share/containers/storage" [storage.options] mount_program = "/usr/bin/fuse-overlayfs"Lingering (running services without login / after logout)
https://github.com/containers/podman/issues/12001
https://unix.stackexchange.com/questions/462845/how-to-apply-lingering-immedeately#462867
sudo loginctl enable-linger <user>Do you need to set lingering for all container users you set up? Does it restart all services in your compose files without issue?
Yes all users that have containers running, that should keep running need lingering.
The Services do not restart themself. I have cronjob that executes
podman start --allat reboot for my “podman user”.
Not sure if it makes a difference and not quite your question but I’ve just switched away from nextcloud-aio to just having my own docker compose, so I have better control and know what’s going on more. I always found it funny and when installing on a new VPS decided to try. It was surprisingly straightforward and Ive been able to install everything I need.
Let me know if my docker compose would help. I still need to add the backup solution but it’s going to be straightforward as well.
I would love to see your compose file. I already have to run special steps on my nextcloud-aio to use it with a reverse proxy so I’m interested in moving away from it.
Hopefully this works and you can see the compose file. I’ve put a few things in [square brackets] to hide some stuff, probably overly cautiously. I have an external network linked to NPM and in that, I use nextcloud-server for IP address and 80 for the port (it’s the inside container port, not 8080 on the system - that took me a long time to figure out!). Add a .env file with everything referenced in the compose file, then (hopefully!) Away you go
I switched and was very glad to do so. You increase your security and so far I haven’t seen any downside. Every container I’ve tried has worked without issues, even complex ones.
Was this with podman or rootless docker?
I also would like to switch to rootless, I have some experience with podman and, while I generally like it, it’s not 100% compatible with (rootful) docker, and can have performance issues if you’re not careful, especiallt with certain file systems like btrfs. I wonder if rootless docker is now better than podman, or preferred for some other reason.
Rootless Podman :) It requires you to learn a little bit of new syntax, for example, the way you mount volumes and pass environment variables can be slightly different, but there’s nothing that hasn’t worked for me.
I’m using this on uBlue uCore, which I would also strongly recommend for security reasons.
Can you expand on why you chose uCore? I was considering CoreOS until just now
and the idea of setting up ignition config serving seems overkill for running only one server at home.ignition is still required the same way as CoreOSMainly for security. I was originally looking at CoreOS but I liked the additional improvements by the UBlue team. Since I only want it to run containers, it is a huge security benefit to be immutable and designed specifically for that workflow.
The Ignition file is super easy to do, even for just one server (substitute
dockerforpodmandepending which you have):Take a copy of the UCore butane file:
https://github.com/ublue-os/ucore/blob/main/examples/ucore-autorebase.butane
Update it with your SSH public key and a password hash by using this command:
# Get a password hash podman run -ti --rm quay.io/coreos/mkpasswd --method=yescryptThen host the butane file in a temporary local webserver:
# Convert Butane file to Ignition file podman run -i --rm quay.io/coreos/butane:release --pretty --strict < ucore-autorebase.butane > ignition.ign # Serve the Igition file using a temp webserver podman run -p 5080:80 -v "$PWD":/var/www/html php:7.2-apacheDuring UCore setup, type in the address of the hosted file, e.g.
http://your_ip_addr:5080/ignition.ignThat’s it - UCore configures everything else during setup.___
One of the main reason I switched to podman was its compatibility with firewalld. Haven’t used rootless docker, but podman and podman-compose gets the job done for me.
Any reason why you use compose and not quadlets?
Do you run anything like fail2ban with that compatibility?
deleted by creator
