I want to reset my server soon and I’m toying with the idea of using a different operating system. I am currently using Ubuntu Server LTS. However, I have been toying with the idea of using Fedora Server (I use Fedora on my laptop and made good experiences with it) or even Fedora CoreOS. I also recently installed NixOS on my desktop computer and find the declarativeness pretty cool (but I’m still a complete beginner) and could imagine that it would fit well into a server setup.
I have quite a few services running on my server, such as Nextcloud, Conduit (Matrix), Jellyfin, etc. and all in containers. I would also rather not install programs without containers, because 1. compose is super easy to maintain and set up, 2. it remains very clear with containers (and compose) and 3. I believe that containers are more secure. But since I also want to make the services inside the containers available, I currently have Nginx installed as a reverse proxy (not in the container, but on the system) and always create certificates with certbot so that I can use HTTPS encryption.
In the paragraph above I actually described exactly the use-case of Fedora CoreOS, but I have no experience with the system and how it works. That’s why I’m still a bit hesitant at considering the OS at the moment. I can imagine that NixOS with its declarative nature seems well suited, since, as I have heard, you can configure containers as well as Nginx and with Nginx also https certificates declaratively. But I could also use a base system like before (Fedora Server or Ubuntu Server) and simply install podman, nginx and certbot and manage everything that way.
Have you had any experience with Fedora Server, Fedora CoreOS, NixOS or a completely different operating system for servers and what are/were your impressions with this setup? Or do you just want to share your knowledge here? I would be delighted.
See you back on Debian in a few months
Or Proxmox
Or, better yet, LXD/Incus.
You next OS will be… Debian. Because you care about your time and you want stuff to be stable.
Debian. Server? Debian.
Not sure what works best in your case. I’m a Debian cat myself but I have been considering openbsd as a future option.
I’m a long time user of Debian myself too. No cutting edge fuzz, just a working, stable OS all of the time. What else do you need for a server? It always did the job.
But then I stumbled on FreeBSD, and man, that’s a server OS. Simple design and blazing fast. No Docker but I never liked it anyway. My Docker is called Jails and in my opinion is they’re superior. Service isolation on the next level.
On my laptop? Debian due to hardware and software support. And I’ll stick to that for now. I feel home on that distro.
I can’t say anything about OpenBSD as I never tried it but it sure is a perfect fit for a server as well depending on your needs and preferences. BSD just rocks!
I love Debian too. Could you tell me what you mean FreeBSD being a faster and better server OS? Is there such a difference in speed in operations?
TBH I’d run alpine VMs on Bhyve to get K8S running and that’s it.
My feeling is that there is. I think it all started with the speed I can login over ssh. Debian always seems to have a short delay but FreeBSD feels instant. When it comes to rating FreeBSD as a better OS for servers I may be biased as Debian has served me so well over the years. I was never a Docker fan but instantly liked Jails for isolating services. Then we have native ZFS support which simplifies my backup needs. A simple zfs send | zfs receive and you have an exact copy of your service instance on a remote node. Everything feels integrated and not stacked. Again, just a personal opinion.
OpenBSD has native limitations on hypervisors. Disabling default measures will lessen security, unfortunately.
I have used all three! I started with Server then went to CoreOS running Kubernetes and settled on NixOS which I have been very happy with for about a year now. I run about 25-30 services all using built in modules.
Regarding security, if you are using well crafted modules on NixOS, there should be good systemd hardening in place. That being said there is no reason you can’t just use containers on NixOS.
I also find deploying NixOS far superior to butane/ignition used by CoreOS/Fedora. I use nixos-anywhere and can deploy my entire server in a few minutes without manual intervention.
Give Gentoo a shot. It’s super stable and you will understand everything in your system. Also it now supports binary packages
And by now you mean for the past decade at least.
Huh?
Portage has supported binary packages since forever, back in 2012 I had some binary packages on my system, I clearly remember because it was a pain in the ass to compile certain things, for those I installed the binary version. It’s like Debian supporting source packages, it’s been there since forever but people don’t know about it.
I mean it’s had -k/-K since mid 2000s from what I remember but it’s changed
The point is that they have recently focused on better binary package availability. Sure they always had support for binary packages but most software needed to be compiled.
Slackware is a great, simple OS that does what it does and does it well. There will be some getting used to, but when it clicks, it makes sense and doesn’t do anything you wouldn’t expect. It is great if you want to use containers as it provides you with the stable, simple base to run all your containers on top of.
I use Fedora CoreOS on my homeserver and a bunch of VPSs. Migrated the homeserver just recently, but I’ve migrated the first VPSs a bit more than a year ago. So far, I had no problems with it. There’s a low-traffic mailing list where the devs inform about security issues and breaking changes to the whole container stack.
I used debian before for some years, but at some point became tired of manually updating the system (which is probably one of the biggest benefit of FCOS). It takes, however, quite some time to put your first Ignition config together, and debugging is tedious as you have to redeploy to see if a bug / error is now gone (I’ve used a VM for that).
I use podman on some, Docker on other servers (you can’t use both at the same time). Both have been working well so far.
I’d recommend it, but would also recommend taking a look at Flatcar Linux which is more or less the same without the IBM dependency (which makes my stomach hurt sometimes).
I’d recommend it, but would also recommend taking a look at Flatcar Linux which is more or less the same without the IBM dependency (which makes my stomach hurt sometimes).
Why exactly are the IBM dependencies a problem for you?
I used debian before for some years, but at some point became tired of manually updating the system (which is probably one of the biggest benefit of FCOS). It takes, however, quite some time to put your first Ignition config together, and debugging is tedious as you have to redeploy to see if a bug / error is now gone (I’ve used a VM for that).
I can’t really find good resources on how FCOS is working and what are the benefits. Is it updating the system/kernel automatically as well as the containers? And what are generally, in your opinion, the advantages of FCOS?
Why exactly are the IBM dependencies a problem for you?
I guess I just like independent, community-driven distros, since there’s less space for financially motivated enshittification. Just shortly after I decided to go with FCOS, RedHat / IBM decided to close down CentOS, for example.
I can’t really find good resources on how FCOS is working and what are the benefits. Is it updating the system/kernel automatically as well as the containers?
The system & kernel yes. The whole system is basically a read-only system “image” for which the devs make sure all the packages play nicely together. Packages are not updated individually, but whole system “image” are released periodically, which the system then downloads automatically and reboots (you decide when it actually reboots through the config). If anything goes wrong, the system is rolled back to the previous “image”.
When you go with podman, there’s a systemd service you can enable which will update the containers (i.e. pull the specified image tag). I’m not aware of a similar mechanism for Docker, which is why I use watchtower for that which has been working smoothly so far.
Edit:
And what are generally, in your opinion, the advantages of FCOS?
For me, it’s the (quite safely designed) auto-updates of the base system (I just feel like having to do less repetitive work), infrastructure-as-code aspect, and the container mindset (as I containerize everything anyways). Also I just have a weakness for new, fancy stuff.
Thanks, I might give it a try. I am not settled yet but FCOS sounds very promising. We will see.
I have not used Fedora server yet but like their desktop is. Currently my two VMs in unraid are Rocky Linux. Been using centos and now Rocky for the last 5-6 years and haven’t had any complaints
I also recommend a stable/LTS distro like Debian or AlmaLinux (or other RHEL-based distros). Or just keep using Ubuntu Server LTS.
The OS packages being hopelessly outdated doesn’t really matter when you’re running most services inside containers.
If you mainly use containers perhaps OpenSUSE Micro OS is of interest to you. Other then that pretty much any distro will do. I use rocky Linux my self for a few different things.
If you want to try out many different distros virtualization is also a option. KVM or something like XCP-NG with XO or proxmox are great options.
What is the difference/benefit to Fedora CoreOS?
I really really like Fedora Server, but any RHEL derivative is my go to for servers. I use Rocky Linux when I need something closer to RHEL, and Fedora server for pretty much everything else. I highly recommend Cockpit as well (main reason I like Fedora server) as it has allowed me to so easily manage all of my servers from a single point.
