lyte.dev

My NixOS/Nix Journey

Posted on Apr 15 2025

I operate computers and I enjoy it. However, I operate a lot of computers, both as an occupation and in my personal life. As I operate computers, I’m often thinking about ways to improve the process of how I operate computers. This fundamental drive in me often pushes me to learn new things, experiment or find different approaches to common problems. I always try to think that perhaps what I have now is a local maximum worth pushing past, especially since, if I play my cards right, I can always revert back to whatever setup I had previously - computers are neat that way.

I originally wrote this for another project. I’ve decided to post it here even though it’s maybe a bit of a dry wall of text. I hope enjoy it! Also, if you don’t care about any of this and are just curious about my Nix configuration, you can check it out here.

Anyways, one day, I was updating my laptop and happened to break something, which made me realize I was going to have to spend at least 15 minutes not doing whatever I actually wanted to do and would instead have to troubleshoot some problem and fix it before I could do whatever it was I wanted. That was the day I finally caved and installed NixOS (as my second or third attempt, I might add) as an experiment and the start of my exploration into Nix. I put it on my laptop and resolved to get myself stuck with it, which is usually how I try to explore something: to force myself to try and live with it and ignore similar tools or products entirely.

Why NixOS? Well, in this particular case, I knew that updates with Nix had a rollback mechanism baked into the machine. If you didn’t purposefully delete the previous version (generation) then you can roll right back into it and it’s baked into the bootloader. I would never have to troubleshoot a broken update if I didn’t want to at that exact moment - I could come back to it later when it was more convenient for me.

Now, it’s true that different systems offer this same promise without being NixOS, so the question still stands, why NixOS? Well, NixOS also lets me configure the system and its applications in a way that plays nicely with my preferred workflow: a text editor and a version control system as opposed to clicking through menus. I would be able to “re-use” configuration elements without needing to repeat actions.

Finally, Nix also offered an alternative build system for software that had certain promises around reproducibility and required me to fully declare all of the build’s inputs. This meant I couldn’t build a piece of software without also guaranteeing that I would be able to build it again in a different environment without doing something particularly weird, which appealed to me greatly as somebody who occasionally opened old projects (whether my own or somebody else’s) only to find that I would need to perform certain involved setup steps before I could build the software – or worse, that those steps were not actually written down anywhere and this knowledge was common (in the face of my ignorance) or completely lost, in which case… good luck.

So I had this weird operating system on my laptop. I struggled to get some basic things setup and for the most part just duplicated my previous mostly-shell-scripts configuration. I “installed” software the “wrong” way (for NixOS) using nix profile install a lot at first and did not feel like I was benefitting much.

Eventually, I discovered some new piece of software I wanted to try and saw it was already available in nixpkgs and even had a NixOS module for configuring it. I tried to enable it and it warned me there might be issues because of some other configuration values I had set up. I realized that I could still try this and simply rollback if it didn’t work. Now I was feeling something. I YOLO’d it, rebooted, and it failed completely. I rebooted and selected the previous generation and it was like I had never YOLO’d. Could… could I YOL… T? Or YOL♾️? Now I was getting excited.

I started investigating how this module was “aware” of the rest of my configuration and the potential problem I was creating, why it existed, and how to make these two things play nice together. I learned more about Nix’s module system and how the lazy evaluation mechanism (similar to many of Rust’s iterators and futures) worked and made it possible. I learned that it enabled a kind of recursion I hadn’t seen outside of Haskell, enabling modules to look at any configuration values set by any other modules. I learned that Nix was able to reconcile multiple values by secretly keeping track of all of them and taking the highest priority one and that this enabled Nix to let you know if you had conflicting values.

I was blown away at how this simplistic language I previously deemed annoying was actually incredible for the purpose it was serving. I began tinkering in earnest at this point. I was certain I could find a way to get these two conflicting components to play nicely and I could have both things at once or toggle between them or something.

I was wrong, but I learned that there was a good reason for it. I also learned I didn’t need them to play nice and I could get what I wanted ultimately by using something Nix calls “specializations” or simply changing the config whenever I wanted to do the other thing instead. For some reason, it wasn’t until this moment that it totally clicked for me that I could freely try just about anything without fear.

Until I put NixOS on my home router and made a configuration change that accidentally closed the firewall while I was away from the house. Also I had a power outage and my idiot self had been using btrfs software RAID on top of a hardware RAID card and I had lost data and I wasn’t monitoring my backups well. Nix doesn’t do everything. But I started writing a simple utility program to run my updates with a process that basically went like this:

1. SSH into the remote host and start a background job that would start by waiting for an update to start.
2. SSH into the remote host and start an update. This would trigger the background job to then start waiting for the update to finish. If it errored out, it would rollback the update with Nix.
3. Once the update finished, the background job on the remote host would then wait a set amount of time for a signal from me that I could still talk to it. If it didn’t, it would ensure a full rollback happened which presumably would restore my connectivity to the remote host (assuming it wasn’t an actual network problem).
4. My local host would attempt to reach out and let the remote host know everything was fine.

As I started writing this, it seemed so simple and straightforward that I realized that surely somebody else had built such a tool. I was correct and discovered deploy-rs and immediately integrated it into my workflow for updating remote hosts. I haven’t broken a remote host since then!

For persistent data, I wanted to also make sure I never lost data again as well. I whipped up a NixOS module with my new skills that used restic for daily encrypted backups for all paths containing data I deemed important to keep, and set up some remote nodes to backup to. I still need to set up some canary mechanism for this, but I at least know when backup jobs fail and check on them.

Since I was using Nix for my system configuration, I began exploring how it is used to build software, but found getting into that a little more difficult. I had also realized and read that Nix was so useful at fetching and running software that it makes for a fantastic tool for setting up development environments: installing and managing compiler toolchains, tools for various programming languages, and all that jazz.

I realized that I might never have to write a readme file again and instead everything would be encoded into the build system itself. This has turned me on to encoding documentation into your systems as a general concept, something I now think is one of the next big shifts in software, especially as we derive more and more information from contexts using popular machine learning tools.

So I began putting together Nix flakes, which you might simplify to a set of inputs that produces outputs, in this case, the inputs might be the nixpkgs software repository and other flakes (each of which might have their own inputs) and the outputs being a “devShell”, with a specific PATH which has references to all the specific versions of the necessary software to perform builds, run pre-commit check such as format code, run tests, kick off linters, and more. Once I had done this a few times for a few projects and hooked it up to direnv such that whenever I opened up a project it would automatically set everything up for me, I realized I was completely sold out for this stuff.

I was ready to build software with Nix. I learned that different languages have different struggles here. Since Nix requires that you know all your inputs and their contents (via their hash) in advance, this makes working with some package managers (such as pip or cargo) more difficult than others. I learned about the various “*2nix” projects out there that make this process simpler, where you take the package manager’s lockfile and derive the necessary Nix expressions from them to enable dependency fetching for Nix-based builds.

This gave me so much more insight into the intricacies of various programming language toolchains, how they work, how they manage and process dependencies, and so much more. Nix forced me to think about these things as they are primitive underpinnings I needed to understand to reap the benefits I was wanting.

I even began contributing to the main nixpkgs repository, which happens to be one of the single largest software repositories in the world (depending on how you measure things). I learned about derivations, overrides, wrapping, overlays.

I suddenly started seeing Flakes for all kinds of software pop up. I created a few of my own, including one for a piece of software I use to play Super Smash Bros. Melee online. I learned how NixOS approaches integration tests and how amazing the system is. I want to have unit tests for my main server software and my home router someday with fancy virtual networking and a virtual ISP and everything. Maybe it would help me get back to my internet exchange peering project! Anyways, my journey with Nix has been one of exploration with fun and rewarding results. I feel so much more confident with anything I put together that has Nix behind it.

If you want to, you can check out the git repo where I’ve worked on all of this!