Back

Nix: Taming Unix with Functional Programming

288 points10 daystweag.io
infogulch10 days ago

Nix is running face-first into complexities of build and package managers. As an observer it looks like Python's package ecosystem in particular is a giant mess. This affects Nix disproportionately because Nix actually integrates all package updates into one channel which nobody else anywhere does (and I guess package authors often don't care to fix), and it's even worse because a lot of projects use python as a build dependency (?!) which then cascades these issues even farther.

I've been following this thread [1] about the issue which has valid interested parties including users, package authors, and nix package maintainers, and contains various proposals to solve or alleviate the problems.

[1]: https://discourse.nixos.org/t/nixpkgss-current-development-w...

SuperSandro200010 days ago

The python ecosystem is not a giant mess, its just dependency hell. You need to work with what you get from upstream.

> Nix actually integrates all package updates into one channel which nobody else anywhere does

If you are using the python system packages then Arch, Debian and probably more are doing the same.

> it's even worse because a lot of projects use python as a build dependency (?!) which then cascades these issues even farther.

In practice this is not a problem at all. Build systems that use python underneath have usually very little dependencies. The bigger problems we are facing are big python (web) applications and anything that moves (very) slowly upstream like (sadly) many AI/ML projects.

FYI: I am very active NixOS maintainer and one of my foci is the python packages in NixOS.

infogulch10 days ago

Thank you for your input!

What do you think about the linked thread?

> The python ecosystem is not a giant mess, its just dependency hell.

There are dozens (?) of actively used package and environment management systems, with no consistency and no lockfiles, many packages ignore semver, and having multiple versions of a package installed causes weird issues. I'm not sure why "a giant mess" is an invalid descriptor, I guess it's just arguing semantics.

SuperSandro20009 days ago

> What do you think about the linked thread?

I already posted multiple posts in the thread under the same username. Other than adding important consumers of a dependency to its tests, the only solution I could come up with is to move AI/ML packages into their own repository where they can move at their own pace and do pins/overrides more freely and/or drastically reduce the amount of python packages in nixpkgs.

aragilar5 days ago

Uh, what? I've not used Nix with python (so I don't know if there's something particularly bad there), but most of the complexity comes from handling non-python dependencies of python packages (which different people have different needs and hence opinions on, and so seeking consistency there is like expecting all the linux distros to merge). At best there are two major systems that people will talk about (pip+venv and conda), and the reality is conda isn't a python package manager, but a more general system (and every other python system you could name is built on the pip+venv system).

viraptor9 days ago

Semver doesn't really apply here since nix always points at a specific version of each dependency. Multiple package versions are not visible to the same app. And you can include your own lockfile-like list.

So in practice, it's not really a bigger problem for nix than for any other release. If you need to freeze versions for an app, you freeze them. If you don't or if you're packaging a library, you can (usually) rely on automated PR testing to catch breakages and deal with them there.

+2
infogulch9 days ago
baby9 days ago

I’m guessing you’d just have to create a derivation for the python package you’re trying to use.

jnxx10 days ago

> As an observer it looks like Python's package ecosystem in particular is a giant mess.

Most people would say it is. The better question is how to avoid such a mess.

My take is that maintaining backward compatibility is a core principle which needs to be strictly observed to solve that problem. And yeah, that has also become a cultural issue with Python, as the Python2/3 breakage shows.

So, could one sum it up in that Nix magnifies unsolved compatibility issues in packaging systems? Because if there were a single core Python distribution, like say, Anaconda, but nothing else, these issues would not exist. Of course, people can avoid the issues if they only use a handful of packages. But putting all into a single channel makes the problem much more acute.

lamontcg10 days ago

> My take is that maintaining backward compatibility is a core principle which needs to be strictly observed to solve that problem.

Given that software developers never guess the correct design up front this means that you always have architecturally buggy software, and a bunch of complaining about why buggy-looking edge conditions are never fixed. There has to be some kind of release valve for software to evolve and break backwards compatibility.

btschaegg10 days ago

I don't disagree, but it doesn't only have to go in this one direction. One of the most interesting things about Rust for example is how it tackles experimental implementations and has concepts in the compiler etc. that make unstable language features "first class". I'd say this will definitely yield better results than "well a couple of guys hacked around on some prototype forks of a compiler, and now we're stuck with the result".

Of course, they also make very impressive backwards compatibility guarantees for stable stuff (cf. Rust's "editions").

+1
lamontcg9 days ago
jnxx9 days ago

> There has to be some kind of release valve for software to evolve and break backwards compatibility.

You kinda insinuate that breaking backwards compatibility is kinda necessary at times.

This is not the case. Projects like

* the Linux kernel, or

* the GNU C library, or

* the Numeric -> Numpy transition around Python 2.0, or

* Common Lisp (which is much older than Python) adopting Unicode

are good examples that this is not necessary. It is not true that you have to break backward compatibility.

There are domains where breaking backwards compatibility in libraries is not acceptable at all, like vendor libraries in industrial automation. You don't throw away a 15-year old printing machine or a chemical plant just because the vendor of the automation software is tired of supporting its old interfaces.

That might sound strong, but others have expressed it more strongly. Read this: https://linuxreviews.org/WE_DO_NOT_BREAK_USERSPACE

How it is done? It starts with well-designed interfaces. And when interfaces are changed, the old interfaces are kept and become special cases of the new ones. Numeric/Numpy is a good example.

Here is a talk, brilliant as always, by Rich Hickey which explains why and how:

https://www.youtube.com/watch?v=oyLBGkS5ICk

It is highly relevant to Nix and Guix.

Python3 could have gone the same way - keeping the interpreter compatible to Python2 code, making the semantics dependent on whether a source file has a *.py or a *.py3 extension, and so on. It would have been more work but the transition would have been nearly painless, and I guess much faster. Support for old stuff does not need to go on forever - for example, Linux does not support any more Intel 386 CPUs.

It boils down to whether keeping stuff backwards-compatible is a goal of the project leaders or not.

+1
lamontcg9 days ago
infogulch10 days ago

Yes they've invented that: It's called bumping the major version.

+2
lamontcg10 days ago
jnxx9 days ago

> Yes they've invented that: It's called bumping the major version.

Rich Hickey's Talk:

https://www.youtube.com/watch?v=oyLBGkS5ICk

explains why this is not a solution. It makes a difference, yes. But it is the difference between "the incompatible changes in my library are going to break your application" and "the incompatible changes in my library are going to break your application, and I am telling you this beforehand".

infogulch10 days ago

> Nix magnifies unsolved compatibility issues in packaging systems ... putting all into a single channel makes the problem much more acute

Well said.

zaphirplane9 days ago

Is the discussion here that Nix doesn’t support multiple side by side installations of different versions making it difficult to install two packages depending on the same package but different versions ?

I thought in Nix you can have that, and python can have venv for independent library versions, is it that no one has done the work to combine the two ?

pxc9 days ago

Nope. All Python programs in Nix are effectively venv'd, so-to-speak.

The problem is that if two packages A & B each depend on the same library L but they require different versions of it, any Python code that imports both as libraries will end up with two, potentially incompatible, copies of the same library in its import path. And Python doesn't support this, so it uses the same version of L with all the Python code running in that process. So now depending on whether some package P which requires both A and B imports A first or B first, A will end up using B's version of L or vice-versa. This sometimes does nothing, and it sometimes causes very weird, very subtle breakage.

Thus for all of the Python libraries in Nixpkgs to be usable in any combination by any package in Nixpkgs, there can only be one version of each Python library in Nixpkgs.

Once your package collection is large enough, you start actually encountering versioning conflicts as described above in the transitive Python dependencies of your end-user application packages. That's why Linux distros run into these integration issues. Application developers generally don't because their applications development environments are much smaller.

SuperSandro20009 days ago

> I thought in Nix you can have that, and python can have venv for independent library versions, is it that no one has done the work to combine the two ?

A python environment is effectively a different take on a venv. Python packages is one giant python environment. For programs that life outside of python packages but use the packages from there we are free to apply overrides how we want. So it is possible to have different versions of a package in a different python environment.

pxc10 days ago

Yeah. This is covering stuff you know if you've read that thread, but:

Putting all Python libs into a single channel is something Nixpkgs does because Python can't handle different versions of the same library in a single process. The Python libs that are used in actual applications in NixOS, then, need to be compatible or they might cause weird issues when a new or existing package tries to leverage them at the same time. Other distros do run into this but it might be worse in Nix. (Some (all?) C libs have this same issue, but they don't have rampant integration problems from version to version like the Python ecosystem does, so having just one copy of them is fine.)

And yeah, Python developers seem wary of the kind of vendorization that would fix this, and some Python package authors are very hostile about integration issues that Linux distros are more likely to hit than developers of individual downstream applications are. (I guess they don't feel like distro integrators are truly 'using their code' in the way that application or library developers would be, and those integration issues can be a lot of work to figure out for what feels like some 'fake user's' configuration problem.)

Python packaging has been an incredible mess for many, many years. I don't think the community's processes and institutions have the means to meaningfully fix it so that downstream consumers of Python packaging infrastructure don't have to use a big pile of hacks to successfully package Python applications.

Distros will probably see another language replace Python for internal tooling and sysadmin applications before they see Python packaging unified around something that behaves deterministically, works offline, lets programs reason about the dependencies of packages that are not present/installed/built, pins versions with cryptographic hashes by default, disallows arbitrary scripts at install time, sanely describes dependencies on native libraries, etc.

PS: This is not a knock on Python. The Python community faces some really tough institutional/governance/adoption problems because:

  - the language is mature and the ecosystem has a lot of valuable code in it, already packaged in various ways
  - the language was born without a modern package management story, because it's very old!
  - big community-wide changes are democratically governed and there are a lot of stakeholders who are bound to have opinions
Consensus will be really hard to build and legacy packaging processes will stick around for a long time. And some changes that could really help, like in-process changes to library loading behavior, are likely to be seen (perhaps correctly!) as too radical/disruptive to be in the best interest of the majority of the existing community.
SuperSandro200010 days ago

Applications can overwrite dependency versions they want to use. It's just not possible in the python package set because if ever two versions of the same package are in one environment very strange things happen and everything falls apart.

pxc10 days ago

Yep, Nixpkgs freely relies on that kind of vendorization to solve compatibility problems in other programming languages' ecosystems when it has to.

And Nix users can totally override specific Python library versions as one-offs for their own Python packages or environments.

It's just really unfortunate that some of Python's internals make that unfeasible for Nixpkgs' collection of Python libraries as a whole. :(

jnxx10 days ago

> The Python community faces some really tough institutional/governance/adoption problems

The Linux kernel people could have the same problems... it is genius how they have solved them, by developing git but also by establishing processes that scale.

qbasic_forever10 days ago

In the Linux kernel world Linus is the final arbiter and decision-maker--he takes input from people he trusts but ultimately he and only he has the final commit. Python used to have this model too with Guido at the top as the "BDFL" (benevolent dictator for life), however when he stepped down they replaced it with a consensus of a steering committee: https://peps.python.org/pep-0013/ Both projects have very different governance as a result.

jnxx10 days ago

> This affects Nix disproportionately because Nix actually integrates all package updates into one channel which nobody else anywhere does

Does Arch not do that?

infogulch10 days ago

Maybe it would be more correct to say that "Nix integrates all package updates into one channel which is unique to system integrators (like Nix and Arch) but an uncommon workflow for regular application developers".

pxc10 days ago

Yep, I think that revision is perfect.

JonChesterfield9 days ago

Python is _really_ tempting as a build (or test) dependency. It's an easy way to throw together code generators or similar glue that is likely to run on whatever platform is building your software.

nixpulvis10 days ago

Nix needs a new porcelain interface for it's CLIs.

Moreover, this post sums up one of my biggest issues stopping me from trying Nix for real again: https://ianthehenry.com/posts/how-to-learn-nix/ambiguous-pac...

Finally, needing to rewrite everything in Nix is nice for poorly written configurations or undocumented packages in general, but seems redundant for well maintained software. Has anyone else come up with a sane Nix strategy to avoid the overhead?

jnxx10 days ago

I have tried Guix (as a package manager), and it seems much better documented.

I also really like the fact that Guix uses a well-established, minimalistic, well-implemented, functional-preferred configuration language, which is Guile, the GNU implementation of Scheme, which is very much tailored to be extended with and embedded in other software, for example written in C. In part, my love comes from having had to use the alternatives: Huge configuration files written in YAML, for example, with no real documentation what all the keywords really mean, or things such as Conan, which appear declarative and are.... whatever.

ryukafalz10 days ago

I much prefer Guix UI-wise but it has some downsides:

— I’ve had more jank on Guix System as a desktop OS than on NixOS. Specifically some dbus-related stuff like notifications and appindicators (when running sway + waybar) has been very unreliable for me under Guix in ways that it hasn’t been on any other distro I’ve tried including NixOS. Still haven’t figured out why.

- Guix is slow compared to Nix. This is especially noticeable on older/weaker hardware.

- Nix home-manager has a lot more options than Guix’s equivalent - it’s really nice being able to rely on it for things like sway configuration.

That said, between the two I do lean towards Guix because I do gravitate towards Scheme more than the Nix DSL. I just wish it were a bit more polished.

mattarm10 days ago

Nix also runs on macOS. I think it gets a lot more traction among devs than Guix does for this reason alone.

jnxx10 days ago

> - Guix is slow compared to Nix. This is especially noticeable on older/weaker hardware.

Yeah, I have noted it is not the fastest snail on the lawn. On the other hand, I have seen so much time wasted on integration and reproducibility issues, that I'd be happy to run one day a week nothing else than a Guix install and not have any of these issues.

rahen10 days ago

GNU Guix, much like Emacs, is a set of LISP (guile) scripts running on top of a small C runtime. Nix is mostly C++ in comparison.

https://git.savannah.gnu.org/cgit/guix.git/tree/guix

SuperSandro200010 days ago

Guix is also very verbose compared to nix.

+1
jnxx9 days ago
Ericson23149 days ago

> minimalistic

The Nix language is far more minimalistic than any Scheme. I have nothing against Scheme but let's be clear, Nix out-Schemes Scheme in this regard.

pxc9 days ago

People complain about learning 'the language' when the thing they've really been trying to learn is actually:

  - the language
  - the stdlib (nixpkgs.lib)
  - the NixOS module system
  - several particular packaging ecosystems (stdenv, buildGoModule, buildPythonApplication, etc.)
  - the hooks and stuff that get exposed as variables and functions in bash-based builders
all at once! (plus maybe even the derivation format)

It makes sense as a shorthand, and I'm sure it describes the feeling, that people say 'learning the Nix language was overwhelming at first', because using the language in the context of all of those other things (essentially libraries and applications written in Nix) is the context in which one generally tries to learn it. And that really can be a lot to take in at once.

But I think when someone says 'learning the Nix language was hard' or something similar, it does sometimes mislead others about the complexity of the Nix language itself. So there's this widespread misconception that Nix-the-language has a lot to it.

But like you say, the language itself is actually super minimal. (And really good for its intended purpose, imo.)

Ericson23149 days ago

I definitely agree.

I want Nix to "show off" its layering a bit more for many reasons, but one of them is to hopefully allow for a more gradual curriculum.

jnxx9 days ago

For somebody who knows Lisp and Scheme, it is however more than nice to already know the language and be able to just read the interface description.

To give an extreme example of the opposite direction (no, I do not want to bash Nix here), take CMake and its configuration language which has no real definition. It is just painful to use.

pxc10 days ago

I'm still running Nix and only ever mess around with Guix, but I agree: the Guix CLI is beautiful and the documentation for it is quite good.

pca00613210 days ago

I think nix probably needs a more beginner friendly documentation as well, by beginner friendly I mean Linux beginners. Beginners will not read through nix pills (assuming they can understand it) before trying nix, they will just give up. And there are a lot of Linux users who don't have much technical knowledge about how binaries are being linked, etc.

I think the ecosystem is now mature enough for beginner users to ignore the detailed packaging issues, just rely on home manager and nixos options for most of the setup. And I think it is should be possible to create something like a GUI for home manager, to lower the entry barrier. If people are looking for a distribution rather than a build system, we shouldn't be teaching them how to use nix as a build system, we should show them some config that just works.

umanwizard9 days ago

Why is it necessary for Nix to be accessible to Linux beginners?

Sure it’d be nice to have, all else being equal, but I don’t see how it helps advance the goals of the project for the people who are using it today.

pca0061329 days ago

Because why not? For things like installing (commonly used) packages, setup some services, I think Nix is simpler than things like Debian or Arch. You don't need to understand all the nitty-gritty to install a package. The problem is about the documentation, some information is old (nix-env for example which should be deprecated IMO), may need better discoverability (options that you don't know they even exists), and we should be able to just give some examples for users to modify instead of asking them to learn the whole language.

For advancing the goals of the project, I think increasing adaptation will be beneficial, for example to let more software provide a nix script for building, due to more users using it. And perhaps if more users are using it, more companies and universities will adapt it.

SuperSandro200010 days ago

Nix is a programming language and there is no visual editor for it yet.

Ericson23149 days ago

I think we could do a GUI for a subset; would love to do that someday.

pca0061329 days ago

Yes, but for simple usages it is somewhat similar to things like JSON, they don't have to learn a language to install some commonly used packages.

grumbel10 days ago

> Nix needs a new porcelain interface for it's CLIs.

It already has one with the 'nix' command, it just needs to be manually enabled under 'experimental-features', but once done there is basically no reason to ever touch any of the old commands.

mikepurvis10 days ago

Yup. I started with Nix a little over a year ago on Nix 2.3, and have only ever used flakes and the new CLI. It's complete from my point of view— the main issue is just that the pills and all the official documentation still refer to the old commands.

I wish they'd flip it over to being the default and update the docs. I know that's not trivial, and it's hard when all the long-time community members have the legacy commands in their muscle memory, but IMO the current state of affairs is actively hurting the onboard experience.

senorsmile9 days ago

I started with a single system on NixOS around February of this year. I just recently saw an example of `nix search` instead of `nix-env -qaP`. I haven't seen any documentation for all of this new stuff. Any place to go, or do we just have to go read the source code?

grumbel9 days ago

`nix --help`, `nix search --help`, ... will show a man page, but a look at source can't hurt, as there are a couple of rough corners with the `nix` command that can be rather confusing.

For example `nix profile remove REGEX` will only match against the attribute part of the URL, which for Flakes is often just "defaultPackage.x86_64-linux", completely missing the name of the actual package, thus making it impossible to remove packages by name (using the index number will work).

The source is pretty readable:

https://github.com/NixOS/nix/tree/master/src/nix

mikepurvis7 days ago

I think it's basically just the man pages online, but I've found this resource to be pretty helpful:

https://nixos.org/manual/nix/stable/command-ref/new-cli/nix....

mattarm10 days ago

> it just needs to be manually enabled under 'experimental-features'

Same with flakes. My impression is that Nix is either on the cusp of a major paradigm and usability change, or the status quo will be forever in a state of having "wrong defaults."

pxc10 days ago

> My impression is that Nix is either on the cusp of a major paradigm and usability change, or the status quo will be forever in a state of having "wrong defaults."

I think probably both. The Nix community is host to very diverse and partially overlapping experimentation, and features like rollbacks and version pinning make the bleeding edge feel relatively safe, further fostering such experimentation.

The new Nix CLI will be huge for new users and for adoption once it's finalized. But there will probably always be a bunch of Nix users doing weird, cool shit that everyone kinda wishes was 'here already' for mainstream use.

+1
easygenes10 days ago
pca00613210 days ago

I think the really crazy default is that flakes by default will not include git submodules, you need to pass a special flag to enable it, which is undocumented and have to dig through github issues: https://github.com/NixOS/nix/issues/4423

yencabulator7 days ago

Which is not without its own warts, in ways that just don't give me any hope. Simplest example:

nix run nixpkgs#cargo --help

gives you a pager viewing the manpage of `nix run`.

alberth10 days ago

> Nix needs a new porcelain interface for it's CLIs.

The analogy I'd use is that Nix needs what "Github did for Git". Meaning, git is actually unnecessarily complex. But Github made git easy and accessible.

nixpulvis10 days ago

No doubt that is needed too! The "human" interface, if you will (that which sits on top of the porcelain).

Still, I think Nix could make a better interface for us power users.

rowanG07710 days ago

For well maintained software it's generally a few lines of a nix definition. It precisely gets hard when it's not properly maintained. Patches, insane build configuration (I'm looking at you Intel...), binary dependencies etc.

kaba010 days ago

> Has anyone else come up with a sane Nix strategy to avoid the overhead?

Didn’t need to use it for a long time now, because nowadays almost everything is packaged, but there is steam-run CLI app that will run the specified executable in a “standard-like” environment. It still doesn’t support everything, but could come in handy.

In the very early days I also just had a debian file system laying around and I machinectl-d into it (very lightweight virtual machines)

tomberek9 days ago

If something is well-maintained, it is usually pretty easy to make Nix derivation for it. The strict discipline that Nix imposes usually becomes an issue exactly when something is not well maintained or has fundamental issues with how it is managed, but this would be a problem for adoption or port to any other package manager.

SuperSandro200010 days ago

That blog post is using nix-env which you shouldn't do if you are trying to achieve more than the bare minimum.

fesc10 days ago

Man, there’s a lot of negative comments here. Just to add a different experience: my company loves Nix, it makes it really easy to integrate new tools into the dev/build environment without needing to document which packages, configuration, … a developer needs to apply to their machine manually.

There’s nothing like it, really.

bogota10 days ago

I dont think anyone is being negative at least not the top level comments i read. I like nix but i gave it a test run at my company and no one could figure out how to make any change that wasn’t a copy and paste of what i had done.

The docs don’t help much unless you really go diving into them and most people who just want the software to run don’t want to spend the time learning it. I don’t blame them.

This is a very valid criticism of any software. Its why things like docker (containerization) win out even when it was technically around for years before. Someone made it easy to use so people used it.

seqizz10 days ago

Same here. Using Nix and Nixos for years, I have prepared nix-shell and packaging files for our internal tools and committed to internal gitlab. No one ever touched it even after multiple presentations from me. No way to find your way out, especially if you don't have functional programming background.

octoberfranklin10 days ago

> most people who just want the software to run don’t want to spend the time learning it

these people are not nix's target audience

bogota10 days ago

I personally think that is a bit of a cop out. Most developers should be able to do basic tasks in a system that has been setup by an expert user.

I think bazel is an ok example of this. Its a pretty complex build system but expert users can build macros and rules that the average developer can consume without having to know a ton about everything that is happening.

IMO the ability to do the above at some level is the sign of well crafted software.

qudat10 days ago

Relegated to a niche subgroup, I see. There's a significant difference between spending time learning a tool and sinking dozens and dozens of hours into a tool in order to perform tasks that are so basic for other package managers.

+1
rgoulter9 days ago
oneplane10 days ago

There are many things like it. For example, managed operating environments where the user doesn't need to do anything (and actually can't do anything). Or disposable environments like VMs and containers.

Sure, it's not the same as massaging a special pet operating system over and over, but most people that need to produce software hopped off of that bandwagon years ago.

I get that companies that do functional programming and linux and linux on the desktop exist, but I have yet to find any company that does that at scale, at a good profit, versus competition. That's not to say that "therefore, Nix is bad", it's just that the problem isn't a technical one that nix suddenly fixes. It seems to be only a problem if you're stuck in yum/apt all day and need to get a fix to get out of that.

otabdeveloper410 days ago

Nix is not an operating system. It can be used to build operating systems easily, though.

Also, there isn't anything "functional" about Nix. It's a nice sales pitch, but underneath it's just a thin layer over bash scripts and environment variables.

octoberfranklin10 days ago

Nix is a metaprogramming language for bash.

The metalanguage is indeed purely functional. The object language (bash) isn't.

+1
otabdeveloper410 days ago
whateveracct10 days ago

Nix attracts whiners - 90% people happy with it are happy using it and not bothering to waste energy arguing with bad attitudes online.

rgoulter9 days ago

Well, it's easier to be contrarian online.

Though I anticipate better discussion from "nix didn't suit me" than "nix works".

Looking at Nix's community survey, there's been a big growth in the community over the last year or two. I think most who try nix like it, and see it as so obviously a good technology.

whateveracct9 days ago

My pet peeve (which is present in this thread!) is when people link Nix discourse or github discussions and say "look how ridiculous it is to do XYZ in Nix."

The community discussions are one of the best parts of Nix. People are super helpful and work together to solve novel problems all the time!

In fact, those threads are people doing something about "nix didn't work for me."

hamandcheese10 days ago

Are you hiring?

alch-10 days ago

I'm not GP, but my company's experience[0] is the same, and we're definitely hiring. Seeing Nix in the tech stack was one of my reasons for applying :)

[0] https://www.channable.com/tech/nix-is-the-ultimate-devops-to...

agentultra10 days ago

Nix seems great for build servers. This is a great introduction to the motivations behind it.

I'm not sold on using it for managing developer environments (another use case it is often used for). It "solves" the problem that developers might be using different versions of libraries or compilers on their machines... but it comes at the cost of having to learn a whole new programming language, a configuration language, a whole new jargon, and workflow. It's a bit like using Docker as a development environment. It introduces a non-trivial amount of friction.

Some folks get excited about package management and configuration. Personally I don't care for it enough to over-come such a high learning curve. And I don't particularly like the workflow it enforces.

However it is pretty great for reproducible CI/CD systems like Hydra: https://github.com/NixOS/hydra

KingMachiavelli10 days ago

In my experience Nix works pretty great for developer environments when you don't previously have anything more than manually configured dev servers. (In general Nix is much easier to buy into if you don't already extensively use Docker, k8s, etc.)

For smaller teams with less experienced developers any tool is going to have a fairly high learning curve. Instead of having everyone learn Nix or Docker, it's very easier to have a few, more Linux experienced devs configure servers and devshells using Nix while providing simple TUI for developers to access various tools.

There are some headaches, if you want to use a full NixOS environment then there are some complications with tools like VSCode and NodeJS that download dynamically linked binaries but its terrible difficult to workaround.

xyzzy_plugh10 days ago

Hydra is arguably one of the scariest parts of Nix: A giant, bespoke CI system written in C++.

Arguably it shouldn't exist. Nix can and should easily slot into any CI tool.

octoberfranklin10 days ago

And, unlike nixpkgs, Hydra is pretty unfriendly to contributions.

There is one master trusted public key for nix:

    6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
It is hardwired into the nix source code and every (unpatched) build of nix from the last decade or so. There is no revocation system. There is no public key infrastructure. If that key gets compromised, there is no backup plan. I love Nix, but this is batshit crazy.

The Hydra instance has access to the corresponding private key. So the people who merge changes to Hydra are understandably paranoid. Unfortunately this has turned the codebase into a mess.

infogulch9 days ago

> There is one master trusted public key .. It is hardwired into the nix source code .. There is no revocation system .. The Hydra instance has access to the private key

> it is built in C++

pxc10 days ago

There are couple of CI/CD systems being produced by startups that aim to fill that role, based on BuildKit/Docker. For now, Nix is still way better than they are in many ways, with the notable exception of Windows support.

Maybe visible interest in them can push forward Nix community developer interest in polishing Nix for the same use cases.

hoosieree10 days ago

A few years ago I heard about Nix and NixOS.

I loved the idea, but I did not enjoy the experience. Maybe the problem was I was trying to make it work on a less well-supported platform (I think it was ARM32). But the packages I wanted to install either weren't available, or I kept getting incompatibility errors.

I still love the idea, but these days I feel like environment managers like Anaconda make (mutable) Python development a little more manageable, and things like Docker make (mutable) Linux development a little more manageable. Basically these both make it less painful to start over with a fresh "thing" when the system starts to get crufty.

In my view, there's a spectrum from "immutable and annoyingly rigid" to "mutable by default and annoyingly unpredictable". And the sweet spot is not at either end, but something like "immutable by default, but mutation is possible".

smoldesu10 days ago

I use Nix on an array of devices (any combo of x86/aarch64 and Linux/Darwin machines you can imagine), and while I really do love the experience, you're right that ARM is a sticking point. It's been getting better over the past few months, but still not close to x86 packaging parity.

On the other side of that coin, I tried switching back to Arch a few weeks ago after ~4 months of NixOS. Maybe it's the sunken-cost fallacy, but Arch didn't make me feel all starry-eyed anymore. Nix feels like a really dependable piece of my workflow now, and it's difficult to imagine myself going back to Homebrew/pacman.

> the sweet spot is not at either end, but something like "immutable by default, but mutation is possible".

Flatpak tried that, you're welcome to draw your own conclusions on how that turned out. The problem is that your modifications now require build hooks for every update, and you're no longer guaranteed a comprehensive runtime. With Nix, these hooks get re-written into derivations, which (in my experience) provides a more stable, sane alternative to Docker images and Flatpaks. It's also not packaged hermetically, which means that not all Flatpaks will behave the same on all machines. Something as subtle as different environment variables or display server implementations can cause your application not to launch.

mmcdermott10 days ago

Is there a way to get it so that UI applications installed with Nix show up in Spotlight Search? I remember that being my big annoyance when I tried using Nix instead of Homebrew on MacOS.

pxc9 days ago

Yeah, if you use a module system like Nix-Darwin or Home Manager: https://github.com/nix-community/home-manager/issues/1341#is...

(Hopefully comparable modules are on their way onto the defaults for each of those module systems)

smoldesu10 days ago

Oh, I wouldn't know (my Mac is a dumb terminal for testing and nothing else). They should have appropriate .desktop files for their Linux counterparts, though.

goosedragons10 days ago

I recently tried Guix which is somewhat similar to Nix in theory. While I loved the idea I did not love dealing with cryptic problems resulting from it. For example I ran into an issue where one R package would only work properly if I installed other packages in a specific order, doing an update though could mess up that order breaking the package. Later updates to R broke support for certain things entirely that I don't see on non-Guix setups either. Plus getting things that required binaries I can't (or easily) compile myself to work is a pain.

I just can't justify fiddling with my package manager so much.

mattarm10 days ago

This is my experience with both Nix and Guix. They have advantages but they are different to the point of being unusual.

I very much want the benefits of these kinds of systems, but they both produce a run time system (shell environment, whatever) that is not typical of how most people use software. So, while most other people are helping each other out with the "usual" problems, Nix/Guix users have a different set of problems. Sure, they are reproducible problems often shared by all other Nix/Guix users, but those communities are niche compared to what is typical.

jnxx10 days ago

I am not sure, but it is possible that R want to be its own package manager which interferes with Guix? A similar problem seems to appear in Python, where there on Distributions like Debian there is kind of a struggle which package manager - the system-defined packages, or the user-defined ones like pip - have the last word.

I have used Guix a bit with Common Lisp libraries, and that works like a charm. I also found it useful to be able to use new Emacs packages like the newest version of Magit, without having to install or supersede the OS installation.

goosedragons10 days ago

R does have it's own package manager but I was explicitly not using it because I read there could be issues, instead only installing packages through Guix. It was mainly a problem compiling things within R as if the order of how gcc-toolchain and the R package were setup it straight up wouldn't work. With later versions it just broke entirely and I never managed to fix it.

+1
wdkrnls10 days ago
+1
JHonaker8 days ago
wdkrnls10 days ago

I have not experienced that for what it's worth... atleast not after I rebuilt the whole profile.

rekado5 days ago

I would be very happy to see details on your R problems, as I have never encountered anything like that in the many years I have been packaging R things for Guix and supporting R users at the research institute where I work.

rowanG07710 days ago

I mean if you don't want to fiddle with it I really don't get why you would choose Guix to test drive this way of managing your packages. It's the way less mature and less supported Nix.

rekado5 days ago

Not when it comes to R. I'm terribly biased, but R packaging in Guix is --- in my opinion --- very high quality, purely from source (including bundled minified JavaScript), and the tooling is great too.

Unlike R in Nix, Guix does not just automatically wrap R packages, which would lead to build and runtime errors. I would not use R from Nix.

goosedragons10 days ago

I did try Nix briefly. It was even more confusing to configure for use with R IMO.

easygenes10 days ago

For x86, the package availability is almost unparalleled.

NixOS also has tools and options which are not all pure and declarative, they're just not considered, "the way."

I find NixOS to have a bit of a high learning curve, but worth it for the power and reproducibility.

tjoff10 days ago

> * For x86, the package availability is almost unparalleled *

No it is not. Please don't spread that nonsense.

Tons of packages don't work. Many are not maintained. Debugging packages are a huge pain.

I've never had so much problems on any distro related to packages as on nixos.

Which is expected! Nixos is novel and a niche, few develop for it and tons of stuff break because assumptions that work on all other distros don't.

But please don't try to give people the idea that nixos package situation is great or even good. It is only hurting the cause.

rgoulter10 days ago

> For x86, the package availability is almost unparalleled.

https://repology.org/repositories/statistics/total In terms of total number of packages, nixpkgs unstable is at 72k, while AUR is at 68k.

I'd bet there are many caveats, though.

enoyls10 days ago

I'm not sure what point you're trying to make.

AUR is a supplementary set of packages, and it looks like you're comparing it to the total number that Nix supports.

Wouldn't a more fair comparison would be official Arch packages + AUR to nixpkgs?

soraminazuki10 days ago

No, because that would be comparing the number of official packages + third party packages with duplicates in Arch against the number of official packages in Nixpkgs. That's as fair as comparing the number of official Nixpkgs packages + every unofficial Nix package on GitHub against the number of official Arch packages.

soraminazuki10 days ago

A direct comparison with AUR is meaningless though. AUR is very unusual because it allows anyone to freely upload packages without going through review. So there are many duplicates and packaging quality varies wildly.

hailwren10 days ago

An unfortunate huge one is any way to interact w/ Intel Optimus. I can't be the only dev that wants to eat my gpu cake and have my battery life optimization cake too.

Bumblebee and optimus-manager both solve this in the aur.

+1
pxc10 days ago
pca00613210 days ago

I don't think the problem you encountered is due to immutability, the problem is due to the implementation, which is now improved due to better documentation and more package support.

The possibility of mutation alone will break a lot of assumptions and make program analysis a lot harder. I personally prefer no mutation at all or only when wrapped inside a cell (UnsafeCell), similar to rust. For the latter kind, we can treat the states as immutable if we don't have a cell, which can help analysis.

hoosieree10 days ago

That's fair. I blamed immutability when I should have blamed the implementation. Thanks for helping me realize my mistake. :)

Availability of packages is what makes or breaks a distribution, though. If I can't (easily) install the software I need to do my job, I choose a distribution that can. My home Ubuntu server isn't bringing me joy, so maybe now's a good time to give Nix another shot.

Fingers crossed for Nvidia driver support...

dpatterbee10 days ago

In my experience the Nvidia driver support has been pretty painless (once you figure out the correct settings in configuration.nix of course), and the availability of packages is by far the best of any linux distro I've used.

jcelerier9 days ago

I've been working on a project that uses Nix, and from an archlinux host with Nvidia hardware it has been very far from painless, with nixGL breaking every time Arch updates glibc

pca00613210 days ago

I think if you don't use some relatively niche packages, nix is fine. However there are some quirks such as setting LD_LIBRARY_PATH for CUDA that may need some tinkering.

If you want to give nix another try, I strongly recommend you to use nix flakes and home manager. Nix flakes allows you to pin dependency versions, and home manager provides a lot of configurations for commonly used packages.

euroclydon10 days ago

I built my router and NAS from NixOS. It was a mostly pleasant experience. Being able to sit in an IDE on my laptop, and build up a server, incrementally pushing changes to it, with rollback if necessary, was great, and I wouldn't want to go back to anything else.

I wrote about the router here. It's pretty heavy on router stuff, and my own thoughts though...

https://www.jjpdev.com/posts/home-router-nixos/

mistahchris10 days ago

Wow that is cool. I have been wanting to build both a router and a NAS and run nixos on them since I run nixos for everything else I do. Thanks for writing and sharing about your experience!

VTimofeenko9 days ago

Excellent read, thank you! Just in time, I have been eyeing one of those tinyminimicro pcs to replace OpenWRT with Nix.

Could you share more details on push_to_router.sh? Is it a wrapper around calling nixos-rebuild through ssh?

euroclydon9 days ago

Yes, although later I learned I could do it in a one-liner. Here nixconfig is a folder with all my nix files:

  tar -czf - nixconfig | ssh 192.168.1.1 \
    'tar -zxf - && sudo cp -r ./nixconfig/* /etc/nixos/ && sudo nixos-rebuild --show-trace '"${rebuild_flag} ${name_flag}"
yencabulator7 days ago

If you're ok with building locally and not on the target host, this is simpler:

nixos-rebuild --flake .#foo --target-host root@foo --build-host localhost switch

+1
VTimofeenko6 days ago
mr33710 days ago

This is fantastic reading and never really through to use nix for a router.

rgoulter10 days ago

Neat article.

For me, the key takeaways are:

1. 'Nix is to `tar -xf && make && make install` as C/C++ is to assembly'. In many ways, Nix applies the same kinds of improvements that other technologies have.

2. Nix does try and create an elegant programming model of Unix systems.. while the Nix programming language is pure, it interfaces with the Unix system by reading files and outputting files.

I'm mixed on to what extent articles like this get to the goal of make Nix more accessible, though. It seems like preaching to the choir to me: if you like the idea of making analogies between "software is files, is like dealing with raw pointers", you'll prob'ly love diving into Nix as is anyway.

revscat10 days ago

Same. I read almost every nix-related post that I see pop up on HN. I want to be convinced. I have not yet been. It seems like `brew install` with (many) extra steps, with little meaningful gain.

rgoulter10 days ago

I'd describe many of the benefits for developers as like "docker, without containers".

e.g. if you want to try out helix, you could run `nix run nixpkgs#helix`, and it would download + run helix without installing it. (Or you could run `nix shell nixpkgs#helix` to add helix to the PATH in the current shell, without installing helix, etc.).

One use case I'm excited about for developers is the ability to declare the dependencies needed to build the project. -- So rather than copy-pasting `apt-get install` commands, you'd rely on nix to fetch the installed dependencies. (e.g. I love that I don't have to worry about what packages to install to work on qmk_firmware, or repos which provide a nix shell).

VSCode's Remote Containers supports a similar workflow to the latter.. but, it relies on containers.

yakshaving_jgt10 days ago

Important difference: Docker is repeatable, but not reproducible. Nix is reproducible.

revscat10 days ago

We use Brewfiles to install binary dependencies needed by various projects. This is only for developer’s machines, but it’s lightweight and fast: `brew bundle` and you’re done.

+1
noahmasur10 days ago
thomastjeffery9 days ago

Nix is an abstraction around `tar -xf && make && make install`. In fact, those commands are even executed by nix, just in a sterile reproducible environment.

What sets nix apart from other package managers is that you are never running `make install` on your root filesystem, but `make` can still dynamically link to libraries (that also aren't installed on the root filesystem) without editing the Makefile directly to find them.

This way, you can't break existing packages, you can trivially roll back changes (because updates are new instances), and you can always start over fresh.

The only problem is that you have to wrap every package in a derivation, then publish that derivation somewhere. Right now, all derivations are tracked in a single git repo (with dozens of branches), all coordinated over GitHub Issues, and referenced by nix itself by an arbitrary (versionless) name in a global namespace in this file: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/...

That last bit can be avoided by using pinning and flakes, but it's still the default way to use nixpkgs, and documentation doesn't clarify much or offer a better consistent UX paradigm.

jnxx10 days ago

The article talks about the file system abstraction and that NixOS is parting ways with it:

> It is not even a new idea for Nix to propose parting ways with one of the most pervasive skeuomorphisms in computing, the file system, which naturally followed from an era where everything was a piece of paper.

What I am wondering is if this is not extremely similar to the way that plan9 handles files? As far as I understand, in plan9, there is still a file system - but there is no common root, every process can have an own view what is, for example, in /bin.

r3trohack3r10 days ago

If you haven't read the original LISA paper on nix, it's a great read. So far it's the best resource I've found that documents what nix is, why it exists, and what it's solving: https://edolstra.github.io/pubs/nspfssd-lisa2004-final.pdf

idealmedtech10 days ago

Back in college, the engineering school had a farm of Linux servers from which you could access all your documents, as well as log in to your account from any number of computer labs around campus. Back when I was first learning Linux (not that long ago, 2008), I fell in love with how many things you could customize, and really create your own experience. When I got to college, I wanted to replicate this ability (install ANY program! use ANY desktop environment!), but predictably, their machines were pretty locked down.

I went snooping around the internet, and found that there was this magical software called Nix, which would let you have a package manager without root! The Linux computers in the lab didn't have root, but they did have GCC. I started learning all about build systems (mainly that you could specify a custom --prefix and essentially create your own filesystem within your filesystem), and got to work compiling Nix (or, i think, GUIX) from source. It's actually a fantastic amount of work to go from GCC all the way to a functional GUIX, even with access to every source tarball on the Internet

Eventually some admin emailed me about this project and I stopped working on making it happen shortly after, but it was such a formative experience in my tech life that I always think back fondly on it when Nix pops up.

pxc10 days ago

This is also how I got into Nix! I just used a 'normal' Nix install with a fake chroot (pivot_root/proot) environment, though, so I didn't have to build from source and I got to leverage the binary caching. There were some quirks, but overall it worked well.

jrmurray10 days ago

While nix can be very intimidating to get going, I think for just getting developer environments spun up it can be somewhat simple. I highly reccomend trying to add a `flake.nix` to your projects. It makes on boaring new devs a breeze https://medium.com/immuta-engineering/nix-and-skaffold-for-p...

baby9 days ago

Been using nix for a few weeks and while I feel like it really has some great ideas, here are what my pain points have been:

- not great documentation, especially for newer features like flakes

- nix wants to replace rustup when rustup is already doing great

- nix doesn’t seem to work that well on mac. Not sure if it’s our config our that it’s painful on mac in general

- the biggest issue: it doesn’t work well with tools (vscode, sublime merge, etc.) as you need to launch them within a nix shell and that doesnt work well (at least on mac). Now I’m wondering if it’d make sense to install tools within the flake dev shell…

rgoulter9 days ago

> the biggest issue: it doesn’t work well with tools (vscode, sublime merge, etc.) as you need to launch them within a nix shell and that doesnt work well (at least on mac).

In what sense?

In terms of "some nix shell provides some tools, and VSCode can't see those".. direnv is one way to work with this. e.g. direnv integrates with nix to integrate the nix shell at that path, and a direnv plugin for VSCode etc. can pick up the direnv file, so that it loads the nix shell appropriately.

baby9 days ago

I tried everything from direnv (with the nix direnv thing and a diverse direnv extensions for vscode) to opening vscode from a flake dev shell, nothing works on my mac

ParetoOptimal9 days ago

Vscode doesn't let you specify plugin load order and nix-env-selector needs to load before anything else or you can have problems last I checked.

baby9 days ago

Ah that might be why I had these problems. Do you know what a solution could be?

jnxx10 days ago

> Which other results from programming language theory and mathematics will we be able to leverage to make software build quickly, work reliably, and further tame Unix?

I think Apenwarr's redo (https://redo.readthedocs.io/en/latest/), based on an idea from D.J. Bernstein, is a very interesting development, because it also has the "purely functional" principle at its core - and this allows for much faster parallel builds.

jnxx10 days ago

To add, there is another angle on immutability / purely functional definitions, which is compatibility of APIs. Rich Hickey (the creator of Clojure) has made a talk titled "Spec-ulations", in which he pointed out that there are certain operations on an API, like adding functions, adding symbols (like enumeration values), loosening preconditions (like, for example, adding keyword arguments to a function), tightening post-conditions (like, for example, removing possible error exit codes), and so on, that keep an API compatible, while others, such as removing functions, tightening preconditions, widening post-conditions (such as adding error codes or exception types), break compatibility.

And then he points out that the API itself can be seen like a persistent data structure, like a dictionary where you can add new things but not remove old things, because that would break client code. And I think this is a very important idea.

GNOMES10 days ago
olodus9 days ago

I have tried starting nix several times never got all things working. However, after being stuck in several dependency hells both in personal projects and at work, I knew I wanted what nix was proposing.

This last time I tried it actually clicked much better. I think what flakes has done is not just provided the technical solutions for why it was created, but it also made it much easier to understand a nix repo and to a newbie like me it almost seems like it results in cleaner code (I now much more often end up understanding what a nix file is doing). That together with the updated nix command in general makes it much more intuitive in most of the cases.

So I just wanted to say that to the nix team that your focus on UI is paying off for newbies like me.

phaer10 days ago

Great introduction and overview on the theoretic foundations of the nix ecosystem!

For people new to it, I am trying to provide a quick glossary of terms here, as I understand them after about 2 years of using nix.

* nix: a language to create derivations and the interpreter/package-manager which provides the implementation of said language. It currently offers two command-line interfaces, the stable on with hyphenated commands like "nix-build", "nix-shell", etc. And the newer, "experimental" one which includes support for nix flakes and so on, without hyphens: nix build, nix shell, nix run, etc.

   repo: https://github.com/nixos/nix
   docs: https://nixos.org/manual/nix/stable/
* nixpkgs & nixos is a huge mono-repo containing instructions how to fetch the source of tenthousands of software packages and how to build them on supported platforms. It also contains the whole nixos operating system and tooling to support all of that.

repo: https://github.com/NixOS/nixpkgs docs: https://nixos.org/manual/nixpkgs/stable/ docs nixos: https://nixos.org/manual/nixos/stable/

This tooling includes higher-level helpers for language-/environment-specific packaging, like "buildGoModule", "buildRustPackage" and so on, as well as e.g. tooling to run integration tests in a whole cluster of inter-connected linux VMs!

Packages which are submitted to nixpkgs must fulfill certain criteria, such as not using "IFD" (input-from-derivation, to simplify: "letting nix evaluate nix-code which was generated by another deriviation/"nix package".

nixpkgs is alive and well with lots of daily contribution and an everlasting effort to keep Hydra, the nix-specific CI/CD system and public binary caches up to date and responsive. Thanks to all maintainers & contributors!

* flakes are an approach to standardize a way to package nix code outside of nixpkgs but to still keep it re-usable. They are still "experimental" as the details are figured out, but nevertheless used in production. There are some frame-works to keep boilerplate low, like "flake-utils", "flake-parts" and others, as well as e.g. deployment tools like "colmena" and "deploy-rs" and re-usable helpers for system-configuration like e.g. https://github.com/nix-community/impermanence

There's lots of other stuff in the community, things like home-manager, direnv + flakes and devshells changed my workflow fundamentally to the better since I've switched. If you got the time and are still interested, join us on matrix or elsewhere :) https://github.com/nix-community/awesome-nix

eulenteufel10 days ago

I'll consider flakes usable for packaging software when they support passing options. The respective issue [0] has been closed unfortunately. Perhaps I am misunderstanding what flakes are meant to be (a more formalized standard way to define nix packages and apps), but a lot packages in nixpkgs have a plethora of parameters that as of now can not really be mapped to a functionality in nix flakes.

[0] https://github.com/NixOS/nix/issues/2861

pxc10 days ago

Depending on your specific needs, the channelsConfig (modify arguments passed to Nixpkgs) and overlay tooling in flake-utils-plus may be adequate: https://github.com/gytis-ivaskevicius/flake-utils-plus/tree/...

You can take a package from a flake input and call it with other args in your overlays, for example

eulenteufel10 days ago

It's nice that there is a workaround, but passing build options is not something that should require a library. There should be a well documented standard way to do it.

+1
pxc10 days ago
zuzuleinen9 days ago

I think Nix is great to add dependencies to your project without relying on your local env or separate Docker containers for that. We use it for bob[1] and so far Nixpkgs proved very valuable. It's amazing how many packages are pushed by the maintainers, there are over 80 000 packages there.

One problem would be when you don't find the package on Nixpkgs and have to write your own expressions to build a package.

https://bob.build/docs/getting-started/package-management

Bolkan9 days ago

If a lot of people used NixOS, the world would be a better place to live - Tommy Wiseau

candiddevmike10 days ago

Does Nix still build a lot of things from source? When I tried it a while ago everything took forever to install because it was compiling locally. Do they have the concept of repos and repo mirroring?

breakds10 days ago

Yes it by default build everything from source. However, most of the packages will have binary cache on https://cache.nixos.org/ so that your installation or update will download them instead of building them. Also you can setup your own binary cache (https://nixos.wiki/wiki/Binary_Cache) and building machine to make your own projects build faster.

pxc10 days ago

> When I tried it a while ago everything took forever to install because it was compiling locally

tl;dr: This is probably due to incompleteness of the binary cache. This is pretty rare in general, but it used to be relatively easy to hit on macOS on Nixpkgs unstable before the community added some channels for use on macOS. Check out the darwin stable release channels of Nixpkgs to avoid this issue if the current defaults don't show enough improvement for you, and see below for a more complete explanation

> Do they have the concept of repos and repo mirroring?

Nix is fundamentally a source-based package manager. This means it does not use binary artifacts enriched witb metadata to perform dependency calculations at install time. This, in turn, means that it doesn't have a use for binary artifact repos of the same kind as you see for DEB or RPM.

However, Nix does support caching and distributing binary artifacts in a different way. Since all Nix builds are deterministic modulo (hopefully inconsequential) indeterminism in upstream build processes, once Nix is right about to build a source package— it has figured out all of the build parameters and source tarballs to use and so on, for that package and recursively for all dependencies— it can just ask a remote server 'Hey, do you have anything for these?'. And the remote server can answer without storing or understanding any metadata about dependencies, or statefully storing a collection of packages at a particular collective repo version, or anything like that. If the remote server answers 'no', then instead of just choking, like a binary packages manager must when a repository is missing a package, Nix just chugs along like 'ok, I'll build it myself, then!'.

So with Nix, there are hosted collections of binary artifacts, but the metadata associated with them is more minimal, and they play a much less crucial role in the install process.

The 'repo mirroring' thing likewise has an equivalent: Nixpkgs' build artifacts are uploaded to S3 and then distributed via CDN. There's no syncing mirrors because there's no state to sync (multiple copies of different versions are hosted in the same place at once, since they're quasi-content addressed). And the CDN hopefully takes care of the local mirror issue for you, but you can set up your own Nix build cache as well, or add custom binary caches. If the CI/CD system you use to do this has 'substituters' (binary caching) enabled, then it will just download packages from the main CDN instead of building them, just like your local machine would! So aside from serving the binary cache publicly, 'building' Nixpkgs is the same as mirroring it.

For third-party efforts outside Nixpkgs, it's common to use the 'free tier' offered by Cachix, a proprietary, freemium SaaS binary cache for Nix builds which is free for open-source projects.

Overall, I think this is better than the old-school setup with binary package managers and their repos. But one thing that is possible here is binary cache misses, where your collection of package recipes includes some recipes that have never been publically built and cached.

Nix uses the notion of release channels to deal with this: a Nixpkgs channel is a snapshot of Nixpkgs which only advances to a new version when every recipe in some collection has been successfully built (and cached!) by CI/CD. This lets you get the best of both worlds: binary caching for everything you could want by default, and totally transparent integration when you want to install a specific package with your own patches, customized build parameters, etc.

Generally speaking, the 'default' channels for Nixpkgs are configured based on collections succeeding on Linux/NixOS builds, so the recipes on them may not always be 'in sync' with the macOS binary caches. If you use one of the channels tested against macOS, you avoid this possible mismatch. Nowadays this is the default, and there's even a stable release channel for macOS. But this was not always so, and consequently you used to get kind of a lot of cache misses on macOS.

cerved10 days ago

disappointing that there was no example :/

atemerev10 days ago

Nix is React for system configuration.

revscat10 days ago

So… over-engineered and expensive to maintain?

pxc10 days ago

Overengineered? Maybe, depending on your use case. NixOS is pretty popular as a desktop OS within the community, for example, and I could see a case that its guarantees and strictures are overkill there.

But maintenance is really easy. You're basically never forced to rewrite or throw away tons of config. Doing literally years worth of updates at once is typically pretty painless. (Adding new packages to Nixpkgs or new features to NixOS can range from trivial to very hard, just depends on the details.)

atemerev10 days ago

I tried using NixOS a few years ago. Everything went fine, except Python packages. There has been quite a lot of incompatibilities, sadly.

+1
pxc10 days ago
atemerev10 days ago

I don't know, 32kB minified is quite good. And everything is React now anyway.