Dapp tools and the Nix package manager

We're in love with the package manager Nix and we're using it to distribute our Ethereum developer tools. It's not just yet another package manager—it's a uniquely principled approach with huge benefits.

This post explains how to use Nix to install our tools and then goes into some more detail about how it all works.

Here's how Eelco Dolstra introduces Nix in his 2006 Ph.D. thesis, The Purely Functional Deployment Model:

The development of principles and tools to support the deployment process has largely been relegated to industry, system administrators, and Unix hackers. This has resulted in a large number of often ad hoc tools that typically automate manual practices but do not address fundamental issues in a systematic and disciplined way.

This is evidenced by the huge number of mailing list and forum postings about deployment failures, ranging from applications not working due to missing dependencies, to subtle malfunctions caused by incompatible components. Deployment problems also seem curiously resistant to automation: the same concrete problems appear time and again. Deployment is especially difficult in heavily component-based systems—such as Unix-based open source software—because the effort of dealing with the dependencies can increase super-linearly with each additional dependency.

This thesis describes a system for software deployment called Nix that addresses many of the problems that plague existing deployment systems.

Basic setup

Installing Nix and our tools on your GNU/Linux or OS X (or Windows?) computer is unobtrusive, and easy to completely undo should you so wish. First run the Nix installer:

$ curl https://nixos.org/nix/install | sh

The installer will tell you to add a line to your shell configuration. Do that, then continue:

$ nix-channel --add https://nix.dapphub.com/pkgs/dapphub
$ nix-channel --update
$ nix-env -iA dapphub.dapp
$ dapp --version
dapp 0.5.7  

Because of Nix's design, the dependencies of dapp are kept encapsulated. Exactly one command will be exposed in your PATH without any other visible impact on your system outside of the /nix store.

To make the whole dapp tool suite available, including the latest solc, do:

$ nix-env -iA dapphub.{dapp,seth,hevm,solc}

Our channel has signed binary packages for Linux and OS X and features

  • the latest versions of our developer tools,
  • versions of the Solidity compiler going back to 0.4.5,
  • the latest Geth, and
  • a growing set of Ethereum-related tools.

If you have any problems or questions, join our chat! There's also the Nix manual with details on using and developing with Nix.

More about Nix

Nix is actually an implementation of a package specification language, which happens to be a simple purely functional language. A package is defined as a function that takes its dependencies as arguments and returns an unambiguous build description called a derivation.

If you haven't programmed with a purely functional language before, you can basically think of the Nix language as a kind of YAML or JSON, but with simple functions. So packages are mostly made up of nested records that define properties, but you can also define functions to reduce duplication.

For an example of why it's useful to write packages with a functional language, you could look at solc-versions.nix in the DappHub overlay, which defines many versions of the Solidity compiler.

All files installed via Nix are guaranteed to end up under a single self-contained directory, usually /nix. If you look there after you've installed some packages, you'll see that derivations are stored in paths that include hashes. The hashes are derived from the expansion of each derivation including the build script and all dependencies. Files in the /nix store are treated as immutable.

The pervasive hash-based referencing means that there is no problem with having several different versions of a library depended on by different programs, or one version but compiled with different flags. The different variants will end up with different hashes and each program will link exactly to the version it needs. If several packages depend on the same exact variant, there is no duplication.

Nixpkgs and NixOS

Nix as a base layer is a very general build and installation tool. The repository of packages, Nixpkgs, is conceptually separate. It has more than 100,000 commits and is one of the most active projects on GitHub.

As an example of a Nix package in Nixpkgs you might look at the Z3
package definition
which provides the SMT logic solver used by recent versions of Solidity. The package is defined in terms of the location and hash of its source code, which Nixpkgs dependencies are needed, and some simple modifications to the default build procedure.

Vendors can provide overlays based on Nixpkgs. For example, Mozilla has its own overlay with cutting edge versions of Rust, Servo, Firefox, and so on. The DappHub channel is also an overlay, pinned to a specific revision of Nixpkgs.

Nixpkgs is structurally similar to FreeBSD's ports repository, and it acts like a version controlled history of an entire software distribution. You can install packages from any revision of Nixpkgs, which is great for reproducibility.

There's also NixOS, a Linux distribution fully based on Nix. With NixOS, your entire system configuration is a derivation, and all changes to your setup are both atomic and reversible.

Plans for Nix in dapp

With the infrastructure in place, we'll go on to integrate Nix with the dapp toolkit. The first use case is the ability to build and test dapps with different Solidity compiler versions without any tedious manual management.

We have more use cases in mind, like the ability to easily spin up a private geth testnet with your dapp deployed, and we even want to experiment with defining Solidity packages as Nix derivations.

It's exciting to finally have a really good solution to package management, and we hope you'll try it out.