DISCLAIMER: This is just an experiment

I made this fork of nixpkgs, which uses the module system for all mkDerivation based packages. All changes are contained in the latest commit.

Done:

• in make-derivation.nix replace call to builtins.derivation with lib.evalModules
• declare interface using the module system
• its based on Robert Hensing (roberth) 's WIP record type for performance reasons (submodule+freeform was quite slow)
• the defined options are incomplete -> unknown fields default to types.raw
• added modules argument to mkDerivation allowing user to pass extra modules (cool, but still requires calling overrideAttrs which is weird)
• pass through .mix function for each derivation, that allows passing a single module to extend the derivation.

Example usage:

Overriding

overriding before:

firefox.overrideAttrs (old: {buildInputs = old.buildInputs or [] ++ [hello];})

overriding after:

firefox.mix {derivation.buildInputs = [hello];}

• no need to pass function
• no need to handle previous value
• no need to handle default
• no need to implement merge

Type Checking

nix-repl> firefox.mix {derivation.buildInputs = {};}
error:
[...]
       error: A definition for option `derivation.buildInputs' is not of type `list of (null or package or path)'.
       [...]

Performance

(None of these findings are thoroughly tested or statistically sound)

• Overhead on packages with small closure is little (~10% for pkgs.hello)
• Overhead on packages with large closure is high: (~100% for pkgs.firefox)
• Overhead when evaluating all packages is between 100 to 200%

Other findings:

• Increasing the number of type checked fields didn't seem to increase the overhead significantly

My main question is: What are the next steps? Is there something useful to make out of this?
Some Ideas:

• deliver this as an overlay to allow experimentation on any nixpkgs version
• finish the interface declaration and make a CI job for nixpkgs that does type checking for all packages (we did notice that some packages pass weird types to some fields, like a list of lists for build inputs. It just doesn't end up being a problem because of how stuff gets converted to strings in derivation)
• do some more sophisticated benchmarks that can be reproduced and used to optimize the module system.

DavHau: Nice! This would be great to have in a GitHub issue, in Matrix it's easily lost.

I'd like some more detailed benchmarks, including what exactly is being benchmarked (is it the same that ofborg uses? that would be a good indication). Also not just time (I assume that's what you measured), but also memory usage.

Just from the numbers you gave though, I think this clearly demonstrates that it's too inefficient and we really need to improve performance of the module system to make this work.