!VRULIdgoKmKPzJZzjj:nixos.org

Nix Package Manager development

856 Members
For people hacking on Nix: https://github.com/NixOS/nix Nix maintainers can be reached here.180 Servers

Load older messages


SenderMessageTime
18 Oct 2025
@dramforever:matrix.orgdramforever --substitute-on-destination definitely happens in practice 15:32:11
@fzakaria:one.ems.hostfzakariaanyone want to pair/hack tonight?20:11:25
@fzakaria:one.ems.hostfzakaria

Question I was confused by Jeremy Fleischman (jfly) :

When calculating store-paths, Nix does "Hashing modulo fixed-output derivations" -- this makes sense since you want the store-path to be stable for multiple drvs for the FOD.

His question:
Can we replace all the inputDrvs with their store-path and also achieve the same thing?
(replace via the same massaging that we do for FODs)

20:54:21
@fzakaria:one.ems.hostfzakariaimage.png
Download image.png
21:04:07
@fzakaria:one.ems.hostfzakariaThe part we are confused about: B.drv may have many different A.drv ; That's OK because eventually A.drv is replaced with A.outPath... BUT C depends on B.drv ... wouldn't that path have changed since the contents of the drv can have N A.drv paths21:04:58
@jfly:matrix.orgJeremy Fleischman (jfly) My question is stronger than "can we?". I'm saying "mustn't we?" 21:42:26
19 Oct 2025
@lovesegfault:matrix.orglovesegfault

I'd appreciate some feedback on https://github.com/NixOS/nix/pull/14298, in particular regarding this:

One open question is that, with this fix, we accept copies of paths with any signature, even if that signature is not trusted. Perhaps we ought to only accept trusted signatures?

01:59:05
@cedricgc:matrix.orgcedricgc joined the room.03:38:04
@raboof:matrix.orgraboof nix-store -q --graphml (of nix-store -q --tree) on a derivation shows what derivations it depends on, but not which specific outputs. That seems fairly easy to add (famous last words), should I? Or is there a better way to get this information anyway? 13:10:27
@raboof:matrix.orgraboof * nix-store -q --graphml (or nix-store -q --tree) on a derivation shows what derivations it depends on, but not which specific outputs. That seems fairly easy to add (famous last words), should I? Or is there a better way to get this information anyway? 13:10:46
@azahi:azahi.cc@azahi:azahi.cc left the room.15:46:49
20 Oct 2025
@fzakaria:one.ems.hostfzakaria is that --outputs or something 03:03:28
@fzakaria:one.ems.hostfzakariai can never remember all the query flags03:03:33
@flacks:matrix.orgJean changed their profile picture.04:36:54
@mschwaig:matrix.orgMartin Schwaighofer
In reply to @jfly:matrix.org
My question is stronger than "can we?". I'm saying "mustn't we?"

I think if you're not looking at the paths or derivations the semantics of that would be the same.

Maybe it would be a conceptual simplification.

17:26:05
@mschwaig:matrix.orgMartin SchwaighoferI really like the idea of having a more explicit resolution step, that all derivations go through, which could model this and the distinction between content- and input-addressed derivations. To me, this is is the way to make this stuff easier to understand in Nix. We would teach users about two dependency trees, an unresolved one, and a resolved one, and the resolution step to get from one to the other. The unresolved one is based on recursive hashes of inputs, like input-addressed paths, the resolved one is based on content hashes of inputs, like content-addressed paths. You can model both input-addressed derivations and content-addressed derivations uniformly this way, it's just that how you address things on disk is different, and which versions of the derivations have more abstract or explicit placeholders for which data. The unresolved tree would contain the unresolved input hash of the FOD, the resolved tree would contain the hard-coded content hash. Expanding on that view, and in practice, you would have to do resolution in two steps. First resolve all of the FODs to their hard-coded content hashes and base IA paths on that (equivalent to the hash-modulo thing), and then interactively resolve and build all of other derivations between the closest satisfied FODs and the build target. You could do ``` nix derivation show --{unresolved,fod-resolved,resolved} nixpkgs#hello ``` to understand and teach this. CA derivations have resolved versions already, but it's not surfaced that nicely to users yet. You can also drop input-addressing, or not change it, but still expose and teach it that way.17:28:10
@mschwaig:matrix.orgMartin SchwaighoferI think the hash modulo thing is meant to prevent the path of C changing as well.17:36:46
@mschwaig:matrix.orgMartin Schwaighofer* I really like the idea of having a more explicit resolution step, that all derivations go through, which could model this and the distinction between content- and input-addressed derivations. To me, this is is the way to make this stuff easier to understand in Nix. We would teach users about two dependency trees, an unresolved one, and a resolved one, and the resolution step to get from one to the other. The unresolved one is based on recursive hashes of inputs, like input-addressed paths, the resolved one is based on content hashes of inputs, like content-addressed paths. You can model both input-addressed derivations and content-addressed derivations uniformly this way, it's just that how you address things on disk is different, which versions of the derivations have more abstract or explicit placeholders for which data, and how strict they are with resolution. The unresolved tree would contain the unresolved input hash of the FOD, the resolved tree would contain the hard-coded content hash. Expanding on that view, and in practice, you would have to do resolution in two steps. First resolve all of the FODs to their hard-coded content hashes and base IA paths on that (equivalent to the hash-modulo thing), and then interactively resolve and build all of other derivations between the closest satisfied FODs and the build target. You could do ``` nix derivation show --{unresolved,fod-resolved,resolved} nixpkgs#hello ``` to understand and teach this. CA derivations have resolved versions already, but it's not surfaced that nicely to users yet. You can also drop input-addressing, or not change it, but still expose and teach it that way.17:40:30
@lovesegfault:matrix.orglovesegfaultAlright, there's a lot of room for bike shedding, but the RequiredSignatures in nix-cache-info idea is up: https://github.com/NixOS/nix/pull/1431322:32:52
21 Oct 2025
@fzakaria:one.ems.hostfzakaria

Sergei Zimmerman (xokdvium): that makes sense
but isn't this 2 step exactly what has to happen now anyways with FODs?

The confusing part is that the inputDrvs for non-FODs are not replaced with their output path but FODs are when caculating the out paths.

The drv's path though is just the hash of the ATerm ?This has the side-effect of causing brand new drvs for the whole system tree since drvs are just the hash of their ATerm without substitution?

00:02:43
@fzakaria:one.ems.hostfzakariaI was going to explore the Katai struct. Is nars.sh and nar-access.sh the only NAR tests?00:04:46
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)
In reply to @fzakaria:one.ems.host
I was going to explore the Katai struct. Is nars.sh and nar-access.sh the only NAR tests?
Some invalid cases in libutil-tests/archive.cc also
00:05:36
@fzakaria:one.ems.hostfzakaria should nix nar ls work on NAR that only have a single file ? 04:28:31
@fzakaria:one.ems.hostfzakariaI started the kaitai spec here: https://github.com/fzakaria/nix-nar-kaitai-spec/tree/main tested directories and regular files so far. There is a test program that is mostly AI generated i need to fix05:08:27
@fzakaria:one.ems.hostfzakariahttps://github.com/fzakaria/nix-nar-kaitai-spec/blob/main/NAR.ksy05:08:55
@raboof:matrix.orgraboof no, --outputs will list all outputs of the derivation. --tree and --graph(ml) are basically recursive versions of --references which shows the inputs (but only shows 'derivation X depends on derivation Y', not 'derivation X depends on output Z of derivation Y') 11:13:45
@juliankuners:matrix.orgJulian

Hey there,

I am currently investigating nix assertion errors that consistently appear once in a while in our CI. Only MacOS machines are affected by this and the error persists until the nix store is garbage collected. It also only persists on the same user on the machine.

[...]
using cache entry 'fetchToStore:{"fingerprint":"3f0bf132df94785670cb324fbb759e85ff5ad0b6","method":"nar","name":"source","path":"/","store":"/nix/store"}' -> 'null', '/nix/store/72alw9m62226brv4v4m98fqrk31mlp34-source'
store path cache hit for '«github:runtimeverification/k/3f0bf132df94785670cb324fbb759e85ff5ad0b6?narHash=sha256-SYrGoV4PuOWux7d4nYhluWD515%2B9sh0DjAMcEDbREVM%3D»/nix/store/8ck644nfh5mb5dqkzgvqhfbznsk1n4df-source'
custom debug: storePath='72alw9m62226brv4v4m98fqrk31mlp34-source' originalInput.computeStorePath(*state.store)='8ck644nfh5mb5dqkzgvqhfbznsk1n4df-source'
Assertion failed: (!originalInput.getNarHash() || storePath == originalInput.computeStorePath(*state.store)), function copyInputToStore, file ../flake.cc, line 40.

This is the assertion that is violated though the check has since been moved in master. We currently run nix version 2.31.2.

So essentially the two nix store paths differ due to existing and non-existing submodule directories, without their content and also somewhat their parent directory, see:

% diff -qr expected instead | sort
Only in expected/haskell-backend/src/main: native
Only in expected/llvm-backend/src/main: native
Only in expected/web: k-web-theme

Does anybody see what could be wrong here or where to look for bugs in nix? I will probably continue to debug this, but there's a lot going on in nix that I am not aware of yet.

13:29:59
@xokdvium:matrix.orgSergei Zimmerman (xokdvium) Generally flake inputs with submodules should not be cached for this reason (the fetchToStore cache entry from the fingerprint -> store path). Maybe that’s the broken part? 13:53:43
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)does it end up in the cache with submodules enabled and disabled?13:56:32
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)For the same git revision13:56:51

Show newer messages


Back to Room ListRoom Version: 6