| 19 Jul 2025 |
 ma27 | In reply to @connorbaker:matrix.org
I know that one of the expensive parts of CI with Nix monorepos is evaluation on each commit in part because there's no re-use of evaluation caches across commits.
I see that a new AttrDb is created for each fingerprint (https://github.com/NixOS/nix/blob/b8d223a2106c7dbdb0c61f822288195ee9e26e9b/src/libexpr/eval-cache.cc#L75).
For those with knowledge about evaluation caching as it is currently implemented: would it be feasible to rewrite it such that entries in Attributes use the hash of their source files as part of their primary key?
As an example, if the hashes of the files read to evaluate an attribute doesn't change across two commits, the result from the first evaluation would be returned for the second (it would be cached). I think part of the problem is that you can't really say which files are the sourcefiles of an attribute without an evaluation, right? | 17:41:11 |
| 20 Jul 2025 |
 connor (he/him) (UTC-7) | I think you could if using the flake interface. I've lookged through the eval-cache.cc file a bit, here's an idea.
Consider the portion which involves getting the keys of an attribute set. If before and after forcing a value you were snapshot and then compare the results of the import cache, that could be a start. | 03:48:39 |
 Robert Hensing (roberth) | I plan to work on that starting Sep/Oct. Might work regardless of flakes :: Bool | 15:16:39 |
 magic_rb | In reply to @ma27:nicht-so.sexy
I think part of the problem is that you can't really say which files are the sourcefiles of an attribute without an evaluation, right? Evaluation should form a tree of visited files theoretically. So you could go and rehash all the files and once you find the first divergence you need to reeval (if we could take incremental snapshots of the evaluator state and flatten the visited files into a list on the time axis, you could even pick up the eval maybe from where it left off? Essentially if you change only a leaf file, you only reimport the leaf) | 15:31:04 |
 Sergei Zimmerman (xokdvium) | It's about time the compile times got better. Yeeted https://github.com/NixOS/nix/pull/13510 and https://github.com/NixOS/nix/pull/13512. | 19:25:10 |
| Sergei Zimmerman (xokdvium) | This smells much like Boost.Spirit or Boost.X3 | 20:41:54 |
| 21 Jul 2025 |
| connor (he/him) (UTC-7) | If you’re going to NixCamp or NixCon this year, I’d love to find out more about your thoughts on how that could be implemented! | 02:34:48 |
| Robert Hensing (roberth) | I'll be at NixCon. Basic idea is: refactor evaluation so that it's an interaction between these three actors and "communication links": CLI - Evaluator - OS, and nothing else. Then MITM the Evaluator on both sides by recording the interactions of previous calls to the evaluator, and replaying them if possible. | 09:05:43 |
| Robert Hensing (roberth) | It's somewhat of a research project; nothing is guaranteed | 09:07:02 |
| magic_rb | Thats kinda what i was proposing https://matrix.to/#/!VRULIdgoKmKPzJZzjj:nixos.org/$FvXA39aRGz0iER7ZM-bRtUkJCuorYa0Ecy_YupgUPC8?via=nixos.org&via=matrix.org&via=nixos.dev ill be at nixcon too. Definitely will be around for this | 09:08:24 |
|  andiandi 🐈 changed their display name from andiandi@hadr25 🏰🏞️ to andiandi 🐈. | 11:04:23 |
| connor (he/him) (UTC-7) | I’d love to talk more about this with you all!
Any chance C++ has something akin to Haskell’s STM? | 14:11:52 |
| connor (he/him) (UTC-7) | tomberek you might be interested in this — I know I talked with you about having an import cache which stores the processed AST instead of the file itself. (One idea I had was to key on the hash of the AST so formatting wouldn’t cause cache misses.) | 14:44:35 |
 toonn | .oO(Unison...) | 14:45:29 |
 emily | see https://matrix.to/#/!VRULIdgoKmKPzJZzjj:nixos.org/$ynuYm1cT2gUMemb4CiyueY7KlkO0yqppb8i9xb6ENJ8?via=nixos.org&via=matrix.org&via=nixos.dev for my attempts | 14:48:07 |
| emily | unfortunately to trace builtins.readFile you need scopedImport which slows things down a bunch | 14:48:23 |
| magic_rb | In reply to @emilazy:matrix.org
unfortunately to trace builtins.readFile you need scopedImport which slows things down a bunch Well, it would have to be done at the c++ level and the evaluator would have to become serializable for my idea to work | 14:54:00 |
| emily | yes, that would be better | 14:54:20 |
| emily | serializing eval state will be really hard and complicated. I'd recommend focusing on just smarter cache keys | 14:54:42 |
| emily | roberth | 14:54:53 |
| emily | oops | 14:54:56 |
| emily | roberth's proposal is basically what strace/preload/FUSE/etc. build systems do | 14:55:09 |
| emily | storing a log of "syscalls" and using that to cache builds | 14:55:33 |
| emily | but more major refactor than just "log the obvious places files are accessed and use those as part of a cache key". but the latter is fatal if you miss anything anywhere, because we end up back in make clean territory | 14:56:05 |
| emily | (that's why I was trying it for the direnv layer, where we already accept imperfection) | 14:58:49 |
 pveierland | Using nix repl 2.28.3 the following evaluates: { "foo" = 42; } however the following does not: { ''foo'' = 42; } and { https://foo = 42; } | 21:44:41 |
| pveierland | Are string literals used in names for attribute sets limited to the double quoted form? | 21:45:48 |
| pveierland | Based on the documentation it appears that any string should parse: https://nix.dev/manual/nix/2.30/language/identifiers.html?highlight=name#names | 21:46:38 |
| Sergei Zimmerman (xokdvium) | Attribute keys in the grammar can only be identifiers, strings or dynamic attributes (interpolation inside dollar curlies). https://github.com/NixOS/nix/blob/6ec50ba73664838993d645dc78c936542eb2012c/src/libexpr/parser.y#L485-L494 | 22:29:45 |
| Sergei Zimmerman (xokdvium) | So yeah, string literals are limited to double quotes. | 22:30:48 |
