| 15 Mar 2025 |
 raitobezarius | https://varlink.org/ | 13:37:33 |
 emily | is varlink any better than JSON for byte canonicity? | 14:51:30 |
| emily | or is that no longer a requirement? | 14:52:16 |
| emily | ah, I guess this is separate :) | 14:52:25 |
| emily | fwiw, (c) seems like the most elegant solution to me. | 14:55:05 |
| emily | though it's a bit weird to be able to chain it like that. | 14:55:32 |
 roberth | Something does seem elegant about (c), putting this capability in its own separate derivation, but it isn't clear to me which is a better UX and whether there's differences in performance between the solutions, especially wrt binary cache roundtrips. I feel like (d) is closer to the "deep realisation" idea, but I don't know whether that's inconsequential, better or worse. | 15:29:24 |
| roberth | I think byte canonicity might not be a problem in this case, as long as clients in the builder don't try to do their own hashing, which they really don't need to when the endpoint returns the relevant hashes anyway | 15:30:37 |
| roberth | Only the Nix implementations need to get this right, which seems feasible | 15:31:25 |
| emily | how does outputOf currently get represented in the derivation? | 15:34:39 |
| roberth | just in the inputs. We use a semistructured ATerm for the derivation references field, and iirc store-path-shaped placeholders in the other attributes | 15:38:54 |
| emily | if the separate derivation is considered a problem, perhaps "join the output" could be a field on the .drv itself? | 15:40:02 |
| emily | you would still need a cp derivation for the arbitrary case to reconstruct (c) though | 15:40:12 |
| roberth | Not a fan of the Go types schema language, but otherwise it looks alright. Could be combined with JSON schema perhaps, via varlink's catch-all object | 15:42:50 |
| roberth | That's sort of what I had in mind for (d), declare statically with an "advanced attr" that an output is a redirect (or may be a rewrite) and then the builder would write to $out or $outRewriteor something.echo $drv^out >$outRewrite` | 15:50:30 |
| roberth | * That's sort of what I had in mind for (d), declare statically with an "advanced attr" that an output is a redirect (or may be a rewrite) and then the builder would write to $out or $outRewriteor something.echo $drv^out >$outRewrite\ | 15:50:41 |
| roberth | * That's sort of what I had in mind for (d), declare statically with an "advanced attr" that an output is a redirect (or may be a rewrite) and then the builder would write to $out or $outRewriteor something.echo $drv^out >$outRewrite | 15:50:49 |
| emily |
especially wrt binary cache roundtrips
the output of a join drv would be the actual final output, right? so if the output is cached no additional roundtrips? I may be misunderstanding though, is this when a cache has .drvs but no outputs? is that common?
| 16:20:06 |
| emily | my feeling is that having a zillion tiny derivations should be made to be fast/good anyway because the more fancy incremental stuff you do the more you'll want that anyway | 16:20:50 |
| emily | or do you mean when pushing outputs? | 16:29:41 |
| 16 Mar 2025 |
 jade_ | i currently do not have the brain space to fully think out how the dyndrv ideas work, but i will say that, as puck mentioned in lix dev a while ago, it could be the case in the future that we allow for input addressed drvs with different drv hashes to resolve to the same output hash for stuff like provenance metadata and other use cases like that where it does not affect the build itself, in the same way as fods currently allow.
so having to generate the drvs from inside a dyndrv could be a trouble for the extension of the drv format? | 01:48:45 |
| roberth | you're considering this? https://github.com/NixOS/nix/issues/10780
| 10:29:28 |
| roberth | it's already the case that drv hashes aren't 1:1 with input addressed output hashes, due to modulo hashing and FODs that occur in the derivation closure | 10:29:37 |
 puck | basically, yeah. i think .drv files (and, to some degree, their format) are distinct from "derivations" in a way that makes sense to rethink what they are and look like, because .drv files' hashes aren't stable to begin with | 10:31:42 |
| roberth | I don't think generating drvs inside there has to be a problem, as presumably you could canonicalize them on the nix side? | 10:31:44 |
| roberth | yeah, the issue tries to be more incremental in terms of change, but yeah, we basically could have a pipeline like exprs -eval-> instantiations of any suitable format -canonicalize-> derivations -build-> outputs | 10:34:34 |
| roberth | or variations of that | 10:35:04 |
|  Martin Charles joined the room. | 17:58:53 |
 John Ericson | hexa: We would like to no more about the bug that caused the builders to go to Lix | 18:59:34 |
| John Ericson | that earlier github thread went of the rails, but the original idea of having some more information written down still stands | 19:00:02 |
