!RbXGJhHMsnQcNIDFWN:nixos.org

Haskell in Nixpkgs/NixOS

726 Members
For discussions and questions about Haskell with Nix, cabal2nix and haskellPackages in nixpkgs | Current Docs: https://haskell4nix.readthedocs.io/146 Servers

Load older messages


SenderMessageTime
26 May 2021
@sternenseemann:systemli.orgsterni (he/him)like the channel will be unable to advance until we have rebuild enough of the haskell packages set to build cachix13:03:24
@sternenseemann:systemli.orgsterni (he/him)if it takes a day to rebuild we can just do it on the branch and have the cache fully populated already when we merge13:03:46
@maralorn:maralorn.demaralorn
In reply to @sternenseemann:systemli.org
It's not just code complexity it's also semantic complexity you have to kinda be aware of when contributing
We actually already have the case that broken flags are kinda opaque to drive by contributers. Sometimes packages are marked broken without an easily understandable reason and cdepillabout pings me to figure out why. (this is more of a statement of fact. I am not arguing in any specific direction.)
13:05:26
@maralorn:maralorn.demaralorn
In reply to @sternenseemann:systemli.org
like the channel will be unable to advance until we have rebuild enough of the haskell packages set to build cachix
I understand your point, but this has nothing to do with cachix. The channel will only advance when all jobs in the evaluation are finished. This includes all haskellPackages. Or are we talking about unstable-small and cachix is included in that? I assume it isn't.
13:07:18
@maralorn:maralorn.demaralorn
In reply to @sternenseemann:systemli.org
if it takes a day to rebuild we can just do it on the branch and have the cache fully populated already when we merge
I am torn on this point. The fact is if our merge would mean a 500+ rebuild we would have to target staging. (and haskell builds aren't the fastest.) So we should basically adhere to that. Otoh the longer we push off a merge the higher is the risk that hydra does redundant additional work.
13:10:15
@sternenseemann:systemli.orgsterni (he/him)the channel will advance when all constituents of a certain aggregate job are built not when all of nixpkgs is built13:11:13
@sternenseemann:systemli.orgsterni (he/him)and cachix is part of the nixpkgs:unstable job13:11:23
@sternenseemann:systemli.orgsterni (he/him)https://hydra.nixos.org/job/nixpkgs/trunk/unstable13:11:59
@maralorn:maralorn.demaralorn
In reply to @sternenseemann:systemli.org
and cachix is part of the nixpkgs:unstable job
Are you telling me that haskellPackages is not part of the nixos unstable job?
13:12:11
@maralorn:maralorn.demaralornWhat we actually need is ofBorg or some other tooling to tell us if the new derivations resulting from a merge are already cached in hydra. That should be totally doable by pulling the nar files.13:12:43
@sternenseemann:systemli.orgsterni (he/him)nixos:trunk-combined:tested tests a bunch of VM tests which are much more elaborate, so unstable is usually behind nixpkgs-unstable13:13:08
@sternenseemann:systemli.orgsterni (he/him)it probably depends on haskell in some way as well13:13:14
@sternenseemann:systemli.orgsterni (he/him) haskellPackages is part of the jobsets, but not directly of the aggregate job cnotrolling channel advancements 13:13:39
@sternenseemann:systemli.orgsterni (he/him)
In reply to @maralorn:maralorn.de
What we actually need is ofBorg or some other tooling to tell us if the new derivations resulting from a merge are already cached in hydra. That should be totally doable by pulling the nar files.
you could just check the drv outpaths before and after test merging — if they differ we have no cache
13:14:29
@maralorn:maralorn.demaralornYeah.true.13:14:56
@sternenseemann:systemli.orgsterni (he/him)I don't see why we shouldn't merge master into haskell-updates prior to a merge: It causes the same outcome as if we merged the branch into master, but we migitate the risks by doing the rebuilds without slowing the channel down and we even can catch unlikely regressions before the ruin someone's day13:15:57
@mjrosenb:matrix.orgMarty Rosenberghas anyone tried to use miso on a modern macos? the pinned nixpkgs don't seem happy.13:16:20
@sternenseemann:systemli.orgsterni (he/him) maralorn: cdepillabout: there has been a staging-next merge since the last time we merged master including changes to LLVM for example, so there will be rebuilds 13:17:08
@maralorn:maralorn.demaralorn
In reply to @sternenseemann:systemli.org
I don't see why we shouldn't merge master into haskell-updates prior to a merge: It causes the same outcome as if we merged the branch into master, but we migitate the risks by doing the rebuilds without slowing the channel down and we even can catch unlikely regressions before the ruin someone's day

Well my argument against it are 1. it will slow us down which is might be annoying for our process and for the person doing the merge who wants to get it done and 2. this could actually mean more unnecessary rebuilds. Imagine we have a change to a core package like random which leads to mass rebuilds and a small change to master which leads to us postponing the merge and then another change hits master leading to rebuild all packages again with the old "random" and then we need to rebuilt everything with the new random before we can merge. (I kinda dread the loop of merging in master, rebuilding, merging in master, rebuilding and never getting to actually merge. That’s theoretically possible.)

But those are only arguments why it isn‘t completely free to merge in master. When our merges causes a lot of rebuilds, yes, then we need to obey the same rules as the rest of nixpkgs and build them before merging into master.

And now that I think about it: Merging in master, retriggering an eval, seeing how many builds will be rebuild and merging when it’s less then 100 or something seems to be a matter of 10 minutes and is probably totally sane.

13:25:50
@domenkozar:matrix.orgDomen Kožar+1 for treating haskell-updates similar to staging-next, mass rebuilds so make sure there are big regressions and everything builds13:26:21
@domenkozar:matrix.orgDomen Kožarit does add overhead, but it's no worse than finding out the same stuff in master13:27:25
@domenkozar:matrix.orgDomen Kožarit's not so much overhead but rather delay13:28:45
@sternenseemann:systemli.orgsterni (he/him)I think not doing it risks a “merge and forget” practice were interactions between haskell-updates and master we couldn't have been aware us ruin other people's day on master and we may not even notice it13:29:13
@maralorn:maralorn.demaralornAgain, cases where merging master and haskell-updates will lead to a build fail that didn‘t exist on one off the branches before seem super unlikely to me. But yeah, I mean we have this job set exactly to not merge build errors into master and we have actually improved on our quality assurance their in the last weeks.13:29:51
@sternenseemann:systemli.orgsterni (he/him)either way we would need to fix it right away in order to get our new haskell-updates branch working again13:30:03
@maralorn:maralorn.demaralorn * Again, cases where merging master and haskell-updates will lead to a build fail that didn‘t exist on one of the branches before seem super unlikely to me. But yeah, I mean we have this job set exactly to not merge build errors into master and we have actually improved on our quality assurance their in the last weeks.13:30:08
@sternenseemann:systemli.orgsterni (he/him)the difference is largely for how long master would be affected by whatever trouble we'd run into13:30:21
@domenkozar:matrix.orgDomen KožarI'd just kindly ask to avoid what Peti did to Cachix, when the build broke, it was removed from channel blockers13:31:29
@domenkozar:matrix.orgDomen Kožarif I have a heads up I can help out resolve issues with packages I maintain13:31:47
@sternenseemann:systemli.orgsterni (he/him) Domen Kožar: on a related note we should probably add you to the maintainer list for your haskell packages 13:32:54

Show newer messages


Back to Room ListRoom Version: 6