!RbXGJhHMsnQcNIDFWN:nixos.org

Nix Haskell

622 Members
For discussions and questions about Haskell with Nix, cabal2nix and haskellPackages in nixpkgs | Current Docs: https://haskell4nix.readthedocs.io/127 Servers

Load older messages


SenderMessageTime
6 Oct 2024
@maralorn:maralorn.demaralorn
In reply to @emilazy:matrix.org
another con is that it's unintuitive for contributors
Only for ones which already understand the staging workflow.
12:26:33
@eldritchcookie:matrix.orgeldritchcookieoh makes sense guess i just got salty that my fix took 2 weeks but i never had a contribution that wasn't merged on master rather than staging so 12:26:42
@emilazy:matrix.orgemilyand it's harder to coordinate Haskell changes with stuff in the usual master/staging workflow12:26:49
@maralorn:maralorn.demaralorn
In reply to @emilazy:matrix.org
and it's harder to coordinate Haskell changes with stuff in the usual master/staging workflow
Yeah, that is true.
12:27:10
@emilazy:matrix.orgemily
In reply to @maralorn:maralorn.de
Only for ones which already understand the staging workflow.
well, it's unintuitive for people who are used to sending things to master too :)
12:27:10
@emilazy:matrix.orgemilywhich AIUI is not good for Haskell even for changes that don't cause a lot of rebuilds?12:27:31
@maralorn:maralorn.demaralorn
In reply to @emilazy:matrix.org
which AIUI is not good for Haskell even for changes that don't cause a lot of rebuilds?
Well, we don’t generally forbid it, but it really helps our QA to only merge into haskell-updates, because then we always have the hydra check. Sometimes packages unexpectedly break a dependency. And ofc there is also merge conflicts …
12:29:17
@maralorn:maralorn.demaralorn
In reply to @emilazy:matrix.org
I've definitely avoided touching Haskell stuff in the past because the procedures are different and unfamiliar and I've seen multiple others express the same sentiment
That’s a bit sad. I promise it is totally fine to just open a PR and we will be very helpful in guiding people in the right direction. And that quickly, easy PRs often get dealt with within a day because we have relatively clear responsibilities.
12:31:07
@maralorn:maralorn.demaralorn
In reply to @emilazy:matrix.org
I've definitely avoided touching Haskell stuff in the past because the procedures are different and unfamiliar and I've seen multiple others express the same sentiment
* That’s a bit sad. I promise it is totally fine to just open a PR and we will be very helpful in guiding people in the right direction, which is really not that complicated. And that quickly, easy PRs often get dealt with within a day because we have relatively clear responsibilities.
12:31:22
@me:linj.techlinj
In reply to @maralorn:maralorn.de

linj:
Cons:

  • It costs considerable extra build time, which is a scarce resource, and honestly sometimes I think people are eyeing us whether that’s really necessary. It’s also a problem for us, because sometimes the hydra scheduler just lets us hanging dry for days.
  • Whenever staging-next lands in master it gets merged into haskell-updates which basically always triggers a full rebuild of our package set effectively staling is for a few days.
  • We need to be careful not to merge in those moments, because more generally it is our responsibility to only merge into master when the cache is already populated.

Pros:

  • We have our own decoupled timing. We open a new branch, bump everything at the start, then we fix stuff and can merge at the end. We are heavily contributor time constraint and being able to set our own pace is super helpful.
  • Also of course we can have lower roundtrip times until stuff reaches master.
  • Having our own hydra jobset, which we can e.g. restart or cancel makes it much easier to have a well-defined workflow around our package set. We have e.g. tooling which currently heavily relies on that.
  • I assume that in staging our dependencies would more often wait for rebuild or actually be broken, that are time windows in which we are flying blind and cannot fix stuff that easily.
  • Related to that: If we added all our ci jobs to the staging test set, they would probably see more rebuilds and thus increase CI load the other way around. Hard to say which solution results in more builds, but it certainly wouldn’t speed up the already very work intensive staging cycle.

I have really never thought about this. To be honest I think it would not be impossible to integrate our workflow into the staging workflow, e.g. bumping the hackage snapshot regularly on staging and then stabilizing on staging-next, but it would be quite a lot upfront cost and probably more friction in the day-to-day.

Thanks! Now I think speeding up hydra is more promising to get more frequent emacs package updates. I hope one day those updates can go directly into master. Building them only takes 20 minutes. Hydra bottlenecks on compression, scheduling and maybe other things.
12:32:32
@emilazy:matrix.orgemily
In reply to @maralorn:maralorn.de
Well, we don’t generally forbid it, but it really helps our QA to only merge into haskell-updates, because then we always have the hydra check. Sometimes packages unexpectedly break a dependency. And ofc there is also merge conflicts …
right. so the result is that Haskell is kind of a silo in which the normal monorepo properties ("you can change something and fix all its users at the same time") don't really apply
12:32:35
@emilazy:matrix.orgemilyand you end up with an experience more like a multirepo12:32:45
@emilazy:matrix.orgemilywhere you have to phase deprecations and send multiple coordinated PRs, etc.12:32:55
@emilazy:matrix.orgemilymost changes probably aren't cross-cutting in that way but it's annoying when they are :)12:33:06
@maralorn:maralorn.demaralorn
In reply to @eldritchcookie:matrix.org
oh makes sense guess i just got salty that my fix took 2 weeks but i never had a contribution that wasn't merged on master rather than staging so
I totally get that. To be fair the team of people doing those merges has shrunk by 50% in the last two years. And back then we basically had a pace of one merge per week. We need to get better with onboarding and will put out a call for volunteers shortly.
12:33:09
@emilazy:matrix.orgemily I do wonder if master/staging wouldn't be faster overall. staging would be slower, of course, but you can't get faster than merging directly into master when things don't cause a bunch of rebuilds 12:33:48
@maralorn:maralorn.demaralorn
In reply to @emilazy:matrix.org
and you end up with an experience more like a multirepo
Yeah, those are good points.
12:33:52
@emilazy:matrix.orgemily
In reply to @maralorn:maralorn.de
That’s a bit sad. I promise it is totally fine to just open a PR and we will be very helpful in guiding people in the right direction, which is really not that complicated. And that quickly, easy PRs often get dealt with within a day because we have relatively clear responsibilities.
yeah, it is unfortunate, but I also get why people feel that way compared to the language ecosystems that follow the normal Nixpkgs processes more
12:34:18
@emilazy:matrix.orgemilythough there are of course advantages to batching up updates in a jobset too :)12:34:33
@maralorn:maralorn.demaralorn
In reply to @emilazy:matrix.org
I do wonder if master/staging wouldn't be faster overall. staging would be slower, of course, but you can't get faster than merging directly into master when things don't cause a bunch of rebuilds
Well then semi-frequently people would do fixes on master which already don’t apply anymore on staging …
12:35:30
@emilazy:matrix.orgemilytrue12:36:50
@emilazy:matrix.orgemilystaging people are used to handling merge issues though12:36:56
@maralorn:maralorn.demaralornHonestly I can see the appeal of not special casing haskell-updates.12:36:58
@emilazy:matrix.orgemily (I just ran into an annoying one that I have to deal with today: someone added a package using OpenJDK 22 on master, which is already EOL. but my PR to do JDK updates, remove OpenJDK 22, and add OpenJDK 23 is waiting on staging. so after master merged back into staging, eval broke, and the person who made the PR has no obvious way to test that 23 will work.) 12:37:53
@emilazy:matrix.orgemily(that sort of thing is unavoidable in any kind of branching workflow though)12:38:00
@maralorn:maralorn.demaralornotoh, I think that hexa switched from staging to python-updates for larger python updates, so their are clearly forces pushing in the other direction.12:38:12
@maralorn:maralorn.demaralorn * otoh, I think that hexa switched from staging to python-updates for larger python updates, so there are clearly forces pushing in the other direction.12:38:47
@emilazy:matrix.orgemily my understanding is that python-updates is for mass batch updates, whereas most Python changes still go to master/staging 12:39:00
@emilazy:matrix.orgemilyso it's a bit more nuanced. OTOH Python does those batch updates less often than Haskell because they're a pain, I think12:39:26
@emilazy:matrix.orgemily but maybe haskell-updates could continue to be used for Stackage updates and other things are handled on the main branches? not sure. 12:39:44

Show newer messages


Back to Room ListRoom Version: 6