| 28 May 2021 |
 andi- | It is independent of architectures. Your jobset gets an amount of shares of the total shares. The scheduling looks at the last 24h and prefers jobs that have more unused build time within the 24h weighted by their share ratio. | 12:30:24 |
 sterni (he/him) | it seems to me that it is a bit all or nothing? seemingly some jobsets just win and get all build capacity and another jobset is just stuck for a day | 12:32:02 |
| andi- | In prometheus speech it is something like sum(build_duration[24h]}/share_ratio group by (jobset) the list of jobs will then be sorted so that the jobs with the least usage are preferred. | 12:32:22 |
 maralorn | In reply to @andi:kack.it
It is independent of architectures. Your jobset gets an amount of shares of the total shares. The scheduling looks at the last 24h and prefers jobs that have more unused build time within the 24h weighted by their share ratio. What’s unused build time? Like, what’s the total? | 12:34:49 |
| maralorn | In reply to @andi:kack.it
In prometheus speech it is something like sum(build_duration[24h]}/share_ratio group by (jobset) the list of jobs will then be sorted so that the jobs with the least usage are preferred. Okay the formula makes sense. | 12:35:51 |
| andi- | https://github.com/NixOS/hydra/blob/master/src/hydra-queue-runner/dispatcher.cc#L139-L184 | 12:36:03 |
| maralorn | sterni (he/him): If that formula is true at least theoretically the following could happen: Queues are empty, haskell-updates get’s all the build time. 12h suddenly queues are full, haskell-updates has already used a lot of build time in the last 24h so now we get starved. | 12:37:17 |
| andi- | Exactly. That has happened in the past with normal release jobs as well. | 12:37:41 |
| andi- | I guess you could probably workaround that issue by bumping the "mergable" job to the front of the queue but that would be cheating the "fairness" system :) | 12:38:42 |
| andi- | * I guess you could probably workaround that issue by bumping the "mergeable" job to the front of the queue but that would be cheating the "fairness" system :) | 12:38:51 |
| maralorn | In reply to @andi:kack.it
I guess you could probably workaround that issue by bumping the "mergeable" job to the front of the queue but that would be cheating the "fairness" system :) Well bumping only the aggreggate job wouldn‘t do anything harmful or does it transitively affect dependencies? | 12:39:33 |
| maralorn | Ah, I should have read that comment.^^ | 12:39:56 |
| maralorn | Anyways, I think we three don‘t even have the bump to front right and I actually don‘t think we need it. | 12:40:23 |
| andi- | A situation where it might be useful is where only aarch64-darwin (or whatever arch) jobs are running and most of our builds are idle. At least then we could make use of the other compute power that is idle at that point in time. | 12:52:33 |
| maralorn | Oh, damn. I interpreted your "it’s independent of architectures" completely wrong. | 12:53:37 |
| maralorn | You are saying that we can have darwin builders idle because we had a large rebuild on x86-linux 12h ago? | 12:54:24 |
|  marinelli joined the room. | 19:46:11 |
| marinelli changed their display name from Giorgio to marinelli. | 19:59:52 |
| 29 May 2021 |
 cdepillabout | In reply to @maralorn:maralorn.de
Anyways, I think we three don‘t even have the bump to front right and I actually don‘t think we need it. It would be convenient a situation like the following: We are trying to fix up the mergeable job, and there are only one or two failed child jobs. You believe that the jobs have "incorrectly" failed, so you restart them and bump them to the front. They finish building in a minute or so and you can then go ahead with merging haskell-updates into master. You could basically do it all in one sitting, without having to wait a few hours for the build to start. | 03:12:07 |
| cdepillabout | So basically a situation where you are literally sitting at your computer and waiting to take some action based on the outcome of a job. | 03:13:18 |
 joe (he/him) | cdepillabout: I'll release exact-real with a "fix" tonight\ | 10:23:33 |
| joe (he/him) | * cdepillabout: I'll release exact-real with a "fix" tonight | 10:23:36 |
| joe (he/him) | The fix is changing the arbitrary instance for CReal to generate less problematic numbers https://github.com/expipiplus1/exact-real/pull/38 | 10:24:01 |
| joe (he/him) | * The fix is changing the arbitrary instance for CReal to generate less problematic numbers https://github.com/expipiplus1/exact-real/pull/38 LOL | 10:24:09 |
| joe (he/him) | * The fix is changing the arbitrary instance for CReal to generate less problematic numbers https://github.com/expipiplus1/exact-real/pull/38 lol | 10:24:13 |
| joe (he/him) | (that's not laughing out loud, it's actually me throwing my hands up) | 10:24:33 |
| sterni (he/him) | still haven't accumulated the necessary hydra karma to get darwin builds it seems | 11:31:53 |
| sterni (he/him) | well any minute now :p | 11:31:58 |
|  justinrestivo changed their display name from justinrestivo to oh caml >>=. | 12:20:58 |
| justinrestivo changed their profile picture. | 12:22:01 |
