| 6 Oct 2021 |
 Sandro | In reply to @vcunat:matrix.org
The IP belongs to Hetzner, according to whois. The sources of this data are not always up to date and in the past I already got really bogus results. Like 3 results from 3 services about location, owner and so on | 21:38:34 |
 Vladimír Čunát | ... though theoretically it is possible to be some Fastly DNS problem. | 21:38:40 |
 vidister / fiona | I'm not running systemd-resolved and the machine is using 1.1.1.1/8.8.8.8. I don't know where that result came from | 21:39:11 |
| vidister / fiona | this is so weird | 21:39:14 |
| vidister / fiona | well, thanks for helping, I'm super confused and I'll just flush my dns cache and stop investigating here.. | 21:40:44 |
| Vladimír Čunát | Perhaps, if it seems like a one-time issue, but it's weird. | 21:42:27 |
| 7 Oct 2021 |
 Jonas Chevalier | there is always the cosmic ray that can flip a bit :) | 08:12:02 |
| Jonas Chevalier | I'm investigating channel updates. nixos-unstable stopped pushing. | 08:12:52 |
| Jonas Chevalier | * I'm investigating channel updates. nixos-unstable stopped pushing. (investigation: https://github.com/NixOS/nixos-org-configurations/issues/180) | 08:17:03 |
| Jonas Chevalier | did anybody cancel builds recently? It looks like it was just caused by a bunch of cancelled builds. | 08:44:20 |
| Vladimír Čunát | This case was probably me. | 08:53:57 |
| Vladimír Čunát | If some jobs are required for channel updates to succeed, shouldn't we make the critical job (tested here) depend on all those jobs? | 08:54:39 |
| Vladimír Čunát | * This case was probably me. | 08:55:23 |
| Vladimír Čunát | Well, I wonder where to reply... seeing the question on one issue and two chat channels. | 08:56:36 |
| Vladimír Čunát | ☝️ Jonas Chevalier | 08:57:23 |
| Jonas Chevalier | that's fine, I'm just trying to better understand what happened | 08:58:38 |
| Jonas Chevalier | I'm not sure super familiar with Hydra, and the channel-update logic | 08:59:37 |
| Jonas Chevalier | Vladimír Čunát: what are typical conditions where builds get canceled? | 09:00:31 |
| Vladimír Čunát | I do it in a few cases | 09:01:28 |
| Vladimír Čunát |
- replaced builds: if we have newer builds of the same jobs (scheduled) and keeping the older ones doesn't seem very useful anymore. (makes sense when they're many, especially if combined with high load on Hydra)
| 09:02:52 |
| Vladimír Čunát |
- channel waiting on bad test. Sometimes the channel would update, but it waits to finish all builds. And sometimes there's a few tests that never succeeded recently and just wait for a long time-out. Cancelling those stragglers can speed up channel update. (makes sense if the channel is quite old currently, e.g. after some period of channel blockers)
| 09:04:34 |
| Jonas Chevalier | Is anybody else keeping a pulse on Hydra like you do? | 09:11:43 |
| Vladimír Čunát | I'm not sure. I do watch https://status.nixos.org/ (or a script of mine with differently defined timestamps, based on committer time) If some channel is getting over three days, I'm looking at what's wrong; typically it's something easy to solve or speed up. | 09:14:00 |
| Vladimír Čunát | I mean, there certainly are other cases of people "unblocking channels", based on what happens, but I have no real insight into that. | 09:22:08 |
| Jonas Chevalier | thanks | 09:22:58 |
| Jonas Chevalier | it would be useful if those user interventions were logged somehow, so that we could know who is doing what | 09:24:29 |
| Vladimír Čunát | It's not the first time there was this idea: https://github.com/NixOS/hydra/issues/786 | 09:27:23 |
| 8 Oct 2021 |
| Jonas Chevalier | I added some templates to the repo and looking for feedback: https://github.com/NixOS/nixos-org-configurations/issues/new/choose
The goal is to make it easier for users to report issues with the infrastructure, and in the future request access or new resources to be deployed. | 07:33:59 |
| 10 Oct 2021 |
 hexa | There seems to be a caching issue with channels.nixos.org https://github.com/NixOS/nixos-org-configurations/issues/169#issuecomment-939478669 | 14:17:48 |
 sterni | this has been like this for over a week (two?) | 14:18:58 |
