!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

271 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.86 Servers

Load older messages


SenderMessageTime
11 Oct 2024
@emilazy:matrix.orgemily
In reply to @vcunat:matrix.org
We might consider splitting it up. At least the queue-runner away from the public web (hydra.nixos.org). I don't feel too comfortable having the signing keys on same machine as a public web. I think these only communicate through the DB which is on a separate machine already.
maybe the website could continue being on the x86 server?
07:34:24
@emilazy:matrix.orgemilyor, right, the idea was to upgrade the x8607:34:39
@emilazy:matrix.orgemilythen maybe the site could move to one of the AArch64 boxes? :)07:34:48
@emilazy:matrix.orgemilybtw, maybe should upgrade Hydra's Nix to 2.24 if it's moving to a new server anyway. since Hydra supports it now and it would solve that Nix bug that didn't get a fix backported because of only 2.18 and 2.24 being supported versions.07:51:44
@joerg:thalheim.ioMic92
In reply to @emilazy:matrix.org
btw, maybe should upgrade Hydra's Nix to 2.24 if it's moving to a new server anyway. since Hydra supports it now and it would solve that Nix bug that didn't get a fix backported because of only 2.18 and 2.24 being supported versions.
Yes. that was the plan. We recently fixed 2.24 support in hydra.
07:59:38
@joerg:thalheim.ioMic92
In reply to @fricklerhandwerk:matrix.org

http://sectracker.nixpkgs.lahfa.xyz/

Config here: https://github.com/Nix-Security-WG/nix-security-tracker/tree/main/staging

ok. Seems to be down just now.
07:59:56
@emilazy:matrix.orgemily(btw I never realized that the cache signing key is on the machine that hosts the hydra.nixos.org site)08:00:49
@emilazy:matrix.orgemily(that terrifies me)08:00:52
@emilazy:matrix.orgemily(strong support for any plan that fixes that)08:01:35
@fricklerhandwerk:matrix.orgfricklerhandwerk
In reply to @joerg:thalheim.io
ok. Seems to be down just now.
Yeah we're debugging it, there's some weird Django issues where some stupid script hangs itself to death
08:04:49
@dgrig:erethon.comdgrig
In reply to @joerg:thalheim.io
@fricklerhandwerk:matrix.org: where is this security tracker deployed just now? I think it would help if Erethon would have a chat with us, so we can make sure it get integrated with the rest of the infra to some extent.
I'm catching up with the channel now, for some reason Element didn't ping me about this mention (maybe because it was capitalized). Just want to point out that what fricklerhandwerk linked too is also Raito's work, don't want to take credit for it. At this point I'm not sure what's the best way to move forward, since it seems that Raito has already done a lot of the work needed to deploy this properly?
08:05:26
@fricklerhandwerk:matrix.orgfricklerhandwerk Mic92: back online 08:06:32
@joerg:thalheim.ioMic92What is your expected update cadence?08:06:41
@joerg:thalheim.ioMic92 dgrig: what is blocking you specifically? 08:07:55
@dgrig:erethon.comdgrig
In reply to @joerg:thalheim.io
dgrig: what is blocking you specifically?
I don't have a "blocker" per se from the nixos infra team. I've been experimenting locally with the security tracker and some other software that fricklerhandwerk wants deployed in an official namespace and manner. On the security tracker front I have some thing to figure out still, but for others (say Odoo if it's ok for us in the end) I want to sync with someone at some point on how we best want it deployed (i.e. does it belong on the non-critical infra, how do we want to backup the database, etc).
08:21:11
@joerg:thalheim.ioMic92Sure. Do Thursday, 18:00 CEST the next week work for you?08:23:58
@joerg:thalheim.ioMic92 * Sure. Does Thursday, 18:00 CEST the next week work for you?08:24:09
@dgrig:erethon.comdgrig
In reply to @joerg:thalheim.io
Sure. Does Thursday, 18:00 CEST the next week work for you?
Yes, I've blocked all the nixos infra meetings in my calendar so I can attend them.
08:24:39
@fricklerhandwerk:matrix.orgfricklerhandwerk
In reply to @joerg:thalheim.io
What is your expected update cadence?
Hm, good question. That will depend on whether we get follow-up funding and how much, but say something between 1 week and 1 month
08:25:23
@fricklerhandwerk:matrix.orgfricklerhandwerk
In reply to @fricklerhandwerk:matrix.org
Hm, good question. That will depend on whether we get follow-up funding and how much, but say something between 1 week and 1 month
There is already automation in place to do continuous deployment to staging, and we'll re-use that for production.
08:26:14
@joerg:thalheim.ioMic92What is s3 used for?08:29:56
@joerg:thalheim.ioMic92Just seems to be backup as far as I can see08:30:55
@rosscomputerguy:matrix.orgTristan Ross
In reply to @emilazy:matrix.org
wouldn't that run into the atomics and platform purity problems wrt the evaluator?
@tomberek:matrix.org: and I were discussing this a bit last night and we're not entirely sure atomics is an actual problem. How does it affect Hydra? Wouldn't this be an issue with a C++ compiler. Hydra appears to run fine from what I've heard when running on aarch64-linux. The purity thing though, as long as the system is passed through and things are expected right, shouldn't be a concern?
13:40:41
@vcunat:matrix.orgvcunat Tristan Ross: I believe the point was around x86 being strict around reordering of instructions while ARM is not. On language level you then need to be careful around https://en.cppreference.com/w/cpp/atomic/memory_order 14:02:11
@rosscomputerguy:matrix.orgTristan Ross
In reply to @vcunat:matrix.org
Tristan Ross: I believe the point was around x86 being strict around reordering of instructions while ARM is not. On language level you then need to be careful around https://en.cppreference.com/w/cpp/atomic/memory_order
Wouldn't this affect the nix cli itself and literally everything?
14:08:29
@k900:0upti.meK900No14:08:42
@k900:0upti.meK900hydra-queue-runner is like 3k lines of C++14:09:00
@k900:0upti.meK900On top of the normal Nix things14:09:06
@k900:0upti.meK900It's those bits I'm worried about, not Nix14:09:15
@rosscomputerguy:matrix.orgTristan RossOh, is there a way to test the queue runner in a way to trigger it breaking because of this on aarch64?14:10:29

Show newer messages


Back to Room ListRoom Version: 6