| 13 Feb 2022 |
 @grahamc:nixos.org | Specifically this migration | 17:16:24 |
 das_j | yess, it works now | 17:19:13 |
| das_j | thanks! | 17:19:14 |
| @grahamc:nixos.org | Great! | 17:26:53 |
| 14 Feb 2022 |
| das_j | I know this solves a solved problem but I wrote a new Hydra exporter in a proper language that can be used for non-official Hydras without patching. Exports all metrics from the queue runner status + reexports the notify metrics | 12:51:24 |
| das_j | Sadly not really compatible with the python exporter because the metrics are now properly namespaced and a bit more consistent :/ | 12:51:46 |
|  @test:boba.best changed their display name from Tseb to Tseb (Old). | 12:52:11 |
 hexa | NIH - not invented at helsinki systems! | 12:53:55 |
| hexa | * NIH - not invented at helsinki systems! 😜 | 12:54:18 |
| das_j | Kinda, yeah :D | 12:54:35 |
| das_j | Also if the code looks like I really wanted to learn some go reflectino - that's what the project was initially for :D | 12:55:02 |
| das_j | * Also if the code looks like I really wanted to learn some go reflectino - that's what the project was initially for | 12:55:04 |
| das_j | * Also if the code looks like I really wanted to learn some go reflection - that's what the project was initially for | 12:55:09 |
| hexa | this looks much more complete from what I remember when I touched the upstream prometheus instance a few days ago 🙂 | 12:59:32 |
| das_j | Not by a lot actually. There's some stuff like nrUnsupportedSomething but the upstream exporter is mostly complete | 13:00:27 |
| das_j | Maybe you want --web.disable-exporter-metrics? :D | 13:00:49 |
|  Tseb joined the room. | 13:34:33 |
| @grahamc:nixos.org | nice | 14:15:18 |
| @grahamc:nixos.org | I'm hoping we could add a proper exporter directly to the evaluator and queue runner some day, but this looks great | 14:15:37 |
| das_j | yeah I was hoping the evaluator somehow dumps its state into the db as well but I couldn't find anything | 14:16:19 |
| @grahamc:nixos.org | one tricky bit of that is of course the process breakdown of hydra-evaulator calling hydra-evaluate-jobset calling hydra-eval-jobs, and potentially each of these could feasibly contain their own exporter, which would also be weird | 14:27:52 |
| das_j | What about this: The main daemon opens a fd for its subprocesses (FD 5 or something), and the subprocesses write their metrics there, maybe as kv pairs? Would be shitty but better than nothing | 14:29:04 |
| das_j | The main process can then collect these and serve them | 14:29:27 |
| @grahamc:nixos.org | I've tried to do that for ofborg in the past but was sort of a mess | 14:29:31 |
| @grahamc:nixos.org | but that was quite a bit more complicated | 14:29:55 |
| das_j | or each component just writes its stats into the main db and we scrape from there | 14:29:59 |
| @grahamc:nixos.org | so it could very well work here | 14:30:00 |
|  @pedrohlc:mozilla.org joined the room. | 15:13:45 |
| 15 Feb 2022 |
|  @packetgrill:automattic.com joined the room. | 13:32:31 |
 Linux Hackerman | So I'm trying to update the hydra package in nixpkgs, and ran into migrations failing because the pgcrypto postgres extension was missing. Seems odd to me though that this wouldn't have been caught by NixOS tests. Has everyone who runs a hyra done CREATE EXTENSION pgcrypto; manually, or am I missing a place where that's done within hydra (a plain rg pgcrypto on the hydra repo didn't yield any results), or what might be happening here? | 16:45:49 |
