| 19 Jun 2026 |
 Mic92 | should be fine in this case | 10:29:53 |
 hexa (signing key rotation when) | silly me tried --delete-old | 10:30:11 |
| hexa (signing key rotation when) | will go for --ensure-free next time round | 10:30:16 |
| Mic92 | it will read the sqlite db into memory in any case. | 10:36:09 |
| hexa (signing key rotation when) | # nix run github:mic92/fast-nix-gc -- --ensure-free 100G
[INFO ] freeing at least 97.67 GiB to reach 100.00 GiB free
[INFO ] remounted /nix/store read-write
[INFO ] loading store graph...
[INFO ] 15409584 total valid paths
[INFO ] finding garbage collector roots...
[INFO ] found 1078966 roots
[INFO ] computing alive closure...
[INFO ] 4701564 alive paths
[INFO ] 10708020 dead paths
[INFO ] 251 unknown paths on disk not in DB
[INFO ] deleting garbage...
| 10:37:29 |
| hexa (signing key rotation when) | do we have an idea, why it loaded multiple hundred GB into its nix store? | 10:40:56 |
| hexa (signing key rotation when) | 
Download | 10:41:45 |
| hexa (signing key rotation when) | ~400GB | 10:42:25 |
| Mic92 | I suspect this was my build product fix: https://github.com/NixOS/hydra/commit/a38b8062f8bd2f1a5e4dbb2efdd25afdb02261a6 | 10:43:56 |
| Mic92 | From my understanding this was only to be triggered as a fallback if there is no db entry. | 10:45:27 |
| hexa (signing key rotation when) | Ah, ok. | 10:46:14 |
| hexa (signing key rotation when) | in the meantime I restarted fast-nix-gc with --ensure-free, so I can leave it running and leave | 10:47:02 |
| hexa (signing key rotation when) | it sits at "deleting garbage..." for minutes | 10:47:20 |
| hexa (signing key rotation when) | I'm sure it is fast, faster than nix-gc, but is mimas the endboss here? :D | 10:47:42 |
| Mic92 | Yeah I think so. | 10:49:00 |
| Mic92 | I run perf on it to see where it hangs | 10:49:18 |
| Mic92 | In good news, I think my scheduler reconciliation fixes seem to work in staging hydra. | 10:54:17 |
| Mic92 | I am doing the perf now on the fast-nix-gc | 10:54:32 |
| Mic92 | hexa (signing key rotation when): oh, here is the catch: --no-vacuum | 10:55:24 |
| Mic92 | you need this | 10:55:26 |
| Mic92 | I think something is still accessing sqlite | 10:55:35 |
| hexa (signing key rotation when) | hah, can you add a log? | 10:55:45 |
| Mic92 | did it print this?
log::info!("vacuuming database ({freelist} of {pages} pages free)..."); | 10:56:33 |
| Mic92 | ? | 10:56:35 |
| hexa (signing key rotation when) | [root@mimas:~]# nix run github:mic92/fast-nix-gc -- --ensure-free 100G
[INFO ] freeing at least 97.67 GiB to reach 100.00 GiB free
[INFO ] remounted /nix/store read-write
[INFO ] loading store graph...
[INFO ] 15409584 total valid paths
[INFO ] finding garbage collector roots...
[INFO ] found 1078966 roots
[INFO ] computing alive closure...
[INFO ] 4701564 alive paths
[INFO ] 10708020 dead paths
[INFO ] 251 unknown paths on disk not in DB
[INFO ] deleting garbage...
^C
| 10:56:44 |
| Mic92 | mhm. | 10:57:11 |
| Mic92 | than it's something else | 10:57:15 |
| Mic92 | I will take over fast-nix-gc. I want to try some other options | 11:01:24 |
 dgrig | If it's related, after lots of GCs without a vacuum on the nix sqlite we saw something similar on the security tracker last year. I had opened https://github.com/NixOS/nixpkgs/pull/435438 to at the very least document this (linking to a patch by Trofi on Nixos/nix) | 12:02:52 |
| hexa (signing key rotation when) | yeah, 16 GB is like factor 10 of my busy personal machines | 12:03:56 |
