| 21 Jan 2022 |
 kenran_ | So far, I've set up the Hydra (on NixOS btw) and added the imperative and the declarative examples from the Wiki page, but nothing seems to be happening: localhost is supposedly Idle. How can I see "what it is doing"? | 07:23:46 |
| kenran_ | Ah, something has happened right now, the evaluations were probably just taking a long time (or the restart of my server actually changed something). | 07:27:42 |
| kenran_ | But only the declarative example is working, the imperative one stays at 0%. That's not as important for me though. | 07:28:08 |
| kenran_ | Got it, the branch nixos-18.03 doesn't exist anymore! | 07:41:47 |
| kenran_ | (I updated the wiki accordingly :)) | 08:48:31 |
| kenran_ | * (I updated the wiki accordingly) | 08:48:46 |
| kenran_ | But phew, I keep having new questions, sorry for all the text:
- I noticed (after a couple of trivial builds) that
/var/lib/hydra/scm is already at 14GB, (this is a small testing machine with only 20GB of disk space). Is there documentation about what goes where and how to clean it up?
- Are these files maybe the
nixpkgs variants that are pulled when building? If so, are they garbage-collected at some point?
| 08:59:58 |
 das_j | In reply to @kenran_:matrix.org
But phew, I keep having new questions, sorry for all the text:
- I noticed (after a couple of trivial builds) that
/var/lib/hydra/scm is already at 14GB, (this is a small testing machine with only 20GB of disk space). Is there documentation about what goes where and how to clean it up?
- Are these files maybe the
nixpkgs variants that are pulled when building? If so, are they garbage-collected at some point?
There is a button to clean the SCM cache in the admin menu. It should contain all SCM repos like git that are input of any evaluations | 09:06:10 |
| das_j | It's git repos so you can probably clean them with git commands as the appropriate user but I haven't ever done this | 09:06:35 |
| kenran_ | Ah thanks, I can't see that right now, as the disk ran full and thus I couldn't access the web interface anymore :D | 09:06:51 |
| das_j | But 14GB is strange… we have been using Hydra for a couple of years now and we only have 9GB | 09:07:34 |
| kenran_ | But is it normal that this escalates that quickly w.r.t. disk usage? https://github.com/NixOS/hydra/issues/421 shows a value of 20GB for a much bigger thing. | 09:07:49 |
| das_j | In reply to @kenran_:matrix.org
But is it normal that this escalates that quickly w.r.t. disk usage? https://github.com/NixOS/hydra/issues/421 shows a value of 20GB for a much bigger thing. That's from 2016 btw… that's pretty old in Hydra terms ;) | 09:08:42 |
| kenran_ | Ok, thanks, seems like it isn't normal then. I read somewhere that dumps could be quite big if they happen. | 09:08:46 |
| kenran_ | In reply to @janne.hess:helsinki-systems.de
That's from 2016 btw… that's pretty old in Hydra terms ;) Haha yes, but still: they have 100+GB of logs, I have KBs | 09:09:03 |
| kenran_ | I'd expect that everything there would be much bigger numbers. | 09:09:18 |
| das_j | Yeah there's currently no automated method to collect logs 👀 they are only compressed | 09:09:27 |
| kenran_ | That's good to know! Right now the experimental machine I'm on has a separate volume attached that I can increase in size, but /var is still on the primary disk, which is quite small. So that won't scale at all sadly. | 09:14:27 |
| das_j | Currently running du for that on our machine. But you should also worry about your postgres database (no retention here) and your nix store (there is gcroot handling but the store can temporarily grow pretty large) | 09:16:03 |
| das_j | * Currently running du for that on our machineThe Hydra directory is only 20G for us. But you should also worry about your postgres database (no retention here) and your nix store (there is gcroot handling but the store can temporarily grow pretty large) | 09:16:30 |
| kenran_ | (I mean, I could do the same thing I'm doing with /nix/store and mount -o bind,ro ... some directory on the volume "over" /var as well, but that seems error-prune.) | 09:16:46 |
| kenran_ | In reply to @janne.hess:helsinki-systems.de
Currently running du for that on our machineThe Hydra directory is only 20G for us. But you should also worry about your postgres database (no retention here) and your nix store (there is gcroot handling but the store can temporarily grow pretty large) Thanks for looking :) | 09:17:11 |
| kenran_ | So far I tried making the one 13GB-dir that's the reason for the bloat smaller, to no avail. Maybe I'll just try deleting it. | 09:17:40 |
| kenran_ | How would I go about pulling git sources from a custom GitLab instance? | 10:07:25 |
| das_j | Like you would use git | 10:07:47 |
| das_j | * Like you would use git clone | 10:07:54 |
| kenran_ | Oh sorry, didn't mean to send it already, I was still researching and accidentally hit Enter. I mean a private GitLab instance. | 10:08:05 |
| das_j | ah auth? | 10:08:17 |
| das_j | hydra will just call git with HOME=/var/lib/hydra as the hydra user | 10:08:50 |
| das_j | This means that this git will pick up private keys in /var/lib/hydra/.ssh if they are owned by the hydra user | 10:09:09 |
