| 21 Jan 2022 |
 kenran_ | Ok, thanks, seems like it isn't normal then. I read somewhere that dumps could be quite big if they happen. | 09:08:46 |
| kenran_ | In reply to @janne.hess:helsinki-systems.de
That's from 2016 btw… that's pretty old in Hydra terms ;) Haha yes, but still: they have 100+GB of logs, I have KBs | 09:09:03 |
| kenran_ | I'd expect that everything there would be much bigger numbers. | 09:09:18 |
 das_j | Yeah there's currently no automated method to collect logs 👀 they are only compressed | 09:09:27 |
| kenran_ | That's good to know! Right now the experimental machine I'm on has a separate volume attached that I can increase in size, but /var is still on the primary disk, which is quite small. So that won't scale at all sadly. | 09:14:27 |
| das_j | Currently running du for that on our machine. But you should also worry about your postgres database (no retention here) and your nix store (there is gcroot handling but the store can temporarily grow pretty large) | 09:16:03 |
| das_j | * Currently running du for that on our machineThe Hydra directory is only 20G for us. But you should also worry about your postgres database (no retention here) and your nix store (there is gcroot handling but the store can temporarily grow pretty large) | 09:16:30 |
| kenran_ | (I mean, I could do the same thing I'm doing with /nix/store and mount -o bind,ro ... some directory on the volume "over" /var as well, but that seems error-prune.) | 09:16:46 |
| kenran_ | In reply to @janne.hess:helsinki-systems.de
Currently running du for that on our machineThe Hydra directory is only 20G for us. But you should also worry about your postgres database (no retention here) and your nix store (there is gcroot handling but the store can temporarily grow pretty large) Thanks for looking :) | 09:17:11 |
| kenran_ | So far I tried making the one 13GB-dir that's the reason for the bloat smaller, to no avail. Maybe I'll just try deleting it. | 09:17:40 |
| kenran_ | How would I go about pulling git sources from a custom GitLab instance? | 10:07:25 |
| das_j | Like you would use git | 10:07:47 |
| das_j | * Like you would use git clone | 10:07:54 |
| kenran_ | Oh sorry, didn't mean to send it already, I was still researching and accidentally hit Enter. I mean a private GitLab instance. | 10:08:05 |
| das_j | ah auth? | 10:08:17 |
| das_j | hydra will just call git with HOME=/var/lib/hydra as the hydra user | 10:08:50 |
| das_j | This means that this git will pick up private keys in /var/lib/hydra/.ssh if they are owned by the hydra user | 10:09:09 |
| kenran_ | Yes exactly. I personally pull via SSH, so I guess I'd have to put some SSH key somewhere. | 10:09:25 |
| kenran_ | In reply to @janne.hess:helsinki-systems.de
This means that this git will pick up private keys in /var/lib/hydra/.ssh if they are owned by the hydra user Currently skimming https://github.com/NixOS/hydra/issues/555, which seems to go well with what you wrote. Thanks for clarifying who does the pull. | 10:09:57 |
| kenran_ | I found out I can eval the jobset interactively as said one linked issue further, which makes debugging easier :) | 10:10:14 |
