| 18 Jun 2026 |
 K900 | Very likely | 07:24:49 |
 Vladimír Čunát | .narinfo seems OK
$ curl -s https://cache.nixos.org/gypp7g48p2fcpjx952i84dhw4ak95npl.narinfo | grep NarHash
NarHash: sha256:033w5cd8z7r02h4r2p8lha6z81qqvky69a7aiahjpd15smss4wjp
| 07:26:26 |
| Vladimír Čunát | This is simplified by being a fixed-output derivation. | 07:27:18 |
| Vladimír Čunát | $ git grep 033w5cd8z7r02h4r2p8lha6z81qqvky69a7aiahjpd15smss4wjp | cat
pkgs/tools/typesetting/tex/texlive/fixed-hashes.nix: run = "033w5cd8z7r02h4r2p8lha6z81qqvky69a7aiahjpd15smss4wjp";
| 07:27:37 |
| Vladimír Čunát | * $ git grep -B1 033w5cd8z7r02h4r2p8lha6z81qqvky69a7aiahjpd15smss4wjp | cat
pkgs/tools/typesetting/tex/texlive/fixed-hashes.nix- lpform-36918 = {
pkgs/tools/typesetting/tex/texlive/fixed-hashes.nix: run = "033w5cd8z7r02h4r2p8lha6z81qqvky69a7aiahjpd15smss4wjp";
| 07:28:32 |
| K900 | The narinfos are uploaded from the coordinator | 07:28:37 |
| K900 | So that presumably avoids whatever confusion is going on | 07:28:48 |
 Ryan Burns | The texliveFull package itself has not changed since 6/10 afaict: https://hydra.nixos.org/build/331466803/evals
So I assume that means either something has changed in the narinfo, or a server that stands between the cache and the user, or PEBCAK on my part? | 07:30:51 |
 Mic92 | In reply to @vcunat:matrix.org
The situation doesn't seem sustainable to me for more than a couple days. Even -small channels get blocked because of the abortions and retrying doesn't seem to help, e.g.:
https://hydra.nixos.org/build/331658358 Could please make issue reports and ping me here on them? Scraping from matrix is a bit of messy situation | 07:31:43 |
| Mic92 | Especially now that we have old and new reports in this thread | 07:33:52 |
| Mic92 | @r-burns:matrix.org: same for you | 07:35:08 |
| Mic92 | I will do another triage now to check if I can identify more sources of aborts just now. But but the next proper fixing session has to wait for the evening. I would do it in a staging environment, but my Aws servers didn't show a lot of the symptoms we are seeing now | 07:38:34 |
| Vladimír Čunát | Posted as
https://github.com/NixOS/hydra/issues/1804
https://github.com/NixOS/hydra/issues/1803 | 07:39:29 |
| Vladimír Čunát | Feel free to edit, add info, etc. | 07:39:41 |
| Vladimír Čunát | To recover we simply remove this NAR+narinfo from S3, right? | 07:43:12 |
| Vladimír Čunát | As it's a fixed-output derivation, it should be cheap to "build" locally, and I expect Hydra will fill it up at some point automatically anyway if it will be missing in S3. | 07:43:47 |
| Mic92 | Yes. | 07:44:02 |
| Vladimír Čunát | * As it's a fixed-output derivation, it should be cheap to "build" locally, and I expect Hydra will fill it up at some point automatically anyway if it will be missing in S3. | 07:44:09 |
| Mic92 | I have a look at the logs first | 07:44:16 |
| Mic92 | and check if we I can root cause this | 07:44:20 |
| Vladimír Čunát | Sure. Though removing it from S3 hopefully wouldn't damage these traces. | 07:44:56 |
| Vladimír Čunát | It also seems likely that this wasn't the only damaged NAR. | 07:45:15 |
|  buterfloro joined the room. | 07:45:24 |
| Vladimír Čunát | (In my experience, most failures do not get reported.) | 07:45:44 |
| buterfloro | I've got full logs that I can provide on request, unlike the truncated ones in the Discourse discussion | 07:46:55 |
| buterfloro | * I've got full logs on my end that I can provide on request, unlike the truncated ones in the Discourse discussion | 07:47:14 |
| K900 | It's not really going to help much, we know the wrong thing is uploaded | 07:48:35 |
| K900 | Figuring out why would require looking at infra, not at your system | 07:48:47 |
| Vladimír Čunát | Also the Unsupported system type:
https://github.com/NixOS/hydra/issues/1805 | 07:53:22 |
| Mic92 | thx | 08:03:10 |
