| 6 Mar 2026 |
|  DavHau joined the room. | 15:22:20 |
| DavHau | I wanted to report this since some time, but didn't get around to do it. cache.nixos.org is consistently very very slow (down to 200 Kb/s) in Thailand. https://discourse.nixos.org/t/download-speed-in-nixos-update-is-too-slow/68138/6?u=davhau
Please let me know if I can help with debugging this. I am pulling +110 Mb/s from other CDNs, so it is not an issue with my setup.
It is liekly that other people in asia are experiencing this as well and if so then it is a real adoption blocker for nixos in asia. | 15:26:50 |
| DavHau | * I wanted to report this since some time, but didn't get around to do it. cache.nixos.org is consistently very very slow (down to 200 Kb/s) in Thailand. https://discourse.nixos.org/t/download-speed-in-nixos-update-is-too-slow/68138/6?u=davhau
Please let me know if I can help with debugging this. I am pulling +110 Mb/s from other CDNs, so it is not an issue with my setup.
It is likely that other people in asia are experiencing this as well and if so then it is a real adoption blocker for nixos in asia. | 15:27:14 |
| DavHau | (My provider is AIS which is one of the main ones in the region AFAIK) | 15:28:47 |
 hexa (signing key rotation when) | what is your path to cache.nixos.org? | 16:43:09 |
| hexa (signing key rotation when) | per https://www.fastly.com/network-map there should be a pop in bangkok | 16:43:44 |
|  Theuni changed their display name from Christian Theune to Theuni. | 19:58:06 |
| 7 Mar 2026 |
| DavHau | [nix-shell:~]# mtr --report cache.nixos.org
Start: 2026-03-07T13:31:01+0000
HOST: bam Loss% Snt Last Avg Best Wrst StDev
1.|-- console.gl-inet.com 0.0% 10 0.5 0.6 0.5 1.0 0.2
2.|-- 192.168.1.1 0.0% 10 1.1 1.8 1.0 7.7 2.1
3.|-- 100.67.0.1 0.0% 10 2.7 3.0 2.6 3.3 0.3
4.|-- ae8.fcr01j.tls1.myaisfibr 0.0% 10 12.8 22.3 12.1 32.3 7.9
5.|-- ve474.cgn05.tls1.myaisfib 0.0% 10 12.0 12.1 11.8 12.8 0.3
6.|-- ae23.bgw01j.tls1.myaisfib 0.0% 10 13.6 16.2 12.4 44.0 9.8
7.|-- 49-228-5-0.24.tls1.myaisf 70.0% 10 12.7 12.7 12.2 13.1 0.4
8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
9.|-- 119.31.4.123 0.0% 10 39.8 40.0 39.6 40.5 0.3
10.|-- 151.101.193.91 0.0% 10 14.3 13.9 13.4 14.3 0.3
[nix-shell:~]# mtr -6 --report cache.nixos.org
Start: 2026-03-07T13:31:22+0000
HOST: bam Loss% Snt Last Avg Best Wrst StDev
1.|-- GL-MT3000.lan 0.0% 10 0.6 0.7 0.5 0.9 0.2
2.|-- 2405-9800-b900.44.pool1.c 0.0% 10 1.5 1.4 1.3 1.6 0.1
3.|-- 2405:9800:70::5 0.0% 10 3.0 2.7 2.2 3.2 0.3
4.|-- 2405:9800:b005:96::97 0.0% 10 12.3 13.3 11.7 21.9 3.1
5.|-- 2405:9800:b005:100::100 0.0% 10 12.6 13.4 12.0 21.4 2.8
6.|-- 2405:9800:b005:118::118 0.0% 10 12.1 12.2 11.6 12.9 0.4
7.|-- 2405:9800:9802::32e 0.0% 10 14.2 14.8 14.2 15.3 0.3
8.|-- 2405:9800:9802::31c 0.0% 10 42.9 42.8 42.3 43.2 0.3
9.|-- 2a04:4e42:200::347 0.0% 10 13.5 13.8 13.3 14.4 0.3
hexa (signing key rotation when)
| 13:32:07 |
| DavHau | * [nix-shell:~]# mtr --report-wide cache.nixos.org
Start: 2026-03-07T13:33:24+0000
HOST: bam Loss% Snt Last Avg Best Wrst StDev
1.|-- console.gl-inet.com 0.0% 10 0.5 0.6 0.5 0.8 0.1
2.|-- 192.168.1.1 0.0% 10 1.6 1.2 1.1 1.6 0.1
3.|-- 100.67.0.1 0.0% 10 3.3 3.1 2.5 3.6 0.4
4.|-- ae8.fcr01j.tls1.myaisfibre.com 0.0% 10 11.8 13.3 11.8 19.8 2.4
5.|-- ve474.cgn05.tls1.myaisfibre.com 0.0% 10 11.6 12.1 11.5 12.7 0.4
6.|-- ae23.bgw01j.tls1.myaisfibre.com 0.0% 10 13.0 12.7 12.2 13.3 0.3
7.|-- 49-228-5-0.24.tls1.myaisfibre.com 70.0% 10 12.4 12.5 12.4 12.8 0.3
8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
9.|-- 119.31.4.124 0.0% 10 38.9 39.1 38.7 39.5 0.3
10.|-- 151.101.65.91 0.0% 10 14.6 14.3 13.7 14.7 0.3
[nix-shell:~]# mtr -6 --report-wide cache.nixos.org
Start: 2026-03-07T13:33:58+0000
HOST: bam Loss% Snt Last Avg Best Wrst StDev
1.|-- GL-MT3000.lan 0.0% 10 1.0 0.7 0.5 1.0 0.2
2.|-- 2405-9800-b900.44.pool1.cmbp-mser01.myaisfibre.com 0.0% 10 1.6 1.4 1.2 1.6 0.1
3.|-- 2405:9800:70::5 0.0% 10 5.3 3.1 2.5 5.3 0.8
4.|-- 2405:9800:b005:96::97 0.0% 10 12.1 14.3 11.8 23.0 4.0
5.|-- 2405:9800:b005:100::100 0.0% 10 12.9 12.7 12.2 13.7 0.4
6.|-- 2405:9800:b005:118::118 0.0% 10 12.3 12.2 11.8 12.9 0.4
7.|-- 2405:9800:9802::32e 0.0% 10 14.7 15.0 14.5 15.5 0.3
8.|-- 2405:9800:9802::31d 0.0% 10 39.4 39.7 39.0 40.2 0.3
9.|-- 2a04:4e42:400::347 0.0% 10 15.0 14.6 13.6 15.1 0.5
hexa (signing key rotation when)
| 13:34:36 |
| hexa (signing key rotation when) | that looks like a local pop, but if that one is slow then redistribution within fastlys network is the culprit | 13:36:10 |
 Lun | Can I get restart jobs perms on hydra? | 15:37:38 |
| hexa (signing key rotation when) | Yes, what is your hydra account? | 15:45:30 |
| Lun | git@lunnova.dev | 15:50:51 |
| hexa (signing key rotation when) | done | 15:59:01 |
| Lun | thanks! | 15:59:18 |
| 8 Mar 2026 |
| DavHau | OK, is there anything we can do? Can we escalate this issue to fastly? | 05:24:13 |
 K900 | staging-nixos merged, kconfig builder complained about, all channels booped | 10:00:18 |
| K900 | Me tired | 10:01:17 |
 eveeifyeve | A good tip, sometimes you need to take it easy on yourself.. | 12:32:40 |
| eveeifyeve | * A good tip, sometimes you need to take it easy on yourself and take a break. | 12:33:05 |
| hexa (signing key rotation when) | you can open a ticket with fastly here https://support.fastly.com/s/contactsupport | 13:23:58 |
| hexa (signing key rotation when) | this is better than us opening one, since you can collect the details and reproduce the issue | 13:24:26 |
|  SomeoneSerge (matrix works sometimes) changed their display name from SomeoneSerge (back on matrix) to SomeoneSerge (matrix works sometimes). | 23:34:24 |
| 10 Mar 2026 |
|  amadaluzia joined the room. | 01:46:02 |
 Picnoir | 👋
triton-llvm is currently failing on hydra for aarch64 with "output limit exceeded": https://hydra.nixos.org/build/323005905
x86 seems fine https://hydra.nixos.org/build/323005904
The embeddings are pretty big :/
This is a pretty expensive package to build, is there a way to increase this limit without splitting this package? | 16:39:05 |
| hexa (signing key rotation when) | Already poked serge a week ago or so 😬 | 16:51:59 |
| hexa (signing key rotation when) | It does affect a package I maintain, but that's hardly good enough a reason to bump the limit. | 16:52:33 |
| hexa (signing key rotation when) | Someone needs to answer the question why it is bigger on arm than x86 and by how much | 16:53:03 |
| hexa (signing key rotation when) | And ideally show growth over time | 16:53:15 |
| Lun | I can take a look | 17:07:38 |
