!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

386 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.120 Servers

Load older messages


SenderMessageTime
22 Jul 2025
@drupol:matrix.org@drupol:matrix.orgimage.png
Download image.png
13:02:03
@drupol:matrix.org@drupol:matrix.orgKinda frustrating.13:02:10
@drupol:matrix.org@drupol:matrix.orgThey do everything to keep their workers at home :D13:02:18
@hexa:lossy.networkhexaalso a very funny route13:03:05
@hexa:lossy.networkhexagiven that you are presumably sitting in brussels, but handing of the traffic to british telecom13:03:16
@hexa:lossy.networkhexafor a detour via londoin13:03:20
@hexa:lossy.networkhexa* for a detour via london13:03:22
@hexa:lossy.networkhexaonly to go back to amsterdam13:03:34
@drupol:matrix.org@drupol:matrix.orgyes indeed, sitting in brussels at the moment13:03:39
@hexa:lossy.networkhexa* given that you are presumably sitting in brussels, but handing off the traffic to british telecom13:03:53
@drupol:matrix.org@drupol:matrix.orgIs there anything else I could do to try to spot where the issue comes from?13:04:00
@hexa:lossy.networkhexathis really looks like the reverse path from fastly to your AS is broken13:04:16
@hexa:lossy.networkhexathe solution is … peer with fastly somewhere and cut out the middleman13:04:42
@drupol:matrix.org@drupol:matrix.orgWhat worries me is this: the behaviour is strange: downloading just 30 MB takes around 15 minutes. It seems to follow a repeating pattern: the connection works normally for 1–2 seconds, then pauses for about 30 seconds, and this cycle continues indefinitely until the end. 13:04:52
@hexa:lossy.networkhexayeah, a route can flap13:05:15
@drupol:matrix.org@drupol:matrix.org

peer with fastly somewhere and cut out the middleman

how can I do that?

13:05:43
@hexa:lossy.networkhexahttps://bgp.tools/prefix/147.67.128.0/17#connectivity13:05:48
@hexa:lossy.networkhexasingle homed behind vodafone 🫠13:05:56
@hexa:lossy.networkhexaif you're not at any exchange you likely cannot13:06:48
@drupol:matrix.org@drupol:matrix.orgI guess the network here is quite protected indeed.13:07:15
@drupol:matrix.org@drupol:matrix.organyway, thanks for the help and the link I'll keep it :)13:09:12
@federicodschonborn:matrix.org@federicodschonborn:matrix.org changed their display name from The Worm 🏳️‍🌈 (he/they) to Wormy McWormface 🏳️‍🌈 (he/they).18:27:19
@solidhal:matrix.orgsolideva joined the room.20:41:27
23 Jul 2025
@vcunat:matrix.orgVladimír Čunát

In the past couple hours we get lots of

mimas hydra-queue-runner[1040202]: AWS error '' (curlCode: 28, Timeout was reached; request id: ), will retry in NN ms

I see nothing at https://health.aws.amazon.com/health/status

10:28:29
@vcunat:matrix.orgVladimír ČunátHmm, we only fit less than two days of log for that unit, so who knows how rare this is over long term, but the build throughput does seem unusually low in the past couple hours (which is why I was looking at logs).10:31:15
@vcunat:matrix.orgVladimír Čunát *

In the past couple hours we get thousands of

mimas hydra-queue-runner[1040202]: AWS error '' (curlCode: 28, Timeout was reached; request id: ), will retry in NN ms

I see nothing at https://health.aws.amazon.com/health/status

10:32:16
24 Jul 2025
@joerg:thalheim.ioMic92 hexa (signing key rotation when) are we on jitsi or something else today? 16:04:51
@hexa:lossy.networkhexahttps://meet.cccda.de/zqb-dylq-bcp16:05:49
@jfly:matrix.orgJeremy Fleischman (jfly)we're there now16:08:28
@maxcf:matrix.orgMax joined the room.16:08:43

Show newer messages


Back to Room ListRoom Version: 6