!CcTBuBritXGywOEGWJ:matrix.org

NixOS Binary Cache Self-Hosting

161 Members
About how to host a very large-scale binary cache and more55 Servers

Load older messages


SenderMessageTime
23 Aug 2023
@linus:schreibt.jetzt@linus:schreibt.jetztah, curling the URL does the same thing09:17:56
@linus:schreibt.jetzt@linus:schreibt.jetzt
$ curl -v http://attic.geruest.sphalerite.tech:8080/magic/nar/ja7cry6cb9wwclhlphmffgg4fv0ky4cd.nar >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying [2a01:4f9:1a:f600:5650::36]:8080...
* Connected to attic.geruest.sphalerite.tech (2a01:4f9:1a:f600:5650::36) port 8080 (#0)
> GET /magic/nar/ja7cry6cb9wwclhlphmffgg4fv0ky4cd.nar HTTP/1.1
> Host: attic.geruest.sphalerite.tech:8080
> User-Agent: curl/8.1.1
> Accept: */*
>
< HTTP/1.1 200 OK
< x-attic-cache-visibility: public
< transfer-encoding: chunked
< date: Wed, 23 Aug 2023 09:17:52 GMT
<
{ [0 bytes data]
* transfer closed with outstanding read data remaining
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Closing connection 0
curl: (18) transfer closed with outstanding read data remaining
09:18:06
@linus:schreibt.jetzt@linus:schreibt.jetztI guess I'll open an attic issue09:18:10
@linus:schreibt.jetzt@linus:schreibt.jetztoh nvm09:22:42
@linus:schreibt.jetzt@linus:schreibt.jetztturns out the backing s3 bucket was configured wrong and the chunks were missing09:23:30
@linus:schreibt.jetzt@linus:schreibt.jetztthough attic should probably recognise that error and report it, at least in its own log ^^09:23:44
@julienmalka:matrix.orgJulienI get a lot of « InternalServerError: The server encountered an internal error or misconfiguration. » in the middle of my attic push. Anyone here had the same issue ?11:05:56
@julienmalka:matrix.orgJulien(Usually if I relaunch the same command it will just work fine)11:06:29
@linus:schreibt.jetzt@linus:schreibt.jetztcheck the atticd logs11:07:30
@linus:schreibt.jetzt@linus:schreibt.jetztI've been having that when it fails to acquire a db connection from the pool, probably because all the connections in the pool are busy11:07:47
@linus:schreibt.jetzt@linus:schreibt.jetztI think it has a 30s timeout11:07:52
@julienmalka:matrix.orgJulienI yes11:16:12
@julienmalka:matrix.orgJulien* Ah yes11:16:21
@julienmalka:matrix.orgJulienIt looks that that’s the problem11:16:29
@julienmalka:matrix.orgJulien* It looks like that’s the problem11:16:39
@linus:schreibt.jetzt@linus:schreibt.jetztI think the suggested chunking params are pretty heavy on the database, so you might need to adjust the chunking, or you might need a more powerful or maybe just better tuned postgres server11:19:24
@linus:schreibt.jetzt@linus:schreibt.jetztthat is, adjust the chunking so that you get fewer chunks, but also less effective deduplication11:19:41
@raitobezarius:matrix.orgraitobezariuspostgres defaults are extremely conservative11:23:32
@raitobezarius:matrix.orgraitobezariustuning postgres is probably the easiest11:23:38
@linus:schreibt.jetzt@linus:schreibt.jetztI'm trying that out now11:30:32
@linus:schreibt.jetzt@linus:schreibt.jetztusing https://pgtune.leopard.in.ua/ and the web application profile11:30:44
@linus:schreibt.jetzt@linus:schreibt.jetztthough maybe desktop is more suitable given that it's not a dedicated machine11:31:39
@linus:schreibt.jetzt@linus:schreibt.jetztwell, I'll find out :)11:31:47
@julienmalka:matrix.orgJulienI’ll use my Raito script to tune postgres11:50:43
@raitobezarius:matrix.orgraitobezariuskrkrkrkrkr11:54:46
@linus:schreibt.jetzt@linus:schreibt.jetzthm, it seems to result in more successes, but doesn't entirely eliminate the failures12:03:42
@raitobezarius:matrix.orgraitobezariuswhat is the nature of pgsql connections that attic does?12:04:52
@raitobezarius:matrix.orgraitobezariusread or write?12:04:55
@linus:schreibt.jetzt@linus:schreibt.jetztI could also imagine that atticd doesn't reuse connections well enough12:05:01
@raitobezarius:matrix.orgraitobezariusin some cases, a pooler could help12:05:04

Show newer messages


Back to Room ListRoom Version: 10