| 6 Mar 2024 |
 edef | In reply to @raitobezarius:matrix.org
But I don't think we need to convince folks of the fact that SLO is a budget and you burn into it so like, one of the things i'm curious about is what the actual distributions are | 17:18:41 |
|  @blitz:chat.x86.lol joined the room. | 17:19:30 |
| edef | eg is this whole-object loss, object corruption, is this uncorrelated across objects or correlated within prefixes | 17:19:41 |
 Jonas Chevalier | One thing I am excited about if we end up self-hosting is that it will be easier to do experiments, like a smart narinfo database that can answer to queries. We can also introduce a log of all the new entries, to make it easier for other caches to mirror the main cache. | 17:21:24 |
| edef | yeah, exactly | 17:21:36 |
| edef | one of the things in the works is incremental updating of the narinfo dataset from S3 | 17:21:50 |
 Wanja Hentze | and dedup, of course | 17:22:19 |
| Jonas Chevalier | if we can be smart about it, it has the potential to have better latency responses on the 404s | 17:22:24 |
| Jonas Chevalier | yeah and dedup will be easier to try, without worrying to pay 10k | 17:22:39 |
| Jonas Chevalier | * yeah and dedup will be easier to try, without worrying to pay 10k to AWS | 17:22:45 |
| edef | yeah. i have not worked on scaling deduplication compute much further because we didn't have the timeline to support that effort | 17:23:18 |
| edef | like. getting it into cold storage as-is is my current goal because i need more time budget, instead of frantically working to meet ever-shifting deletion deadlines | 17:24:09 |
| edef | as much as i am personally motivated to make this happen, this is a very stressful and strangely constrained working environment, and budget thus far would motivate no sane person to even get out of bed. i mostly do this because my sense of purpose is borderline anime protagonist levels | 17:27:44 |
 @patka_123:matrix.org | It's unasked for, but please take care of yourself | 17:29:10 |
| Wanja Hentze | make what exactly happen? self-hosted binary cache? or the more advanced stuff like dedup | 17:29:13 |
| edef | In reply to @zimbatm:numtide.com
One thing I am excited about if we end up self-hosting is that it will be easier to do experiments, like a smart narinfo database that can answer to queries. We can also introduce a log of all the new entries, to make it easier for other caches to mirror the main cache. i would like to have separate narinfo serving, and we are in a great position to do that. we can quite easily answer questions like "get me the entire closure of this path" in milliseconds now, if we want | 17:29:17 |
| edef | In reply to @whentze:matrix.org
make what exactly happen? self-hosted binary cache? or the more advanced stuff like dedup all of the above | 17:29:30 |
| edef | like, i'm not getting the past ~six months of my life back, my bank account nearly hit 0 doing this, other people have (probably wisely) dropped out because this is nuts and stressful | 17:31:44 |
| edef | having enthusiasm for self-hosting appear, nh2 show up to volunteer to help with Ceph, etc has been very good news to me | 17:32:20 |
| @patka_123:matrix.org | In reply to @edef1c:matrix.org
like, i'm not getting the past ~six months of my life back, my bank account nearly hit 0 doing this, other people have (probably wisely) dropped out because this is nuts and stressful Do you personally take donations? | 17:32:50 |
| edef | and if people offering funding follow through, especially long-term funding for hardware and hosting, that really changes the game | 17:33:16 |
| edef | In reply to @patka_123:matrix.org
Do you personally take donations? https://github.com/sponsors/edef1c is the default path, and any payment rail know to mankind can be arranged as well | 17:34:39 |
