| 7 Mar 2024 |
 edef | but deduped cache replication might be positive-sum here | 00:38:38 |
 raitobezarius | i'm okay funding deduped cache replication via my own requester pays stuff to avoid putting any churn on the foundation | 00:38:58 |
| edef | In reply to @edef1c:matrix.org
to elaborate on this more: we basically have three tiers. frontend cache, semi-hot fast-enough storage, cold storage / DR one of the other considerations is like, the ingestion path leads straight into the AWS prison | 00:39:03 |
| edef | In reply to @edef1c:matrix.org
to elaborate on this more: we basically have three tiers. frontend cache, semi-hot fast-enough storage, cold storage / DR * one of the other considerations is like, the ingestion path leads straight into the AWS prison right now | 00:39:08 |
| raitobezarius | but my stuff is mostly a tangent and i will go to bed | 00:39:24 |
| edef | i'd much rather have that piped into somewhere cheap and from there to S3, or replicated into two places | 00:39:40 |
| edef | there just isn't a separate ingestion mechanism right now, the build farm hits S3 directly | 00:39:53 |
| raitobezarius | to be discussed next meeting I guess | 00:40:00 |
| edef | i'm not even sure it fetches from Fastly | 00:40:02 |
| raitobezarius | I need to hammer a bit more the software | 00:40:04 |
| raitobezarius | to make this more tangible | 00:40:07 |
 Zhaofeng Li | <offtopic>Ah, Requester Pays... I always wondered why AWS/Google don't provide an easy way for non-technical people to access those buckets (i.e., enter credit card and bucket name, get web access)</offtopic> | 01:02:24 |
| edef | hmm, that seems buildable | 01:02:53 |
| edef | i think their ToS might have some stuff about reselling but i'm not sure how much that applies here | 01:03:32 |
| edef | * i think their ToS might have some stuff about reselling their services too directly but i'm not sure how much that applies here | 01:03:48 |
| edef | but you can probably go forth and be the change in the world you want to see | 01:04:11 |
| raitobezarius | aka build a startup on the top of that | 01:04:25 |
| Zhaofeng Li | It would be nice if such an UI can avoid being in the middle of the payment process | 01:05:03 |
| Zhaofeng Li | like research teams can self-host the UI and not have to worry about fees, tax implications etc | 01:05:36 |
| edef | you can totally sign up for an AWS account and use the bucket explorer UI, i think? | 01:05:39 |
| Zhaofeng Li | * like research teams (dataset publishers) can self-host the UI and not have to worry about fees, tax implications etc | 01:05:55 |
| edef | and we have UI on eg https://releases.nixos.org | 01:06:01 |
| edef | but for cache.nixos.org there's basically no easy UI on top of the raw objects that would be that useful | 01:06:22 |
| Zhaofeng Li | you totally can, but the setup process can be daunting/annoying for those who have never used AWS before and just want to download one dataset | 01:06:35 |
| edef | we have the data to build a cache explorer now though :p | 01:06:35 |
| edef | the counterparty risk is different doing credit card processing for long-lived accounts vs one-offs | 01:07:15 |
| edef | like, it may well be a profitable business, it's just a different one | 01:07:26 |
| raitobezarius | is there anything like frontend clickhouse | 01:07:30 |
| raitobezarius | where people can type clickhouse queries in their browser and get results | 01:07:39 |
| edef | In reply to @raitobezarius:matrix.org
is there anything like frontend clickhouse it has a UI on the same port | 01:07:40 |
