| 21 Aug 2023 |
 Julien | To be honnest I thought that that was what people were doing on the group but I was only following the conversation from afar. Iām interested into doing experiments on that field though. | 08:57:40 |
 @linus:schreibt.jetzt | I guess raitobezarius is the most likely to know :D | 08:58:26 |
 @elvishjerricco:matrix.org | Linux Hackerman: Any suggestion on special_small_blocks value? I was thinking 8K since anything <=8K is worst case space efficiency on raidz2 (ashift=12) | 09:04:45 |
| Julien | In reply to @linus:schreibt.jetzt
attic=# select sum(file_size) / (select sum(nar_size) from nar) from chunk;
?column?
------------------------
0.20098691256561418637
(1 row)
On my own attic deployment I get 0.17052771681498935 | 09:04:49 |
| @linus:schreibt.jetzt | In reply to @elvishjerricco:matrix.org
Linux Hackerman: Any suggestion on special_small_blocks value? I was thinking 8K since anything <=8K is worst case space efficiency on raidz2 (ashift=12) sounds reasonable, and I don't think many narinfos will be bigger than 8K | 09:05:31 |
| @elvishjerricco:matrix.org | plus I think optane has 512B sectors, so ashift=9 on that vdev š | 09:05:39 |
| @linus:schreibt.jetzt | but are you going to use raidz2 on your special too? | 09:05:39 |
| @elvishjerricco:matrix.org | you can't use raidz on special I'm pretty sure | 09:05:53 |
| @elvishjerricco:matrix.org | ashift=9 on the special just means that small things get even smaller | 09:06:41 |
| @linus:schreibt.jetzt | In reply to @elvishjerricco:matrix.org
you can't use raidz on special I'm pretty sure ah yeah | 09:07:03 |
| @elvishjerricco:matrix.org | I've only got two of these optanes in here, so my redundancy level is sort of different. But it's optane, so it should resilver fast in the event of a failure, and they should be durable as hell. So it's not too big a worry | 09:08:00 |
| @linus:schreibt.jetzt | yeah, and losing a binary cache isn't the end of the world I think? | 09:08:31 |
| @linus:schreibt.jetzt | and you'd have backups of any more important stuff, right? ;) | 09:08:43 |
| @elvishjerricco:matrix.org | right; granted there's other stuff I like on this pool but none of it matters | 09:08:45 |
| @elvishjerricco:matrix.org | and yes :P | 09:08:53 |
| @elvishjerricco:matrix.org | other machine has a single very big hard drive as a send/recv target | 09:09:28 |
| @linus:schreibt.jetzt | In reply to @julienmalka:matrix.org
On my own attic deployment I get 0.17052771681498935 you can also replace file_size with chunk_size to see the ratio for dedup-only (without compression) | 09:09:47 |
| @linus:schreibt.jetzt | which is like 47% for me | 09:09:55 |
| @linus:schreibt.jetzt | * which is like 45% for me | 09:10:05 |
| Julien | In reply to @linus:schreibt.jetzt
which is like 45% for me Yeah 43% | 09:11:33 |
| Julien | So this is still a big time storage economy | 09:12:15 |
| @elvishjerricco:matrix.org | Linux Hackerman: Oh wow yea. Tested out copying a big closure from the cache and the "querying path" stuff was very noticeably stop-and-go and took a good amount of time.
I've now added optane special metadata and migrated the data with a send/recv to and from the same pool. I used a completely different closure to make sure it wasn't just remembering the cache hit, and that querying part is effectively instant now
| 11:24:55 |
| @linus:schreibt.jetzt | \o/ | 11:25:09 |
| @elvishjerricco:matrix.org | optane is kind of insane | 11:25:27 |
| @elvishjerricco:matrix.org | there was a huge influx of supply like 6 months ago or something when intel announced they were killing the division, so I managed to pick up four 110GB nvme sticks at a quarter their normal price | 11:26:02 |
| @linus:schreibt.jetzt | nice | 11:26:15 |
| 22 Aug 2023 |
| @elvishjerricco:matrix.org | whoa; just did a full system update with the "new" cache storage, and while optane made the querying instant, the download bandwidth was way reduced. zpool iostat 1 showed a pretty steady read bandwidth of ~80MB/s but I wasn't getting anywhere near that over the wire; whereas on the SSD I saturated the gigabit easily | 05:30:57 |
| @elvishjerricco:matrix.org | (also this pool can easily do sequential reads upwards of 500MB/s) | 05:32:30 |
| @linus:schreibt.jetzt | huh | 06:38:37 |
| @linus:schreibt.jetzt | did you rewrite the nars as well? | 06:38:41 |
