| 27 Jan 2025 |
 Leonardo Santiago | also, some misunderstandings of mine about the state of gc: there are 2 possible implementations of gc? if so, is the bdwgc included by default? do I need to include any dependencies and enable flags? | 13:43:57 |
 Benedikt | In reply to @o-santi:matrix.org
just so I can understand, am I on a dead end here? shouldn' I expect that at least some memory is freed during the runtime? it seems to me that the memory isn't being free at all, not even when the whole state/store is freed. Are you expecting that the memory is returned to the system? To my knowledge free simply tells the allocator, that it can reuse this memory | 13:44:31 |
| Benedikt | So allocating a new eval stat should not increase your total memory | 13:44:56 |
| Benedikt | * | 13:45:53 |
| Leonardo Santiago | tbh not 100% sure what I'm expecting, just trying to clear my confusion. though, it is of my expectation that after clearing all nix related objects from my program, the resident memory winds down to the state it was before, even if during evaluation it's not that | 13:46:14 |
| Leonardo Santiago | I'm mostly envisioning a use case where I'd like to evaluate a quite complex expression that takes multiple GB, that builds a package that I'd like to use, and those GB are forever entrapped as "nix evaluation memory" even though I'm not going to be evaluating anything else. | 13:47:21 |
| Leonardo Santiago | also, this is not the case, if I do the whole evaluate file -> evaluate attribute of attribute set -> free all of them twice the memory roughly doubles | 13:49:38 |
| Leonardo Santiago | goes from 470MB in the first cycle to 700MB in the next cycle, even if I free the evaluation state | 13:50:08 |
| Leonardo Santiago | and if I do it a third time, it jumps to 1,0GB | 13:51:41 |
| Leonardo Santiago | * tbh not 100% sure what I'm expecting, just trying to clear my confusion. though, it is of my expectation that after clearing all nix related objects from my program, the resident memory winds down to the state it was before, even if during evaluation it's not exactly perfect | 13:52:00 |
| Leonardo Santiago | I don't know how to interpret this other than nix gc isn't actually gc'ing anything:
- maybe because of something very dumb I'm doing
- maybe because of an implementation problem
- maybe I did not compile with the bdwgc (as I'm not doing anything explicit, and don't know if that's necessary)
| 13:54:01 |
| Leonardo Santiago | * I don't know how to interpret this other than nix gc isn't actually gc'ing anything:
- maybe because of something very dumb I'm doing
- maybe because of an implementation issue in nix itself
- maybe I did not compile with the bdwgc (as I'm not doing anything explicit, and don't know if that's necessary)
| 13:54:22 |
| Leonardo Santiago | * also, this is not the case, if I do the whole evaluate file -> evaluate attribute of attribute set -> free all of them twice the memory roughly doubles | 13:55:14 |
| Leonardo Santiago | maybe this is related to the GC_register_my_thread stuff, but I don't quite understand how that works | 14:07:21 |
| Leonardo Santiago | my only reference is $nixops4/rust/nix-expr/src/eval_state.rs stuff, which I will try to copy and see if it changes anything | 14:07:49 |
| Leonardo Santiago | * maybe this is related to the GC_register_my_thread stuff (that I'm doing 0 of), but I don't quite understand how that works | 14:16:35 |
| Leonardo Santiago | don't think that's it either, even after enabling register of threads, registering the thread and calling gc_now after every one of these drops, the problem still remains | 14:29:42 |
| Leonardo Santiago | so yeah, I'm mostly just looking for pointers on if this is expected, if this is an implementation error on my end or if this is a flaw of nix GC | 14:34:12 |
| Leonardo Santiago | * my only reference is
$nixops4/rust/nix-expr/src/eval_state.rs, which I will try to copy and see if it changes anything | 14:34:58 |
| Leonardo Santiago | * my only reference is
`$nixops4/rust/nix-expr/src/eval_state.rs`, which I will try to copy and see if it changes anything | 14:35:07 |
| Leonardo Santiago | * don't think that's it either, even after enabling registering of threads, registering the thread and calling
gc_now after every one of these drops, the problem still remains | 14:35:20 |
| Benedikt | In reply to @o-santi:matrix.org
also, this is not the case, if I do the whole evaluate file -> evaluate attribute of attribute set -> free all of them twice the memory roughly doubles Well that's unfortunate. I've probably missed something | 16:11:53 |
| 29 Jan 2025 |
| Leonardo Santiago | Robert Hensing (roberth): can I get your opinion on this? | 16:30:20 |
| 30 Jan 2025 |
 Robert Hensing (roberth) | do you also call nix_gc_now()? | 11:57:37 |
| Robert Hensing (roberth) | iirc the cache of parsed files is not released along with the eval state. This is usually not an issue, because it just dies with the process, but this could be improved | 12:00:43 |
| Robert Hensing (roberth) | shouldn't be necessary if you're doing everything on the main thread | 12:01:09 |
| Leonardo Santiago | In reply to @roberthensing:matrix.org
do you also call nix_gc_now()? wasn't, but even if I do, nothing changes, the behavior is the same | 13:05:28 |
| Leonardo Santiago | In reply to @roberthensing:matrix.org
iirc the cache of parsed files is not released along with the eval state. This is usually not an issue, because it just dies with the process, but this could be improved but its reallocated when a new eval state is instantiated | 19:48:29 |
| Leonardo Santiago | In reply to @roberthensing:matrix.org
iirc the cache of parsed files is not released along with the eval state. This is usually not an issue, because it just dies with the process, but this could be improved but is it allocated again once a new evalstate is instantiated? | 19:49:07 |
| 31 Jan 2025 |
|  raboof changed their display name from raboof to raboof@FOSDEM. | 23:54:45 |
