!lTqhfnNvIkcEackxxk:matrix.org

Nix Bindings

71 Members
Nix C API https://github.com/NixOS/nix/blob/master/doc/manual/src/c-api.md, language-specific bindings, and their use cases (currently broken link: https://nixos.org/manual/nix/unstable/c-api)19 Servers

Load older messages

SenderMessageTime
7 Apr 2025
@o-santi:matrix.orgLeonardo Santiago *

This is what I did, and if I change nix-expr-c for nix.libs.nix-expr-c it goes back to segfaulting

inputs = { 
  nix.url` = "github:nixos/nix/2.27.0;
};
outputs = { ... } : { 
...
nix = inputs.nix.packages.${system}.nix;
mkDebug = p: p.overrideAttrs (old: old // {                                                                                                           
          dontStrip = true;                                                                                                                                   
          mesonBuildType = "debugoptimized";                                                                                                                  
          NIX_CFLAGS_COMPILE = toString (old.NIX_CFLAGS_COMPILE or "") + " -ggdb -Og";                                                                        
});                                                                                                                                                   
nix-expr-c = (mkDebug nix.libs.nix-expr-c).override { nix-expr = mkDebug nix.libs.nix-expr; };
nix-deps = map mkDebug [ nix.libs.nix-store-c nix-expr-c nix.libs.nix-util-c ];
16:24:48
@o-santi:matrix.orgLeonardo Santiago *

This is what I did, and if I change nix-expr-c for nix.libs.nix-expr-c it goes back to segfaulting

inputs = { 
  nix.url` = "github:nixos/nix/2.27.0";
};
outputs = { ... } : { 
...
nix = inputs.nix.packages.${system}.nix;
mkDebug = p: p.overrideAttrs (old: old // {                                                                                                           
          dontStrip = true;                                                                                                                                   
          mesonBuildType = "debugoptimized";                                                                                                                  
          NIX_CFLAGS_COMPILE = toString (old.NIX_CFLAGS_COMPILE or "") + " -ggdb -Og";                                                                        
});                                                                                                                                                   
nix-expr-c = (mkDebug nix.libs.nix-expr-c).override { nix-expr = mkDebug nix.libs.nix-expr; };
nix-deps = map mkDebug [ nix.libs.nix-store-c nix-expr-c nix.libs.nix-util-c ];
16:24:52
@o-santi:matrix.orgLeonardo Santiago *

This is what I did, and if I change nix-expr-c for nix.libs.nix-expr-c it goes back to segfaulting

inputs = { 
  nix.url = "github:nixos/nix/2.27.0";
};
outputs = { ... } : { 
...
nix = inputs.nix.packages.${system}.nix;
mkDebug = p: p.overrideAttrs (old: old // {                                                                                                           
          dontStrip = true;                                                                                                                                   
          mesonBuildType = "debugoptimized";                                                                                                                  
          NIX_CFLAGS_COMPILE = toString (old.NIX_CFLAGS_COMPILE or "") + " -ggdb -Og";                                                                        
});                                                                                                                                                   
nix-expr-c = (mkDebug nix.libs.nix-expr-c).override { nix-expr = mkDebug nix.libs.nix-expr; };
nix-deps = map mkDebug [ nix.libs.nix-store-c nix-expr-c nix.libs.nix-util-c ];
16:24:58
@o-santi:matrix.orgLeonardo Santiago Robert Hensing (roberth): Is this UB? 16:27:29
@o-santi:matrix.orgLeonardo Santiago * Robert Hensing (roberth): Could this be UB? 16:41:45
@o-santi:matrix.orgLeonardo Santiago

Removing mesonBuildType and NIX_CFLAGS_COMPILE and keeping only dontStrip finally made it segfault with debug symbols:

#0  0x00007ffff68ac8d7 in nix::EvalState::callFunction (this=0x5b7b00, fun=..., args=..., vRes=..., pos=...) at <artificial>:1507                           
#1  0x00007ffff68b034c in nix::ExprCall::eval (this=0x648580, state=..., env=..., v=...) at ../stl_iterator.h:90                                            
#2  0x00007ffff68af62b in nix::EvalState::forceValue (pos=..., v=..., this=0x5b7b00) at /build/source/src/libexpr/build/value.hh:96                         
#3  nix::ExprSelect::eval (this=0x648540, state=..., env=..., v=...) at <artificial>:1443                                                                   
#4  0x00007ffff68af158 in nix::EvalState::forceValue (pos=..., v=..., this=0x5b7b00) at /build/source/src/libexpr/build/value.hh:96                         
#5  nix::ExprVar::eval (this=<optimized out>, state=..., env=..., v=...) at <artificial>:1372                                                               
#6  0x00007ffff68af3d9 in nix::ExprSelect::eval (this=0x17f49f0, state=..., env=..., v=...) at <artificial>:1402                                            
#7  0x00007ffff68b0ed1 in nix::ExprOpConcatLists::eval (this=0x17f4a30, state=..., env=..., v=...) at <artificial>:1948                                     
#8  0x00007ffff6905617 in nix::EvalState::forceValue (pos=..., v=..., this=<optimized out>, this=<optimized out>, v=..., pos=...)                           
    at /nix/store/9c9h0zbqa9xfk6pk4387l1h9pj28r692-nix-util-2.27.0-dev/include/nix/aligned_buffer.h:96                                                      
#9  nix::EvalState::forceList (errorCtx=..., pos=..., v=..., this=<optimized out>, this=<optimized out>, v=..., pos=..., errorCtx=...)                      
    at /nix/store/9c9h0zbqa9xfk6pk4387l1h9pj28r692-nix-util-2.27.0-dev/include/nix/aligned_buffer.h:136                                                     
#10 nix::prim_foldlStrict (state=..., pos=..., args=0x7fffff670c00, v=...) at /build/source/src/libexpr/build/new_allocator.h:3485
16:48:55
@roberthensing:matrix.orgRobert Hensing (roberth) I've never seen something like <artificial>:1507 before 20:23:52
@roberthensing:matrix.orgRobert Hensing (roberth) maybe it's just eval.cc? 20:25:14
@roberthensing:matrix.orgRobert Hensing (roberth) can't glean much from this, but maybe this happens due to a missing garbage collection root (erroneous decref / missing incref?) or any other kind of corruption 20:26:35
@roberthensing:matrix.orgRobert Hensing (roberth)a change to code layout may well have knock-on effects that cause a corruption to happen (all else equal). That's regardless of whether your overrides produce a valid binary, which I do not know.20:28:39
@roberthensing:matrix.orgRobert Hensing (roberth) It's like overrideAttrs but applies to all the components 20:29:11
8 Apr 2025
@o-santi:matrix.orgLeonardo Santiagome neither hahaha12:18:49
@o-santi:matrix.orgLeonardo Santiago I don't think so, I'm relying on Drop from Rust and am not really messing with the GC of nix. 12:19:47
@o-santi:matrix.orgLeonardo Santiago This only seems to happen when I use it through the pyo3 adapter, from the python side, which kind of hints me that's something there. The part where I'm most afraid is that pyo3 requires all public structs to be Send, and mine clearly aren't as they hold raw pointers, so I wrapped everything public in an Arc<Mutex<T>> and it seemed to work fine until now. 12:21:43
@roberthensing:matrix.orgRobert Hensing (roberth)Ohh, Rust wasn't enough? ^^'12:22:27
@o-santi:matrix.orgLeonardo SantiagoWdym? If I try to evaluate the exact same expression through the Rust side it works fine.12:22:49
@roberthensing:matrix.orgRobert Hensing (roberth)Didn't mean anything specific with that12:23:11
@roberthensing:matrix.orgRobert Hensing (roberth)Did you publish your Rust bindings btw? Might be interesting to merge efforts12:24:19
@o-santi:matrix.orgLeonardo SantiagoYes, github.com/o-santi/nix-forall12:24:38
@o-santi:matrix.orgLeonardo SantiagoI took some inspiration from yours at some places hahaha, specially at the read_string/read_hashmap callbacks12:25:26
@o-santi:matrix.orgLeonardo SantiagoAnd I think yours is much more careful when handling thread locking and GC. I don't really understand the constraints there so maybe there's something wrong I did that caused this.12:26:45
@roberthensing:matrix.orgRobert Hensing (roberth) If everything is on the main thread you're fine, and nix_value registers/deregisters itself with the GC just fine (if all is well), but the GC may not be happy if it needs to operate from a thread it doesn't know about. That includes allocation, so for all intents and purposes that's the whole of nix-expr that should be called from registered threads (or the main thread) only 12:29:59
@roberthensing:matrix.orgRobert Hensing (roberth) You'd get an error message along the lines of "trying to GC from unknown thread". I don't think it can cause corruption necessarily 12:30:37
@roberthensing:matrix.orgRobert Hensing (roberth)but maybe my assumption about GC roots is wrong, and my code does rely on stack scanning regardless12:31:07
@o-santi:matrix.orgLeonardo Santiago I think the problem may lie there, pyo3 requires that all your structs be freely movable between threads, as python is not really a single threaded interpreter, it just heavily relies on the GIL. I'm not doing anything multithreaded from the python side, much to the contrary, I'm just state.eval_file('path').get(attr) but I wouldn't say it isn't moving it to another thread either 12:32:48
@roberthensing:matrix.orgRobert Hensing (roberth)Thing is, you might get away with coincidentally not triggering GC in your other threads / stacks12:32:58
@o-santi:matrix.orgLeonardo SantiagoOne more weird detail is that the segfault is really consisten, happens everytime I run the program, but it doesn't happen always at the same place.12:33:59
@o-santi:matrix.orgLeonardo Santiago Sometimes it happens at nix::ExprSelect::eval, sometimes at nix::ExprAttr::eval, in the original message I think it happened at nix::ExprVar::eval 12:34:32
@o-santi:matrix.orgLeonardo Santiago So indeed this may be related to something that is shared/passed to all of them, most likely the EvalState itself, as I may be doing something incorrectly with it. 12:35:21
@o-santi:matrix.orgLeonardo SantiagoBut I don't understand what it is yet, I'll dig further12:35:50

Show newer messages

Back to Room ListRoom Version: 10