!lTqhfnNvIkcEackxxk:matrix.org

Nix Bindings

72 Members
Nix C API https://github.com/NixOS/nix/blob/master/doc/manual/src/c-api.md, language-specific bindings, and their use cases (currently broken link: https://nixos.org/manual/nix/unstable/c-api)19 Servers

Load older messages

SenderMessageTime
8 Apr 2025
@o-santi:matrix.orgLeonardo SantiagoOne more weird detail is that the segfault is really consisten, happens everytime I run the program, but it doesn't happen always at the same place.12:33:59
@o-santi:matrix.orgLeonardo Santiago Sometimes it happens at nix::ExprSelect::eval, sometimes at nix::ExprAttr::eval, in the original message I think it happened at nix::ExprVar::eval 12:34:32
@o-santi:matrix.orgLeonardo Santiago So indeed this may be related to something that is shared/passed to all of them, most likely the EvalState itself, as I may be doing something incorrectly with it. 12:35:21
@o-santi:matrix.orgLeonardo SantiagoBut I don't understand what it is yet, I'll dig further12:35:50
@o-santi:matrix.orgLeonardo Santiago=> if this was a race condition I don't think it would be this reproducible, it would sometimes fail and sometimes not.12:36:47
@o-santi:matrix.orgLeonardo Santiago* One more weird detail is that the segfault is really consistent, happens everytime I run the program, but it doesn't happen always at the same place.12:36:55
@roberthensing:matrix.orgRobert Hensing (roberth)yeah12:37:53
@o-santi:matrix.orgLeonardo Santiago Guess what? Setting ulimit -s unlimited made it work. 16:29:35
@o-santi:matrix.orgLeonardo SantiagoIt was an uncaught stack overflow.16:29:45
@o-santi:matrix.orgLeonardo SantiagoDidn't even occur to me until now.16:29:57
10 Apr 2025
@o-santi:matrix.orgLeonardo Santiago @roberth how does nix circunvent this issue in their main binary? I see I can try leveraging ld's -z stack_size=X but it only seems to work if you set it in the entry point elf binary, which I can't do as it's python! I didn't want to bleed this problem elsewhere, like force people to set ulimit -s unlimited, but I don't see many other ways around it, and surely nix has had to deal with this; though it is the elf entry point. Any tips or hints? 13:21:26
@o-santi:matrix.orgLeonardo SantiagoThere's the possiblity of spawning a new thread with an increased stack size, but that adds the context switching overhead to every nix evaluation, which is something I'd like to avoid if possible.13:22:44
@o-santi:matrix.orgLeonardo Santiago * @roberth how does nix circunvent this issue in their main binary? I see I can try leveraging ld's -z stack_size=X but it only seems to work if you set it in the entry point elf binary, which I can't do as it's python and I'm merely offering a .so extension library. I didn't want to bleed this problem elsewhere, like force people to set ulimit -s unlimited, but I don't see many other ways around it, and surely nix has had to deal with this; though it is the elf entry point. Any tips or hints? 13:23:25
@o-santi:matrix.orgLeonardo Santiago There's also the possibility of using setrlimit to increase the process's own stack size, that seems to me like the most graceful solution 14:09:58
@o-santi:matrix.orgLeonardo Santiago And indeed, that seems the solution that nix's binary uses, src/nix/main.cc calls nix::setStackSize(64MB) which internally calls setrlimit with that value. Awesome to know, most likely I'll try going down this route. 14:12:10
15 Apr 2025
@o-santi:matrix.orgLeonardo SantiagoI notice I may be bringing all the stupid problems and ideas to the chat, but would it be possible to static link against the nix C libraries?15:55:05
@o-santi:matrix.orgLeonardo Santiago* I notice I may be the onebringing all the stupid problems and ideas to the chat, but would it be possible to static link against the nix C libraries?15:56:12
@o-santi:matrix.orgLeonardo Santiago* I notice I may be the one bringing all the stupid problems and ideas to the chat, but would it be possible to static link against the nix C libraries?15:56:14
@o-santi:matrix.orgLeonardo SantiagoI'm trying to optimize the performance of a custom eval cache using the C API I wrote and the cache hit is at around 20ms, which seems very good, but knowing what it does I know it's not really that impressive, since it's hashing some 20 files and querying on a sqlite file, 20ms is actually very poor performance if you consider it.15:57:49
@o-santi:matrix.orgLeonardo Santiago I tried perf recording and most of the time (15ms~ish) seems to be spent on do_lookup_x, which seems to be a libc function related to finding the dynamic libraries. 15:59:04
@o-santi:matrix.orgLeonardo Santiago * I tried perf recording and most of the time (15ms~ish) seems to be spent on do_lookup_x, which seems to be a libc function related to finding the dynamic libraries, and there are ~66 linked libraries, most related to nix stuff like libaws-c-sdkutils.so.1.0.0 which shouldn't even be used in this case but are loaded anyway before the program starts. 16:01:47
@o-santi:matrix.orgLeonardo Santiago If I remove most of the C API usage and just compile a simple binary to query the sqlite file it reduces down to 4 linked libraries and returns in ~6ms~ in --release mode, which seems to hint that indeed most of the time is spent finding the libraries 16:03:47
@o-santi:matrix.orgLeonardo Santiago

I'm using pkg-config to find and link against the C libraries, and it seems to have a .statik(true) option to static link each of them, but the moment I try to set it, it complains that it can't find them:

  The file `nix-expr-c.pc` needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
  PKG_CONFIG_PATH_FOR_TARGET contains the following:
      - /nix/store/f2q12pxzicc2739nsxg35yfzqqja1jbg-python3-3.12.8-env/lib/pkgconfig
      - /nix/store/7qxaap76irik1jqmqzgjwkrw3qs7mpcw-nix-store-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/wizqfb99xs3f4fh856bgp5qzvs76scbr-nix-util-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/9c9h0zbqa9xfk6pk4387l1h9pj28r692-nix-util-2.27.0-dev/lib/pkgconfig
      - /nix/store/b5fg5mqb5y1r1yvzf9paql1cr1iqy4ma-boost-1.81.0-dev/lib/pkgconfig
      - /nix/store/47zbszclyyy55nb5z7dvpcki4x5g99v4-libarchive-3.7.7-dev/lib/pkgconfig
      - /nix/store/rwrb0afm7z79720kb2kb84vyzcbslc5i-attr-2.5.2-dev/lib/pkgconfig
      - /nix/store/gsddsc9hs8l3j26r5958xcc5hhdf6hlf-acl-2.3.2-dev/lib/pkgconfig
      - /nix/store/c303g1m646jv9rir4zv49q1ggphsh9b5-nlohmann_json-3.11.3/share/pkgconfig
      - /nix/store/n29xgyyb4ig1sc275j8d9h7blx3g3jam-nix-store-2.27.0-dev/lib/pkgconfig
      - /nix/store/v8paqfgzs6gyznsxwmgwa798nr10yk11-nix-util-2.27.0-dev/lib/pkgconfig
      - /nix/store/xlw57h1x6r9l1kj8f25ij0n4j22i094n-nix-expr-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/x980i1fsgak6392wj9jxnr7d1yzcmpsd-nix-store-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/1fxkf2bvj1mcvrydx053bw7kyqnnplkg-nix-store-2.27.0-dev/lib/pkgconfig
      - /nix/store/0iv0kkf8hyzjv7rnxhbkvg8s0gqnk3h7-nix-expr-2.27.0-dev/lib/pkgconfig
      - /nix/store/sdx89ykxak1rdzxlg6qpb70n5bar354m-nix-fetchers-2.27.0-dev/lib/pkgconfig
      - /nix/store/6mn1mdcvv6rgyj8q2wh5q3v0riv3z3z1-boehm-gc-8.2.8-dev/lib/pkgconfig
      - /nix/store/ah7ylnwnsaibh71dnkxranzqahypy714-nix-util-c-2.27.0-dev/lib/pkgconfig
16:06:07
@o-santi:matrix.orgLeonardo Santiago *

I'm using pkg-config to find and link against the C libraries inside the build.rs, and it seems to have a .statik(true) option to static link each of them, but the moment I try to set it, it complains that it can't find them:

  The file `nix-expr-c.pc` needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
  PKG_CONFIG_PATH_FOR_TARGET contains the following:
      - /nix/store/f2q12pxzicc2739nsxg35yfzqqja1jbg-python3-3.12.8-env/lib/pkgconfig
      - /nix/store/7qxaap76irik1jqmqzgjwkrw3qs7mpcw-nix-store-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/wizqfb99xs3f4fh856bgp5qzvs76scbr-nix-util-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/9c9h0zbqa9xfk6pk4387l1h9pj28r692-nix-util-2.27.0-dev/lib/pkgconfig
      - /nix/store/b5fg5mqb5y1r1yvzf9paql1cr1iqy4ma-boost-1.81.0-dev/lib/pkgconfig
      - /nix/store/47zbszclyyy55nb5z7dvpcki4x5g99v4-libarchive-3.7.7-dev/lib/pkgconfig
      - /nix/store/rwrb0afm7z79720kb2kb84vyzcbslc5i-attr-2.5.2-dev/lib/pkgconfig
      - /nix/store/gsddsc9hs8l3j26r5958xcc5hhdf6hlf-acl-2.3.2-dev/lib/pkgconfig
      - /nix/store/c303g1m646jv9rir4zv49q1ggphsh9b5-nlohmann_json-3.11.3/share/pkgconfig
      - /nix/store/n29xgyyb4ig1sc275j8d9h7blx3g3jam-nix-store-2.27.0-dev/lib/pkgconfig
      - /nix/store/v8paqfgzs6gyznsxwmgwa798nr10yk11-nix-util-2.27.0-dev/lib/pkgconfig
      - /nix/store/xlw57h1x6r9l1kj8f25ij0n4j22i094n-nix-expr-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/x980i1fsgak6392wj9jxnr7d1yzcmpsd-nix-store-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/1fxkf2bvj1mcvrydx053bw7kyqnnplkg-nix-store-2.27.0-dev/lib/pkgconfig
      - /nix/store/0iv0kkf8hyzjv7rnxhbkvg8s0gqnk3h7-nix-expr-2.27.0-dev/lib/pkgconfig
      - /nix/store/sdx89ykxak1rdzxlg6qpb70n5bar354m-nix-fetchers-2.27.0-dev/lib/pkgconfig
      - /nix/store/6mn1mdcvv6rgyj8q2wh5q3v0riv3z3z1-boehm-gc-8.2.8-dev/lib/pkgconfig
      - /nix/store/ah7ylnwnsaibh71dnkxranzqahypy714-nix-util-c-2.27.0-dev/lib/pkgconfig
16:06:18
@o-santi:matrix.orgLeonardo Santiago *

I'm using pkg-config to find and link against the C libraries inside the build.rs, and it seems to have a .statik(true) option to static link each of them, but the moment I try to set it, it complains that it can't find them even though the lib/pkgconfig path seems to be correctly configured:

  The file `nix-expr-c.pc` needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
  PKG_CONFIG_PATH_FOR_TARGET contains the following:
      - /nix/store/f2q12pxzicc2739nsxg35yfzqqja1jbg-python3-3.12.8-env/lib/pkgconfig
      - /nix/store/7qxaap76irik1jqmqzgjwkrw3qs7mpcw-nix-store-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/wizqfb99xs3f4fh856bgp5qzvs76scbr-nix-util-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/9c9h0zbqa9xfk6pk4387l1h9pj28r692-nix-util-2.27.0-dev/lib/pkgconfig
      - /nix/store/b5fg5mqb5y1r1yvzf9paql1cr1iqy4ma-boost-1.81.0-dev/lib/pkgconfig
      - /nix/store/47zbszclyyy55nb5z7dvpcki4x5g99v4-libarchive-3.7.7-dev/lib/pkgconfig
      - /nix/store/rwrb0afm7z79720kb2kb84vyzcbslc5i-attr-2.5.2-dev/lib/pkgconfig
      - /nix/store/gsddsc9hs8l3j26r5958xcc5hhdf6hlf-acl-2.3.2-dev/lib/pkgconfig
      - /nix/store/c303g1m646jv9rir4zv49q1ggphsh9b5-nlohmann_json-3.11.3/share/pkgconfig
      - /nix/store/n29xgyyb4ig1sc275j8d9h7blx3g3jam-nix-store-2.27.0-dev/lib/pkgconfig
      - /nix/store/v8paqfgzs6gyznsxwmgwa798nr10yk11-nix-util-2.27.0-dev/lib/pkgconfig
      - /nix/store/xlw57h1x6r9l1kj8f25ij0n4j22i094n-nix-expr-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/x980i1fsgak6392wj9jxnr7d1yzcmpsd-nix-store-c-2.27.0-dev/lib/pkgconfig
      - /nix/store/1fxkf2bvj1mcvrydx053bw7kyqnnplkg-nix-store-2.27.0-dev/lib/pkgconfig
      - /nix/store/0iv0kkf8hyzjv7rnxhbkvg8s0gqnk3h7-nix-expr-2.27.0-dev/lib/pkgconfig
      - /nix/store/sdx89ykxak1rdzxlg6qpb70n5bar354m-nix-fetchers-2.27.0-dev/lib/pkgconfig
      - /nix/store/6mn1mdcvv6rgyj8q2wh5q3v0riv3z3z1-boehm-gc-8.2.8-dev/lib/pkgconfig
      - /nix/store/ah7ylnwnsaibh71dnkxranzqahypy714-nix-util-c-2.27.0-dev/lib/pkgconfig
16:06:52
16 Apr 2025
@alexandrutocar:matrix.orgAlexandru Tocar joined the room.11:47:53
1 May 2025
@rosariopulella:matrix.orgRosario Pulella changed their display name from Rosario Pulella to Rosuavio.20:08:52
2 May 2025
@zmitchell:matrix.orgzmitchell joined the room.20:13:39
17 May 2025
@terrorjack:matrix.orgterrorjack set a profile picture.08:53:49
29 May 2025
@o-santi:matrix.orgLeonardo Santiago

I'm getting this very weird error when trying to update nix to the latest version:

 error: linking with `cc` failed: exit status: 1
(...)
/nix/store/yrrg35ii81mlcfkymv45amlxshd2a9sa-nix-util-2.29.0pre/lib/libnixutil.so: undefined reference to `__cxa_call_terminate@CXXABI_1.3.15'
13:25:32

Show newer messages

Back to Room ListRoom Version: 10