| 10 Apr 2025 |
 Leonardo Santiago | @roberth how does nix circunvent this issue in their main binary? I see I can try leveraging ld's -z stack_size=X but it only seems to work if you set it in the entry point elf binary, which I can't do as it's python! I didn't want to bleed this problem elsewhere, like force people to set ulimit -s unlimited, but I don't see many other ways around it, and surely nix has had to deal with this; though it is the elf entry point. Any tips or hints? | 13:21:26 |
| Leonardo Santiago | There's the possiblity of spawning a new thread with an increased stack size, but that adds the context switching overhead to every nix evaluation, which is something I'd like to avoid if possible. | 13:22:44 |
| Leonardo Santiago | * @roberth how does nix circunvent this issue in their main binary? I see I can try leveraging ld's -z stack_size=X but it only seems to work if you set it in the entry point elf binary, which I can't do as it's python and I'm merely offering a .so extension library. I didn't want to bleed this problem elsewhere, like force people to set ulimit -s unlimited, but I don't see many other ways around it, and surely nix has had to deal with this; though it is the elf entry point. Any tips or hints? | 13:23:25 |
| Leonardo Santiago | There's also the possibility of using setrlimit to increase the process's own stack size, that seems to me like the most graceful solution | 14:09:58 |
| Leonardo Santiago | And indeed, that seems the solution that nix's binary uses, src/nix/main.cc calls nix::setStackSize(64MB) which internally calls setrlimit with that value. Awesome to know, most likely I'll try going down this route. | 14:12:10 |
| 15 Apr 2025 |
| Leonardo Santiago | I notice I may be bringing all the stupid problems and ideas to the chat, but would it be possible to static link against the nix C libraries? | 15:55:05 |
| Leonardo Santiago | * I notice I may be the onebringing all the stupid problems and ideas to the chat, but would it be possible to static link against the nix C libraries? | 15:56:12 |
| Leonardo Santiago | * I notice I may be the one bringing all the stupid problems and ideas to the chat, but would it be possible to static link against the nix C libraries? | 15:56:14 |
| Leonardo Santiago | I'm trying to optimize the performance of a custom eval cache using the C API I wrote and the cache hit is at around 20ms, which seems very good, but knowing what it does I know it's not really that impressive, since it's hashing some 20 files and querying on a sqlite file, 20ms is actually very poor performance if you consider it. | 15:57:49 |
| Leonardo Santiago | I tried perf recording and most of the time (15ms~ish) seems to be spent on do_lookup_x, which seems to be a libc function related to finding the dynamic libraries. | 15:59:04 |
| Leonardo Santiago | * I tried perf recording and most of the time (15ms~ish) seems to be spent on do_lookup_x, which seems to be a libc function related to finding the dynamic libraries, and there are ~66 linked libraries, most related to nix stuff like libaws-c-sdkutils.so.1.0.0 which shouldn't even be used in this case but are loaded anyway before the program starts. | 16:01:47 |
| Leonardo Santiago | If I remove most of the C API usage and just compile a simple binary to query the sqlite file it reduces down to 4 linked libraries and returns in ~6ms~ in --release mode, which seems to hint that indeed most of the time is spent finding the libraries | 16:03:47 |
| Leonardo Santiago | I'm using pkg-config to find and link against the C libraries, and it seems to have a .statik(true) option to static link each of them, but the moment I try to set it, it complains that it can't find them:
The file `nix-expr-c.pc` needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
PKG_CONFIG_PATH_FOR_TARGET contains the following:
- /nix/store/f2q12pxzicc2739nsxg35yfzqqja1jbg-python3-3.12.8-env/lib/pkgconfig
- /nix/store/7qxaap76irik1jqmqzgjwkrw3qs7mpcw-nix-store-c-2.27.0-dev/lib/pkgconfig
- /nix/store/wizqfb99xs3f4fh856bgp5qzvs76scbr-nix-util-c-2.27.0-dev/lib/pkgconfig
- /nix/store/9c9h0zbqa9xfk6pk4387l1h9pj28r692-nix-util-2.27.0-dev/lib/pkgconfig
- /nix/store/b5fg5mqb5y1r1yvzf9paql1cr1iqy4ma-boost-1.81.0-dev/lib/pkgconfig
- /nix/store/47zbszclyyy55nb5z7dvpcki4x5g99v4-libarchive-3.7.7-dev/lib/pkgconfig
- /nix/store/rwrb0afm7z79720kb2kb84vyzcbslc5i-attr-2.5.2-dev/lib/pkgconfig
- /nix/store/gsddsc9hs8l3j26r5958xcc5hhdf6hlf-acl-2.3.2-dev/lib/pkgconfig
- /nix/store/c303g1m646jv9rir4zv49q1ggphsh9b5-nlohmann_json-3.11.3/share/pkgconfig
- /nix/store/n29xgyyb4ig1sc275j8d9h7blx3g3jam-nix-store-2.27.0-dev/lib/pkgconfig
- /nix/store/v8paqfgzs6gyznsxwmgwa798nr10yk11-nix-util-2.27.0-dev/lib/pkgconfig
- /nix/store/xlw57h1x6r9l1kj8f25ij0n4j22i094n-nix-expr-c-2.27.0-dev/lib/pkgconfig
- /nix/store/x980i1fsgak6392wj9jxnr7d1yzcmpsd-nix-store-c-2.27.0-dev/lib/pkgconfig
- /nix/store/1fxkf2bvj1mcvrydx053bw7kyqnnplkg-nix-store-2.27.0-dev/lib/pkgconfig
- /nix/store/0iv0kkf8hyzjv7rnxhbkvg8s0gqnk3h7-nix-expr-2.27.0-dev/lib/pkgconfig
- /nix/store/sdx89ykxak1rdzxlg6qpb70n5bar354m-nix-fetchers-2.27.0-dev/lib/pkgconfig
- /nix/store/6mn1mdcvv6rgyj8q2wh5q3v0riv3z3z1-boehm-gc-8.2.8-dev/lib/pkgconfig
- /nix/store/ah7ylnwnsaibh71dnkxranzqahypy714-nix-util-c-2.27.0-dev/lib/pkgconfig
| 16:06:07 |
| Leonardo Santiago | * I'm using pkg-config to find and link against the C libraries inside the build.rs, and it seems to have a .statik(true) option to static link each of them, but the moment I try to set it, it complains that it can't find them:
The file `nix-expr-c.pc` needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
PKG_CONFIG_PATH_FOR_TARGET contains the following:
- /nix/store/f2q12pxzicc2739nsxg35yfzqqja1jbg-python3-3.12.8-env/lib/pkgconfig
- /nix/store/7qxaap76irik1jqmqzgjwkrw3qs7mpcw-nix-store-c-2.27.0-dev/lib/pkgconfig
- /nix/store/wizqfb99xs3f4fh856bgp5qzvs76scbr-nix-util-c-2.27.0-dev/lib/pkgconfig
- /nix/store/9c9h0zbqa9xfk6pk4387l1h9pj28r692-nix-util-2.27.0-dev/lib/pkgconfig
- /nix/store/b5fg5mqb5y1r1yvzf9paql1cr1iqy4ma-boost-1.81.0-dev/lib/pkgconfig
- /nix/store/47zbszclyyy55nb5z7dvpcki4x5g99v4-libarchive-3.7.7-dev/lib/pkgconfig
- /nix/store/rwrb0afm7z79720kb2kb84vyzcbslc5i-attr-2.5.2-dev/lib/pkgconfig
- /nix/store/gsddsc9hs8l3j26r5958xcc5hhdf6hlf-acl-2.3.2-dev/lib/pkgconfig
- /nix/store/c303g1m646jv9rir4zv49q1ggphsh9b5-nlohmann_json-3.11.3/share/pkgconfig
- /nix/store/n29xgyyb4ig1sc275j8d9h7blx3g3jam-nix-store-2.27.0-dev/lib/pkgconfig
- /nix/store/v8paqfgzs6gyznsxwmgwa798nr10yk11-nix-util-2.27.0-dev/lib/pkgconfig
- /nix/store/xlw57h1x6r9l1kj8f25ij0n4j22i094n-nix-expr-c-2.27.0-dev/lib/pkgconfig
- /nix/store/x980i1fsgak6392wj9jxnr7d1yzcmpsd-nix-store-c-2.27.0-dev/lib/pkgconfig
- /nix/store/1fxkf2bvj1mcvrydx053bw7kyqnnplkg-nix-store-2.27.0-dev/lib/pkgconfig
- /nix/store/0iv0kkf8hyzjv7rnxhbkvg8s0gqnk3h7-nix-expr-2.27.0-dev/lib/pkgconfig
- /nix/store/sdx89ykxak1rdzxlg6qpb70n5bar354m-nix-fetchers-2.27.0-dev/lib/pkgconfig
- /nix/store/6mn1mdcvv6rgyj8q2wh5q3v0riv3z3z1-boehm-gc-8.2.8-dev/lib/pkgconfig
- /nix/store/ah7ylnwnsaibh71dnkxranzqahypy714-nix-util-c-2.27.0-dev/lib/pkgconfig
| 16:06:18 |
| Leonardo Santiago | * I'm using pkg-config to find and link against the C libraries inside the build.rs, and it seems to have a .statik(true) option to static link each of them, but the moment I try to set it, it complains that it can't find them even though the lib/pkgconfig path seems to be correctly configured:
The file `nix-expr-c.pc` needs to be installed and the PKG_CONFIG_PATH environment variable must contain its parent directory.
PKG_CONFIG_PATH_FOR_TARGET contains the following:
- /nix/store/f2q12pxzicc2739nsxg35yfzqqja1jbg-python3-3.12.8-env/lib/pkgconfig
- /nix/store/7qxaap76irik1jqmqzgjwkrw3qs7mpcw-nix-store-c-2.27.0-dev/lib/pkgconfig
- /nix/store/wizqfb99xs3f4fh856bgp5qzvs76scbr-nix-util-c-2.27.0-dev/lib/pkgconfig
- /nix/store/9c9h0zbqa9xfk6pk4387l1h9pj28r692-nix-util-2.27.0-dev/lib/pkgconfig
- /nix/store/b5fg5mqb5y1r1yvzf9paql1cr1iqy4ma-boost-1.81.0-dev/lib/pkgconfig
- /nix/store/47zbszclyyy55nb5z7dvpcki4x5g99v4-libarchive-3.7.7-dev/lib/pkgconfig
- /nix/store/rwrb0afm7z79720kb2kb84vyzcbslc5i-attr-2.5.2-dev/lib/pkgconfig
- /nix/store/gsddsc9hs8l3j26r5958xcc5hhdf6hlf-acl-2.3.2-dev/lib/pkgconfig
- /nix/store/c303g1m646jv9rir4zv49q1ggphsh9b5-nlohmann_json-3.11.3/share/pkgconfig
- /nix/store/n29xgyyb4ig1sc275j8d9h7blx3g3jam-nix-store-2.27.0-dev/lib/pkgconfig
- /nix/store/v8paqfgzs6gyznsxwmgwa798nr10yk11-nix-util-2.27.0-dev/lib/pkgconfig
- /nix/store/xlw57h1x6r9l1kj8f25ij0n4j22i094n-nix-expr-c-2.27.0-dev/lib/pkgconfig
- /nix/store/x980i1fsgak6392wj9jxnr7d1yzcmpsd-nix-store-c-2.27.0-dev/lib/pkgconfig
- /nix/store/1fxkf2bvj1mcvrydx053bw7kyqnnplkg-nix-store-2.27.0-dev/lib/pkgconfig
- /nix/store/0iv0kkf8hyzjv7rnxhbkvg8s0gqnk3h7-nix-expr-2.27.0-dev/lib/pkgconfig
- /nix/store/sdx89ykxak1rdzxlg6qpb70n5bar354m-nix-fetchers-2.27.0-dev/lib/pkgconfig
- /nix/store/6mn1mdcvv6rgyj8q2wh5q3v0riv3z3z1-boehm-gc-8.2.8-dev/lib/pkgconfig
- /nix/store/ah7ylnwnsaibh71dnkxranzqahypy714-nix-util-c-2.27.0-dev/lib/pkgconfig
| 16:06:52 |
| 16 Apr 2025 |
|  Alexandru Tocar joined the room. | 11:47:53 |
| 1 May 2025 |
|  Rosuavio changed their display name from Rosario Pulella to Rosuavio. | 20:08:52 |
| 2 May 2025 |
|  zmitchell joined the room. | 20:13:39 |
| 17 May 2025 |
|  terrorjack set a profile picture. | 08:53:49 |
