| 24 May 2025 |
 Randy Eckenrode | Only 1%? | 17:59:21 |
 Tristan Ross | Would be hard to say if that's truly the performance impact | 18:00:28 |
| Tristan Ross | Best thing is to compare the variants stdenv with the cross stdenv that matches along and see how the evaluation compares. | 18:00:58 |
| 25 May 2025 |
| Tristan Ross | So I have a comparison between pkgsMusl.stdenv and crossStdenv.configs.aarch64-unknown-linux-musl | 01:37:56 |
| Tristan Ross | crossStdenv.configs.aarch64-unknown-linux-musl
{
"cpuTime": 0.43762001395225525,
"envs": {
"bytes": 5428488,
"elements": 369695,
"number": 308866
},
"gc": {
"cycles": 1,
"heapSize": 402915328,
"totalBytes": 79396896
},
"list": {
"bytes": 717400,
"concats": 4609,
"elements": 89675
},
"nrAvoided": 294217,
"nrExprs": 135112,
"nrFunctionCalls": 295668,
"nrLookups": 152167,
"nrOpUpdateValuesCopied": 2227726,
"nrOpUpdates": 12343,
"nrPrimOpCalls": 114176,
"nrThunks": 263414,
"sets": {
"bytes": 42449136,
"elements": 2626304,
"number": 26767
},
"sizes": {
"Attr": 16,
"Bindings": 16,
"Env": 8,
"Value": 24
},
"symbols": {
"bytes": 321885,
"number": 32150
},
"time": {
"cpu": 0.43762001395225525,
"gc": 0.043000000000000003,
"gcFraction": 0.09825876017793679
},
"values": {
"bytes": 20363664,
"number": 848486
}
}
pkgsMusl.stdenv
{
"cpuTime": 0.5026220083236694,
"envs": {
"bytes": 4656672,
"elements": 324861,
"number": 257223
},
"gc": {
"cycles": 1,
"heapSize": 402915328,
"totalBytes": 125050816
},
"list": {
"bytes": 601264,
"concats": 4389,
"elements": 75158
},
"nrAvoided": 262676,
"nrExprs": 131023,
"nrFunctionCalls": 242926,
"nrLookups": 125858,
"nrOpUpdateValuesCopied": 3930750,
"nrOpUpdates": 13551,
"nrPrimOpCalls": 104327,
"nrThunks": 322854,
"sets": {
"bytes": 73163712,
"elements": 4542912,
"number": 29820
},
"sizes": {
"Attr": 16,
"Bindings": 16,
"Env": 8,
"Value": 24
},
"symbols": {
"bytes": 320546,
"number": 32025
},
"time": {
"cpu": 0.5026220083236694,
"gc": 0.07200000000000001,
"gcFraction": 0.14324880090335151
},
"values": {
"bytes": 32448624,
"number": 1352026
}
}
| 01:38:49 |
 emily | looks still expensive enough that we'd want to avoid it wherever possible? | 02:29:09 |
| emily | it's not clear to me if Firefox could just use an unwrapped compiler though | 02:29:22 |
| Tristan Ross | crossStdenv.predicated.useLLVM.configs.aarch64-unknown-linux-gnu
{
"cpuTime": 0.8687459826469421,
"envs": {
"bytes": 12327288,
"elements": 839812,
"number": 701099
},
"gc": {
"cycles": 1,
"heapSize": 402915328,
"totalBytes": 124386272
},
"list": {
"bytes": 1597240,
"concats": 14954,
"elements": 199655
},
"nrAvoided": 651102,
"nrExprs": 190283,
"nrFunctionCalls": 673020,
"nrLookups": 340854,
"nrOpUpdateValuesCopied": 2985262,
"nrOpUpdates": 30307,
"nrPrimOpCalls": 244291,
"nrThunks": 553576,
"sets": {
"bytes": 58833760,
"elements": 3619956,
"number": 57154
},
"sizes": {
"Attr": 16,
"Bindings": 16,
"Env": 8,
"Value": 24
},
"symbols": {
"bytes": 445341,
"number": 42001
},
"time": {
"cpu": 0.8687459826469421,
"gc": 0.061,
"gcFraction": 0.07021615203806976
},
"values": {
"bytes": 31742712,
"number": 1322613
}
}
pkgsLLVM.stdenv
{
"cpuTime": 1.1511069536209106,
"envs": {
"bytes": 10916216,
"elements": 771025,
"number": 593502
},
"gc": {
"cycles": 1,
"heapSize": 402915328,
"totalBytes": 258382960
},
"list": {
"bytes": 3100896,
"concats": 16000,
"elements": 387612
},
"nrAvoided": 624277,
"nrExprs": 189716,
"nrFunctionCalls": 558370,
"nrLookups": 220405,
"nrOpUpdateValuesCopied": 7006441,
"nrOpUpdates": 39748,
"nrPrimOpCalls": 191955,
"nrThunks": 873127,
"sets": {
"bytes": 148522576,
"elements": 9080341,
"number": 202320
},
"sizes": {
"Attr": 16,
"Bindings": 16,
"Env": 8,
"Value": 24
},
"symbols": {
"bytes": 444992,
"number": 41976
},
"time": {
"cpu": 1.1511069536209106,
"gc": 0.202,
"gcFraction": 0.17548325927889744
},
"values": {
"bytes": 66434280,
"number": 2768095
}
}
| 02:33:13 |
| Tristan Ross | Yeah, ideally no cross when possible. | 02:33:26 |
| Tristan Ross | From what I was able to gleam when trying to test the wasm cross stdenv, it looks like it's only necessary for sandboxing things in a way? We could provide an option but idk if it's worth it. | 02:34:48 |
| Tristan Ross | crossStdenv.predicated.useLLVM.configs.wasm32-unknown-wasi
{
"cpuTime": 0.8086940050125122,
"envs": {
"bytes": 9276688,
"elements": 643991,
"number": 515595
},
"gc": {
"cycles": 1,
"heapSize": 402915328,
"totalBytes": 113596128
},
"list": {
"bytes": 1192696,
"concats": 14033,
"elements": 149087
},
"nrAvoided": 507605,
"nrExprs": 191950,
"nrFunctionCalls": 489091,
"nrLookups": 246990,
"nrOpUpdateValuesCopied": 2948940,
"nrOpUpdates": 28801,
"nrPrimOpCalls": 197474,
"nrThunks": 490984,
"sets": {
"bytes": 57271296,
"elements": 3526449,
"number": 53007
},
"sizes": {
"Attr": 16,
"Bindings": 16,
"Env": 8,
"Value": 24
},
"symbols": {
"bytes": 446123,
"number": 42055
},
"time": {
"cpu": 0.8086940050125122,
"gc": 0.101,
"gcFraction": 0.12489272750134622
},
"values": {
"bytes": 29123448,
"number": 1213477
}
}
pkgsCross.wasi32.stdenv
{
"cpuTime": 1.1445599794387817,
"envs": {
"bytes": 11093168,
"elements": 793274,
"number": 593372
},
"gc": {
"cycles": 1,
"heapSize": 402915328,
"totalBytes": 257411904
},
"list": {
"bytes": 3087112,
"concats": 17218,
"elements": 385889
},
"nrAvoided": 634315,
"nrExprs": 191436,
"nrFunctionCalls": 554713,
"nrLookups": 216783,
"nrOpUpdateValuesCopied": 6831370,
"nrOpUpdates": 46625,
"nrPrimOpCalls": 194292,
"nrThunks": 908524,
"sets": {
"bytes": 146253904,
"elements": 8929195,
"number": 211674
},
"sizes": {
"Attr": 16,
"Bindings": 16,
"Env": 8,
"Value": 24
},
"symbols": {
"bytes": 446471,
"number": 42080
},
"time": {
"cpu": 1.1445599794387817,
"gc": 0.20500000000000002,
"gcFraction": 0.17910813210550902
},
"values": {
"bytes": 67181760,
"number": 2799240
}
}
| 02:36:27 |
| emily | RLBox is important, I mean what's the reason it can't use an unwrappd multi-target Clang? | 02:50:40 |
| emily | it uses libcxx for WASM, I guess? but does it use anything else from pkgsCross? | 02:50:52 |
| Tristan Ross | Yeah, idk | 02:51:04 |
| Tristan Ross | I think it needs wasilibc | 02:52:01 |
| Tristan Ross | Ok yeah, it needs a wasm toolchain. It looks like upstream uses the wasi-sdk. | 02:54:37 |
| Randy Eckenrode | How does crossStdenv handle cross-compiled dependencies? DXVK needs SPIRV and Vulkan headers and windows.pthreads. | 13:39:07 |
| Randy Eckenrode | Also, how does it handle NIX_ vars? Does this even work with Darwin since Darwin needs an SDK? | 13:44:42 |
| Tristan Ross | In reply to @reckenrode:matrix.org
Also, how does it handle NIX_ vars? Does this even work with Darwin since Darwin needs an SDK? Suffix salt should work with the env vars. Idk about Darwin but it could be made to work. | 14:28:05 |
| Tristan Ross | In reply to @reckenrode:matrix.org
How does crossStdenv handle cross-compiled dependencies? DXVK needs SPIRV and Vulkan headers and windows.pthreads. You'll have to override the stdenv for the dependencies. | 14:28:23 |
| Randy Eckenrode | In reply to @rosscomputerguy:matrix.org
You'll have to override the stdenv for the dependencies. That doesn’t seem like an improvement. | 14:29:31 |
| Tristan Ross | In reply to @reckenrode:matrix.org
That doesn’t seem like an improvement. I don't see really any other way of doing cross without either creating a new nixpkgs instance (what we already have) or only provide cross compilation stdenv's | 14:30:30 |
| Randy Eckenrode | DXVK uses a custom MinGW (with the Win32 thread model instead of the POSIX one). When you build dxvk on Darwin or Linux, it uses the cross-compiled 32- and 64-bit MinGW DLLs.
https://github.com/NixOS/nixpkgs/blob/6fb6fd3ceff99de72d881f1a97773647c94e5a82/pkgs/by-name/dx/dxvk/package.nix#L28-L44
DXVK itself has dependencies: spirv-headers and vulkan-headers. Fortunately, those do not have transitive dependencies. (Arguably, DXVK has more dependencies, but I didn’t bother to de-vendor them.)
https://github.com/NixOS/nixpkgs/blob/6fb6fd3ceff99de72d881f1a97773647c94e5a82/pkgs/by-name/dx/dxvk_2/package.nix#L74-L81
The replacement sounds like I need to manually enumerate the transitive dependencies of DXVK, replace all of their stdenvs and make sure they get the right dependencies themselves (thankfully easy for DXVK since they have none themselves).
| 15:08:53 |
| Randy Eckenrode | crossStdenv is still expensive, but now the maintainability of my package has gone down because I have more work to do to build my package instead of just referencing pkgsCross.mingw32.dxvk_2 and pkgsCross.mingwW64.dxvk_2. | 15:09:59 |
| Randy Eckenrode | Note that it’s not possible to write a function to replace inputs that works in all cases because not all dependencies are in buildInputs (or propagatedBuildInputs). It’s the same problem that overrideSDK had. | 15:15:22 |
| Randy Eckenrode | * Note that it’s not possible to write a function to replace inputs that works in all cases because not all dependencies are in buildInputs (or propagatedBuildInputs). It’s the same problem that overrideSDK had. You’ll always need to know what dependencies your inputs have to make sure there are no problems. | 15:15:44 |
| Tristan Ross | In reply to @reckenrode:matrix.org
Note that it’s not possible to write a function to replace inputs that works in all cases because not all dependencies are in buildInputs (or propagatedBuildInputs). It’s the same problem that overrideSDK had. Yeah, I have thought about a recursiveStdenvOverride function which might work here | 15:30:31 |
| Tristan Ross | It would go through and apply the new stdenv where it should apply. | 15:30:54 |
| Randy Eckenrode | That can’t be done in the general case. Would it be possible to have a function like yesReallyDoInstantiateAnotherNixpkgsWithStdenc for non-trivial cases? | 15:31:47 |
| Randy Eckenrode | * That can’t be done in the general case. Would it be possible to have a function like yesReallyDoInstantiateAnotherNixpkgsWithStdenv for non-trivial cases? | 15:31:59 |
