| 24 Dec 2024 |
trofi | Surprisingly adding a no-op change like:
--- a/pkgs/servers/sql/postgresql/generic.nix
+++ b/pkgs/servers/sql/postgresql/generic.nix
@@ -447,0 +448,2 @@ let
+ FOO = "42";
+
fixes the build. As if there is some ordering issue in nix to discover known outputs.
| 22:25:47 |
matthewcroughan @fosdem | There is currently no way to pass auth headers into filetransfer.cc, open PRs/Issues about it, years of history about authentication being unsolved, how come? | 22:29:02 |
matthewcroughan @fosdem | https://ec.haxx.se/libcurl-http/auth.html#bearer | 22:29:16 |
matthewcroughan @fosdem | All that must be done is passing CURLOPT_XOAUTH2_BEARER when present | 22:29:31 |
matthewcroughan @fosdem | yet this is only supported for gitlab fetches | 22:29:47 |
| 25 Dec 2024 |
| Ananya S Kaligal joined the room. | 13:20:21 |
| Tanja (she/her) changed their display name from Tanja (she/her) to Tanja (she/her) [DECT 6929]. | 14:48:10 |
| hackerman_1337 changed their display name from hackerman to hackerman_1337. | 19:06:51 |
trofi | Attempted to improve an error message as https://github.com/NixOS/nix/pull/12105. Looks like nix losk man output when built a goal with man output. I wonder if outputChecks does not specify some kind of dependency for string references luke plain "man". | 22:07:02 |
trofi | * Attempted to improve an error message as https://github.com/NixOS/nix/pull/12105. Looks like nix lotk man output when built a goal with man output. I wonder if outputChecks does not specify some kind of dependency for string references luke plain "man". | 22:07:19 |
| 26 Dec 2024 |
| @10leej:matrix.org left the room. | 01:41:45 |
| gasu joined the room. | 05:33:52 |
trofi | Fun fact: -D_GLIBCXX_DEBUG crashes nix-env as std::stable_sort predicate does not maintain !(a < a) property: https://github.com/NixOS/nix/issues/12106 | 07:46:10 |
Sergei Zimmerman (xokdvium) | What should builtins.sort (a: b: true) [ 1 2 3 ]even do? Should this be a nix language-level UB? Currently manual does not provide any preconditions https://nix.dev/manual/nix/2.25/language/builtins.html?highlight=builtins.sort#builtins-sort.
This would be a tricky thing to fix. Failing gracefully would require validating the strict weak ordering or using sorting algorithms that do not explode when the comparator is broken. std::sort can easily explode since it's something like qsort. stable_sort is actually a combination of insertion sort with merge sort on top. https://gcc.gnu.org/onlinedocs/libstdc++/libstdc++-html-USERS-4.4/a01347.html#l03409
I think using a custom merge sort would be safe even for broken predicates and won't crash nix evaluator, but this would still need to be nix UB (such calls to sort are ill-formed and diagnosing it is prohibitively expensive).
Either way this calls for a custom sorting implementation that does not assume that the provided predicate is strict weak ordering.
| 10:17:33 |
Sergei Zimmerman (xokdvium) | What's more worrying is the fact that somewhere in nixpkgs there are broken comparator predicates. | 10:27:59 |
@aloisw:julia0815.de | In reply to @xokdvium:matrix.org
What should builtins.sort (a: b: true) [ 1 2 3 ]even do? Should this be a nix language-level UB? Currently manual does not provide any preconditions https://nix.dev/manual/nix/2.25/language/builtins.html?highlight=builtins.sort#builtins-sort.
This would be a tricky thing to fix. Failing gracefully would require validating the strict weak ordering or using sorting algorithms that do not explode when the comparator is broken. std::sort can easily explode since it's something like qsort. stable_sort is actually a combination of insertion sort with merge sort on top. https://gcc.gnu.org/onlinedocs/libstdc++/libstdc++-html-USERS-4.4/a01347.html#l03409
I think using a custom merge sort would be safe even for broken predicates and won't crash nix evaluator, but this would still need to be nix UB (such calls to sort are ill-formed and diagnosing it is prohibitively expensive).
Either way this calls for a custom sorting implementation that does not assume that the provided predicate is strict weak ordering.
It currently triggers C++ UB. | 10:29:30 |
Sergei Zimmerman (xokdvium) | Yup, that's clearly a huge problem and this should be fixed by not using standard sorting algorithms:
Either way this calls for a custom sorting implementation that does not assume that the provided predicate is strict weak ordering.
What I'm trying to say is that this should still be UB in the nix language, but not crash and burn the evaluator.
| 10:31:14 |
@aloisw:julia0815.de | I don't think there should be UB in the Nix language, unspecified result should be good enough. | 10:32:15 |
Sergei Zimmerman (xokdvium) | Yeah, that makes more sense. Though I hope there could be a better way to find the broken usages in nixpkgs. Unspecified result makes the eval non-deterministic by nature. Does this mean that if this bug gets fixed and replaced with unspecified order instead of C++ UB nixpkgs would suddenly eval differently? | 10:37:58 |
@aloisw:julia0815.de | I don't think it is possible to determine in general whether a given comparator is broken. And I think literally depending on C++ UB is one of the cases where breaking evaluation compatibility is acceptable. | 10:40:06 |
Sergei Zimmerman (xokdvium) | At the very least we could first trace the problematic usages with a custom cppnix build and evaluate the scope of the disaster. Checking strict weak order properties by hand + random shuffling before the sort should surface at least some problems. | 10:43:29 |
Sergei Zimmerman (xokdvium) | Though random shuffling would have to preserve the order, since the sort is stable. | 10:44:30 |
@aloisw:julia0815.de | How do you know what the problematic usages are in the first place? | 10:44:43 |
Sergei Zimmerman (xokdvium) | See the original https://github.com/NixOS/nix/issues/12106#issue-2759417825 issue. I just reproduced the problem on my end on nixpkgs master (9a90c6b6218860ab29f79ecd6213c3a547c15c7e):
../nixpkgs-check-strict-weak-order/result/bin/nix-env -qa --json --arg config 'import pkgs/top-level/packages-config.nix'
/nix/store/4KRAB2H0HD4WVXXMSCXRW21PL77J4I7J-gcc-13.3.0/include/c++/13.3.0/bits/stl_algo.h:5117:
In function:
void std::stable_sort(_RAIter, _RAIter, _Compare) [with _RAIter =
nix::Value**; _Compare = nix::prim_sort(EvalState&, PosIdx, Value**,
Value&)::<lambda(nix::Value*, nix::Value*)>]
Error: comparison doesn't meet irreflexive requirements, assert(!(a < a)).
Objects involved in the operation:
instance "functor" @ 0x7ffff3cb8c90 {
type = nix::prim_sort(nix::EvalState&, nix::PosIdx, nix::Value**, nix::Value&)::{lambda(nix::Value*, nix::Value*)#1};
}
iterator::value_type "ordered type" {
type = nix::Value*;
}
[1] 2675868 abort (core dumped) ../nixpkgs-check-strict-weak-order/result/bin/nix-env -qa --json --arg config
| 11:14:58 |
Sergei Zimmerman (xokdvium) | I'm not sure that if that actually explodes with normal usage though | 11:26:03 |
Sergei Zimmerman (xokdvium) | A quick glance also shows that this gets triggered in tests.coq, so this is not a one-off problem:
nix-instantiate -A tests.coq
/nix/store/4KRAB2H0HD4WVXXMSCXRW21PL77J4I7J-gcc-13.3.0/include/c++/13.3.0/bits/stl_algo.h:5117:
In function:
void std::stable_sort(_RAIter, _RAIter, _Compare) [with _RAIter =
nix::Value**; _Compare = nix::prim_sort(EvalState&, PosIdx, Value**,
Value&)::<lambda(nix::Value*, nix::Value*)>]
Error: comparison doesn't meet irreflexive requirements, assert(!(a < a)).
Objects involved in the operation:
instance "functor" @ 0x7ffc7a9bef10 {
type = nix::prim_sort(nix::EvalState&, nix::PosIdx, nix::Value**, nix::Value&)::{lambda(nix::Value*, nix::Value*)#1};
}
iterator::value_type "ordered type" {
type = nix::Value*;
}
[1] 2713634 abort (core dumped) nix-instantiate -A tests.coq
| 11:44:28 |
trofi | https://bpa.st/raw/JNGQ allows a bit better exception reporting. like https://bpa.st/raw/NBNQ | 11:52:31 |
Sergei Zimmerman (xokdvium) | Thanks! I guess it's just up to someone to rain nixmas gifts of bugfixes on nixpkgs. | 11:59:34 |
| elikoga changed their display name from elikoga to elikoga (@38c3 📞448{0,1}. | 15:21:56 |
| elikoga changed their display name from elikoga (@38c3 📞448{0,1} to elikoga (@38c3 📞448{0,1}). | 15:26:08 |