!VRULIdgoKmKPzJZzjj:nixos.org

Nix Hackers

923 Members
For people hacking on the Nix package manager itself193 Servers

Load older messages


SenderMessageTime
24 Dec 2024
@trofi:matrix.orgtrofi

Surprisingly adding a no-op change like:

--- a/pkgs/servers/sql/postgresql/generic.nix
+++ b/pkgs/servers/sql/postgresql/generic.nix
@@ -447,0 +448,2 @@ let
+      FOO = "42";
+

fixes the build. As if there is some ordering issue in nix to discover known outputs.

22:25:47
@matthewcroughan:defenestrate.itmatthewcroughan @fosdemThere is currently no way to pass auth headers into filetransfer.cc, open PRs/Issues about it, years of history about authentication being unsolved, how come?22:29:02
@matthewcroughan:defenestrate.itmatthewcroughan @fosdemhttps://ec.haxx.se/libcurl-http/auth.html#bearer22:29:16
@matthewcroughan:defenestrate.itmatthewcroughan @fosdem All that must be done is passing CURLOPT_XOAUTH2_BEARER when present 22:29:31
@matthewcroughan:defenestrate.itmatthewcroughan @fosdemyet this is only supported for gitlab fetches22:29:47
25 Dec 2024
@ananya.kaligal:matrix.orgAnanya S Kaligal joined the room.13:20:21
@tanja:catgirl.cloudTanja (she/her) changed their display name from Tanja (she/her) to Tanja (she/her) [DECT 6929].14:48:10
@chukfinley:matrix.orghackerman_1337 changed their display name from hackerman to hackerman_1337.19:06:51
@trofi:matrix.orgtrofi Attempted to improve an error message as https://github.com/NixOS/nix/pull/12105. Looks like nix losk man output when built a goal with man output. I wonder if outputChecks does not specify some kind of dependency for string references luke plain "man". 22:07:02
@trofi:matrix.orgtrofi * Attempted to improve an error message as https://github.com/NixOS/nix/pull/12105. Looks like nix lotk man output when built a goal with man output. I wonder if outputChecks does not specify some kind of dependency for string references luke plain "man". 22:07:19
26 Dec 2024
@10leej:matrix.org@10leej:matrix.org left the room.01:41:45
@gasu:matrix.orggasu joined the room.05:33:52
@trofi:matrix.orgtrofi Fun fact: -D_GLIBCXX_DEBUG crashes nix-env as std::stable_sort predicate does not maintain !(a < a) property: https://github.com/NixOS/nix/issues/12106 07:46:10
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)

What should builtins.sort (a: b: true) [ 1 2 3 ]even do? Should this be a nix language-level UB? Currently manual does not provide any preconditions https://nix.dev/manual/nix/2.25/language/builtins.html?highlight=builtins.sort#builtins-sort.

This would be a tricky thing to fix. Failing gracefully would require validating the strict weak ordering or using sorting algorithms that do not explode when the comparator is broken. std::sort can easily explode since it's something like qsort. stable_sort is actually a combination of insertion sort with merge sort on top. https://gcc.gnu.org/onlinedocs/libstdc++/libstdc++-html-USERS-4.4/a01347.html#l03409

I think using a custom merge sort would be safe even for broken predicates and won't crash nix evaluator, but this would still need to be nix UB (such calls to sort are ill-formed and diagnosing it is prohibitively expensive).

Either way this calls for a custom sorting implementation that does not assume that the provided predicate is strict weak ordering.

10:17:33
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)What's more worrying is the fact that somewhere in nixpkgs there are broken comparator predicates.10:27:59
@aloisw:julia0815.de@aloisw:julia0815.de
In reply to @xokdvium:matrix.org

What should builtins.sort (a: b: true) [ 1 2 3 ]even do? Should this be a nix language-level UB? Currently manual does not provide any preconditions https://nix.dev/manual/nix/2.25/language/builtins.html?highlight=builtins.sort#builtins-sort.

This would be a tricky thing to fix. Failing gracefully would require validating the strict weak ordering or using sorting algorithms that do not explode when the comparator is broken. std::sort can easily explode since it's something like qsort. stable_sort is actually a combination of insertion sort with merge sort on top. https://gcc.gnu.org/onlinedocs/libstdc++/libstdc++-html-USERS-4.4/a01347.html#l03409

I think using a custom merge sort would be safe even for broken predicates and won't crash nix evaluator, but this would still need to be nix UB (such calls to sort are ill-formed and diagnosing it is prohibitively expensive).

Either way this calls for a custom sorting implementation that does not assume that the provided predicate is strict weak ordering.

It currently triggers C++ UB.
10:29:30
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)

Yup, that's clearly a huge problem and this should be fixed by not using standard sorting algorithms:

Either way this calls for a custom sorting implementation that does not assume that the provided predicate is strict weak ordering.

What I'm trying to say is that this should still be UB in the nix language, but not crash and burn the evaluator.

10:31:14
@aloisw:julia0815.de@aloisw:julia0815.deI don't think there should be UB in the Nix language, unspecified result should be good enough.10:32:15
@xokdvium:matrix.orgSergei Zimmerman (xokdvium) Yeah, that makes more sense. Though I hope there could be a better way to find the broken usages in nixpkgs. Unspecified result makes the eval non-deterministic by nature. Does this mean that if this bug gets fixed and replaced with unspecified order instead of C++ UB nixpkgs would suddenly eval differently? 10:37:58
@aloisw:julia0815.de@aloisw:julia0815.deI don't think it is possible to determine in general whether a given comparator is broken. And I think literally depending on C++ UB is one of the cases where breaking evaluation compatibility is acceptable.10:40:06
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)At the very least we could first trace the problematic usages with a custom cppnix build and evaluate the scope of the disaster. Checking strict weak order properties by hand + random shuffling before the sort should surface at least some problems.10:43:29
@xokdvium:matrix.orgSergei Zimmerman (xokdvium) Though random shuffling would have to preserve the order, since the sort is stable. 10:44:30
@aloisw:julia0815.de@aloisw:julia0815.deHow do you know what the problematic usages are in the first place?10:44:43
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)

See the original https://github.com/NixOS/nix/issues/12106#issue-2759417825 issue. I just reproduced the problem on my end on nixpkgs master (9a90c6b6218860ab29f79ecd6213c3a547c15c7e):

../nixpkgs-check-strict-weak-order/result/bin/nix-env -qa --json --arg config 'import pkgs/top-level/packages-config.nix'
/nix/store/4KRAB2H0HD4WVXXMSCXRW21PL77J4I7J-gcc-13.3.0/include/c++/13.3.0/bits/stl_algo.h:5117:
In function:
    void std::stable_sort(_RAIter, _RAIter, _Compare) [with _RAIter =
    nix::Value**; _Compare = nix::prim_sort(EvalState&, PosIdx, Value**,
    Value&)::<lambda(nix::Value*, nix::Value*)>]

Error: comparison doesn't meet irreflexive requirements, assert(!(a < a)).

Objects involved in the operation:
    instance "functor" @ 0x7ffff3cb8c90 {
      type = nix::prim_sort(nix::EvalState&, nix::PosIdx, nix::Value**, nix::Value&)::{lambda(nix::Value*, nix::Value*)#1};
    }
    iterator::value_type "ordered type"  {
      type = nix::Value*;
    }
[1]    2675868 abort (core dumped)  ../nixpkgs-check-strict-weak-order/result/bin/nix-env -qa --json --arg config
11:14:58
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)I'm not sure that if that actually explodes with normal usage though11:26:03
@xokdvium:matrix.orgSergei Zimmerman (xokdvium)

A quick glance also shows that this gets triggered in tests.coq, so this is not a one-off problem:

nix-instantiate -A tests.coq

/nix/store/4KRAB2H0HD4WVXXMSCXRW21PL77J4I7J-gcc-13.3.0/include/c++/13.3.0/bits/stl_algo.h:5117:
In function:
    void std::stable_sort(_RAIter, _RAIter, _Compare) [with _RAIter =
    nix::Value**; _Compare = nix::prim_sort(EvalState&, PosIdx, Value**,
    Value&)::<lambda(nix::Value*, nix::Value*)>]

Error: comparison doesn't meet irreflexive requirements, assert(!(a < a)).

Objects involved in the operation:
    instance "functor" @ 0x7ffc7a9bef10 {
      type = nix::prim_sort(nix::EvalState&, nix::PosIdx, nix::Value**, nix::Value&)::{lambda(nix::Value*, nix::Value*)#1};
    }
    iterator::value_type "ordered type"  {
      type = nix::Value*;
    }
[1]    2713634 abort (core dumped)  nix-instantiate -A tests.coq
11:44:28
@trofi:matrix.orgtrofihttps://bpa.st/raw/JNGQ allows a bit better exception reporting. like https://bpa.st/raw/NBNQ11:52:31
@xokdvium:matrix.orgSergei Zimmerman (xokdvium) Thanks! I guess it's just up to someone to rain nixmas gifts of bugfixes on nixpkgs. 11:59:34
@elikoga:matrix.orgelikoga changed their display name from elikoga to elikoga (@38c3 📞448{0,1}.15:21:56
@elikoga:matrix.orgelikoga changed their display name from elikoga (@38c3 📞448{0,1} to elikoga (@38c3 📞448{0,1}).15:26:08

Show newer messages


Back to Room ListRoom Version: 6