!OqhvaDMJdKYUicLDiE:nixos.org

Nixpkgs Stdenv

228 Members
75 Servers

Load older messages


SenderMessageTime
25 Nov 2024
@philiptaron:matrix.orgPhilip Taron (UTC-8)Hope that clarifies rather than obfuscates.00:58:13
@reckenrode:matrix.orgRandy Eckenrode
In reply to @emilazy:matrix.org
I think things below 18 can go.
Does Wine build on staging? I haven’t tried building it yet, but it was broken with Clang 18. Unlike other .cfi_startproc usage, Wine’s changes would be non-trivial.
01:19:28
@emilazy:matrix.orgemilyI believe I built it. I'm not sure.01:19:46
@emilazy:matrix.orgemilycould be misremembering01:19:47
@trofi:matrix.orgtrofi
In reply to @philiptaron:matrix.org
I notice basically everyone who tackles something with bootstrap breaks something. Why? What can we do to make this less common?
My experience is that everyone has their own vision of how bootstrap is supposed to work in nixpkgs. Everyone's vision is usually slightly flawed in one way or another (as in does not achieve basic goals like "no references to the bootstrap tools (or copies) in the final stdenv", or does not actually deliver fully working gcc, or does not allow mixing gccs, etc.).
16:55:40
@philiptaron:matrix.orgPhilip Taron (UTC-8)

Yes, trofi, exactly this. What we have today works but requires a large seed, is quite entangled with the actual tools themselves, and has pitfalls of many specified and unspecified varieties.

I don't have a vision of my own. I'm just the poor sod in the meme who says we should improve matters somewhat. I hope (at some unspecified future time) to put my labor into trying to define what better looks like and to making it so.

18:47:28
@k900:0upti.meK900We have the minimal bootstrap work18:51:13
@k900:0upti.meK900That's very good and just needs someone to get everything in place to ship it18:51:30
@philiptaron:matrix.orgPhilip Taron (UTC-8)"just" is doing a lot of load-bearing work in that sentence!18:52:22
@philiptaron:matrix.orgPhilip Taron (UTC-8)For sure: minimal bootstrap's path has been paved substantially, not least of which by GUIX.18:52:50
@emilazy:matrix.orgemilyit exists in-tree18:56:27
@trofi:matrix.orgtrofi I personally think minimal bootstrap is an orthogonal effort to bootstrap structure: minimal bootstrap changes how binary seed is provided, but does not change the way how it's used to get an up-to-date toolchain. For example it does not magically untangle circular dependency between g++ and libstdc++.so. 21:33:46
@alex:tunstall.xyzAlex

Dealing with circular dependencies seems like a part of shrinking a binary seed though?

Because otherwise you need to include one of the dependencies in the seed.

21:43:01
@trofi:matrix.orgtrofi g++ uses libstdc++.so as part of it's implementation. libstdc++ uses g++ to build itself. It's not directly related to the seed. Even after you got the seed you need to build both of those. You need to build both in a single derivation to get them linked against one another (gcc usually builds it in 2 stages internally to untagle from builder's gcc/libstdc++.so, which nixpkgs disables and suffers afterwardsm fresh example is https://github.com/NixOS/nixpkgs/issues/342904). 21:47:27
@trofi:matrix.orgtrofi * g++ uses libstdc++.so as part of it's implementation. libstdc++ uses g++ to build itself. It's not directly related to the seed. Even after you got the seed you need to build both of those. You need to build both in a single derivation to get them linked against one another (gcc usually builds it in 2 stages internally to untagle from builder's gcc/libstdc++.so, which nixpkgs disables and suffers afterwards, fresh example is https://github.com/NixOS/nixpkgs/issues/342904). 21:47:55
@alex:tunstall.xyzAlex

Isn't this kind of problem exactly why the Guix bootstrap involves multiple versions of GCC?

Obviously, you want to build as few versions as possible and it seems like someone else has already done the hard work of finding a usable path from stage0 to modern GCC.

I suppose the main difficulty is porting the build steps to Nix?

21:58:52
@emilazy:matrix.orgemilythat has already been done21:59:24
@emilazy:matrix.orgemilyit doesn't address trofi's concern, which comes up within a GCC build21:59:30
@alex:tunstall.xyzAlexRight, I see now. Nix needs a DAG but it requires hacking into the GCC build system to untangle libstdc++ from g++.22:00:54
@trofi:matrix.orgtrofi Yeah, or patching built binaries after the fact to redirect to fresher libstdc++. 22:23:26
26 Nov 2024
@yannis:mozilla.orgyannis changed their display name from yannis|pto_back_nov_26 to yannis.07:49:11
27 Nov 2024
@Ericson2314:matrix.orgJohn Ericson
In reply to @trofi:matrix.org
g++ uses libstdc++.so as part of it's implementation. libstdc++ uses g++ to build itself. It's not directly related to the seed. Even after you got the seed you need to build both of those. You need to build both in a single derivation to get them linked against one another (gcc usually builds it in 2 stages internally to untagle from builder's gcc/libstdc++.so, which nixpkgs disables and suffers afterwards, fresh example is https://github.com/NixOS/nixpkgs/issues/342904).
I think it is still related
06:34:27
@Ericson2314:matrix.orgJohn EricsonNix wants a dag, but the from-scratch bootstrap forces a dag06:34:38
@Ericson2314:matrix.orgJohn Ericsonin practice, reality does dictate it's a dag06:34:47
@Ericson2314:matrix.orgJohn Ericsonthe way the world get around that in practice is bootstrapping to a fixed point06:35:02
@Ericson2314:matrix.orgJohn Ericson * the way the world get around that in practice is bootstrapping to a fixed point, either byte-for-byte or just the ABI06:35:20
@Ericson2314:matrix.orgJohn Ericson
In reply to @philiptaron:matrix.org

I'm up for a Zoom or other video meetup.

Here's the list of things I'm tracking in stdenv world.

  1. __structuredAttrs https://github.com/NixOS/nixpkgs/issues/205690. Mostly driven by @wolfgangwalthier. Thank you!
  2. Various improvements to fetchgit,fetchGitHub, etc, mostly around rev vs. tag to increase the expressiveness of the function.
  3. Improvements to nixpkgs configuration options to document and regularize those options (https://github.com/NixOS/nixpkgs/pull/330753, but the PR author has left nixpkgs)
  4. Structured logging (@nix { "action": "msg", "level": "1", "msg": ...}) support in the stdenv. https://github.com/NixOS/nixpkgs/pull/331794
  5. Thorough removal of substituteAll in the nixpkgs codebase: https://github.com/NixOS/nixpkgs/issues/237216
  6. Continued progress on clarifying what the environment is and what guarantees are possible across "releases".
  7. Linters and checkers (like nixpkgs-vet) that assist with in-tree and out-of-tree users to provide diagnostics and migrations for necessary changes.
  8. Breaking apart the knot of bootstrapping on every platform to enable a stdenv that's "bootstrap-free" -- not strangely constrained by bootstrap concerns, but able to be changed without breaking bootstrap.
  9. Making sense of how cross-compilation depends and does not depend on stdenv. (This is a personal goal, since I just don't understand it today.)

There are other small things, but this is the majority effort.

Good list! Would like to we the GCC repackaging on that list too :)
06:35:57
@emilazy:matrix.orgemilywhat repackaging?06:36:33
@trofi:matrix.orgtrofi
In reply to @Ericson2314:matrix.org
in practice, reality does dictate it's a dag
It's a DAG from build step standpoint. But from installation directory standpoint gcc's own bootstrap allows installing libstdc++ and gcc multiple times to make sure both are using one another. nix does not allow for in place file updates and one has to resort to file copies from one store path into another and requires binary patching to achieve the same effect (which is fragile for self-references and effectively requires CA compatibility as a precondition).
06:56:31
@trofi:matrix.orgtrofi Luckily gcc is CA-compatible, but glibc is not. As a result I have to use hacks like https://bpa.st/raw/5QWA just to make glibc work in CA (and in cases when I have to rewrite references to relocate glibc) :) 07:02:13

Show newer messages


Back to Room ListRoom Version: 9