| 20 Jan 2025 |
 Tristan Ross | I don't see any other alternative option until we have the fine grain control and test every case (example is something like doCheck in pkgs/by-name/li/libseccomp/package.nix). | 18:43:35 |
| Tristan Ross | The benefit of staging the platform stuff is then we can get the fine grain control, handle non default cases, and still have things work as they do now. | 18:44:44 |
 emily | that means that there'll be conditionals for toolchain == "llvm" or toolchain == "gnu" that just fail if you try to vary any of the variables that they're shorthand for, because it'll become null despite actually wanting to check e.g. unwinder library. having a staging system that separates inputs and outputs so that toolchain = "llvm"; could be used as a shorthand but is never present in the final structure would be okay, but seems like a more major rework and probably not worth it just to save ~5 lines when you define a new platform | 18:45:29 |
| emily | we just have to investigate the existing ambiguous uses and figure out what they actually care about, whether that's through pinging maintainers, experimenting, or diving into the relevant source code. it's not necessarily easy, but it's what's required to actually make it a cleanup and deconfusing rather than just adding more non-orthogonal variables that won't actually work properly, which will make a bigger mess | 18:46:06 |
| emily | e.g. do libseccomp tests run successfully with Clang + libstdc++? with Clang + libc++? is it the unwinder? what errors happen when we try? | 18:46:40 |
| emily | having difficult-to-figure-out conditionals like this where it's not clear what they're checking for is the result of having overly-broad flags like the toolchain your PR adds | 18:47:11 |
| emily | moving away from that is good – especially since Darwin already shows that useLLVM is kinda meaningless – but we have to actually move away from it, not just spread around the confusion a bit more | 18:47:32 |
| emily | looking at the log you linked in that libseccomp PR, some of those failures seem pretty scary, there's cases where things trying to ensure that seccomp denies an action incorrectly result in ALLOW results… | 18:49:29 |
| emily | so I don't think glossing over that is the right solution, those could be legitimate security vulnerabilities in pkgsLLVM | 18:50:05 |
| emily | coordinating with upstream seems like the best option there, since it seems like a pretty gnarly issue. I'm not sure we should have turned off tests in the first place at all without investigating the cause | 18:50:42 |
| emily | but even if we just want to keep disabling them and ignoring the potential issues, the nice thing about splitting up the variables is that it should make it easier to test various combinations of them to see which ones actually cause the issue | 18:51:13 |
| Tristan Ross | Ok, what about only accepting toolchain for user input and not relying upon it inside nixpkgs unless it's for a default value? | 18:51:45 |
| Tristan Ross | In reply to @emilazy:matrix.org
so I don't think glossing over that is the right solution, those could be legitimate security vulnerabilities in pkgsLLVM Yeah, I can understand that. Just don't have the bandwidth to investigate that. I believe there's an upstream issue already. I eventually want all tests running. | 18:52:35 |
| emily | I'm not sure what you mean by default value, but I think that allowing it for input only would be good in principle but that we have no way to ensure people don't actually condition on it without automated linting. if it's that important to avoid a handful of lines of boilerplate for defining platform toolchains (I think that given that we already have shorthand for pkgsLLVM it's not a big deal and it's good to be explicit), then we should ensure it doesn't appear in the elaborated structure, but I'm not sure what that would break since as sterni said on the issue the idea of elaboration just adding more stuff is pretty baked in | 18:54:33 |
| Tristan Ross | Now that I think about it, not using toolchain would mean we'd need a different attribute to rely upon checking whether to use the LLVM bootstrapping or GCC bootstrapping. | 18:58:18 |
| emily | it should hopefully be possible to bootstrap any combination of the variables that actually works? | 18:59:43 |
| emily | or if you mean from a choice of predefined bootstrap tools to start the chain with, then I guess that would have to be a separate variable, yeah (it could just point directly to an attribute of the right format for flexibility?) | 18:59:52 |
| Tristan Ross | Yeah, I'm meaning bootstrapping from nothing. | 19:00:13 |
| Tristan Ross | I don't want to add another attribute which defines what bootstrap should be used. | 19:00:48 |
| Tristan Ross | It's kinda why I was making this PR to serve as the base to start from for getting LLVM bootstrapping started. | 19:02:10 |
| emily | I think it's the correct thing to determine bootstrap tools at least in principle. e.g., you can bootstrap a native GCC system from an LLVM toolchain and vice versa | 19:05:54 |
| emily | Randy Eckenrode's notion of native bootstrap as cross would help make bootstraps support more toolchains without causing a combinatorial explosion of special cases, I think | 19:06:19 |
| emily | though in practice there are likely to be infelicities | 19:06:25 |
| emily | if we end up stuck with multiple monolithic bootstraps that bake in a bunch of assumptions about what you set all the variables to then I think a separate field that explicitly signals that's what it's varying would be best as it would also make it obvious that it shouldn't be used for checks in regular packages | 19:07:48 |
| emily | but ideally that would be avoidable… the drift between native and cross bootstrap is already pretty sad | 19:08:03 |
| emily | cc John Ericson who I think was working on cleaning that stuff up in the past | 19:08:17 |
| Tristan Ross | Yeah, I'm not fully concerned with bootstrapping until the platform changes have been merged. | 19:10:00 |
| Tristan Ross | Maybe a single attribute which defines the bootstrap environment to use would make sense? I'm just not sure. My thinking was to use the toolchain attribute. | 19:11:27 |
| emily | you can bootstrap a native "all GNU" toolchain from an "all LLVM" bootstrap toolchain, just like our "all LLVM toolchain" will get bootstrapped from our existing "all GNU" toolchain | 19:13:18 |
| emily | in principle, native bootstrap gets a stdenv made from bootstrap tools (that may have Clang or GCC or whatever) and cross-compiles new stdenvs until all the bootstrap tools have been washed away | 19:13:28 |
