!ayCRiZriCVtuCUpeLp:nixos.org

Nix Cross Compiling

578 Members
129 Servers

Load older messages


SenderMessageTime
8 Feb 2025
@mindtree:matrix.orgmindtree *

Update: while the above^ works on aarch64-darwin as the build machine, I'm now running into issues when using x86_64-linux as the build machine.

It looks like even when just building a hello-world rust program with rustPlatform.buildrustPackage via pkgs.pkgsCross.wasm32-unknown-none.callPackage, it starts compiling LLVM and then rustc from scratch.

This was kind of expected after my experience on my mac, where the same happened, however on the mac they at least complete successfully after an hour or two. On my linux build machine, rustc fails to build during the bootstrap phase with a libunwind .so not found error.

Shouldn't llvm and rustc be cached for the pkgsCross.wasm32-unknown-none? Perhaps they're not, because there are no packages that actually use pkgsCross.wasm32-unknown-none, so as a result hydra never builds and caches them?

If so, what would be the best way to get hydra testing and building these? Would I need to add some simple hello-world-rust-wasm32-unknown-none package to pkgs/by-name/he/ or something along these lines?

I guess working out why rustc won't build is another issue, but at least having some basic pkg in nixpkgs would help to ensure it stays working.

Also, another thing I'm noticing is that regardless of what I put in meta.platforms, it always says the system is unsupported when building with pkgsCross.wasm32-unknown-none, and I always have to add NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1 and --impure everytime I build 🫠 I might look into addressing this too.

13:04:36
@mindtree:matrix.orgmindtree *

Update: while the above^ works on aarch64-darwin as the build machine, I'm now running into issues when using x86_64-linux as the build machine.

It looks like even when just building a hello-world rust program with rustPlatform.buildRustPackage via pkgs.pkgsCross.wasm32-unknown-none.callPackage, it starts compiling LLVM and then rustc from scratch.

This was kind of expected after my experience on my mac, where the same happened, however on the mac they at least complete successfully after an hour or two. On my linux build machine, rustc fails to build during the bootstrap phase with a libunwind .so not found error.

Shouldn't llvm and rustc be cached for the pkgsCross.wasm32-unknown-none? Perhaps they're not, because there are no packages that actually use pkgsCross.wasm32-unknown-none, so as a result hydra never builds and caches them?

If so, what would be the best way to get hydra testing and building these? Would I need to add some simple hello-world-rust-wasm32-unknown-none package to pkgs/by-name/he/ or something along these lines?

I guess working out why rustc won't build is another issue, but at least having some basic pkg in nixpkgs would help to ensure it stays working.

Also, another thing I'm noticing is that regardless of what I put in meta.platforms, it always says the system is unsupported when building with pkgsCross.wasm32-unknown-none, and I always have to add NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1 and --impure everytime I build 🫠 I might look into addressing this too.

13:22:10
@qyliss:fairydust.spaceAlyssa Rosswell, they can't be cached if they don't build17:55:19
@qyliss:fairydust.spaceAlyssa Rossthe thing with Hydra "testing" things is that it's only helpful if somebody is actively monitoring Hydra to see if something stops building, because Hydra doesn't do anything to notify of failed builds17:56:30
@rosscomputerguy:matrix.orgTristan RossIt would be useful if it could but then emails would be flooded and likely ignored.18:09:11
9 Feb 2025
@tired:fairydust.space@tired:fairydust.space left the room.22:50:47
10 Feb 2025
@rhelmot:matrix.orgrhelmotdoes anyone wanna explain how the splicing internals work to me03:58:07
@artturin:matrix.orgArtturinEasy, it's just adding attrs on top of the package set03:58:47
@artturin:matrix.orgArtturinFrom other offset package sets03:59:12
@rhelmot:matrix.orgrhelmotI mean like the stuff where derivations added to specific lists in mkDerivation automatically get turned into derivations from one of the spliced pkgsets03:59:34
@artturin:matrix.orgArtturin* Easy, it's just adding attrs to package attrs on top of the package set03:59:37
@rhelmot:matrix.orgrhelmotif there's a better word for that process I'd love to hear it lol03:59:53
@artturin:matrix.orgArtturinThat's the simplest part of the whole splicing thing 04:00:39
@artturin:matrix.orgArtturinhttps://github.com/NixOS/nixpkgs/blob/f202c36babad2412fc20a061d56c1f378efa806d/pkgs/stdenv/generic/make-derivation.nix#L34604:00:41
@rhelmot:matrix.orgrhelmotas far as I can tell it pulls stuff out of drv.__spliced.pkgsSomethingSomething which is provided by splice.nix but __spliced isn't present in nix repl04:00:41
@artturin:matrix.orgArtturincallPackage gets packaged from __splicedPackages04:01:05
@artturin:matrix.orgArtturin* callPackage gets packages from __splicedPackages04:01:21
@artturin:matrix.orgArtturin* callPackage gets attrs from __splicedPackages04:01:28
@artturin:matrix.orgArtturinpkgsCross.X.__splicedPackages to access them04:01:55
@rhelmot:matrix.orgrhelmothm04:02:24
@rhelmot:matrix.orgrhelmotwhat's special about pkgsCross that causes __splicedPackages to contain the __spliced attribute but not the toplevel04:03:03
@artturin:matrix.orgArtturinhttps://github.com/NixOS/nixpkgs/blob/f202c36babad2412fc20a061d56c1f378efa806d/pkgs/top-level/splice.nix#L18104:03:36
@artturin:matrix.orgArtturin
In reply to @rhelmot:matrix.org
what's special about pkgsCross that causes __splicedPackages to contain the __spliced attribute but not the toplevel
toplevel is just normal pkgs
04:04:06
@rhelmot:matrix.orgrhelmothmmk... so actuallySplice is adjacentPackages != null04:05:05
@rhelmot:matrix.orgrhelmothere's what I'm trying to do: I want to instantiate nixpkgs with localSystem = A and crossSystem = B, do some builds, transfer the closure onto a B machine, then do the same instantiation so that B doesn't have to do any heavy building of its own... but then import pkgs/build-support/trivial-builders/default.nix with a new stdenv which has both build and host as B so that B can do things like build config files. the problem I'm foreseeing is that it will try to splice its dependencies and end up with stuff that can only run on A. I've been struggling to construct a spliced package set that, well, effectively lies about what e.g. bash.__spliced.buildHost is and actually gives hostHost.05:14:25
@rhelmot:matrix.orgrhelmotthe reason for this task: being able to build nixbsd installer images. A here is Linux and B here is e.g. OpenBSD which doesn't have a real stdenv and can't build anything natively05:16:16
@trofi:matrix.orgtrofi loongarch64's libgcc is broken: https://github.com/NixOS/nixpkgs/pull/381016 23:18:42
12 Feb 2025
@kira:jakira.spaceKira changed their display name from kira to Kira.19:59:44
14 Feb 2025
@bitestring:matrix.orgbitestring

Hi, can anyone please help me how to get started on cross-compilation. Documentation is very confusing and scattered across multiple places. I want to get GHC compiler than can compile my Haskell code to both ARM64 Linux and ARM64 Android. However I do not know where to get started. I tried the following flake.nix

{
  description = "Haskell Playground";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
  };

  outputs =
    {
      self,
      nixpkgs,
    }:
      let
        pkgsBuildSystem = import nixpkgs {
          localSystem = "x86_64-linux"; # buildPlatform
          config = {
            android_sdk.accept_license = true;
            allowUnfree = true;
          };
        };
        pkgs = pkgsBuildSystem.pkgsCross.aarch64-android-prebuilt;
        hs = pkgs.haskell.packages."ghc966"; # need to match Stackage LTS version from stack.yaml resolver
        ghc = hs.ghc.override {
          enableRelocatedStaticLibs = true;
          enableShared = true;
        };
        buildDependencies = with pkgs; [
          # --- System ---
          zlib

          # Haskell deps
          stack
          ghc # GHC compiler in the desired version (will be available on PATH)
        ];
      in
      {
        formatter = pkgs.nixfmt-rfc-style;
        packages.aarch64-android.default = pkgs.mkShell {
          buildInputs = [ ];
          nativeBuildInputs = buildDependencies;
          LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath buildDependencies;
        };
      };
}

However nix develop fails with

./zconf.h:446:14: fatal error: 'sys/types.h' file not found
13:14:55
@bitestring:matrix.orgbitestring *

Hi, can anyone please help me how to get started on cross-compilation. Documentation is very confusing and scattered across multiple places. I want to get GHC compiler than can compile my Haskell code to both ARM64 Linux and ARM64 Android. However I do not know where to get started. I tried the following flake.nix

{
  description = "Haskell Playground";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
  };

  outputs =
    {
      self,
      nixpkgs,
    }:
      let
        pkgsBuildSystem = import nixpkgs {
          localSystem = "x86_64-linux"; # buildPlatform
          config = {
            android_sdk.accept_license = true;
            allowUnfree = true;
          };
        };
        pkgs = pkgsBuildSystem.pkgsCross.aarch64-android-prebuilt;
        hs = pkgs.haskell.packages."ghc966"; # need to match Stackage LTS version from stack.yaml resolver
        ghc = hs.ghc.override {
          enableRelocatedStaticLibs = true;
          enableShared = true;
        };
        buildDependencies = with pkgs; [
          zlib
          stack
          ghc # GHC compiler in the desired version (will be available on PATH)
        ];
      in
      {
        formatter = pkgs.nixfmt-rfc-style;
        packages.aarch64-android.default = pkgs.mkShell {
          buildInputs = [ ];
          nativeBuildInputs = buildDependencies;
          LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath buildDependencies;
        };
      };
}

However nix develop fails with

./zconf.h:446:14: fatal error: 'sys/types.h' file not found
13:16:14

Show newer messages


Back to Room ListRoom Version: 6