o/ hoping for a bit of advice, I'm currently trying to set up a CI flow with crate2nix to test some code.

The tests involve spinning up a test environment with docker compose, though, so simply adding runTests won't work.

I'm using cargo nextest as a test runner, so AIUI in theory I should be able to build my tests (with cargo nextest run --no-run from a high level perspective) with nix, make the test binaries themselves the build output, and then outside of the nix context invoke cargo nextest run to actually run the tests.

Does this seem like a reasonable approach? Could it be something crate2nix is interested in as a supported use case? Any other ideas on how to handle tests like this?

Can we do something about those two not being equal?

RUST_SRC_PATH = "${rustPlatform.rustLibSrc}";

RUST_SRC_PATH = rustPlatform.rustLibSrc;

nix-repl> "${pkgs.rustPlatform.rustLibSrc}"
"/nix/store/r08c2hz6pgag3mcr3pxgwdfjhw4hjidm-rust-lib-src"

nix-repl> pkgs.rustPlatform.rustLibSrc
«derivation /nix/store/i8773apx734nbryahq371jrf9sd98n6d-rust-lib-src.drv»

I'm getting a conflict between the .cargo/config.toml generated by the builder setup hook and the .cargo/config.toml that is included in a project I'm packaging. Does someone know what to do in this case?

error: could not load Cargo configuration

Caused by:
  failed to merge configuration at `/build/.cargo/config.toml`

Caused by:
  failed to merge key `target` between /build/source/rust/.cargo/config.toml and /build/.cargo/config.toml

Caused by:
  failed to merge key `x86_64-unknown-linux-gnu` between /build/source/rust/.cargo/config.toml and /build/.cargo/config.toml

Caused by:
  failed to merge key `rustflags` between /build/source/rust/.cargo/config.toml and /build/.cargo/config.toml

Caused by:
  failed to merge config value from `/build/.cargo/config.toml` into `/build/source/rust/.cargo/config.toml`: expected string, but found array

The generated one contains

"rustflags" = [ "-C", "target-feature=-crt-static" ]

and the upstream one

[target.x86_64-unknown-linux-musl]
rustflags="-C force-frame-pointers=yes"

[target.x86_64-unknown-linux-gnu]
rustflags="-C force-frame-pointers=yes"

Now getting in mitmproxy: = note: Error: error: unexpected argument '-c' found

mitmproxy-linux>    Compiling mitmproxy v0.11.5 (/build/source)
mitmproxy-linux> warning: mitmproxy-linux@0.11.5: error: linking with `bpf-linker` failed: exit status: 1
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:   |
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:   = note: LC_ALL="C" PATH="/nix/store/n3a6bznx02a1dj2fjbqidm1qblinq93a-rustc-1.84.0/lib/rustlib/x86_64-unknown-linux-gnu/bin:/nix/store/n3a6bznx02a1dj2fjbqidm1qblinq93a-rustc-1.84.0/lib/rustlib/x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/nix/store/0l539chjmcq5kdd43j6dgdjky4sjl7hl-python3-3.12.8/bin:/nix/store/48crl8icb7s8bzc4gn7hb46w7iml2wqq-python3.12-wheel-0.45.1/bin:/nix/store/zjbmhch5hwk3gwkf4zza6iypg43y7zp6-bpf-linker-0.9.13/bin:/nix/store/xv8gw6j7cabaqywxgdk2kxx0xfkshizm-maturin-1.8.1/bin:/nix/store/ym4b522im2q6x8v6xbhn7r9i7w5isgnv-cargo-1.84.0/bin:/nix/store/2ny2a7qny2bhr2mp7xk3mfl3hw6v4qdg-rustc-wrapper-1.84.0/bin:/nix/store/f04zhapn8n8w6yrd35s8sd9qmjp8g9ry-patchelf-0.15.0/bin:/nix/store/4ijy8jbsiqmj37avrk83gn2m903486mr-gcc-wrapper-14-20241116/bin:/nix/store/zs2gq6fkglrd28g1nxlb8waqq37cdc2z-gcc-14-20241116/bin:/nix/store/9lcg6rsqbmx6s35jzy86b86pkj0qhxjl-glibc-2.40-66-bin/bin:/nix/store/wdap4cr3bnm685f27y9bb6q5b6q18msl-coreutils-9.5/bin:/nix/store/vrkxj51s4a1awh7m4p4f1w29wad5s20m-binutils-wrapper-2.43.1/bin:/nix/store/5h5ghy2qf6l91l52j6m5vx473zi38vc3-binutils-2.43.1/bin:/nix/store/wdap4cr3bnm685f27y9bb6q5b6q18msl-coreutils-9.5/bin:/nix/store/032xw8dchwjipwqh6b3h70yc3mcmsqld-findutils-4.10.0/bin:/nix/store/dd7xqz1qwl0di4zb8rzj7r1ds8np9xqs-diffutils-3.10/bin:/nix/store/bffnm1211li6y431irplzbjbccr0k884-gnused-4.9/bin:/nix/store/4lbfasv335vpk8rbcf3pgkag4rhg8jx8-gnugrep-3.11/bin:/nix/store/xpzl2sf58fqfpl64b1fy1ihxay7k71li-gawk-5.3.1/bin:/nix/store/zlmk040fc3jax9s3gldwp5rfwc1hhajc-gnutar-1.35/bin:/nix/store/chwdy9qaxd13q8zvl0zd5r7ql2q116di-gzip-1.13/bin:/nix/store/hpppxlcfvjzrvvcvhcm47divp65gbwq1-bzip2-1.0.8-bin/bin:/nix/store/y0akgyz13jgxwm968bs8kay47zbxx638-gnumake-4.4.1/bin:/nix/store/fd118hwh7d1ncib4mdw56ylv3g9k0iyj-bash-5.2p37/bin:/nix/store/apqwjgbjj646wk2jkzr67l26djamn481-patch-2.7.6/bin:/nix/store/rrv4bd5i7rp2m7j8ix4kl8bzijhh8gd3-xz-5.6.3-bin/bin:/nix/store/qraqns84wjffzd8d3dgbdcyxg41czbd6-file-5.46/bin" VSLANG="1033" "bpf-linker" "--export-symbols" "/build/rustcRSO8Ha/symbols" "/build/rustcRSO8Ha/symbols.o" "/build/source/target/x86_64-unknown-linux-gnu/release/build/mitmproxy-linux-e2f52f2d82af6c26/out/mitmproxy-linux-ebpf/bpfel-unknown-none/release/deps/mitmproxy_linux-07a61da24c7de989.mitmproxy_linux.5e8c272d4b543fbf-cgu.0.rcgu.o" "--cpu" "generic" "--cpu-features" "-crt-static" "-o" "/build/source/target/x86_64-unknown-linux-gnu/release/build/mitmproxy-linux-e2f52f2d82af6c26/out/mitmproxy-linux-ebpf/bpfel-unknown-none/release/deps/mitmproxy_linux-07a61da24c7de989" "-O3" "--debug"
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:   = note: Error: error: unexpected argument '-c' found
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:             tip: to pass '-c' as a value, use '-- -c'
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:           Usage: bpf-linker [OPTIONS] --output <OUTPUT> <INPUTS>...
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:           For more information, try '--help'.
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:
mitmproxy-linux> warning: mitmproxy-linux@0.11.5:
mitmproxy-linux> warning: mitmproxy-linux@0.11.5: error: could not compile `mitmproxy-linux-ebpf` (bin "mitmproxy-linux") due to 1 previous error
mitmproxy-linux> error: failed to run custom build command for `mitmproxy-linux v0.11.5 (/build/source/mitmproxy-linux)`
mitmproxy-linux>
mitmproxy-linux> Caused by:
mitmproxy-linux>   process didn't exit successfully: `/build/source/target/release/build/mitmproxy-linux-41df685fa76e9c30/build-script-build` (exit status: 101)
mitmproxy-linux>   --- stdout
mitmproxy-linux>   cargo:rerun-if-changed=/build/source/mitmproxy-linux-ebpf
mitmproxy-linux>   cargo:warning=error: linking with `bpf-linker` failed: exit status: 1
mitmproxy-linux>   cargo:warning=  |
mitmproxy-linux>   cargo:warning=  = note: LC_ALL="C" PATH="/nix/store/n3a6bznx02a1dj2fjbqidm1qblinq93a-rustc-1.84.0/lib/rustlib/x86_64-unknown-linux-gnu/bin:/nix/store/n3a6bznx02a1dj2fjbqidm1qblinq93a-rustc-1.84.0/lib/rustlib/x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/nix/store/0l539chjmcq5kdd43j6dgdjky4sjl7hl-python3-3.12.8/bin:/nix/store/48crl8icb7s8bzc4gn7hb46w7iml2wqq-python3.12-wheel-0.45.1/bin:/nix/store/zjbmhch5hwk3gwkf4zza6iypg43y7zp6-bpf-linker-0.9.13/bin:/nix/store/xv8gw6j7cabaqywxgdk2kxx0xfkshizm-maturin-1.8.1/bin:/nix/store/ym4b522im2q6x8v6xbhn7r9i7w5isgnv-cargo-1.84.0/bin:/nix/store/2ny2a7qny2bhr2mp7xk3mfl3hw6v4qdg-rustc-wrapper-1.84.0/bin:/nix/store/f04zhapn8n8w6yrd35s8sd9qmjp8g9ry-patchelf-0.15.0/bin:/nix/store/4ijy8jbsiqmj37avrk83gn2m903486mr-gcc-wrapper-14-20241116/bin:/nix/store/zs2gq6fkglrd28g1nxlb8waqq37cdc2z-gcc-14-20241116/bin:/nix/store/9lcg6rsqbmx6s35jzy86b86pkj0qhxjl-glibc-2.40-66-bin/bin:/nix/store/wdap4cr3bnm685f27y9bb6q5b6q18msl-coreutils-9.5/bin:/nix/store/vrkxj51s4a1awh7m4p4f1w29wad5s20m-binutils-wrapper-2.43.1/bin:/nix/store/5h5ghy2qf6l91l52j6m5vx473zi38vc3-binutils-2.43.1/bin:/nix/store/wdap4cr3bnm685f27y9bb6q5b6q18msl-coreutils-9.5/bin:/nix/store/032xw8dchwjipwqh6b3h70yc3mcmsqld-findutils-4.10.0/bin:/nix/store/dd7xqz1qwl0di4zb8rzj7r1ds8np9xqs-diffutils-3.10/bin:/nix/store/bffnm1211li6y431irplzbjbccr0k884-gnused-4.9/bin:/nix/store/4lbfasv335vpk8rbcf3pgkag4rhg8jx8-gnugrep-3.11/bin:/nix/store/xpzl2sf58fqfpl64b1fy1ihxay7k71li-gawk-5.3.1/bin:/nix/store/zlmk040fc3jax9s3gldwp5rfwc1hhajc-gnutar-1.35/bin:/nix/store/chwdy9qaxd13q8zvl0zd5r7ql2q116di-gzip-1.13/bin:/nix/store/hpppxlcfvjzrvvcvhcm47divp65gbwq1-bzip2-1.0.8-bin/bin:/nix/store/y0akgyz13jgxwm968bs8kay47zbxx638-gnumake-4.4.1/bin:/nix/store/fd118hwh7d1ncib4mdw56ylv3g9k0iyj-bash-5.2p37/bin:/nix/store/apqwjgbjj646wk2jkzr67l26djamn481-patch-2.7.6/bin:/nix/store/rrv4bd5i7rp2m7j8ix4kl8bzijhh8gd3-xz-5.6.3-bin/bin:/nix/store/qraqns84wjffzd8d3dgbdcyxg41czbd6-file-5.46/bin" VSLANG="1033" "bpf-linker" "--export-symbols" "/build/rustcRSO8Ha/symbols" "/build/rustcRSO8Ha/symbols.o" "/build/source/target/x86_64-unknown-linux-gnu/release/build/mitmproxy-linux-e2f52f2d82af6c26/out/mitmproxy-linux-ebpf/bpfel-unknown-none/release/deps/mitmproxy_linux-07a61da24c7de989.mitmproxy_linux.5e8c272d4b543fbf-cgu.0.rcgu.o" "--cpu" "generic" "--cpu-features" "-crt-static" "-o" "/build/source/target/x86_64-unknown-linux-gnu/release/build/mitmproxy-linux-e2f52f2d82af6c26/out/mitmproxy-linux-ebpf/bpfel-unknown-none/release/deps/mitmproxy_linux-07a61da24c7de989" "-O3" "--debug"
mitmproxy-linux>   cargo:warning=  = note: Error: error: unexpected argument '-c' found
mitmproxy-linux>   cargo:warning=
mitmproxy-linux>   cargo:warning=            tip: to pass '-c' as a value, use '-- -c'
mitmproxy-linux>   cargo:warning=
mitmproxy-linux>   cargo:warning=          Usage: bpf-linker [OPTIONS] --output <OUTPUT> <INPUTS>...
mitmproxy-linux>   cargo:warning=
mitmproxy-linux>   cargo:warning=          For more information, try '--help'.
mitmproxy-linux>   cargo:warning=
mitmproxy-linux>   cargo:warning=
mitmproxy-linux>   cargo:warning=
mitmproxy-linux>   cargo:warning=
mitmproxy-linux>   cargo:warning=error: could not compile `mitmproxy-linux-ebpf` (bin "mitmproxy-linux") due to 1 previous error
mitmproxy-linux>
mitmproxy-linux>   --- stderr
mitmproxy-linux>   thread 'main' panicked at mitmproxy-linux/build.rs:135:5:
mitmproxy-linux>   assertion `left == right` failed: cd "/build/source/mitmproxy-linux-ebpf" && env -u RUSTC -u RUSTC_WORKSPACE_WRAPPER -u RUSTUP_TOOLCHAIN CARGO_CFG_BPF_TARGET_ARCH="x86_64" "cargo" "build" "--bins" "--message-format=json" "--release" "--target" "bpfel-unknown-none" "--target-dir" "/build/source/target/x86_64-unknown-linux-gnu/release/build/mitmproxy-linux-e2f52f2d82af6c26/out/mitmproxy-linux-ebpf" failed: ExitStatus(unix_wait_status(25856))
mitmproxy-linux>     left: Some(101)
mitmproxy-linux>    right: Some(0)
mitmproxy-linux>   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
mitmproxy-linux> warning: build failed, waiting for other jobs to finish...
mitmproxy-linux> 💥 maturin failed
mitmproxy-linux>   Caused by: Failed to build a native library through cargo
mitmproxy-linux>   Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.12-64bit" PYO3_PYTHON="/nix/store/0l539chjmcq5kdd43j6dgdjky4sjl7hl-python3-3.12.8/bin/python3.12" PYTHON_SYS_EXECUTABLE="/nix/store/0l539chjmcq5kdd43j6dgdjky4sjl7hl-python3-3.12.8/bin/python3.12" "cargo" "rustc" "--jobs" "48" "--target" "x86_64-unknown-linux-gnu" "--message-format" "json-render-diagnostics" "--offline" "--manifest-path" "/build/source/mitmproxy-linux/Cargo.toml" "--release" "--bin" "mitmproxy-linux-redirector" "--" "-C" "strip=symbols"`

Found a pretty interesting "problem" with the old fetchCargoTarball implementation: it doesn't check if the lockfile is outdated or not, so when running cargo vendor it would update the lockfile and vendor those libraries. But it would copy the original lockfile to the vendor dir, so the check that decides if the lockfile in the vendor dir and the lockfile in the package's build environment is the same would still succeed.
Looking at the implementation of fetchCargoTarball, it seems kinda intentional. Not sure why it was done.
In any case, weird behaviour IMO.

One package that had an outdated lockfile was discovered after the fetchCargoVendor migration, since it no longer regenerates the lockfile:
https://github.com/NixOS/nixpkgs/issues/382991

Just wanted to share, since it was interesting

Found a pretty interesting "problem" with the old fetchCargoTarball implementation: it doesn't check if the lockfile is outdated or not, so when running cargo vendor it would update the lockfile and vendor the new dependencies. But it would copy the original lockfile to the vendor dir, so the check that decides if the lockfile in the vendor dir and the lockfile in the package's build environment is the same would still succeed.
Looking at the implementation of fetchCargoTarball, it seems kinda intentional. Not sure why it was done.
In any case, weird behaviour IMO.

One package that had an outdated lockfile was discovered after the fetchCargoVendor migration, since it no longer regenerates the lockfile:
https://github.com/NixOS/nixpkgs/issues/382991

Just wanted to share, since it was interesting

Found a pretty interesting "problem" with the fetchCargoTarball implementation: it doesn't check if the lockfile is outdated or not, so when running cargo vendor it would update the lockfile and vendor the new dependencies. But it would copy the original lockfile to the vendor dir, so the check that decides if the lockfile in the vendor dir and the lockfile in the package's build environment is the same would still succeed.
Looking at the implementation of fetchCargoTarball, it seems kinda intentional. Not sure why it was done.
In any case, weird behaviour IMO.

One package that had an outdated lockfile was discovered after the fetchCargoVendor migration, since it no longer regenerates the lockfile:
https://github.com/NixOS/nixpkgs/issues/382991

Just wanted to share, since it was interesting