In reply to @pepe:runas.rocks

I cannot figure - in any way - how to make my environment work. This is my flake:

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/release-24.05";
    powerlifted-flake.url = "github:peperunas/planner8";
    flake-utils.url = "github:numtide/flake-utils";
    poetry2nix = {
      url = "github:nix-community/poetry2nix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { self, nixpkgs, flake-utils, powerlifted-flake, poetry2nix }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        inherit (poetry2nix.lib.mkPoetry2Nix { inherit pkgs; }) mkPoetryEnv defaultPoetryOverrides;

        pkgs = import nixpkgs {
          inherit system;
          config.allowUnfree = true;
        };

        powerlifted = powerlifted-flake.packages.${system}.powerlifted;

        shellWithPkgs = packages:
          pkgs.mkShell {
            inherit packages;
            buildInputs = [
              pkgs.bashInteractive
              (mkPoetryEnv {
                projectDir = ./.;
                preferWheels = true;
                overrides = defaultPoetryOverrides.extend (self: super: {
                  gcs-oauth2-boto-plugin = super.gcs-oauth2-boto-plugin.overridePythonAttrs (old: {
                    buildInputs = (old.buildInputs or [ ]) ++ [ super.setuptools ];
                  });
                  gsutil = super.gsutil.overridePythonAttrs (old: {
                    buildInputs = (old.buildInputs or [ ]) ++ [ super.setuptools ];
                  });
                  python-gflags = super.python-gflags.overridePythonAttrs (old: {
                    buildInputs = (old.buildInputs or [ ]) ++ [ super.setuptools ];
                  });
                });
              })
            ];
          };
        shell = with pkgs; shellWithPkgs [
          powerlifted
          parallel-full
          sqlite
        ];
      in
      {
        devShell = shell;
        defaultPackage = shell;
      });
}

And this is the error I get back when the compilation gets to bcrypt:

error: builder for '/nix/store/lqfg86v33h64hi1dj3ib3i2dr1p0b7q6-python3.11-bcrypt-4.1.3.drv' failed with exit code 1;
       last 10 log lines:
       > Running phase: unpackPhase
       > Executing wheelUnpackPhase
       > Finished executing wheelUnpackPhase
       > Running phase: patchPhase
       > Executing cargoSetupPostPatchHook
       > Validating consistency between /build/src/_bcrypt/Cargo.lock and /Cargo.lock
       > /nix/store/0fw4a3z849azkhyjxnpxbygj4g5qhd0v-diffutils-3.10/bin/diff: /build/src/_bcrypt/Cargo.lock: No such file or directory
       > /nix/store/0fw4a3z849azkhyjxnpxbygj4g5qhd0v-diffutils-3.10/bin/diff: /Cargo.lock: No such file or directory
       > ERROR: Missing Cargo.lock from src. Expected to find it at: /build/src/_bcrypt/Cargo.lock
       > Hint: You can use the cargoPatches attribute to add a Cargo.lock manually to the build.
       For full logs, run 'nix log /nix/store/lqfg86v33h64hi1dj3ib3i2dr1p0b7q6-python3.11-bcrypt-4.1.3.drv'.
error: 1 dependencies of derivation '/nix/store/swgnvl8v6ia6c0xljqgmrdnvqwpk5byc-python3.11-paramiko-3.4.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/dk5k1wk3ms188q5v30045ijam6lwnhs5-python3.11-pwntools-4.11.1.drv' failed to build
error: 3 dependencies of derivation '/nix/store/agw1nq1w6756lml1dmn189zb7xil9r0x-python3-3.11.9-env.drv' failed to build
error: 1 dependencies of derivation '/nix/store/72cyph2439whiqn07xkar3100217n3n8-nix-shell-env.drv' failed to build

I tried multiple approaches - either having the shell not using poetry2nix and just call poetry shell (but then I get into weird libc++ linking issues) and trying to toggle preferWheels on and off. Nothing.

Do you have any pointers on how to make bcrypt / the python environment work? Thank you!

try picking a bcrypt version for which there currently is an override?

          "4.0.0" ,
          "4.0.1,
          "4.1.1" ,
          "4.1.2" ,
          "4.1.3" 

In reply to @pepe:runas.rocks
😭

In reply to @tyberius_prime:matrix.org
I know I build it with python 3.12, bcrypt 4.1.3 and the poetry2nix master like 2 weeks ago.. if you want I can drop you the flake.nix/pyproject.toml/poetry.lock

I tried with this:

                overrides = defaultPoetryOverrides.extend (self: super: {
                  bcrypt = super.bcrypt.overridePythonAttrs (old: {
                    preferWheels = false;
                  });

But it doesn't work :-(

I have a monorepo structured like so:

repo/
    services/
        a/
            pyproject.toml
         b/
            pyproject.toml

Where a imports b as editable with b = {path="../b", develop=true} in the pyproject.toml. I'm having trouble getting my nix development derivation to import b correctly into the shell.

I've tried:

1. a/flake.nix with a derivation like:

        devShells.default = let 
            envShell = mkPoetryEnv {
              projectDir = self;
              overrides = p2n-overrides;
              preferWheels = true;
              editablePackageSources = {
                b = ../b;
              };
            }; 
          in
          pkgs.mkShell {
            name = "A";
            buildInputs = [ envShell ];
            #inputsFrom = [ self.packages.${system}.a ];
            packages = with pkgs; [ poetry docker-compose unzip ruff-lsp stdenv.cc.cc.lib zip awscli2 postgresql];
            LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
        };
      });

My understanding was that nix (like docker) copies the entire sub-directory structure, so it might be missing "sibling" folders (i.e. w/ same parent directory)
2. I've also tried putting flake.nix in services/ with something like

        devShells.default = let 
            envShell = mkPoetryEnv {
              projectDir = ./a;
              overrides = p2n-overrides;
              preferWheels = true;
              editablePackageSources = {
                b = ./b;
                a = ./a;
              };
            }; 
          in
          pkgs.mkShell {
            name = "A";
            buildInputs = [ envShell ];
            packages = with pkgs; [ poetry docker-compose unzip ruff-lsp stdenv.cc.cc.lib zip awscli2 postgresql];
            LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
        };
      });

But I still get the ModuleNotFoundError for b.

What's the expected way for me to make this structure work?

In reply to @notwren:matrix.org

I have a monorepo structured like so:

repo/
    services/
        a/
            pyproject.toml
         b/
            pyproject.toml

Where a imports b as editable with b = {path="../b", develop=true} in the pyproject.toml. I'm having trouble getting my nix development derivation to import b correctly into the shell.

I've tried:

1. a/flake.nix with a derivation like:

        devShells.default = let 
            envShell = mkPoetryEnv {
              projectDir = self;
              overrides = p2n-overrides;
              preferWheels = true;
              editablePackageSources = {
                b = ../b;
              };
            }; 
          in
          pkgs.mkShell {
            name = "A";
            buildInputs = [ envShell ];
            #inputsFrom = [ self.packages.${system}.a ];
            packages = with pkgs; [ poetry docker-compose unzip ruff-lsp stdenv.cc.cc.lib zip awscli2 postgresql];
            LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
        };
      });

My understanding was that nix (like docker) copies the entire sub-directory structure, so it might be missing "sibling" folders (i.e. w/ same parent directory)
2. I've also tried putting flake.nix in services/ with something like

        devShells.default = let 
            envShell = mkPoetryEnv {
              projectDir = ./a;
              overrides = p2n-overrides;
              preferWheels = true;
              editablePackageSources = {
                b = ./b;
                a = ./a;
              };
            }; 
          in
          pkgs.mkShell {
            name = "A";
            buildInputs = [ envShell ];
            packages = with pkgs; [ poetry docker-compose unzip ruff-lsp stdenv.cc.cc.lib zip awscli2 postgresql];
            LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
        };
      });

But I still get the ModuleNotFoundError for b.

What's the expected way for me to make this structure work?

I have some thing like

        devShells.a = pkgs.mkShell {
          packages = with pkgs; [
            (mkPoetryEnv {
              projectDir = a/.;
              python=pkgs.python39;
            })
          ];
        };

        devShells.b = pkgs.mkShell {
          packages = with pkgs; [
            (mkPoetryEnv {
              projectDir = b/.;
              python=pkgs.python39;
            })
          ];
        };

I'm wondering if poetry2nix could be used to build wheels of all my dependencies?
Currently, I use poetry2nix for a lovely dev environment but now I want to build wheels for my production environment.
I'm using docker currently by exporting my poetry.lock to a requirements.txt and then for each package doing
python wheel <packageName==Version>

I have to maintain dependencies manually in the Docker file :(
Feels like I should be able to do this with nix because it already created a reproducible environment to build those packages.
I just need the wheel output?