| 12 Jul 2024 |
TyberiusPrime (smilodon inopinatus) | (deep dark magic = i don't grok it) | 10:50:33 |
@pepe:runas.rocks | Thanks TyberiusPrime (smilodon inopinatus), I really appreciate it | 16:45:07 |
@pepe:runas.rocks | I'm gonna try and report back | 16:45:11 |
@pepe:runas.rocks | This thing is driving me crazy @_@ | 16:45:18 |
@pepe:runas.rocks | I'm trying to get a dev env fully reproducible for a paper I submitted to a conference | 16:45:35 |
@pepe:runas.rocks | And Python is making everything explode badly | 16:45:42 |
@pepe:runas.rocks | 😭 | 16:45:47 |
TyberiusPrime (smilodon inopinatus) | In reply to @pepe:runas.rocks 😭 it does do that.. | 19:33:23 |
| 14 Jul 2024 |
@pepe:runas.rocks | Funnily enough, switching to Python 3.12 seems to alleviate the issue with bcrypt but now I get the same exact error with diffutils... | 19:53:10 |
@pepe:runas.rocks | Actually, no. bcrypt fails in the same way... | 19:53:38 |
@pepe:runas.rocks | In reply to @tyberius_prime:matrix.org I know I build it with python 3.12, bcrypt 4.1.3 and the poetry2nix master like 2 weeks ago.. if you want I can drop you the flake.nix/pyproject.toml/poetry.lock Can you please give me your files? I am going crazy here | 19:53:58 |
@pepe:runas.rocks | Is there a way to use bcrypt from nixpkgs instead? | 19:56:42 |
| 16 Jul 2024 |
TyberiusPrime (smilodon inopinatus) | peppino: send you a PM | 11:08:53 |
@pepe:runas.rocks | Is there a way to force a package - in my case bcrypt - not to use preferWheels = true? | 20:12:56 |
@pepe:runas.rocks | I tried with this:
overrides = defaultPoetryOverrides.extend (self: super: {
bcrypt = super.bcrypt.overridePythonAttrs (old: {
preferWheels = false;
});
But it doesn't work :-(
| 20:13:23 |
@pepe:runas.rocks | Delete delete delete, I RTFM | 20:26:45 |
@pepe:runas.rocks | 🤦♂️ | 20:26:51 |
| 18 Jul 2024 |
| notwren joined the room. | 21:40:02 |
notwren | I have a monorepo structured like so:
repo/
services/
a/
pyproject.toml
b/
pyproject.toml
Where a imports b as editable with b = {path="../b", develop=true} in the pyproject.toml. I'm having trouble getting my nix development derivation to import b correctly into the shell.
I've tried:
a/flake.nix with a derivation like:
devShells.default = let
envShell = mkPoetryEnv {
projectDir = self;
overrides = p2n-overrides;
preferWheels = true;
editablePackageSources = {
b = ../b;
};
};
in
pkgs.mkShell {
name = "A";
buildInputs = [ envShell ];
#inputsFrom = [ self.packages.${system}.a ];
packages = with pkgs; [ poetry docker-compose unzip ruff-lsp stdenv.cc.cc.lib zip awscli2 postgresql];
LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
};
});
My understanding was that nix (like docker) copies the entire sub-directory structure, so it might be missing "sibling" folders (i.e. w/ same parent directory) 2. I've also tried putting flake.nix in services/ with something like
devShells.default = let
envShell = mkPoetryEnv {
projectDir = ./a;
overrides = p2n-overrides;
preferWheels = true;
editablePackageSources = {
b = ./b;
a = ./a;
};
};
in
pkgs.mkShell {
name = "A";
buildInputs = [ envShell ];
packages = with pkgs; [ poetry docker-compose unzip ruff-lsp stdenv.cc.cc.lib zip awscli2 postgresql];
LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
};
});
But I still get the ModuleNotFoundError for b.
What's the expected way for me to make this structure work?
| 21:42:09 |
| 19 Jul 2024 |
Pico | In reply to @notwren:matrix.org
I have a monorepo structured like so:
repo/
services/
a/
pyproject.toml
b/
pyproject.toml
Where a imports b as editable with b = {path="../b", develop=true} in the pyproject.toml. I'm having trouble getting my nix development derivation to import b correctly into the shell.
I've tried:
a/flake.nix with a derivation like:
devShells.default = let
envShell = mkPoetryEnv {
projectDir = self;
overrides = p2n-overrides;
preferWheels = true;
editablePackageSources = {
b = ../b;
};
};
in
pkgs.mkShell {
name = "A";
buildInputs = [ envShell ];
#inputsFrom = [ self.packages.${system}.a ];
packages = with pkgs; [ poetry docker-compose unzip ruff-lsp stdenv.cc.cc.lib zip awscli2 postgresql];
LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
};
});
My understanding was that nix (like docker) copies the entire sub-directory structure, so it might be missing "sibling" folders (i.e. w/ same parent directory) 2. I've also tried putting flake.nix in services/ with something like
devShells.default = let
envShell = mkPoetryEnv {
projectDir = ./a;
overrides = p2n-overrides;
preferWheels = true;
editablePackageSources = {
b = ./b;
a = ./a;
};
};
in
pkgs.mkShell {
name = "A";
buildInputs = [ envShell ];
packages = with pkgs; [ poetry docker-compose unzip ruff-lsp stdenv.cc.cc.lib zip awscli2 postgresql];
LD_LIBRARY_PATH = "${pkgs.stdenv.cc.cc.lib}/lib";
};
});
But I still get the ModuleNotFoundError for b.
What's the expected way for me to make this structure work?
I have some thing like
devShells.a = pkgs.mkShell {
packages = with pkgs; [
(mkPoetryEnv {
projectDir = a/.;
python=pkgs.python39;
})
];
};
devShells.b = pkgs.mkShell {
packages = with pkgs; [
(mkPoetryEnv {
projectDir = b/.;
python=pkgs.python39;
})
];
};
| 08:27:26 |
Pico | I'm wondering if poetry2nix could be used to build wheels of all my dependencies? Currently, I use poetry2nix for a lovely dev environment but now I want to build wheels for my production environment. I'm using docker currently by exporting my poetry.lock to a requirements.txt and then for each package doing
python wheel <packageName==Version>
I have to maintain dependencies manually in the Docker file :( Feels like I should be able to do this with nix because it already created a reproducible environment to build those packages. I just need the wheel output?
| 08:36:40 |
K900 | And your production environment consumes what, wheels directly? | 08:43:53 |
Pico | Yep, I just install them. Production environment installer, will just setup a virtualenv and use pip --no-index --find-link to install all the packages.
Not using nix in production (yet) | 08:50:58 |
Pico | * Yep, Production environment installer, will just setup a virtualenv and use pip --no-index --find-link to install all the packages.
Not using nix in production (yet) | 08:51:35 |
K900 | Yeah the problem with doing that is that Nix-built wheels will reference Nix store paths | 08:51:41 |
K900 | So you can't just install them | 08:51:44 |
Pico | Ah | 08:51:57 |
Pico | It might still be possible to get the right build environment using nix (and perhaps info from poetry2nix) I built some wheels manually on nix using the pip wheel command and I don't see any nix references in the result | 09:00:01 |
K900 | They'll show up if you have native library dependencies | 09:01:49 |
Pico | Ah yes
ldd M2Crypto/_m2crypto.cpython-39-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007ffc1013c000)
libssl.so.3 => /nix/store/gkx5bfa7kf83wgh94s6vr93vn356w081-openssl-3.0.14/lib/libssl.so.3 (0x00007f825e337000)
libcrypto.so.3 => /nix/store/gkx5bfa7kf83wgh94s6vr93vn356w081-openssl-3.0.14/lib/libcrypto.so.3 (0x00007f825de00000)
libc.so.6 => /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libc.so.6 (0x00007f825dc13000)
libdl.so.2 => /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libdl.so.2 (0x00007f825e332000)
libpthread.so.0 => /nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib/libpthread.so.0 (0x00007f825e32b000)
/nix/store/dbcw19dshdwnxdv5q2g6wldj6syyvq7l-glibc-2.39-52/lib64/ld-linux-x86-64.so.2 (0x00007f825e488000)
| 09:03:56 |