9 May 2024 |
| dagenix joined the room. | 04:34:29 |
wk | is it possible to use overrideAttrs inject additional python packages into a package defined with mkPoetryApplication? | 04:38:05 |
dagenix | I'm trying to build an older version of poetry, specifically version 1.4.2. I have this derivation:
{ fetchFromGitHub, poetry2nix }:
poetry2nix.mkPoetryApplication {
projectDir = fetchFromGitHub {
owner = "python-poetry";
repo = "poetry";
rev = "1.4.2";
hash = "sha256-AiRQFZA5+M1niTzj1RO2lx0QFOMmSzpQo1gzauyTblg=";
};
}
The derivation builds fine. However, the resulting package depends on the build module version 1.0.3. This is a problem, since the poetry.lock from that version of poetry requires build version 0.10. As a result, the poetry package doesn't work since that version of poetry breaks with that version of the build package.
I have attempted to override the build package version:
{ fetchFromGitHub, poetry2nix }:
poetry2nix.mkPoetryApplication {
projectDir = fetchFromGitHub {
owner = "python-poetry";
repo = "poetry";
rev = "1.4.2";
hash = "sha256-AiRQFZA5+M1niTzj1RO2lx0QFOMmSzpQo1gzauyTblg=";
};
overrides = poetry2nix.defaultPoetryOverrides.extend (final: prev: {
build = prev.build.overridePythonAttrs (old: {
version = "0.10.0";
src = fetchFromGitHub {
owner = "pypa";
repo = "build";
rev = "0.10.0";
hash = "sha256-kXFrfTb7+68EV+gSENL81IFSR+ue7Fl6R2gsuFFBJhI=";
};
});
});
}
But, when I do that, I get a build error:
> patching script interpreter paths in /nix/store/i2mwa5yrz6lgllq3rvqim6ldwgj2bcfv-python3.11-zipp-3.14.0
> stripping (with command strip and flags -S) in /nix/store/i2mwa5yrz6lgllq3rvqim6ldwgj2bcfv-python3.11-zipp-3.14.0/lib
> Executing pythonRemoveTestsDir
> Finished executing pythonRemoveTestsDir
> Running phase: pythonCatchConflictsPhase
> Found duplicated packages in closure for dependency 'wheel':
> wheel 0.41.1 (/nix/store/k0hkn48yfv05f6a1nf4qfri3ajilxgzb-python3.11-wheel-0.41.1/lib/python3.11/site-packages/wheel-0.41.1.dist-info)
> wheel 0.41.1 (/nix/store/lq0ayfkzrgp69r5g7ia7ln597qa5dv18-python3.11-wheel-0.41.1/lib/python3.11/site-packages/wheel-0.41.1.dist-info)
>
> Package duplicates found in closure, see above. Usually this happens if two packages depend on different version of the same dependency.
I've tried searching the edgecases documentation but didn't find anything that seems to explain what I should do in this case.
Anyway, I was about to open an issue, but saw that it asked that I first post problems to the Matrix chat and I was wondering if anyone had any idea what I was doing wrong or if I should open up an issue.
Thanks in advance!
| 04:39:35 |
| @morgrimm:matrix.org changed their display name from morgrimm to remi-gelinas. | 13:21:58 |
| @morgrimm:matrix.org left the room. | 13:22:52 |
10 May 2024 |
| grw00 joined the room. | 07:16:01 |
11 May 2024 |
Andrea Mangrella | hello, I'm having problems to make some packages build with poetry2nix, by reading various sources online I've found out about overrides and how to add default packages to the build environment of mkPoetry application. I just started using nix 2 weeks ago and I don;t understand what I'm doing wrong with this flake setup, can someone help me out ? | 14:52:03 |
Andrea Mangrella | description = "Application packaged using poetry2nix";
inputs = {
flake-utils.url = "github:numtide/flake-utils";
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable-small";
poetry2nix = {
url = "github:nix-community/poetry2nix";
inputs.nixpkgs.follows = "nixpkgs";
};
};
outputs = { self, nixpkgs, flake-utils, poetry2nix }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = nixpkgs.legacyPackages.${system};
inherit (poetry2nix.lib.mkPoetry2Nix { inherit pkgs; }) mkPoetryApplication overrides;
mk = mkPoetryApplication {
projectDir = ./.;
overrides = overrides.withDefaults (self: super: {
pathspec = super.pathspec.overridePythonAttrs (
old: {
buildInputs = (old.buildInputs or [ ]) ++ [ self.flit-core ];
}
);
});
};
in
{
packages = {
myapp = mk;
default = self.packages.${system}.myapp;
};
# Shell for app dependencies:
# Use this shell for developing your app: nix develop
devShells.default = pkgs.mkShell {
inputsFrom = [ self.packages.${system}.myapp ];
};
# Shell for poetry.
# Use this shell for changes to pyproject.toml and poetry.lock: nix develop .#poetry
devShells.poetry = pkgs.mkShell {
packages = [ pkgs.poetry ];
};
});
}
| 14:52:22 |
Andrea Mangrella | Redacted or Malformed Event | 14:53:29 |
Andrea Mangrella | with the following error: ``` warning: Git tree '/home/kativen/universita/statistical_learning' is dirty error: builder for '/nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv' failed with exit code 2; last 10 log lines: > File "<frozen importlib._bootstrap>", line 1204, in _gcd_import > File "<frozen importlib._bootstrap>", line 1176, in _find_and_load > File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked > File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed > File "<frozen importlib._bootstrap>", line 1204, in _gcd_import > File "<frozen importlib._bootstrap>", line 1176, in _find_and_load > File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked > ModuleNotFoundError: No module named 'flit_core' > > For full logs, run 'nix log /nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv'. error: 1 dependencies of derivation '/nix/store/qhqqkxwv2hn4iyqh6i7bp8yrnz2gazih-python3.11-flit-3.9.0.drv' failed to build error (ignored): error: cannot unlink '/tmp/nix-build-python3.11-pyyaml-6.0.1.drv-0': Directory not empty error: 1 dependencies of derivation '/nix/store/sbhbrhkfx74cwklf0vccjffm659v3wd8-nix-shell-env.drv' failed to build
| 14:54:17 |
Andrea Mangrella | * with the following error: ``` warning: Git tree '/home/kativen/universita/statistical_learning' is dirty
error: builder for '/nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv' failed with exit code 2;
last 10 log lines:
> File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
> File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
> File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
> File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
> File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
> File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
> File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
> ModuleNotFoundError: No module named 'flit_core'
>
>
For full logs, run 'nix log /nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv'.
error: 1 dependencies of derivation '/nix/store/qhqqkxwv2hn4iyqh6i7bp8yrnz2gazih-python3.11-flit-3.9.0.drv' failed to build
error (ignored): error: cannot unlink '/tmp/nix-build-python3.11-pyyaml-6.0.1.drv-0': Directory not empty
error: 1 dependencies of derivation '/nix/store/sbhbrhkfx74cwklf0vccjffm659v3wd8-nix-shell-env.drv' failed to build ``` | 14:54:29 |
Andrea Mangrella | * with the following error: ``` warning: Git tree '/home/kativen/universita/statistical_learning' is dirty error: builder for '/nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv' failed with exit code 2; last 10 log lines:
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import File "<frozen importlib._bootstrap>", line 1176, in _find_and_load File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1204, in _gcd_import File "<frozen importlib._bootstrap>", line 1176, in _find_and_load File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked ModuleNotFoundError: No module named 'flit_core'
For full logs, run 'nix log /nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv'. error: 1 dependencies of derivation '/nix/store/qhqqkxwv2hn4iyqh6i7bp8yrnz2gazih-python3.11-flit-3.9.0.drv' failed to build error (ignored): error: cannot unlink '/tmp/nix-build-python3.11-pyyaml-6.0.1.drv-0': Directory not empty error: 1 dependencies of derivation '/nix/store/sbhbrhkfx74cwklf0vccjffm659v3wd8-nix-shell-env.drv' failed to build ```
| 14:54:46 |
Andrea Mangrella | * with the following error: warning: Git tree '/home/kativen/universita/statistical_learning' is dirty error: builder for '/nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv' failed with exit code 2; last 10 log lines:
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import File "<frozen importlib._bootstrap>", line 1176, in _find_and_load File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "<frozen importlib._bootstrap>", line 1204, in _gcd_import File "<frozen importlib._bootstrap>", line 1176, in _find_and_load File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked ModuleNotFoundError: No module named 'flit_core'
For full logs, run 'nix log /nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv'. error: 1 dependencies of derivation '/nix/store/qhqqkxwv2hn4iyqh6i7bp8yrnz2gazih-python3.11-flit-3.9.0.drv' failed to build error (ignored): error: cannot unlink '/tmp/nix-build-python3.11-pyyaml-6.0.1.drv-0': Directory not empty error: 1 dependencies of derivation '/nix/store/sbhbrhkfx74cwklf0vccjffm659v3wd8-nix-shell-env.drv' failed to build ```
| 14:55:07 |
Andrea Mangrella | * error: builder for '/nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv' failed with exit code 2;
last 10 log lines:
> File "\<frozen importlib.\_bootstrap>", line 1204, in \_gcd\_import
> File "\<frozen importlib.\_bootstrap>", line 1176, in \_find\_and\_load
> File "\<frozen importlib.\_bootstrap>", line 1126, in \_find\_and\_load\_unlocked
> File "\<frozen importlib.\_bootstrap>", line 241, in \_call\_with\_frames\_removed
> File "\<frozen importlib.\_bootstrap>", line 1204, in \_gcd\_import
> File "\<frozen importlib.\_bootstrap>", line 1176, in \_find\_and\_load
> File "\<frozen importlib.\_bootstrap>", line 1140, in \_find\_and\_load\_unlocked
> ModuleNotFoundError: No module named flit\_core
For full logs, run 'nix log /nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv'.
error: 1 dependencies of derivation '/nix/store/qhqqkxwv2hn4iyqh6i7bp8yrnz2gazih-python3.11-flit-3.9.0.drv' failed to build
error (ignored): error: cannot unlink '/tmp/nix-build-python3.11-pyyaml-6.0.1.drv-0': Directory not empty
error: 1 dependencies of derivation '/nix/store/sbhbrhkfx74cwklf0vccjffm659v3wd8-nix-shell-env.drv' failed to build ```
| 14:55:40 |
12 May 2024 |
nim65s | Andrea Mangrella: do you have a link to your repo with your pyproject.toml ? | 09:19:59 |
13 May 2024 |
truh | In reply to @kativen:matrix.org
error: builder for '/nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv' failed with exit code 2;
last 10 log lines:
> File "\<frozen importlib.\_bootstrap>", line 1204, in \_gcd\_import
> File "\<frozen importlib.\_bootstrap>", line 1176, in \_find\_and\_load
> File "\<frozen importlib.\_bootstrap>", line 1126, in \_find\_and\_load\_unlocked
> File "\<frozen importlib.\_bootstrap>", line 241, in \_call\_with\_frames\_removed
> File "\<frozen importlib.\_bootstrap>", line 1204, in \_gcd\_import
> File "\<frozen importlib.\_bootstrap>", line 1176, in \_find\_and\_load
> File "\<frozen importlib.\_bootstrap>", line 1140, in \_find\_and\_load\_unlocked
> ModuleNotFoundError: No module named flit\_core
For full logs, run 'nix log /nix/store/65p128giakz2k87slj0mphllvk9fcn2r-python3.11-docutils-0.21.2.drv'.
error: 1 dependencies of derivation '/nix/store/qhqqkxwv2hn4iyqh6i7bp8yrnz2gazih-python3.11-flit-3.9.0.drv' failed to build
error (ignored): error: cannot unlink '/tmp/nix-build-python3.11-pyyaml-6.0.1.drv-0': Directory not empty
error: 1 dependencies of derivation '/nix/store/sbhbrhkfx74cwklf0vccjffm659v3wd8-nix-shell-env.drv' failed to build ```
to me this looks like you need an override for docutils. In the flake there is only an override for pathspec. | 09:48:13 |
16 May 2024 |
@vengmark2:matrix.org | What do I need to add to this to make sure meson doesn't try to download freetype? It's already part of propagatedBuildInputs . | 03:08:59 |
22 May 2024 |
x10an14 | Hey y'all!
I'm working on nixifying a Python application, and chose to use poetry2nix due to previous familiarity. Originally the project used hatch (not really relevant).
However, the end-result is a docker image (no surprises there). And the docker image previously produces has much smaller size than the one generated by Nix. (I'm leveraging nixpkgs 's dockerTools.buildImage on the mkPoetryApplication ).
| 11:29:42 |
x10an14 | * Hey y'all!
I'm working on nixifying a Python application, and chose to use poetry2nix due to previous familiarity. Originally the project used hatch (not really relevant).
However, the end-result is a docker image (no surprises there). And the docker image previously produces has much smaller size than the one generated by Nix. (I'm leveraging nixpkgs 's dockerTools.buildImage on the mkPoetryApplication ).
We're talking like 125MiB vs 527MiB.
| 11:30:10 |
K900 | Probably optional dependencies somewhere | 11:30:37 |
x10an14 | Some of it might be explained by the python dependencies in the nix docker image not being compressed (as they seem to be in the original one).
But what gives? Any options? | 11:30:43 |
x10an14 | In reply to @k900:0upti.me Probably optional dependencies somewhere I made sure to specify the groups = [] list in mkPoetryApplication , so that can't be it, right? Or am I mistaken? | 11:31:10 |
K900 | Optional dependencies of dependencies | 11:31:21 |
K900 | Possibly | 11:31:23 |
K900 | But it's hard to tell | 11:31:25 |
K900 | nix-tree? | 11:31:26 |
x10an14 | Need to read up on nix tree, I thought i could just give it my results folder, but alas | 11:32:56 |
x10an14 |  Download 2024-05-22T13:44:20,258284425+02:00.png | 11:45:01 |
x10an14 | According to nix-tree, I live in a world of impossibilities 🙃 | 11:45:02 |
x10an14 | Does this make any sense to you? | 11:45:10 |