| 22 Jan 2022 |
adisbladis | In reply to @k900:0upti.me Yeah but ideally you'd want to not have it at all We're always going to have overrides for native deps, so as long as the interface is generic I think it's a really nice addition | 09:07:15 |
mou | something like
overrides = poetry2nix.buildSystemOverrides { aiosqlite = "flit" } | 09:08:23 |
K900 | Mostly I just want to avoid it getting to a point where it's easier to add overrides than it is to fix things upstream | 09:09:14 |
mou | but upstreams are python packages | 09:09:43 |
mou | poetry for example. It's really difficult to propose changes to product used by hundred thousands | 09:10:36 |
mou | PR mentioned above in poetry is still did not accepted ( | 09:11:24 |
adisbladis | There is a very little documented interface that is' a lot better than withDefaults, namely poetry2nix.defaultPoetryOverrides.extend:
(poetry2nix.defaultPoetryOverrides.extend (self: super: {})).extend(self: super: {}) | 09:11:27 |
adisbladis | It actually composes nicely | 09:12:37 |
adisbladis | I'm thinking that might be a decent foundation | 09:12:48 |
adisbladis | https://github.com/nix-community/poetry2nix/pull/528 phew 😓 | 09:31:14 |
adisbladis | (God I hate MacOS) | 09:31:55 |
K900 | adisbladis: the "not a supported wheel" thing is probably because you need 3.10 to get a bundled pip that recognizes aarch64-darwin is at all valid | 09:52:12 |
adisbladis | In reply to @k900:0upti.me adisbladis: the "not a supported wheel" thing is probably because you need 3.10 to get a bundled pip that recognizes aarch64-darwin is at all valid I'm aware, I'm just not motivated enough to solve it :) | 09:52:42 |
adisbladis | At least we have regression testing for darwin now 🤷 | 09:55:11 |
sephi | I’m using mkPoetryEnv for my local dev environment, but it seems I can’t use the poetry add command anymore to manipulate my pyproject.toml file:
❯ poetry add gunicorn
Using version ^20.1.0 for gunicorn
Updating dependencies
Resolving dependencies... (0.2s)
Writing lock file
Package operations: 0 installs, 31 updates, 0 removals
• Updating asgiref (3.5.0 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 3.5.0)
• Updating pytz (2021.3 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 2021.3)
• Updating sqlparse (0.4.2 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 0.4.2)
• Updating certifi (2021.10.8 /nix/store/p9z0r0817qabx2l6nfjl5lar2a836vl0-python3.9-certifi-2021.10.08/lib/python3.9/site-packages -> 2021.10.8)
• Updating charset-normalizer (2.0.5 /nix/store/bn9zx7afgmmz8ai47l5fnkn8f2ndi5jx-python3.9-charset-normalizer-2.0.5/lib/python3.9/site-packages -> 2.0.10)
• Updating django (3.0.14 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 3.0.14)
• Updating idna (3.2 /nix/store/hl5mixqrm35xav0iywxcz8m7mnrzmxzl-python3.9-idna-3.2/lib/python3.9/site-packages -> 3.3)
• Updating six (1.16.0 /nix/store/9xfvadr4vvz5988db78jy2gn3vjdwl9a-python3.9-six-1.16.0/lib/python3.9/site-packages -> 1.16.0)
• Updating urllib3 (1.26.7 /nix/store/lrbpwbc9vnmzggc3qd2nfzipzrw00wxa-python3.9-urllib3-1.26.7/lib/python3.9/site-packages -> 1.26.8)
• Updating webencodings (0.5.1 /nix/store/800af4vm9mi3mi2gmw2sb557aawacz2x-python3.9-webencodings-0.5.1/lib/python3.9/site-packages -> 0.5.1)
• Updating django-taggit (1.5.1 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 1.5.1)
• Updating django-treebeard (4.5.1 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 4.5.1)
• Updating djangorestframework (3.13.1 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 3.13.1)
• Updating draftjs-exporter (2.1.7 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 2.1.7)
• Updating html5lib (1.1 /nix/store/m112dcb1rgamgyy1gnp7jfsn773ap0i1-python3.9-html5lib-1.1/lib/python3.9/site-packages -> 1.1)
• Updating l18n (2020.6.1 -> 2021.3)
• Updating pillow (6.2.2 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 6.2.2): Failed
EnvCommandError
Command ['/home/sephi/.cache/pypoetry/virtualenvs/myproj-rnq942R3-py3.9/bin/pip', 'install', '--no-deps', '-U', 'file:///home/sephi/.cache/pypoetry/artifacts/72/23/b8/d141eebecfc4390e65c825063b71d91e2060738bede0f238b2571ad828/Pillow-6.2.2.tar.gz'] errored with the following return code 1, and output:
...
The headers or library files could not be found for zlib,
a required dependency when compiling Pillow from source.
Using --dry-run to get the version of the package, and then add it manually to pyproject.toml and running poetry lock works though. I’m not sure what the correct workflow is?
| 17:16:44 |
K900 | Well, you're missing zlib, and poetry2nix doesn't know that you're missing zlib because Pillow is not in your lockfile because you're missing zlib | 17:17:44 |
K900 | It's kind of a catch-22 | 17:18:03 |
sephi | But poetry2nix works fine (the local dev env works correctly), it’s the poetry command invocation that doesn’t work. You mean I should just add zlib to my shell? | 17:21:18 |
K900 | The problem isn't that poetry2nix doesn't work | 17:21:53 |
K900 | It's that it doesn't even know you're trying to add stuff | 17:21:59 |
K900 | And you're trying to add stuff that has native dependencies | 17:22:11 |
K900 | And those dependencies need to exist in your shell | 17:22:19 |
K900 | Which is managed by Nix, which doesn't know you're trying to add stuff | 17:22:29 |
sephi | The pillow dependency and all the other dependencies you see are already in my environment, I’m just trying to add gunicorn (which doesn’t have pillow as a dependency). I have no idea why poetry is trying to upgrade all these dependencies. | 17:25:41 |
sephi | Ah it seems I can just do poetry add --lock gunicorn to only update the pyproject.toml & poetry.lock files without trying to install anything | 17:37:23 |
| lelit joined the room. | 20:05:15 |
| 23 Jan 2022 |
mou | i'm a little confused by nix scoping rules and closures. I have this minimal flake file
description = "Application packaged using poetry2nix";
inputs.flake-utils.url = "github:numtide/flake-utils";
inputs.nixpkgs.url = "github:NixOS/nixpkgs";
inputs.poetry2nix.url = "github:nix-community/poetry2nix";
outputs = { self, nixpkgs, flake-utils, poetry2nix }:
{
# Nixpkgs overlay providing the application
overlay = nixpkgs.lib.composeManyExtensions [
poetry2nix.overlay
(final: prev: {
# The application
myapp = prev.poetry2nix.mkPoetryApplication {
projectDir = ./.;
};
})
];
} // (flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [ self.overlay ];
};
in
{
apps = {
myapp = pkgs.myapp;
};
defaultApp = pkgs.myapp;
devShell = poetry2nix.mkPoetryEnv {
projectDir = ./.;
editablePackageSources = {
emptyproject = ./.;
};
};
}));
}
And it's produce this error:
error: attribute 'mkPoetryEnv' missing
at /nix/store/c0qrx1svg4p1xviy2jsmjsqmz09rbdjx-source/flake.nix:33:20:
32| defaultApp = pkgs.myapp;
33| devShell = poetry2nix.mkPoetryEnv {
| ^
34| projectDir = ./.;
(use '--show-trace' to show detailed location information)
I expect what since poetr2nix is an binding references argument of outputs function, it will be available for evaluation of this function expression.
| 14:07:13 |
mou | * i'm a little confused by nix scoping rules and closures. I have this minimal flake file
description = "Application packaged using poetry2nix";
inputs.flake-utils.url = "github:numtide/flake-utils";
inputs.nixpkgs.url = "github:NixOS/nixpkgs";
inputs.poetry2nix.url = "github:nix-community/poetry2nix";
outputs = { self, nixpkgs, flake-utils, poetry2nix }:
{
# Nixpkgs overlay providing the application
overlay = nixpkgs.lib.composeManyExtensions [
poetry2nix.overlay
(final: prev: {
# The application
myapp = prev.poetry2nix.mkPoetryApplication {
projectDir = ./.;
};
})
];
} // (flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [ self.overlay ];
};
in
{
apps = {
myapp = pkgs.myapp;
};
defaultApp = pkgs.myapp;
devShell = poetry2nix.mkPoetryEnv {
projectDir = ./.;
editablePackageSources = {
emptyproject = ./.;
};
};
}));
}
And it's produce this error:
error: attribute 'mkPoetryEnv' missing
at /nix/store/c0qrx1svg4p1xviy2jsmjsqmz09rbdjx-source/flake.nix:33:20:
32| defaultApp = pkgs.myapp;
33| devShell = poetry2nix.mkPoetryEnv {
| ^
34| projectDir = ./.;
(use '--show-trace' to show detailed location information)
I expect what since poetry2nix is an binding references argument of outputs function, it will be available for evaluation of this function expression.
| 14:07:39 |
K900 | poetry2nix the flake doesn't have a mkPoetryEnv attribute | 14:08:56 |
K900 | The poetry2nix package in nixpkgs does | 14:09:46 |