| 22 Jan 2022 |
 sephi | I’m using mkPoetryEnv for my local dev environment, but it seems I can’t use the poetry add command anymore to manipulate my pyproject.toml file:
❯ poetry add gunicorn
Using version ^20.1.0 for gunicorn
Updating dependencies
Resolving dependencies... (0.2s)
Writing lock file
Package operations: 0 installs, 31 updates, 0 removals
• Updating asgiref (3.5.0 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 3.5.0)
• Updating pytz (2021.3 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 2021.3)
• Updating sqlparse (0.4.2 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 0.4.2)
• Updating certifi (2021.10.8 /nix/store/p9z0r0817qabx2l6nfjl5lar2a836vl0-python3.9-certifi-2021.10.08/lib/python3.9/site-packages -> 2021.10.8)
• Updating charset-normalizer (2.0.5 /nix/store/bn9zx7afgmmz8ai47l5fnkn8f2ndi5jx-python3.9-charset-normalizer-2.0.5/lib/python3.9/site-packages -> 2.0.10)
• Updating django (3.0.14 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 3.0.14)
• Updating idna (3.2 /nix/store/hl5mixqrm35xav0iywxcz8m7mnrzmxzl-python3.9-idna-3.2/lib/python3.9/site-packages -> 3.3)
• Updating six (1.16.0 /nix/store/9xfvadr4vvz5988db78jy2gn3vjdwl9a-python3.9-six-1.16.0/lib/python3.9/site-packages -> 1.16.0)
• Updating urllib3 (1.26.7 /nix/store/lrbpwbc9vnmzggc3qd2nfzipzrw00wxa-python3.9-urllib3-1.26.7/lib/python3.9/site-packages -> 1.26.8)
• Updating webencodings (0.5.1 /nix/store/800af4vm9mi3mi2gmw2sb557aawacz2x-python3.9-webencodings-0.5.1/lib/python3.9/site-packages -> 0.5.1)
• Updating django-taggit (1.5.1 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 1.5.1)
• Updating django-treebeard (4.5.1 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 4.5.1)
• Updating djangorestframework (3.13.1 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 3.13.1)
• Updating draftjs-exporter (2.1.7 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 2.1.7)
• Updating html5lib (1.1 /nix/store/m112dcb1rgamgyy1gnp7jfsn773ap0i1-python3.9-html5lib-1.1/lib/python3.9/site-packages -> 1.1)
• Updating l18n (2020.6.1 -> 2021.3)
• Updating pillow (6.2.2 /nix/store/8hy837b1jmqzpy4gcq57l80x2piidiz1-python3-3.9.6-env/lib/python3.9/site-packages -> 6.2.2): Failed
EnvCommandError
Command ['/home/sephi/.cache/pypoetry/virtualenvs/myproj-rnq942R3-py3.9/bin/pip', 'install', '--no-deps', '-U', 'file:///home/sephi/.cache/pypoetry/artifacts/72/23/b8/d141eebecfc4390e65c825063b71d91e2060738bede0f238b2571ad828/Pillow-6.2.2.tar.gz'] errored with the following return code 1, and output:
...
The headers or library files could not be found for zlib,
a required dependency when compiling Pillow from source.
Using --dry-run to get the version of the package, and then add it manually to pyproject.toml and running poetry lock works though. I’m not sure what the correct workflow is?
| 17:16:44 |
 K900 | Well, you're missing zlib, and poetry2nix doesn't know that you're missing zlib because Pillow is not in your lockfile because you're missing zlib | 17:17:44 |
| K900 | It's kind of a catch-22 | 17:18:03 |
| sephi | But poetry2nix works fine (the local dev env works correctly), it’s the poetry command invocation that doesn’t work. You mean I should just add zlib to my shell? | 17:21:18 |
| K900 | The problem isn't that poetry2nix doesn't work | 17:21:53 |
| K900 | It's that it doesn't even know you're trying to add stuff | 17:21:59 |
| K900 | And you're trying to add stuff that has native dependencies | 17:22:11 |
| K900 | And those dependencies need to exist in your shell | 17:22:19 |
| K900 | Which is managed by Nix, which doesn't know you're trying to add stuff | 17:22:29 |
| sephi | The pillow dependency and all the other dependencies you see are already in my environment, I’m just trying to add gunicorn (which doesn’t have pillow as a dependency). I have no idea why poetry is trying to upgrade all these dependencies. | 17:25:41 |
| sephi | Ah it seems I can just do poetry add --lock gunicorn to only update the pyproject.toml & poetry.lock files without trying to install anything | 17:37:23 |
|  lelit joined the room. | 20:05:15 |
| 23 Jan 2022 |
 mou | i'm a little confused by nix scoping rules and closures. I have this minimal flake file
description = "Application packaged using poetry2nix";
inputs.flake-utils.url = "github:numtide/flake-utils";
inputs.nixpkgs.url = "github:NixOS/nixpkgs";
inputs.poetry2nix.url = "github:nix-community/poetry2nix";
outputs = { self, nixpkgs, flake-utils, poetry2nix }:
{
# Nixpkgs overlay providing the application
overlay = nixpkgs.lib.composeManyExtensions [
poetry2nix.overlay
(final: prev: {
# The application
myapp = prev.poetry2nix.mkPoetryApplication {
projectDir = ./.;
};
})
];
} // (flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [ self.overlay ];
};
in
{
apps = {
myapp = pkgs.myapp;
};
defaultApp = pkgs.myapp;
devShell = poetry2nix.mkPoetryEnv {
projectDir = ./.;
editablePackageSources = {
emptyproject = ./.;
};
};
}));
}
And it's produce this error:
error: attribute 'mkPoetryEnv' missing
at /nix/store/c0qrx1svg4p1xviy2jsmjsqmz09rbdjx-source/flake.nix:33:20:
32| defaultApp = pkgs.myapp;
33| devShell = poetry2nix.mkPoetryEnv {
| ^
34| projectDir = ./.;
(use '--show-trace' to show detailed location information)
I expect what since poetr2nix is an binding references argument of outputs function, it will be available for evaluation of this function expression.
| 14:07:13 |
| mou | * i'm a little confused by nix scoping rules and closures. I have this minimal flake file
description = "Application packaged using poetry2nix";
inputs.flake-utils.url = "github:numtide/flake-utils";
inputs.nixpkgs.url = "github:NixOS/nixpkgs";
inputs.poetry2nix.url = "github:nix-community/poetry2nix";
outputs = { self, nixpkgs, flake-utils, poetry2nix }:
{
# Nixpkgs overlay providing the application
overlay = nixpkgs.lib.composeManyExtensions [
poetry2nix.overlay
(final: prev: {
# The application
myapp = prev.poetry2nix.mkPoetryApplication {
projectDir = ./.;
};
})
];
} // (flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [ self.overlay ];
};
in
{
apps = {
myapp = pkgs.myapp;
};
defaultApp = pkgs.myapp;
devShell = poetry2nix.mkPoetryEnv {
projectDir = ./.;
editablePackageSources = {
emptyproject = ./.;
};
};
}));
}
And it's produce this error:
error: attribute 'mkPoetryEnv' missing
at /nix/store/c0qrx1svg4p1xviy2jsmjsqmz09rbdjx-source/flake.nix:33:20:
32| defaultApp = pkgs.myapp;
33| devShell = poetry2nix.mkPoetryEnv {
| ^
34| projectDir = ./.;
(use '--show-trace' to show detailed location information)
I expect what since poetry2nix is an binding references argument of outputs function, it will be available for evaluation of this function expression.
| 14:07:39 |
| K900 | poetry2nix the flake doesn't have a mkPoetryEnv attribute | 14:08:56 |
| K900 | The poetry2nix package in nixpkgs does | 14:09:46 |
| K900 | If you want to use poetry2nix from git instead of the one in nixpkgs, you can use the overlay in the flake | 14:10:10 |
| mou | confusing. can i do import of git hosted flake in nix repl? to explore it and experiment | 14:11:08 |
| K900 | You can with builtins.getFlake | 14:11:23 |
| mou | thanks | 14:11:30 |
| K900 | But basically you want something like
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable-small";
flake-utils.url = "github:numtide/flake-utils";
poetry2nix = {
url = "github:nix-community/poetry2nix";
inputs.nixpkgs.follows = "nixpkgs";
inputs.flake-utils.follows = "flake-utils";
};
};
outputs = { self, nixpkgs, flake-utils, poetry2nix }:
flake-utils.lib.eachDefaultSystem
(system:
let
pkgs = import nixpkgs {
overlays = [ poetry2nix.overlay ];
inherit system;
};
in
{
packages.your-app = pkgs.poetry2nix.mkPoetryApplication {
# stuff
};
}
)
}
| 14:12:38 |
| mou | also expression in lambda passed to eachDefaultSystem contains this binding
23 inherit system;
24 overlays = [ self.overlay ];
25 };
Does it import nixpkgs with revision locked by flake, or just version defined as channel?
| 14:13:24 |
| K900 | import nixpkgs re-imports the nixpkgs input | 14:14:00 |
| K900 | import <nixpkgs> imports the channel | 14:14:05 |
| K900 | (and will fail if you're not using --impure) | 14:14:14 |
| mou | * also expression in lambda passed to eachDefaultSystem contains this binding
22 pkgs = import nixpkgs {
23 inherit system;
24 overlays = [ self.overlay ];
25 };
Does it import nixpkgs with revision locked by flake, or just version defined as channel?
| 14:14:31 |
| mou | thanks for clarification | 14:14:45 |
| mou | So poetry2nix flake made all other functionality available through applying overlay to nixpkgs? So it kind of glue between everything defined in this flake modules and all other nix infrastructure? | 14:37:58 |
| K900 | poetry2nix is included in nixpkgs | 14:38:30 |
| K900 | The overlay just replaces the stable version in nixpkgs with the latest version from git | 14:38:43 |
