!rWxyQqNqMUDLECdsIf:blad.is

Poetry2nix

320 Members
https://github.com/nix-community/poetry2nix60 Servers

Load older messages


SenderMessageTime
23 Jan 2022
@nialov:matrix.orgnialovAh okay, rebuilds are of course fine. Will it use the definition if the version for the package is different?15:41:52
@k900:0upti.meK900Yes 15:42:00
@mou_bugtracker:matrix.orgmou

looks like i almost get my first working flake with dev environment but ended up with code duplication to solve external library build dependency on poetry. And i can not think of good solution to extract duplication. Maybe anybody can suggest good idiomatic solution?

  description = "Exyaru (Atushka) project"; inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs";
  inputs.poetry2nix.url = "github:nix-community/poetry2nix";

  outputs = { self, nixpkgs, flake-utils, poetry2nix }: {
      # Nixpkgs overlay providing the application
      overlay = nixpkgs.lib.composeManyExtensions [ poetry2nix.overlay
        (final: prev: {
          # The application
          exyaru = prev.poetry2nix.mkPoetryApplication {
            projectDir = ./.;
            overrides = prev.poetry2nix.defaultPoetryOverrides.extend ( self: super: {
              uengine = super.uengine.overridePythonAttrs (
                old: {
                  buildInputs = (old.buildInputs or [ ]) ++ [ self.poetry ];
                }
              );
            });
          };
        })
      ];
    } // (flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ self.overlay ];
        };
      in
      rec {
        apps = {
          exyaru = pkgs.exyaru;
        };

        defaultApp = pkgs.exyaru;

        packages = {
          exyaru = pkgs.exyaru;
        };

        defaultPackage = pkgs.exyaru;

        devShell = (pkgs.poetry2nix.mkPoetryEnv {
          projectDir = ./.;
          overrides = pkgs.poetry2nix.defaultPoetryOverrides.extend ( self: super: {
            uengine = super.uengine.overridePythonAttrs (
              old: {
                buildInputs = (old.buildInputs or [ ]) ++ [ self.poetry ];
              }
            );
          });
          editablePackageSources = {
            exyaru = ./.;
          };
        }).env.overrideAttrs (oldAttrs: {
          buildInputs = [ pkgs.poetry ];
        });
      }));
}
15:43:35
@mou_bugtracker:matrix.orgmou *

looks like i almost get my first working flake with dev environment but ended up with code duplication to solve external library build dependency on poetry. And i can not think of good solution to extract duplication. Maybe anybody can suggest good idiomatic solution?

{
  description = "Exyaru (Atushka) project"; inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs";
  inputs.poetry2nix.url = "github:nix-community/poetry2nix";

  outputs = { self, nixpkgs, flake-utils, poetry2nix }: {
      # Nixpkgs overlay providing the application
      overlay = nixpkgs.lib.composeManyExtensions [ poetry2nix.overlay
        (final: prev: {
          # The application
          exyaru = prev.poetry2nix.mkPoetryApplication {
            projectDir = ./.;
            overrides = prev.poetry2nix.defaultPoetryOverrides.extend ( self: super: {
              uengine = super.uengine.overridePythonAttrs (
                old: {
                  buildInputs = (old.buildInputs or [ ]) ++ [ self.poetry ];
                }
              );
            });
          };
        })
      ];
    } // (flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ self.overlay ];
        };
      in
      rec {
        apps = {
          exyaru = pkgs.exyaru;
        };

        defaultApp = pkgs.exyaru;

        packages = {
          exyaru = pkgs.exyaru;
        };

        defaultPackage = pkgs.exyaru;

        devShell = (pkgs.poetry2nix.mkPoetryEnv {
          projectDir = ./.;
          overrides = pkgs.poetry2nix.defaultPoetryOverrides.extend ( self: super: {
            uengine = super.uengine.overridePythonAttrs (
              old: {
                buildInputs = (old.buildInputs or [ ]) ++ [ self.poetry ];
              }
            );
          });
          editablePackageSources = {
            exyaru = ./.;
          };
        }).env.overrideAttrs (oldAttrs: {
          buildInputs = [ pkgs.poetry ];
        });
      }));
}
15:43:57
@mou_bugtracker:matrix.orgmouDuplication is because i need override in mkPoetryApplication and in mkPoetryEnv15:44:29
@mou_bugtracker:matrix.orgmou * Duplication occurred because i need override in mkPoetryApplication and in mkPoetryEnv15:44:56
@tcelferact:matrix.orgtcelferact

Is there a way of overriding the package version specified by poetry and just using whatever is defined in nixpkgs instead? The command I'm imagining might naively look like this:

  overrides = pkgs.poetry2nix.overrides.withDefaults (self: super: {
    debugpy = pkgs.python3Packages.debugpy;
    ...
16:18:23
@k900:0upti.meK900Why do you want that?16:18:40
@tcelferact:matrix.orgtcelferactbecause the poetry build breaks16:18:56
@k900:0upti.meK900Breaks how exactly?16:19:20
@tcelferact:matrix.orgtcelferact

it builds fine, then at runtime I get this Expected: /nix/store/vx25bdp4pqfkmj5zh8vbkxl0z8l05kcd-python3-3.7.12-env/lib/python3.7/site-packages/debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_x86_64.dylib to exist..

It looks like there's some build logic in https://github.com/NixOS/nixpkgs/blob/nixos-21.11/pkgs/development/python-modules/debugpy/default.nix to create the missing file.

I tried adding the preBuild to my overrides:

    debugpy = super.debugpy.overridePythonAttrs (
      old: {
        inherit (pkgs.python3Packages.debugpy) preBuild;
      }
    );

but then I just get a different runtime error /nix/store/qd710abqqj5b1g3zad6m7a4rrq7xa3ym-python3-3.7.12-env/lib/python3.7/site-packages/debugpy/_vendored/force_pydevd.py:20: UserWarning: incompatible copy of pydevd already imported.

Since it's a dev dependency I figured it might be simpler to just use the nix packaged version entirely?

16:22:32
@tcelferact:matrix.orgtcelferactBut perhaps there an alternative to this which just involves adding the correct overrides?16:43:51
@mou_bugtracker:matrix.orgmouRedacted or Malformed Event16:57:44
@mou_bugtracker:matrix.orgmou
In reply to @k900:0upti.me
There's a direnv plugin: https://github.com/fehnomenal/intellij-direnv
I made direnv flawlessly works with my flake in console. But in PyCharm even if force reimport of direnv, nothing happened. It does not detect python in Add interpreter dialog, nor poetry. Can not find any information on how to use this plugin and what effects to expect from it.
16:58:11
@mou_bugtracker:matrix.orgmou
In reply to @k900:0upti.me
There's a direnv plugin: https://github.com/fehnomenal/intellij-direnv
* I made direnv flawlessly works with my flake in console. But in PyCharm even if force reimport of direnv, nothing happens. It does not detect python in Add interpreter dialog, nor poetry. Can not find any information on how to use this plugin and what effects to expect from it.
16:58:31
@k900:0upti.meK900It shouldn't be using Poetry, you should be able to just set it to the interpreter set in your Nix environment16:59:00
@k900:0upti.meK900 You can find out the path with which python 16:59:06
@k900:0upti.meK900It is somewhat annoying16:59:10
@mou_bugtracker:matrix.orgmouNow i understand. So if my env will change path (due to any changes in attributes), i should update my interpreter settings manually? I think i'ts ok for the start.17:01:42
@mou_bugtracker:matrix.orgmou Does poetry2nix respect interpreter version specification from pyproject.toml? 18:22:41
@mou_bugtracker:matrix.orgmou or should i specify it in mkPoetryApplication? 18:23:39
@tcelferact:matrix.orgtcelferact I've confirmed that a poetry2nix build with just the debugpy package experiences errors, and that a shell.nix python environment with just this package does not. Is it ok if I open an issue on the repo for this? 19:56:07
@worldofgeese:one.ems.hostworldofgeese joined the room.20:50:02
@cgroves:matrix.orgcgroves joined the room.22:10:14
@magisa:matrix.orgmagisa joined the room.23:44:23
@magisa:matrix.orgmagisaHello all! I am very new to this so apologies in advance if I ask something very dumb. I want to install lmdb bindings for python. However if i create a simple poetry pyproject.toml only with that dependancy the build fails. I understand why. It's because there is a very dumb line in there (https://github.com/jnwatson/py-lmdb/blob/master/setup.py#L117-L119) which tries to access /usr/bin/patch. That obviously does not exist on NixOS. I am trying to make this work but I am kind of struggling. If I was to build the package imperative stylez I would install lmdb and then export LMDB_FORCE_SYSTEM before calling setup.py. I know how do the former (I think) from extraPackages in mkPoetryEnv but I do not know how do the latter. I cannot find an option to add a bash script before the build. Appreciate any help you can give me. Thx23:52:27
24 Jan 2022
@magisa:matrix.orgmagisanever mind I am dumb sorry for the distrurbance all. And thanks to all devs for the cool work!!!! 00:05:23
@mou_bugtracker:matrix.orgmou
In reply to @magisa:matrix.org
Hello all!
I am very new to this so apologies in advance if I ask something very dumb. I want to install lmdb bindings for python. However if i create a simple poetry pyproject.toml only with that dependancy the build fails. I understand why. It's because there is a very dumb line in there (https://github.com/jnwatson/py-lmdb/blob/master/setup.py#L117-L119) which tries to access /usr/bin/patch. That obviously does not exist on NixOS. I am trying to make this work but I am kind of struggling. If I was to build the package imperative stylez I would install lmdb and then export LMDB_FORCE_SYSTEM before calling setup.py. I know how do the former (I think) from extraPackages in mkPoetryEnv but I do not know how do the latter. I cannot find an option to add a bash script before the build.
Appreciate any help you can give me. Thx

i'm also beginner so my advice may not lead you to solution. But if i faced this problem, in first place i would look into implementation of this module in nixpkgs https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/lmdb/default.nix

As i understand, for poetry2nix you can define override, which will apply changes to default derivation parameters right before building

07:24:56
@mou_bugtracker:matrix.orgmouAs you can see, environment variables are defined as simple attribute in argument set for buildPythonPackage (which is also used inside poetry2nix)07:29:34
@cchalc:matrix.orgcchalcI am trying to install packages with poetry but running into permission errors. ~/Projects/nix-poetry(branch:dev*)(nix-shell)$ poetry add flask Skipping virtualenv creation, as specified in config file. Using version ^2.0.2 for Flask Updating dependencies Resolving dependencies... (0.6s) Writing lock file Package operations: 5 installs, 5 updates, 0 removals • Installing markupsafe (2.0.1): Failed EnvCommandError Command ['/nix/store/5w2swv66p5sl140my96vcipzi2vfrcxq-python3-3.8.12-env/bin/python3.8', '-m', 'pip', 'install', '--no-deps', '/Users/christopher.chalcraft/Library/Caches/pypoetry/artifacts/cd/11/54/fd481eec49cddc06876e526483e7ee8675d0910fed2aa7668a9fc88e62/MarkupSafe-2.0.1-cp38-cp38-macosx_10_9_x86_64.whl'] errored with the following return code 1, and output: Processing /Users/christopher.chalcraft/Library/Caches/pypoetry/artifacts/cd/11/54/fd481eec49cddc06876e526483e7ee8675d0910fed2aa7668a9fc88e62/MarkupSafe-2.0.1-cp38-cp38-macosx_10_9_x86_64.whl Installing collected packages: MarkupSafe ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/nix/store/5w2swv66p5sl140my96vcipzi2vfrcxq-python3-3.8.12-env/lib/python3.8/site-packages/markupsafe' Check the permissions. I have a shell.nix that looks like this: { pkgs ? import <nixpkgs> {} }: with import <nixpkgs> {}; let pythonEnv = python38.withPackages (ps: with ps; [ flake8 black pynvim ipython python-language-server.override { pylint = null; } pip poetry ]); in pkgs.mkShell { buildInputs = with pkgs; [ pythonEnv ]; } 08:16:27

Show newer messages


Back to Room ListRoom Version: 6