!rWxyQqNqMUDLECdsIf:blad.is

Poetry2nix

313 Members
https://github.com/nix-community/poetry2nix58 Servers

Load older messages


SenderMessageTime
23 Jan 2022
@nialov:matrix.orgnialovI have a question, does poetry2nix use nixpkgs to download ANY of the python packages defined in poetry.lock or does it build all from source?15:24:09
@k900:0upti.meK900It does, but exact cache hits are rare 15:30:22
@k900:0upti.meK900So you'll end up building most things from source 15:30:44
@nialov:matrix.orgnialovOkay thanks! So if you create a default.nix for nixpkgs and submit it, then want to use that same package from poetry2nix you will probably have to redefine some build steps and dependencies twice for a complicated package?15:39:35
@k900:0upti.meK900No15:39:58
@k900:0upti.meK900poetry2nix will use the package definitions from nixpkgs15:40:13
@k900:0upti.meK900But the build environment is slightly different usually, so you'll get rebuuldsn15:40:38
@k900:0upti.meK900* But the build environment is slightly different usually, so you'll get rebuilds15:40:45
@nialov:matrix.orgnialovAh okay, rebuilds are of course fine. Will it use the definition if the version for the package is different?15:41:52
@k900:0upti.meK900Yes 15:42:00
@mou_bugtracker:matrix.orgmou

looks like i almost get my first working flake with dev environment but ended up with code duplication to solve external library build dependency on poetry. And i can not think of good solution to extract duplication. Maybe anybody can suggest good idiomatic solution?

  description = "Exyaru (Atushka) project"; inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs";
  inputs.poetry2nix.url = "github:nix-community/poetry2nix";

  outputs = { self, nixpkgs, flake-utils, poetry2nix }: {
      # Nixpkgs overlay providing the application
      overlay = nixpkgs.lib.composeManyExtensions [ poetry2nix.overlay
        (final: prev: {
          # The application
          exyaru = prev.poetry2nix.mkPoetryApplication {
            projectDir = ./.;
            overrides = prev.poetry2nix.defaultPoetryOverrides.extend ( self: super: {
              uengine = super.uengine.overridePythonAttrs (
                old: {
                  buildInputs = (old.buildInputs or [ ]) ++ [ self.poetry ];
                }
              );
            });
          };
        })
      ];
    } // (flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ self.overlay ];
        };
      in
      rec {
        apps = {
          exyaru = pkgs.exyaru;
        };

        defaultApp = pkgs.exyaru;

        packages = {
          exyaru = pkgs.exyaru;
        };

        defaultPackage = pkgs.exyaru;

        devShell = (pkgs.poetry2nix.mkPoetryEnv {
          projectDir = ./.;
          overrides = pkgs.poetry2nix.defaultPoetryOverrides.extend ( self: super: {
            uengine = super.uengine.overridePythonAttrs (
              old: {
                buildInputs = (old.buildInputs or [ ]) ++ [ self.poetry ];
              }
            );
          });
          editablePackageSources = {
            exyaru = ./.;
          };
        }).env.overrideAttrs (oldAttrs: {
          buildInputs = [ pkgs.poetry ];
        });
      }));
}
15:43:35
@mou_bugtracker:matrix.orgmou *

looks like i almost get my first working flake with dev environment but ended up with code duplication to solve external library build dependency on poetry. And i can not think of good solution to extract duplication. Maybe anybody can suggest good idiomatic solution?

{
  description = "Exyaru (Atushka) project"; inputs.flake-utils.url = "github:numtide/flake-utils";
  inputs.nixpkgs.url = "github:NixOS/nixpkgs";
  inputs.poetry2nix.url = "github:nix-community/poetry2nix";

  outputs = { self, nixpkgs, flake-utils, poetry2nix }: {
      # Nixpkgs overlay providing the application
      overlay = nixpkgs.lib.composeManyExtensions [ poetry2nix.overlay
        (final: prev: {
          # The application
          exyaru = prev.poetry2nix.mkPoetryApplication {
            projectDir = ./.;
            overrides = prev.poetry2nix.defaultPoetryOverrides.extend ( self: super: {
              uengine = super.uengine.overridePythonAttrs (
                old: {
                  buildInputs = (old.buildInputs or [ ]) ++ [ self.poetry ];
                }
              );
            });
          };
        })
      ];
    } // (flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ self.overlay ];
        };
      in
      rec {
        apps = {
          exyaru = pkgs.exyaru;
        };

        defaultApp = pkgs.exyaru;

        packages = {
          exyaru = pkgs.exyaru;
        };

        defaultPackage = pkgs.exyaru;

        devShell = (pkgs.poetry2nix.mkPoetryEnv {
          projectDir = ./.;
          overrides = pkgs.poetry2nix.defaultPoetryOverrides.extend ( self: super: {
            uengine = super.uengine.overridePythonAttrs (
              old: {
                buildInputs = (old.buildInputs or [ ]) ++ [ self.poetry ];
              }
            );
          });
          editablePackageSources = {
            exyaru = ./.;
          };
        }).env.overrideAttrs (oldAttrs: {
          buildInputs = [ pkgs.poetry ];
        });
      }));
}
15:43:57
@mou_bugtracker:matrix.orgmouDuplication is because i need override in mkPoetryApplication and in mkPoetryEnv15:44:29
@mou_bugtracker:matrix.orgmou * Duplication occurred because i need override in mkPoetryApplication and in mkPoetryEnv15:44:56
@tcelferact:matrix.orgtcelferact

Is there a way of overriding the package version specified by poetry and just using whatever is defined in nixpkgs instead? The command I'm imagining might naively look like this:

  overrides = pkgs.poetry2nix.overrides.withDefaults (self: super: {
    debugpy = pkgs.python3Packages.debugpy;
    ...
16:18:23
@k900:0upti.meK900Why do you want that?16:18:40
@tcelferact:matrix.orgtcelferactbecause the poetry build breaks16:18:56
@k900:0upti.meK900Breaks how exactly?16:19:20
@tcelferact:matrix.orgtcelferact

it builds fine, then at runtime I get this Expected: /nix/store/vx25bdp4pqfkmj5zh8vbkxl0z8l05kcd-python3-3.7.12-env/lib/python3.7/site-packages/debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_x86_64.dylib to exist..

It looks like there's some build logic in https://github.com/NixOS/nixpkgs/blob/nixos-21.11/pkgs/development/python-modules/debugpy/default.nix to create the missing file.

I tried adding the preBuild to my overrides:

    debugpy = super.debugpy.overridePythonAttrs (
      old: {
        inherit (pkgs.python3Packages.debugpy) preBuild;
      }
    );

but then I just get a different runtime error /nix/store/qd710abqqj5b1g3zad6m7a4rrq7xa3ym-python3-3.7.12-env/lib/python3.7/site-packages/debugpy/_vendored/force_pydevd.py:20: UserWarning: incompatible copy of pydevd already imported.

Since it's a dev dependency I figured it might be simpler to just use the nix packaged version entirely?

16:22:32
@tcelferact:matrix.orgtcelferactBut perhaps there an alternative to this which just involves adding the correct overrides?16:43:51
@mou_bugtracker:matrix.orgmouRedacted or Malformed Event16:57:44
@mou_bugtracker:matrix.orgmou
In reply to @k900:0upti.me
There's a direnv plugin: https://github.com/fehnomenal/intellij-direnv
I made direnv flawlessly works with my flake in console. But in PyCharm even if force reimport of direnv, nothing happened. It does not detect python in Add interpreter dialog, nor poetry. Can not find any information on how to use this plugin and what effects to expect from it.
16:58:11
@mou_bugtracker:matrix.orgmou
In reply to @k900:0upti.me
There's a direnv plugin: https://github.com/fehnomenal/intellij-direnv
* I made direnv flawlessly works with my flake in console. But in PyCharm even if force reimport of direnv, nothing happens. It does not detect python in Add interpreter dialog, nor poetry. Can not find any information on how to use this plugin and what effects to expect from it.
16:58:31
@k900:0upti.meK900It shouldn't be using Poetry, you should be able to just set it to the interpreter set in your Nix environment16:59:00
@k900:0upti.meK900 You can find out the path with which python 16:59:06
@k900:0upti.meK900It is somewhat annoying16:59:10
@mou_bugtracker:matrix.orgmouNow i understand. So if my env will change path (due to any changes in attributes), i should update my interpreter settings manually? I think i'ts ok for the start.17:01:42
@mou_bugtracker:matrix.orgmou Does poetry2nix respect interpreter version specification from pyproject.toml? 18:22:41
@mou_bugtracker:matrix.orgmou or should i specify it in mkPoetryApplication? 18:23:39
@tcelferact:matrix.orgtcelferact I've confirmed that a poetry2nix build with just the debugpy package experiences errors, and that a shell.nix python environment with just this package does not. Is it ok if I open an issue on the repo for this? 19:56:07

Show newer messages


Back to Room ListRoom Version: 6