!rWxyQqNqMUDLECdsIf:blad.is

Poetry2nix

325 Members
https://github.com/nix-community/poetry2nix61 Servers

Load older messages


SenderMessageTime
26 Jan 2024
@sreec:matrix.orgSreekara Chelamalla *

anyone seen this error with the watchfiles package before?

sreec@Sreekaras-MacBook-Pro data-warehouse % nix-shell --show-trace third_party/dag/shell.nix
...
patching sources
no configure script, doing nothing
building
no Makefile or custom buildPhase, doing nothing
installing
Executing pypaInstallPhase
updateAutotoolsGnuConfigScriptsPhase
unpacking sources
configuring
Executing wheelUnpackPhase
cp: -r not specified; omitting directory '/nix/store/q3zw45mf49af911hphcpb8ng348m3qfq-source'
00:33:03
@sreec:matrix.orgSreekara Chelamalla *

anyone seen this error with the watchfiles package before?

% nix log /nix/store/hs6abwalf22xnn8kqpalxc9w5ix7bz00-python3.11-watchfiles-0.21.0.drv
warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '/nix/store/hs6abwalf22xnn8kqpalxc9w5ix7bz00-python3.11-watchfiles-0.21.0.drv^*'
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing wheel setup hook
Using wheelUnpackPhase
Sourcing pypa-install-hook
Using pypaInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing python-catch-conflicts-hook.sh
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
Executing wheelUnpackPhase
cp: -r not specified; omitting directory '/nix/store/q3zw45mf49af911hphcpb8ng348m3qfq-source
00:33:35
@tyberius_prime:matrix.orgTyberiusPrime (smilodon inopinatus)
In reply to @angryant:envs.net
I am having trouble building a Poetry project which has tiktoken as a dependency. I am assuming the issue is related to https://github.com/nix-community/poetry2nix/issues/261 - but I am unsure how to fix it.

I think your issue is that your overlay is expecing there to be a <nixpkgs>.tiktoken, when it would be at least <nixpkgs>.python310Packages.tiktoken or such.

Are you using mkPoetryApplication or the like?
here's a minimal example taht does this for autograd-gamma (and setuptools) in a project of mine

{
  description = "Application packaged using poetry2nix";

  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    poetry2nix = {
      url = "github:TyberiusPrime/poetry2nix/rpds-py";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = {
    self,
    nixpkgs,
    flake-utils,
    poetry2nix,
  }:
    flake-utils.lib.eachDefaultSystem (system: let
      # see https://github.com/nix-community/poetry2nix/tree/master#api for more functions and examples.
      pkgs = nixpkgs.legacyPackages.${system};
      inherit (poetry2nix.lib.mkPoetry2Nix {inherit pkgs;}) mkPoetryApplication overrides;
    in {
      packages = {
        myapp = mkPoetryApplication {
          projectDir = self;
          preferWheels = true;
          overrides =
            overrides.withDefaults
            (
              self: super: {
                autograd-gamma = super.autograd-gamma.overridePythonAttrs (old: {
                  propagatedBuildInputs = (old.propagatedBuildInputs or []) ++ [self.setuptools];
                });
                           }
            );
        };
        default = self.packages.${system}.myapp;
      };

      devShells.default = pkgs.mkShell {
        inputsFrom = [self.packages.${system}.myapp];
        packages = [pkgs.poetry];
      };
    });
}
10:45:02
@angryant:envs.netAngryAnt
In reply to @tyberius_prime:matrix.org

I think your issue is that your overlay is expecing there to be a <nixpkgs>.tiktoken, when it would be at least <nixpkgs>.python310Packages.tiktoken or such.

Are you using mkPoetryApplication or the like?
here's a minimal example taht does this for autograd-gamma (and setuptools) in a project of mine

{
  description = "Application packaged using poetry2nix";

  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    poetry2nix = {
      url = "github:TyberiusPrime/poetry2nix/rpds-py";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = {
    self,
    nixpkgs,
    flake-utils,
    poetry2nix,
  }:
    flake-utils.lib.eachDefaultSystem (system: let
      # see https://github.com/nix-community/poetry2nix/tree/master#api for more functions and examples.
      pkgs = nixpkgs.legacyPackages.${system};
      inherit (poetry2nix.lib.mkPoetry2Nix {inherit pkgs;}) mkPoetryApplication overrides;
    in {
      packages = {
        myapp = mkPoetryApplication {
          projectDir = self;
          preferWheels = true;
          overrides =
            overrides.withDefaults
            (
              self: super: {
                autograd-gamma = super.autograd-gamma.overridePythonAttrs (old: {
                  propagatedBuildInputs = (old.propagatedBuildInputs or []) ++ [self.setuptools];
                });
                           }
            );
        };
        default = self.packages.${system}.myapp;
      };

      devShells.default = pkgs.mkShell {
        inputsFrom = [self.packages.${system}.myapp];
        packages = [pkgs.poetry];
      };
    });
}

Thanks! I am using mkPoetryPackages as I just need the packages in a composite environment where the Poetry setup is only part of things. For now I have bypassed the issue via preferWheels, but I'll give your setup a closer look to see if I can sort things out without.

  poetry = pkgs.poetry2nix.mkPoetryPackages {
    projectDir = pythonPath;
    python = pkgs.python311;
    preferWheels = true;
  };
10:49:38
@angryant:envs.netAngryAnt *

Thanks! I am using mkPoetryPackages as I just need the packages in a composite environment where the Poetry setup is only part of things. For now I have bypassed the issue via preferWheels, but I'll give your setup a closer look to see if I can sort things out without.

  poetry = pkgs.poetry2nix.mkPoetryPackages {
    projectDir = pythonPath;
    python = pkgs.python311;
    preferWheels = true;
  };
10:49:53
@tyberius_prime:matrix.orgTyberiusPrime (smilodon inopinatus)
In reply to @angryant:envs.net

Thanks! I am using mkPoetryPackages as I just need the packages in a composite environment where the Poetry setup is only part of things. For now I have bypassed the issue via preferWheels, but I'll give your setup a closer look to see if I can sort things out without.

  poetry = pkgs.poetry2nix.mkPoetryPackages {
    projectDir = pythonPath;
    python = pkgs.python311;
    preferWheels = true;
  };
mkPoetryPackages appears to take overrides as well.
10:50:37
@angryant:envs.netAngryAntI would assume as much. It is a very promising direction.10:51:17
@tyberius_prime:matrix.orgTyberiusPrime (smilodon inopinatus)Just beware, there's a minor issue with the order that overrides.withDefaults applies, and you might have to read the source and do it the other way around if you want to add an override where poetry2nix already has one.10:51:20
@tyberius_prime:matrix.orgTyberiusPrime (smilodon inopinatus)what llm contraption are you trying to nixify, If I might ask.10:51:39
@angryant:envs.netAngryAntWe're developing a composite python-dotnet chat & project management tool.10:53:00
@angryant:envs.netAngryAntMostly doing a lot of bridging.10:53:28
@tyberius_prime:matrix.orgTyberiusPrime (smilodon inopinatus)Now I have 'python-inside-of-delphi' flashbacks :). Good luck with that.10:54:59
@angryant:envs.netAngryAntThanks :) The composite part has worked well for a long while - we're not really worried there. The new part is moving to Poetry, so we can have as much of python development as possible not require syncing updates to the nix shell script.10:57:44
@willhansen:matrix.orgWill Hansen joined the room.16:19:53
@mroxatron:matrix.orgAndy Mroczkowski joined the room.22:43:32
@mroxatron:matrix.orgAndy Mroczkowski

Hi, I'm a novice nix user. I’m trying to write a flake for a poetry project at work. The existing project has gnureadline in the pyproject.toml dev dependencies:

[tool.poetry.group.dev.dependencies]
gnureadline = "^8.1.2"

When I try to build the flake, I get the following error:

  ld: library not found for -lncurses
  clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
  error: command '/nix/store/2k44jw0kwqnymimzfwq1p53s59rvbvzr-clang-wrapper-16.0.6/bin/clang' failed with exit code 1

A stripped-down example of the issue is here: https://github.com/amrox/poetry2nix-gnureadline

Any ideas how to resolve this?

22:49:46
@angryant:envs.netAngryAnt
In reply to @mroxatron:matrix.org

Hi, I'm a novice nix user. I’m trying to write a flake for a poetry project at work. The existing project has gnureadline in the pyproject.toml dev dependencies:

[tool.poetry.group.dev.dependencies]
gnureadline = "^8.1.2"

When I try to build the flake, I get the following error:

  ld: library not found for -lncurses
  clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
  error: command '/nix/store/2k44jw0kwqnymimzfwq1p53s59rvbvzr-clang-wrapper-16.0.6/bin/clang' failed with exit code 1

A stripped-down example of the issue is here: https://github.com/amrox/poetry2nix-gnureadline

Any ideas how to resolve this?

ncurses is a native dependency of one of your python libraries. Just need to add that in as well.
22:54:23
27 Jan 2024
@mroxatron:matrix.orgAndy Mroczkowski thanks, but I still don't understand how to do it. I found this article https://github.com/nix-community/poetry2nix/blob/master/docs/edgecases.md which helped me get past the build issue with debian-parser. I still don't understand where to add the native dependencies. This article talks about it https://www.tweag.io/blog/2020-08-12-poetry2nix/ but I don't follow exactly how 03:24:09
@mroxatron:matrix.orgAndy MroczkowskiI think I figured it out with inspiration from https://github.com/nix-community/poetry2nix/blob/master/overrides/default.nix03:58:36
31 Jan 2024
@federicodschonborn:matrix.org@federicodschonborn:matrix.org changed their profile picture.03:36:47
@federicodschonborn:matrix.org@federicodschonborn:matrix.org changed their profile picture.06:22:22
1 Feb 2024
@vengmark2:matrix.orgl0b0

I'm getting this error when trying to enter a poetry2nix environment with latest poetry2nix master and gitlint 0.19.1:

ValueError: Unable to determine which files to ship inside the wheel using the following heuristics: https://hatch.pypa.io/latest/plugins/builder/wheel/#default-file-selection

23:15:43
@vengmark2:matrix.orgl0b0 *

I'm getting this error when trying to enter a poetry2nix environment with latest poetry2nix master and gitlint 0.19.1:

ValueError: Unable to determine which files to ship inside the wheel using the following heuristics: https://hatch.pypa.io/latest/plugins/builder/wheel/#default-file-selection

Not sure how to deal with that.

23:19:05
@vengmark2:matrix.orgl0b0 *

I'm getting this error when trying to enter a poetry2nix environment with latest poetry2nix master and gitlint 0.19.1:

ValueError: Unable to determine which files to ship inside the wheel using the following heuristics: https://hatch.pypa.io/latest/plugins/builder/wheel/#default-file-selection

Not sure how to deal with that.

tests/gitlint already runs against 0.19.1, so presumably it has something to do with combining poetry2nix master with nixos-unstable nixpkgs.

23:30:14
@vengmark2:matrix.orgl0b0 *

I'm getting this error when trying to enter a poetry2nix environment with latest poetry2nix master and gitlint 0.19.1:

ValueError: Unable to determine which files to ship inside the wheel using the following heuristics: https://hatch.pypa.io/latest/plugins/builder/wheel/#default-file-selection

Not sure how to deal with that.

tests/gitlint already runs against 0.19.1, so presumably it has something to do with combining poetry2nix master with recent nixpkgs. This fails on both nixos-unstable and nixos-23.11.

23:32:40
5 Feb 2024
@gaivs:matrix.orggaivsHello, I'm currently getting a collision error and I'm not sure how to fix it. The error is: `error: collision between `/nix/store/qnrbkh9xk85xkm5s0h03rihzzp2bv117-python3.10-py3rosmsgs-1.18.2/lib/python3.10/site-packages/genpy/__pycache__/__init__.cpython-310.opt-1.pyc' and `/nix/store/n5ncx7zza0852fawz56r2cn32ydv4yw8-python3.10-genpy-2022.1/lib/python3.10/site-packages/genpy/__pycache__/__init__.cpython-310.opt-1.pyc'`11:07:32
@gaivs:matrix.orggaivsOr if anyone have any pointers to how I can debug this issue, it would be much appreciated 😊11:08:27
@djacu:matrix.org@djacu:matrix.orgWe would need more context. What's your kernel configuration look like?17:03:16
@gaivs:matrix.orggaivsI found a similar problem on the issues page, and ended up removing the offending files through the override, e.g. postInstall='' rm -f $out/lib/python3.10/site-packafes/genpy/__pycache__/__init__.cpython-310.opt.pyc ''; I can post a simplified version of my flake and pyproject tomorrow20:05:14
6 Feb 2024
@phaer:matrix.org@phaer:matrix.orgGoing by the paths alone, it looks like py3rosmsgs vendors genpy, so you'd end up with two versions of that module in PYTHON_PATH. 09:01:53

Show newer messages


Back to Room ListRoom Version: 6