!rWxyQqNqMUDLECdsIf:blad.is

Poetry2nix

326 Members
https://github.com/nix-community/poetry2nix62 Servers

Load older messages


SenderMessageTime
3 Feb 2022
@jyamad:cofree.coffeejyamad joined the room.20:43:40
@lewo:matrix.orglewo joined the room.21:47:31
8 Feb 2022
@graf_blutwurst:matrix.orgDominic Egger joined the room.05:38:54
@graf_blutwurst:matrix.orgDominic EggerGood Morning everyone. I am currently working on a little showcase workbench/projects for python based friends that don't know nix. They work fairly extensively with numpy and pandas so i tried adding those to poetry and so also to my environment. but it seems that that causes pandas and numpy to fully recompile them locally. i also tried pinning the versions to ones specifically available on nix. does anyone have some insight into this?05:41:33
@adis:blad.isadisbladis
In reply to @graf_blutwurst:matrix.org
Good Morning everyone. I am currently working on a little showcase workbench/projects for python based friends that don't know nix. They work fairly extensively with numpy and pandas so i tried adding those to poetry and so also to my environment. but it seems that that causes pandas and numpy to fully recompile them locally. i also tried pinning the versions to ones specifically available on nix. does anyone have some insight into this?
That's fully expected. Even if you would somehow pin the exact same versions that are available in nixpkgs the build closure and resulting derivations are not exactly the same.
05:52:31
@graf_blutwurst:matrix.orgDominic Eggerhm it seems i have to better understand build closures better. i assumed a dependency like that would be "low" enough that it could be picked up from the store/nixpkgs. thanks though! how do people deal with the long resulting build times? push the result from mkPoetryEnv to a binary cache?06:02:07
@adis:blad.isadisbladis
In reply to @graf_blutwurst:matrix.org
hm it seems i have to better understand build closures better. i assumed a dependency like that would be "low" enough that it could be picked up from the store/nixpkgs. thanks though! how do people deal with the long resulting build times? push the result from mkPoetryEnv to a binary cache?
Typically, yes.
06:02:46
@graf_blutwurst:matrix.orgDominic Egger
In reply to @adis:blad.is
Typically, yes.
ok then i was on the right track for that. do you happen to have any resources that'd help me understand what causes the recompile of pandas etc? mostly so i can predict when it happens since those take a significant chunk of time to crunch through
06:06:31
@adis:blad.isadisbladis
In reply to @graf_blutwurst:matrix.org
ok then i was on the right track for that. do you happen to have any resources that'd help me understand what causes the recompile of pandas etc? mostly so i can predict when it happens since those take a significant chunk of time to crunch through

When some of the build inputs changes, or some parameters are different.
poetry2nix tries as much as possible to ignore nixpkgs python packages which will cause the derivations to be different from nixpkgs ~all of the time.

Practially this means that you will have to rebuild when:

  • You update nixpkgs
  • Any of the python packages that pandas/numpy depends on changes in your poetry.lock
06:08:37
@graf_blutwurst:matrix.orgDominic Eggerthat sounds sensible.thanks for your time!06:17:11
12 Feb 2022
@octavioduarte:matrix.orgOctavio Martín Duarte joined the room.15:26:44
@octavioduarte:matrix.orgOctavio Martín Duarte

Hello!
I'm suffering from a problem that seems to have been already detected and fixed...

I get this failure even for the Flask exaple from the Tweag blogpost and the video. I get the message about an infinite recursion and when I use the verbose switch it get to read that flint-core is detected as missing. Adding the package as a dependendency does not stop this from failing.

15:54:06
@mou_bugtracker:matrix.orgmouOctavio, could you provide details about your environment? Error message, pyproject.toml, your nix expression will help anyone to better identify root cause.18:45:31
14 Feb 2022
@octavioduarte:matrix.orgOctavio Martín DuarteDownload pyproject.toml02:09:18
@octavioduarte:matrix.orgOctavio Martín Duarte
In reply to @mou_bugtracker:matrix.org
Octavio, could you provide details about your environment? Error message, pyproject.toml, your nix expression will help anyone to better identify root cause.
Off course! I didn't want to send the files without being asked to but if anybody can help that would be amazing.
02:09:19
@octavioduarte:matrix.orgOctavio Martín DuarteDownload shell.nix02:09:28
@octavioduarte:matrix.orgOctavio Martín DuarteDownload default.nix02:09:37
@octavioduarte:matrix.orgOctavio Martín DuarteThis is the error.02:13:36
@octavioduarte:matrix.orgOctavio Martín DuarteDownload error.md02:13:46
@worldofgeese:one.ems.hostworldofgeeseCould I ask someone to test the Poetry2nix flake template with some trivial Python code and see if it works for them? I'm getting `error: flake 'git+file:///home/worldofgeese/projects/little-bits-of-buddha' does not provide attribute 'packages.x86_64-linux.defaultPackage.x86_64-linux', 'legacyPackages.x86_64-linux.defaultPackage.x86_64-linux' or 'defaultPackage.x86_64-linux'`22:36:56
@worldofgeese:one.ems.hostworldofgeeseI was able to solve by swapping for Serokell's Poetry2nix template22:37:20
@worldofgeese:one.ems.hostworldofgeeseHost system is Arch Linux22:37:26
15 Feb 2022
@asymmetric:matrix.dapp.org.ukasymmetric joined the room.11:46:48
@asymmetric:matrix.dapp.org.ukasymmetric hi! i'm a bit of a python newbie: how would i go about adding an output to my flake, where i run py.test? i can use mkPoetryEnv.env to get a shell with dev dependencies installed, but how do i also get tests to run? 11:48:27
@asymmetric:matrix.dapp.org.ukasymmetric i realized i can pass checkPhase = ... to mkPoetryApplication, but i cannot pass the same attrset to mkPoetryEnv, as mkPoetryEnv doesn't like having unuexpected attrs (and I'd like to not repeat myself, as I have to define overlays). i could filter out the checkPhase attribute from the attrset i pass to both function, but it seems a bit hacky. is there a better way? 12:12:55
@asymmetric:matrix.dapp.org.ukasymmetric
  outputs = { self, nixpkgs, poetry2nix }:
    let
      name = "foo";
      system = "x86_64-linux";

      pkgs = import nixpkgs {
        inherit system;
        overlays = [ poetry2nix.overlay ];
      };

      poetryAttrs = {
        projectDir = ./.;
        python = pkgs.python39;

        checkPhase = ''
          py.test tests/
        '';

        overrides = pkgs.poetry2nix.overrides.withDefaults (final: prev: {
          lmdb = prev.lmdb.overrideAttrs
            (
              old: {
                # we need to patch the hard-coded path to patch,
                # see https://github.com/jnwatson/py-lmdb/blob/80d7ce6fd5531a2b2f586d9ec871fe60126f8aba/setup.py#L117
                patchPhase = ''
                  substituteInPlace ./setup.py --replace /usr/bin/patch ${pkgs.patch}/bin/patch
                '';
              }
            );
        });
      };
    in
    {
      packages.${system} = {
        ${name} = pkgs.poetry2nix.mkPoetryApplication poetryAttrs;
      };

      devShell.${system} =
        (pkgs.poetry2nix.mkPoetryEnv poetryAttrs).env;

    };
...
}
12:15:13
@asymmetric:matrix.dapp.org.ukasymmetric devShell.${system} = (pkgs.poetry2nix.mkPoetryEnv (pkgs.lib.filterAttrs (n: v: n != "checkPhase") poetryAttrs)).env; this is the workaround i've found 12:21:18
@asymmetric:matrix.dapp.org.ukasymmetric^ugly12:21:38
@k900:0upti.meK900Do it the other way around? 12:36:16
@k900:0upti.meK900 Have the shared args in a variable, and pass sharedArgs // { checkPhase = "blah" } to mkPoetryApplication 12:36:55

Show newer messages


Back to Room ListRoom Version: 6