!rWxyQqNqMUDLECdsIf:blad.is

Poetry2nix

329 Members
https://github.com/nix-community/poetry2nix66 Servers

Load older messages


SenderMessageTime
2 Jan 2024
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


Since it's essential to use override instead of overridePythonAttrs to set the value of preferWheel as described in #948, how can you successfully override both preferWheel and, for example, nativeBuildInputs? In this case

              dm-tree = super.dm-tree.override (
                old: {
                  preferWheel = false;
                  nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ pkgs.cmake ];
                }
              );

successfully disables the global setting of preferWheels = true for the target package but it seems nativeBuildInputs is disregarded in several formulations ( or possibly cmake needs to be provided elsewhere and/or with additional configuration since the build fails where the target package setup.py calls cmake via subprocess to check that cmake is available on the PATH ). Attempting to disable the wheel with override and include cmake in the overridePythonAttrs nativeBuildInputs is further down in this thread.

18:13:00
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


If you'd like to combine override and overridePythonAttrs, using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

functions as expected.


** THE REMAINDER BELOW IS ALMOST SURELY NOT WORTH READING **

Since it's essential to use override instead of overridePythonAttrs to set the value of preferWheel as described in #948, how can you successfully override both preferWheel and, for example, nativeBuildInputs? In this case

              dm-tree = super.dm-tree.override (
                old: {
                  preferWheel = false;
                  nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ pkgs.cmake ];
                }
              );

successfully disables the global setting of preferWheels = true for the target package but it seems nativeBuildInputs is disregarded in several formulations ( or possibly cmake needs to be provided elsewhere and/or with additional configuration since the build fails where the target package setup.py calls cmake via subprocess to check that cmake is available on the PATH ). Attempting to disable the wheel with override and include cmake in the overridePythonAttrs nativeBuildInputs is further down in this thread.

18:14:56
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


If you'd like to combine override and overridePythonAttrs, using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

functions as expected.


THE REMAINDER BELOW IS ALMOST SURELY NOT WORTH READING

Since it's essential to use override instead of overridePythonAttrs to set the value of preferWheel as described in #948, how can you successfully override both preferWheel and, for example, nativeBuildInputs? In this case

              dm-tree = super.dm-tree.override (
                old: {
                  preferWheel = false;
                  nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ pkgs.cmake ];
                }
              );

successfully disables the global setting of preferWheels = true for the target package but it seems nativeBuildInputs is disregarded in several formulations ( or possibly cmake needs to be provided elsewhere and/or with additional configuration since the build fails where the target package setup.py calls cmake via subprocess to check that cmake is available on the PATH ). Attempting to disable the wheel with override and include cmake in the overridePythonAttrs nativeBuildInputs is further down in this thread.

18:15:13
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


If you'd like to combine override and overridePythonAttrs, using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

functions as expected.


THE REMAINDER BELOW IS ALMOST SURELY NOT WORTH READING

Since it's essential to use override instead of overridePythonAttrs to set the value of preferWheel as described in #948, how can you successfully override both preferWheel and, for example, nativeBuildInputs? In this case

              dm-tree = super.dm-tree.override (
                old: {
                  preferWheel = false;
                  nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ pkgs.cmake ];
                }
              );

successfully disables the global setting of preferWheels = true for the target package but it seems nativeBuildInputs is disregarded in several formulations ( or possibly cmake needs to be provided elsewhere and/or with additional configuration since the build fails where the target package setup.py calls cmake via subprocess to check that cmake is available on the PATH ). Attempting to disable the wheel with override and include cmake in the overridePythonAttrs nativeBuildInputs is further down in this thread.

18:15:32
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


If you'd like to combine override and overridePythonAttrs, using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

functions as expected.


THE REMAINDER BELOW IS ALMOST SURELY NOT WORTH READING

Since it's essential to use override instead of overridePythonAttrs to set the value of preferWheel as described in #948, how can you successfully override both preferWheel and, for example, nativeBuildInputs? In this case

              dm-tree = super.dm-tree.override (
                old: {
                  preferWheel = false;
                  nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ pkgs.cmake ];
                }
              );

successfully disables the global setting of preferWheels = true for the target package but it seems nativeBuildInputs is disregarded in several formulations ( or possibly cmake needs to be provided elsewhere and/or with additional configuration since the build fails where the target package setup.py calls cmake via subprocess to check that cmake is available on the PATH ). Attempting to disable the wheel with override and include cmake in the overridePythonAttrs nativeBuildInputs is further down in this thread.

18:15:49
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


If you'd like to combine override and overridePythonAttrs, using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

functions as expected.


THE REMAINDER BELOW IS ALMOST SURELY NOT WORTH READING

Since it's essential to use override instead of overridePythonAttrs to set the value of preferWheel as described in #948, how can you successfully override both preferWheel and, for example, nativeBuildInputs? In this case

              dm-tree = super.dm-tree.override (
                old: {
                  preferWheel = false;
                  nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ pkgs.cmake ];
                }
              );

successfully disables the global setting of preferWheels = true for the target package but it seems nativeBuildInputs is disregarded in several formulations ( or possibly cmake needs to be provided elsewhere and/or with additional configuration since the build fails where the target package setup.py calls cmake via subprocess to check that cmake is available on the PATH ). Attempting to disable the wheel with override and include cmake in the overridePythonAttrs nativeBuildInputs is further down in this thread.

18:17:31
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


If you'd like to combine override and overridePythonAttrs, using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

functions as expected.


THE REMAINDER OF THIS THREAD IS ALMOST SURELY NOT WORTH READING

Since it's essential to use override instead of overridePythonAttrs to set the value of preferWheel as described in #948, how can you successfully override both preferWheel and, for example, nativeBuildInputs? In this case

              dm-tree = super.dm-tree.override (
                old: {
                  preferWheel = false;
                  nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ pkgs.cmake ];
                }
              );

successfully disables the global setting of preferWheels = true for the target package but it seems nativeBuildInputs is disregarded in several formulations ( or possibly cmake needs to be provided elsewhere and/or with additional configuration since the build fails where the target package setup.py calls cmake via subprocess to check that cmake is available on the PATH ). Attempting to disable the wheel with override and include cmake in the overridePythonAttrs nativeBuildInputs is further down in this thread.

18:17:58
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


If you'd like to combine override and overridePythonAttrs, using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

functions as expected.

18:19:03
@cameronraysmith:matrix.orgcameronraysmith *

THE REMAINDER OF THIS THREAD IS ALMOST SURELY NOT WORTH READING

Since it's essential to use override instead of overridePythonAttrs to set the value of preferWheel as described in #948, how can you successfully override both preferWheel and, for example, nativeBuildInputs? In this case

              dm-tree = super.dm-tree.override (
                old: {
                  preferWheel = false;
                  nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ pkgs.cmake ];
                }
              );

successfully disables the global setting of preferWheels = true for the target package but it seems nativeBuildInputs is disregarded in several formulations ( or possibly cmake needs to be provided elsewhere and/or with additional configuration since the build fails where the target package setup.py calls cmake via subprocess to check that cmake is available on the PATH ). Attempting to disable the wheel with override and include cmake in the overridePythonAttrs nativeBuildInputs is further down in this thread.


Using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

persists in downloading the wheel and fails with does not appear to contain CMakeLists.txt because it has downloaded the wheel instead of the source. That is successfully disabled, leading to the source download with ONLY dm-tree = super.dm-tree.override {preferWheel = false;}, but then the latter lacks cmake in the build env and fails for that reason at the point linked from the top of the thread.

18:19:32
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


If you'd like to combine override and overridePythonAttrs, using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

functions as expected.


The original question remains in the thread but is mostly confused.

18:20:25
@cameronraysmith:matrix.orgcameronraysmith *

Thank you! ... that seems like one of the variants I tried, but maybe I didn't get it quite right or there's a more subtle issue with this particular package ... I'll try again

I put a bit more of the intended usage context in the thread above, but have now summarized what was useful and pushed the original question which was largely confused below its break.

18:21:49
@cameronraysmith:matrix.orgcameronraysmith *

Thank you! ... that seems like one of the variants I tried, but maybe I didn't get it quite right or there's a more subtle issue with this particular package ... I'll try again

I put a bit more of the intended usage context in the thread above, but have now summarized what was useful and pushed the original question , which was largely confused by equivalent errors with different causes, below its break.

18:22:18
@cameronraysmith:matrix.orgcameronraysmith *

Thank you! ... that seems like one of the variants I tried, but maybe I didn't get it quite right or there's a more subtle issue with this particular package ... I'll try again

I put a bit more of the intended usage context in the thread above, but have now summarized what was useful and pushed the original question, which was largely confused by equivalent errors with different causes, below its break.

18:23:48
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


If you'd like to combine override and overridePythonAttrs, using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

functions as expected, but the above does not work for the referenced package, dm-tree, because this doesn't come close to addressing the issues with its build.


The original question remains in the thread but is mostly confused.

18:25:15
@cameronraysmith:matrix.orgcameronraysmith * Right but preferWheel needs to be set with override ... so I'm missing something about how you combine those two successfully for a single package because it appears to fail to download the source when I compose them. I have tried the obvious constructions I am aware of but still receive the same error related to cmake being absent in the build environment for this package or the wheel is downloaded prior to running cmake when the source should be instead. 18:26:13
@cameronraysmith:matrix.orgcameronraysmith *

But, of course, this doesn't really begin to get to the root of the issue, for which I definitely should've looked upstream first

https://github.com/NixOS/nixpkgs/blob/4e44fb9c2673c5b58c44deb7e05bcf312a5862cf/pkgs/development/python-modules/dm-tree/default.nix#L15

18:27:11
@cameronraysmith:matrix.orgcameronraysmith *

But, of course, this doesn't really begin to get to the root of the issue, for which I definitely should've looked upstream first

https://github.com/NixOS/nixpkgs/blob/4e44fb9c2673c5b58c44deb7e05bcf312a5862cf/pkgs/development/python-modules/dm-tree/default.nix#L15

especially at the cmake patch

https://github.com/NixOS/nixpkgs/blob/4e44fb9c2673c5b58c44deb7e05bcf312a5862cf/pkgs/development/python-modules/dm-tree/cmake.patch

18:28:02
@cameronraysmith:matrix.orgcameronraysmith *

tl; dr: should've looked upstream first regarding the dm-tree package

since even after a heroic effort there this is broken on darwin, if you need crossplatform compatibility, probably one of the better approaches is to ensure chex>=0.1.82 and avoid dm-tree if not otherwise essential


If you'd like to combine override and overridePythonAttrs, using what K900 ⚡️ suggested below

              dm-tree = (super.dm-tree.override {preferWheel = false;}).overridePythonAttrs (
                old: {
                  nativeBuildInputs = (old.nativeBuildInputs or []) ++ [pkgs.cmake];
                }
              );

functions as expected, but the above does not work for the referenced package, dm-tree, because this doesn't come close to addressing the issues with its build. Those are much better approximated in the cmake patch there.


The original question remains in the thread but is mostly confused.

18:29:24
@kennethkoski:matrix.orgknkski joined the room.21:40:07
@kennethkoski:matrix.orgknkski Hey, I 've got an issue with recent versions of pytorch-lightning, and I'm wondering if anyone can help me figure out what's going on 21:41:36
@kennethkoski:matrix.orgknkski

Here's the gist of it:

$ nix log /nix/store/r01bqrznkq4bm9q29qwlrwr7f5vx7q3h-python3.11-pytorch-lightning-2.1.3.drv
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing wheel setup hook
Sourcing pypa-install-hook
Using pypaInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing python-catch-conflicts-hook.sh
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
21:41:59
@kennethkoski:matrix.orgknkski

I've tried a couple different overrides, but none of them seem to take:

pytorch-lightning = pkgs.python311Packages.pytorch-lightning;
pytorch-lightning = super.pytorch-lightning.override { preferWheel = false; };
pytorch-lightning = super.pytorch-lightning.overridePythonAttrs (old: {
  unpackPhase = ''
    set -eux
    echo "THIS SHOULD NOT BUILD!"
    exit 1
  '';
});
21:43:51
@kennethkoski:matrix.orgknkski Those not having an effect might be related to this issue? I tried setting preferWheels = false;, but then cmake was having issues building and I couldn't get that working 21:44:57
@kennethkoski:matrix.orgknkskiI'm building on linux x86-64 for the same platform, so that issue might not be relevant21:45:26
@kennethkoski:matrix.orgknkski

I've tried specifying the dependency as both of these ways:

pytorch-lightning = "2.1.3"
pytorch-lightning = { url = "https://files.pythonhosted.org/packages/f6/95/a662bd12b255ee3b59c975277c67398410b5810ee72ff11e34134e10cc9a/pytorch_lightning-2.1.3-py3-none-any.whl" }

Even when directly pointing it directly at the wheel file, it still gives the same error

21:46:52
@kennethkoski:matrix.orgknkskiI can wrap this all up into a github issue if that would work better, the github issue template pointed towards this room as a start21:49:29
@kennethkoski:matrix.orgknkski I just tried manually removing cmake from my poetry.lock file, so that I could try out preferWheels = false;. Strangely, that doesn't seem to have any effect either, I get the same error 22:15:09
@cameronraysmith:matrix.orgcameronraysmith knkski: there are default overrides for pytorch-lightning that attempt to handle what must have previously been something like a nested tarball. I removed these from a test fork https://github.com/cameronraysmith/poetry2nix/commit/aab624b6f49b5974a57a4be5d034613d6b65c79f and things work as expected when I source it in my flake. I was going to discuss submitting these upstream but it might be necessary to retain them with version guards for backward compatibility 22:28:51
@kennethkoski:matrix.orgknkski cameronraysmith: that worked, thank you! 22:30:28
@cameronraysmith:matrix.orgcameronraysmith* knkski: there are default overrides for pytorch-lightning that attempt to handle what must have previously been something like a nested tarball. I removed these from a test fork https://github.com/cameronraysmith/poetry2nix/commit/aab624b6f49b5974a57a4be5d034613d6b65c79f and things work as expected when I source it in my flake. I was going to discuss submitting these upstream but it might be necessary to retain them with version guards for backward compatibility I didn't manage to bisect where this must have changed in the pytorch-lightning release history but I tried as far back as 1.9.4 or so.22:30:58

Show newer messages


Back to Room ListRoom Version: 6