!rWxyQqNqMUDLECdsIf:blad.is

Poetry2nix

315 Members
https://github.com/nix-community/poetry2nix57 Servers

Load older messages


SenderMessageTime
29 Jan 2022
@mou_bugtracker:matrix.orgmou
In reply to @rolandco:matrix.org
To reproduce, nix develop; graphtage --help
pyproject.toml and poetry.lock needed too
16:54:24
@mou_bugtracker:matrix.orgmouor just pyproject.toml16:54:59
@rolandco:matrix.orgRoland Coeurjoly
In reply to @mou_bugtracker:matrix.org
or just pyproject.toml

[tool.poetry] name = “graphtage-tool” version = “0.1.0” description = “” authors = [“Roland Coeurjoly rolandcoeurjoly@gmail.com”]

[tool.poetry.dependencies] python = “>=3.9,<3.11” graphtage = “^0.2.5” wildq = “^1.1.10”

[tool.poetry.dev-dependencies]

[build-system] requires = [“poetry-core>=1.0.0”] build-backend = “poetry.core.masonry.api”

17:00:06
@rolandco:matrix.orgRoland CoeurjolySorry17:00:34
@rolandco:matrix.orgRoland CoeurjolyHow can you reproduce without poetry.lock? Nix complaints without it17:01:26
@rolandco:matrix.orgRoland Coeurjolyhttps://github.com/RCoeurjoly/nix_graphtage17:11:45
@rolandco:matrix.orgRoland Coeurjoly mou: you can find the repo above 17:12:17
@mou_bugtracker:matrix.orgmouthanks17:12:36
@mou_bugtracker:matrix.orgmouit take time to build17:12:44
@rolandco:matrix.orgRoland CoeurjolyThe package graphtage is in nixpkgs, and I can create a shell with it easily18:52:16
@rolandco:matrix.orgRoland CoeurjolyBut wildq is not in nixpkgs18:52:29
@rolandco:matrix.orgRoland CoeurjolyIs there a way to create a shell with python modules from nixpkgs and also some installed with poetry?18:54:10
@mou_bugtracker:matrix.orgmouunfortunately my time is over for today. I was not able to figure out cause of problem. But i think cause lies in missing native build dependencies like gfortran and similar. Here is definition for scypi in nixpkgs https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/python-modules/scipy/default.nix And 19:36:21
@mou_bugtracker:matrix.orgmouConsider use poetry2nix not from nixpkgs, but latest version from git. because it has pretty haevy override for scipy https://github.com/nix-community/poetry2nix/blob/master/overrides/default.nix#L167119:39:18
@mou_bugtracker:matrix.orgmou

i take standard template for poetry2nix and tried to build. ended up with cythonization fail ((

Processing scipy/optimize/_group_columns.pyx
  Processing scipy/interpolate/interpnd.pyx
  Processing scipy/optimize/_highs/cython/src/_highs_wrapper.pyx
  Processing scipy/optimize/_highs/cython/src/_highs_constants.pyx
  Traceback (most recent call last):
    File "/build/scipy-1.7.3/tools/cythonize.py", line 324, in <module>
      main()
    File "/build/scipy-1.7.3/tools/cythonize.py", line 320, in main
      find_process_files(root_dir)
    File "/build/scipy-1.7.3/tools/cythonize.py", line 309, in find_process_files
  Processing scipy/optimize/_trlib/_trlib.pyx
      for result in pool.imap_unordered(lambda args: process(*args), jobs):
    File "/nix/store/i6vabb4div9iy6lsl642d86k1q8riasn-python3-3.9.9/lib/python3.9/multiprocessing/pool.py", line 870, in next
      raise value
    File "/nix/store/i6vabb4div9iy6lsl642d86k1q8riasn-python3-3.9.9/lib/python3.9/multiprocessing/pool.py", line 125, in worker
      result = (True, func(*args, **kwds))
    File "/build/scipy-1.7.3/tools/cythonize.py", line 309, in <lambda>
      for result in pool.imap_unordered(lambda args: process(*args), jobs):
    File "/build/scipy-1.7.3/tools/cythonize.py", line 243, in process
      processor_function(fromfile, tofile, cwd=path)
    File "/build/scipy-1.7.3/tools/cythonize.py", line 80, in process_pyx
      min_required_version, max_required_version = line.split(split_on)
  ValueError: too many values to unpack (expected 2)
  Processing scipy/optimize/cython_optimize/_zeros.pyx.in
  Cythonizing sources
19:54:23
@mou_bugtracker:matrix.orgmou *

i took standard template for poetry2nix and tried to build. ended up with cythonization fail ((

Processing scipy/optimize/_group_columns.pyx
  Processing scipy/interpolate/interpnd.pyx
  Processing scipy/optimize/_highs/cython/src/_highs_wrapper.pyx
  Processing scipy/optimize/_highs/cython/src/_highs_constants.pyx
  Traceback (most recent call last):
    File "/build/scipy-1.7.3/tools/cythonize.py", line 324, in <module>
      main()
    File "/build/scipy-1.7.3/tools/cythonize.py", line 320, in main
      find_process_files(root_dir)
    File "/build/scipy-1.7.3/tools/cythonize.py", line 309, in find_process_files
  Processing scipy/optimize/_trlib/_trlib.pyx
      for result in pool.imap_unordered(lambda args: process(*args), jobs):
    File "/nix/store/i6vabb4div9iy6lsl642d86k1q8riasn-python3-3.9.9/lib/python3.9/multiprocessing/pool.py", line 870, in next
      raise value
    File "/nix/store/i6vabb4div9iy6lsl642d86k1q8riasn-python3-3.9.9/lib/python3.9/multiprocessing/pool.py", line 125, in worker
      result = (True, func(*args, **kwds))
    File "/build/scipy-1.7.3/tools/cythonize.py", line 309, in <lambda>
      for result in pool.imap_unordered(lambda args: process(*args), jobs):
    File "/build/scipy-1.7.3/tools/cythonize.py", line 243, in process
      processor_function(fromfile, tofile, cwd=path)
    File "/build/scipy-1.7.3/tools/cythonize.py", line 80, in process_pyx
      min_required_version, max_required_version = line.split(split_on)
  ValueError: too many values to unpack (expected 2)
  Processing scipy/optimize/cython_optimize/_zeros.pyx.in
  Cythonizing sources
21:53:16
3 Feb 2022
@jyamad:cofree.coffeejyamad joined the room.20:43:40
@lewo:matrix.orglewo joined the room.21:47:31
8 Feb 2022
@graf_blutwurst:matrix.orgDominic Egger joined the room.05:38:54
@graf_blutwurst:matrix.orgDominic EggerGood Morning everyone. I am currently working on a little showcase workbench/projects for python based friends that don't know nix. They work fairly extensively with numpy and pandas so i tried adding those to poetry and so also to my environment. but it seems that that causes pandas and numpy to fully recompile them locally. i also tried pinning the versions to ones specifically available on nix. does anyone have some insight into this?05:41:33
@adis:blad.isadisbladis
In reply to @graf_blutwurst:matrix.org
Good Morning everyone. I am currently working on a little showcase workbench/projects for python based friends that don't know nix. They work fairly extensively with numpy and pandas so i tried adding those to poetry and so also to my environment. but it seems that that causes pandas and numpy to fully recompile them locally. i also tried pinning the versions to ones specifically available on nix. does anyone have some insight into this?
That's fully expected. Even if you would somehow pin the exact same versions that are available in nixpkgs the build closure and resulting derivations are not exactly the same.
05:52:31
@graf_blutwurst:matrix.orgDominic Eggerhm it seems i have to better understand build closures better. i assumed a dependency like that would be "low" enough that it could be picked up from the store/nixpkgs. thanks though! how do people deal with the long resulting build times? push the result from mkPoetryEnv to a binary cache?06:02:07
@adis:blad.isadisbladis
In reply to @graf_blutwurst:matrix.org
hm it seems i have to better understand build closures better. i assumed a dependency like that would be "low" enough that it could be picked up from the store/nixpkgs. thanks though! how do people deal with the long resulting build times? push the result from mkPoetryEnv to a binary cache?
Typically, yes.
06:02:46
@graf_blutwurst:matrix.orgDominic Egger
In reply to @adis:blad.is
Typically, yes.
ok then i was on the right track for that. do you happen to have any resources that'd help me understand what causes the recompile of pandas etc? mostly so i can predict when it happens since those take a significant chunk of time to crunch through
06:06:31
@adis:blad.isadisbladis
In reply to @graf_blutwurst:matrix.org
ok then i was on the right track for that. do you happen to have any resources that'd help me understand what causes the recompile of pandas etc? mostly so i can predict when it happens since those take a significant chunk of time to crunch through

When some of the build inputs changes, or some parameters are different.
poetry2nix tries as much as possible to ignore nixpkgs python packages which will cause the derivations to be different from nixpkgs ~all of the time.

Practially this means that you will have to rebuild when:

  • You update nixpkgs
  • Any of the python packages that pandas/numpy depends on changes in your poetry.lock
06:08:37
@graf_blutwurst:matrix.orgDominic Eggerthat sounds sensible.thanks for your time!06:17:11
12 Feb 2022
@octavioduarte:matrix.orgOctavio Martín Duarte joined the room.15:26:44
@octavioduarte:matrix.orgOctavio Martín Duarte

Hello!
I'm suffering from a problem that seems to have been already detected and fixed...

I get this failure even for the Flask exaple from the Tweag blogpost and the video. I get the message about an infinite recursion and when I use the verbose switch it get to read that flint-core is detected as missing. Adding the package as a dependendency does not stop this from failing.

15:54:06
@mou_bugtracker:matrix.orgmouOctavio, could you provide details about your environment? Error message, pyproject.toml, your nix expression will help anyone to better identify root cause.18:45:31
14 Feb 2022
@octavioduarte:matrix.orgOctavio Martín DuarteDownload pyproject.toml02:09:18

Show newer messages


Back to Room ListRoom Version: 6