!rWxyQqNqMUDLECdsIf:blad.is

Poetry2nix

329 Members
https://github.com/nix-community/poetry2nix63 Servers

Load older messages


SenderMessageTime
16 Feb 2022
@jyamad:cofree.coffeejyamad
In reply to @jyamad:cofree.coffee
I am trying to use poetry2nix via flakes, but I need to downgrade poetry due to https://github.com/python-poetry/poetry/issues/4523 . Does anyone have an example of a flake that overrides the poetry version that poetry2nix uses?
It looks to me like I may be getting the right version of poetry by making an overlay, but that poetry2nix is maybe not using my overlay version of poetry.
07:18:21
@worldofgeese:one.ems.hostworldofgeese

I have a Poetry flake I can get to reliably build a Nix derivation. I've been banging my head the last two days trying to build a container image from it with dockerTools.streamLayeredImage. The data.json file I use as my "database" can't be found by the Python processing script but only inside the container. Here's the error:

$ docker run docker run -p 5000:5000 --rm -it localhost/lbob:6xwsg8ixlbpaisq1xc7ipvs5qxcs3d1m
 * Serving Flask app 'little_bits_of_buddha.app' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on all addresses.
   WARNING: This is a development server. Do not use it in a production deployment.
 * Running on http://10.0.2.100:5000/ (Press CTRL+C to quit)
[2022-02-16 17:33:30,415] ERROR in app: Exception on / [GET]
Traceback (most recent call last):
  File "/nix/store/wmaqg26cza9vvck1wglp3xywsqabnp7n-python3.9-flask-2.0.2/lib/python3.9/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/nix/store/wmaqg26cza9vvck1wglp3xywsqabnp7n-python3.9-flask-2.0.2/lib/python3.9/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/nix/store/wmaqg26cza9vvck1wglp3xywsqabnp7n-python3.9-flask-2.0.2/lib/python3.9/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/nix/store/wmaqg26cza9vvck1wglp3xywsqabnp7n-python3.9-flask-2.0.2/lib/python3.9/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/nix/store/yr8hjra5czylkiz1pkcsclbbc7pqbyh2-python3.9-little-bits-of-buddha-0.1.0/lib/python3.9/site-packages/little_bits_of_buddha/app.py", line 10, in random_sutta
    return _random_sutta()
  File "/nix/store/yr8hjra5czylkiz1pkcsclbbc7pqbyh2-python3.9-little-bits-of-buddha-0.1.0/lib/python3.9/site-packages/little_bits_of_buddha/data.py", line 8, in _random_sutta
    with open("../data.json") as json_file:
FileNotFoundError: [Errno 2] No such file or directory: './data.json'

But the file is there:

ls /nix/store/yr8hjra5czylkiz1pkcsclbbc7pqbyh2-python3.9-little-bits-of-buddha-0.1.0/lib/python3.9/site-packages/little_bits_of_buddha/
__pycache__  app.py  data.json  data.py  run.py

Am I doing something wrong?

20:22:43
@k900:0upti.meK900You need to actually declare that the file should be installed 20:23:56
@k900:0upti.meK900It won't take the entire directory and drop it in site-packages20:24:12
@k900:0upti.meK900And this is a Python thing, not really a Nix thing 20:24:24
@worldofgeese:one.ems.hostworldofgeese
In reply to @k900:0upti.me
You need to actually declare that the file should be installed
It is in site-packages, if you look at the second command I put
20:26:00
@k900:0upti.meK900Oh, I see 20:26:44
@k900:0upti.meK900Your path is wrong for two reasons 20:26:52
@k900:0upti.meK9001) you're looking one folder above the current directory, when the file is in the same directory as your Python code 20:27:12
@k900:0upti.meK900 And 2) open works relative to current working directory, not source file location 20:27:35
@k900:0upti.meK900 The actually correct way to do this would be https://docs.python.org/3/library/importlib.html#module-importlib.resources 20:29:08
@worldofgeese:one.ems.hostworldofgeese

This is what I thought! But poetry run lbob actually wants the source directory path. Here's what I wrote in this commit explaining the choice:

Poetry has odd behavior around relative file paths. Python scripts must
import starting from the project (`little_bits_of_buddha`) even if they
occupy the same directory. This commit fixes file paths to operate from
"project in".
20:29:24
@worldofgeese:one.ems.hostworldofgeeseOf course I admit my thinking could be wrong here but I couldn't argue with the results20:29:37
@worldofgeese:one.ems.hostworldofgeese As poetry run lbob and the Nix derivation both run successfully 20:29:49
@k900:0upti.meK900So you fixed issue #1, kind of20:31:13
@k900:0upti.meK900But you still have issue #220:31:19
@k900:0upti.meK900 The easy way out would be to do something like pathlib.Path(__file__) / "../data.json" 20:31:50
@k900:0upti.meK900 The correct way out would be to use importlib.resources 20:32:05
@worldofgeese:one.ems.hostworldofgeeseInteresting. Can you give me an example of how I would use importlib.resources in this case?20:32:40
@k900:0upti.meK900 Basically just from importlib import resources; resources.files("your_package_name").join("data.json").read_text() 20:34:19
@worldofgeese:one.ems.hostworldofgeese From inside my data.py file right? It would replace the open() 20:35:20
@k900:0upti.meK900It would replace the whole thing 20:35:52
@worldofgeese:one.ems.hostworldofgeeseThis is my first real Python project, sorry if my questions are very amateur20:36:03
@k900:0upti.meK900 You probably want to then pass the result to json.loads 20:36:04
@k900:0upti.meK900Also, you might want to move that out of the function 20:36:43
@k900:0upti.meK900Because importlib will be slower than just reading the file off the disk 20:37:00
@k900:0upti.meK900Probably not by enough for it to mattrr20:37:10
@k900:0upti.meK900* Probably not by enough for it to matter20:37:13
@k900:0upti.meK900But it's generally good practice to load resources once and then keep them in memory 20:37:26
@k900:0upti.meK900Because your project could be installed as a wheel, or as an egg, or as whatever other weird format whatever other weird Python implementation uses 20:38:02

Show newer messages


Back to Room ListRoom Version: 6