| 16 Feb 2022 |
jyamad | In reply to @jyamad:cofree.coffee I am trying to use poetry2nix via flakes, but I need to downgrade poetry due to https://github.com/python-poetry/poetry/issues/4523 . Does anyone have an example of a flake that overrides the poetry version that poetry2nix uses? It looks to me like I may be getting the right version of poetry by making an overlay, but that poetry2nix is maybe not using my overlay version of poetry. | 07:18:21 |
worldofgeese | I have a Poetry flake I can get to reliably build a Nix derivation. I've been banging my head the last two days trying to build a container image from it with dockerTools.streamLayeredImage. The data.json file I use as my "database" can't be found by the Python processing script but only inside the container. Here's the error:
$ docker run docker run -p 5000:5000 --rm -it localhost/lbob:6xwsg8ixlbpaisq1xc7ipvs5qxcs3d1m
* Serving Flask app 'little_bits_of_buddha.app' (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on all addresses.
WARNING: This is a development server. Do not use it in a production deployment.
* Running on http://10.0.2.100:5000/ (Press CTRL+C to quit)
[2022-02-16 17:33:30,415] ERROR in app: Exception on / [GET]
Traceback (most recent call last):
File "/nix/store/wmaqg26cza9vvck1wglp3xywsqabnp7n-python3.9-flask-2.0.2/lib/python3.9/site-packages/flask/app.py", line 2073, in wsgi_app
response = self.full_dispatch_request()
File "/nix/store/wmaqg26cza9vvck1wglp3xywsqabnp7n-python3.9-flask-2.0.2/lib/python3.9/site-packages/flask/app.py", line 1518, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/nix/store/wmaqg26cza9vvck1wglp3xywsqabnp7n-python3.9-flask-2.0.2/lib/python3.9/site-packages/flask/app.py", line 1516, in full_dispatch_request
rv = self.dispatch_request()
File "/nix/store/wmaqg26cza9vvck1wglp3xywsqabnp7n-python3.9-flask-2.0.2/lib/python3.9/site-packages/flask/app.py", line 1502, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File "/nix/store/yr8hjra5czylkiz1pkcsclbbc7pqbyh2-python3.9-little-bits-of-buddha-0.1.0/lib/python3.9/site-packages/little_bits_of_buddha/app.py", line 10, in random_sutta
return _random_sutta()
File "/nix/store/yr8hjra5czylkiz1pkcsclbbc7pqbyh2-python3.9-little-bits-of-buddha-0.1.0/lib/python3.9/site-packages/little_bits_of_buddha/data.py", line 8, in _random_sutta
with open("../data.json") as json_file:
FileNotFoundError: [Errno 2] No such file or directory: './data.json'
But the file is there:
ls /nix/store/yr8hjra5czylkiz1pkcsclbbc7pqbyh2-python3.9-little-bits-of-buddha-0.1.0/lib/python3.9/site-packages/little_bits_of_buddha/
__pycache__ app.py data.json data.py run.py
Am I doing something wrong?
| 20:22:43 |
K900 | You need to actually declare that the file should be installed | 20:23:56 |
K900 | It won't take the entire directory and drop it in site-packages | 20:24:12 |
K900 | And this is a Python thing, not really a Nix thing | 20:24:24 |
worldofgeese | In reply to @k900:0upti.me You need to actually declare that the file should be installed It is in site-packages, if you look at the second command I put | 20:26:00 |
K900 | Oh, I see | 20:26:44 |
K900 | Your path is wrong for two reasons | 20:26:52 |
K900 | 1) you're looking one folder above the current directory, when the file is in the same directory as your Python code | 20:27:12 |
K900 | And 2) open works relative to current working directory, not source file location | 20:27:35 |
K900 | The actually correct way to do this would be https://docs.python.org/3/library/importlib.html#module-importlib.resources | 20:29:08 |
worldofgeese | This is what I thought! But poetry run lbob actually wants the source directory path. Here's what I wrote in this commit explaining the choice:
Poetry has odd behavior around relative file paths. Python scripts must
import starting from the project (`little_bits_of_buddha`) even if they
occupy the same directory. This commit fixes file paths to operate from
"project in".
| 20:29:24 |
worldofgeese | Of course I admit my thinking could be wrong here but I couldn't argue with the results | 20:29:37 |
worldofgeese | As poetry run lbob and the Nix derivation both run successfully | 20:29:49 |
K900 | So you fixed issue #1, kind of | 20:31:13 |
K900 | But you still have issue #2 | 20:31:19 |
K900 | The easy way out would be to do something like pathlib.Path(__file__) / "../data.json" | 20:31:50 |
K900 | The correct way out would be to use importlib.resources | 20:32:05 |
worldofgeese | Interesting. Can you give me an example of how I would use importlib.resources in this case? | 20:32:40 |
K900 | Basically just from importlib import resources; resources.files("your_package_name").join("data.json").read_text() | 20:34:19 |
worldofgeese | From inside my data.py file right? It would replace the open() | 20:35:20 |
K900 | It would replace the whole thing | 20:35:52 |
worldofgeese | This is my first real Python project, sorry if my questions are very amateur | 20:36:03 |
K900 | You probably want to then pass the result to json.loads | 20:36:04 |
K900 | Also, you might want to move that out of the function | 20:36:43 |
K900 | Because importlib will be slower than just reading the file off the disk | 20:37:00 |
K900 | Probably not by enough for it to mattrr | 20:37:10 |
K900 | * Probably not by enough for it to matter | 20:37:13 |
K900 | But it's generally good practice to load resources once and then keep them in memory | 20:37:26 |
K900 | Because your project could be installed as a wheel, or as an egg, or as whatever other weird format whatever other weird Python implementation uses | 20:38:02 |