| 23 Aug 2022 |
Carl Thomé | In reply to @jboy:utwente.io
with import <nixpkgs> {};
mkShell {
venvDir = "./VENV";
buildInputs = [
cairo
zlib
poetry
python3.pkgs.venvShellHook
];
postShellHook = ''
export LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/:${cairo}/lib/:${zlib}/lib:$LD_LIBRARY_PATH
poetry install
'';
}
Thanks! | 14:35:02 |
Felix | jboy: Sounds like a pragmatic solution if packaging is out of reach. Thanks for the snippet | 14:35:11 |
Felix | carlthome: Have you also a nix file that uses jupyterWith? Where do you see friction with jupyterWith? | 14:36:08 |
Felix | And thanks for the poetry2nix file | 14:36:19 |
Carl Thomé | In reply to @flxai:matrix.org carlthome: Have you also a nix file that uses jupyterWith? Where do you see friction with jupyterWith? TBH not sure what I'm doing really. 🫡
Figured it could be convenient to have a set of regular ipykernels (one for PyTorch, one for scikit-learn, etc.), ready for playing around with ML modelling ideas.
Thus I started just throwing in them as devShells in my ongoing home flake to learn more about jupyterWith.
Here's a wip commit:
https://github.com/carlthome/dotfiles/pull/3/files#diff-206b9ce276ab5971a2489d75eb1b12999d4bf3843b7988cbe8d687cfde61dea0R57
| 14:43:54 |
Felix | Yes I feel you. There are many solutions to this problem. Will try those posted. Thanks again | 14:54:50 |
| 24 Aug 2022 |
| cafkafk joined the room. | 19:15:39 |
| underpantsgnome changed their display name from tinybronca to underpantsgnome. | 23:00:44 |
| 25 Aug 2022 |
| entheogenesis joined the room. | 17:46:09 |
| 30 Aug 2022 |
| sents joined the room. | 13:42:27 |
| aru joined the room. | 14:42:54 |
| aru left the room. | 14:56:16 |
| hacobjinkle joined the room. | 16:51:54 |
| 5 Sep 2022 |
| kotatsuyaki joined the room. | 05:39:36 |
| michelvasconcelos joined the room. | 18:21:51 |
| 6 Sep 2022 |
hacobjinkle | Nix is attractive for reproducibility and provenance tracking. However, for ML workflows one major limitation is that all data produced as part of a derivation resides in /nix/store. It would be great to be able to define flakes for large datasets and use them as inputs to other programs/flakes. For example define a flake for ImageNet and then get train and test splits as outputs of those downstream flakes. Currently such a train/test split would require duplication of the original dataset twice into /nix/store if I'm not mistaken. | 15:34:39 |
hacobjinkle | apologies if there is an existing solution to this kind of data handling with nix that I'm unaware of | 15:35:12 |
hacobjinkle | i guess the splitting could of course be done with symlinks. main question is then is there any hope of handling large datasets at rest on some storage outside /nix | 15:39:06 |
FRidh | Nix master supports impure derivations. Impure derivations have network access, so you could write your result somewhere else. I have not tried this yet, but I imagine this going to be very useful also in CI/CD. | 17:04:49 |
FRidh | https://github.com/NixOS/nix/pull/6227 | 17:05:19 |
hacobjinkle | thanks. I currently use --impure on non-NixOS with nixGL to get cuda working anyway, so that may be the ticket | 17:07:24 |
FRidh | that's not the same though, that's impure evaluation | 17:08:40 |
hacobjinkle | i see that now. should've read before replying.. | 17:09:23 |
| 7 Sep 2022 |
SomeoneSerge (back on matrix) | Has anyone got a working example of micromamba environment? I've tried using it in a bulidFHSUserEnv(Bubblewrap) with this in /etc/profile:
eval "$(micromamba shell hook --shell=bash)"
micromamba activate
...but it keeps segfaulting later when I prompt micromamba activate $envName
What helps is running the eval "$(...)" in the shell manually
| 12:43:40 |
| 15 Sep 2022 |
| m_algery joined the room. | 12:10:46 |
| m_algery left the room. | 12:10:49 |
| 16 Sep 2022 |
| TyberiusPrime (smilodon inopinatus) changed their display name from TyberiusPrime to TyberiusPrime (fiber at home when). | 07:37:08 |
| Suwon Park joined the room. | 18:17:04 |
Suwon Park | In reply to @flxai:matrix.org I am new to NixOS and wondering what's a good way to set up Jupyter. Are there opinions on JupyterWith or maybe some general recommendations? Try using jupyterWith + poetry2nix with flake.nix! It's hard but fun! | 18:33:55 |
Felix | sepiabrown: Thanks, this sounds like a good mix. Do you have configs lieing around somewhere? | 18:38:49 |