!fXpAvneDgyJuYMZSwO:nixos.org

Nix Data Science

290 Members
61 Servers

Load older messages


SenderMessageTime
13 Aug 2022
@yuu:matrix.orgYuu Yin changed their display name from yuu[m] to yuu.05:06:14
15 Aug 2022
@pederbs:pvv.ntnu.nopbsds changed their display name from pederbs to pbsds.23:19:31
16 Aug 2022
@flxai:matrix.orgFelix joined the room.11:48:20
@flxai:matrix.orgFelixHello :)11:51:31
@flxai:matrix.orgFelixI am new to NixOS and wondering what's a good way to set up Jupyter. Are there opinions on JupyterWith or maybe some general recommendations?11:52:27
17 Aug 2022
@greaka:greaka.degreaka joined the room.20:24:12
21 Aug 2022
@greaka:greaka.degreaka changed their display name from greaka ⚡️ to greaka .09:25:58
23 Aug 2022
@callmeecho:matrix.orgEcho joined the room.00:50:04
@jboy:utwente.ioJohn ✒️
In reply to @flxai:matrix.org
I am new to NixOS and wondering what's a good way to set up Jupyter. Are there opinions on JupyterWith or maybe some general recommendations?
These days I use a shell.nix with poetry and a venvHook, and then I just manage project dependencies (including jupyter) using poetry. It would be nice to do everything with nix packages, but inevitably I get to the point where I want to use something that isn't packaged.
13:36:28
@carlthome:matrix.orgCarl Thomé
In reply to @jboy:utwente.io
These days I use a shell.nix with poetry and a venvHook, and then I just manage project dependencies (including jupyter) using poetry. It would be nice to do everything with nix packages, but inevitably I get to the point where I want to use something that isn't packaged.
Could you share an example of such a shell.nix? I'm interested in how to use a venvHook.
14:29:17
@carlthome:matrix.orgCarl Thomé
In reply to @carlthome:matrix.org
Could you share an example of such a shell.nix? I'm interested in how to use a venvHook.

I've been dabbling with jupyterWith but feeling that poetry2nix is easier when one just needs to get stuff done.

But I've only tried poetry2nix for making a library so far, not a notebook environment: https://github.com/carlthome/poetry2nix-test/blob/main/flake.nix

14:31:08
@jboy:utwente.ioJohn ✒️
14:31:57
@jboy:utwente.ioJohn ✒️ *
with import <nixpkgs> {};

mkShell {
  venvDir = "./VENV";
  buildInputs = [
    cairo
    graphviz
    zlib
    poetry
    python3.pkgs.venvShellHook
  ];
  postShellHook = ''
    export LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/:${cairo}/lib/:${zlib}/lib:$LD_LIBRARY_PATH
    poetry install
  '';
}
14:32:05
@jboy:utwente.ioJohn ✒️ *
with import <nixpkgs> {};

mkShell {
  venvDir = "./VENV";
  buildInputs = [
    cairo
    zlib
    poetry
    python3.pkgs.venvShellHook
  ];
  postShellHook = ''
    export LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/:${cairo}/lib/:${zlib}/lib:$LD_LIBRARY_PATH
    poetry install
  '';
}
14:32:26
@carlthome:matrix.orgCarl Thomé
In reply to @jboy:utwente.io
with import <nixpkgs> {};

mkShell {
  venvDir = "./VENV";
  buildInputs = [
    cairo
    zlib
    poetry
    python3.pkgs.venvShellHook
  ];
  postShellHook = ''
    export LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib/:${cairo}/lib/:${zlib}/lib:$LD_LIBRARY_PATH
    poetry install
  '';
}
Thanks!
14:35:02
@flxai:matrix.orgFelixjboy: Sounds like a pragmatic solution if packaging is out of reach. Thanks for the snippet14:35:11
@flxai:matrix.orgFelixcarlthome: Have you also a nix file that uses jupyterWith? Where do you see friction with jupyterWith?14:36:08
@flxai:matrix.orgFelixAnd thanks for the poetry2nix file14:36:19
@carlthome:matrix.orgCarl Thomé
In reply to @flxai:matrix.org
carlthome: Have you also a nix file that uses jupyterWith? Where do you see friction with jupyterWith?

TBH not sure what I'm doing really. 🫡

Figured it could be convenient to have a set of regular ipykernels (one for PyTorch, one for scikit-learn, etc.), ready for playing around with ML modelling ideas.

Thus I started just throwing in them as devShells in my ongoing home flake to learn more about jupyterWith.

Here's a wip commit:

https://github.com/carlthome/dotfiles/pull/3/files#diff-206b9ce276ab5971a2489d75eb1b12999d4bf3843b7988cbe8d687cfde61dea0R57

14:43:54
@flxai:matrix.orgFelixYes I feel you. There are many solutions to this problem. Will try those posted. Thanks again14:54:50
24 Aug 2022
@cafkafk:matrix.cafkafk.comcafkafk joined the room.19:15:39
@tinybronca:sibnsk.netunderpantsgnome changed their display name from tinybronca to underpantsgnome.23:00:44
25 Aug 2022
@entheogenesis:matrix.orgentheogenesis joined the room.17:46:09
30 Aug 2022
@sents:matrix.orgsents joined the room.13:42:27
@aruzeta:matrix.orgaru joined the room.14:42:54
@aruzeta:matrix.orgaru left the room.14:56:16
@hinklejd:matrix.orghacobjinkle joined the room.16:51:54
5 Sep 2022
@kotatsuyaki:matrix.orgkotatsuyaki joined the room.05:39:36
@michelvasconcelos:matrix.orgmichelvasconcelos joined the room.18:21:51
6 Sep 2022
@hinklejd:matrix.orghacobjinkleNix is attractive for reproducibility and provenance tracking. However, for ML workflows one major limitation is that all data produced as part of a derivation resides in /nix/store. It would be great to be able to define flakes for large datasets and use them as inputs to other programs/flakes. For example define a flake for ImageNet and then get train and test splits as outputs of those downstream flakes. Currently such a train/test split would require duplication of the original dataset twice into /nix/store if I'm not mistaken.15:34:39

Show newer messages


Back to Room ListRoom Version: 6