!fXpAvneDgyJuYMZSwO:nixos.org

Nix Data Science

329 Members
64 Servers

You have reached the beginning of time (for this room).


SenderMessageTime
14 Jan 2024
@benoitdr:matrix.orgbenoitdr
In reply to @gambrose:matrix.org
Hi all. New here and am looking for some help. I am trying to get a persistent jupyterhub/lab up and running on nixos. I have made progress, but stuck on two items. First, is it possible in the approach I am using to add jupyterlab extensions declaratively? I saw that its possible if I build a shell to use, but I would rather not take that approach. Second, while I got a python env up and running with some basic data science packages to try out, but I can't get an R to work. Console won't connect to the kernel, even after multiple attempts, including finding and using the full path to the R bin. Pasting my code below, if anyone has pointers who may have tried this. Thanks!
For the extensions, can't you just add them like other packages ? See jupyterlab-widgets, jupyterlab-git, jupyterlab-lsp, ...
09:29:23
15 Jan 2024
@gambrose:matrix.orggambrose
In reply to @benoitdr:matrix.org
For the extensions, can't you just add them like other packages ? See jupyterlab-widgets, jupyterlab-git, jupyterlab-lsp, ...
Yes, that works for some extensions that have official packages (like jupyterlab-lsp), but not all extensions have packages, but I can probably do a derivation to create them from their github repos. But related issue. Jupyterlab-git needs regular git installed, which it obvioulsy is on the base system, but the jupyterhub/lab environment for some reason can't see git. I must need to load it into the hub/lab environment. They are built like this:
02:49:40
@gambrose:matrix.orggambrose
In reply to @benoitdr:matrix.org
For the extensions, can't you just add them like other packages ? See jupyterlab-widgets, jupyterlab-git, jupyterlab-lsp, ...
*

Yes, that works for some extensions that have official packages (like jupyterlab-lsp), but not all extensions have packages, but I can probably do a derivation to create them from their github repos. But related issue. Jupyterlab-git needs regular git installed, which it obvioulsy is on the base system, but the jupyterhub/lab environment for some reason can't see git. I must need to load it into the hub/lab environment. They are built with this:

jupyterhubEnv = pkgs.python311.withPackages (p: with p; [
    jupyterhub
    jupyterhub-systemdspawner
    ...
02:50:18
@gambrose:matrix.orggambrose *

Yes, that works for some extensions that have official packages (like jupyterlab-lsp), but not all extensions have packages, but I can probably do a derivation to create them from their github repos. But related issue. Jupyterlab-git needs regular git installed, which it obvioulsy is on the base system, but the jupyterhub/lab environment for some reason can't see git. I must need to load it into the hub/lab environment. They are built with this:

jupyterhubEnv = pkgs.python311.withPackages (p: with p; [
    jupyterhub
    jupyterhub-systemdspawner
    ...

Is it possible to load non python311pacakges into this enviornment. I tried a few different ways, but couldn't get it to work. Trying to load the regular git nixos package.
Thanks!

02:52:00
@gambrose:matrix.orggambrose * Yes, that works for some extensions that have official packages (like jupyterlab-lsp), but not all extensions have packages, but I can probably do a derivation to create them from their github repos. But related issue. Jupyterlab-git needs regular git installed, which it obvioulsy is on the base system, but the jupyterhub/lab environment for some reason can't see git. I must need to load it into the hub/lab environment. They are built like this: 03:26:01
@gambrose:matrix.orggambrose *

Yes, that works for some extensions that have official packages (like jupyterlab-lsp), but not all extensions have packages, but I can probably do a derivation to create them from their github repos. But related issue. Jupyterlab-git needs regular git installed, which it obvioulsy is on the base system, but the jupyterhub/lab environment for some reason can't see git. I must need to load it into the hub/lab environment. They are built like this:

jupyterhubEnv = pkgs.python311.withPackages (p: with p; [
    jupyterhub
    jupyterhub-systemdspawner
    pip
    ...

It appears you can only load python packages into the underlying environment. Is there a way to load non python packages, such as git?
Thanks.

03:28:00
@gambrose:matrix.orggambrose *

Yes, that works for some extensions that have official packages (like jupyterlab-lsp), but not all extensions have packages, but I can probably do a derivation to create them from their github repos. But related issue. Jupyterlab-git needs regular git installed, which it obviously is on the base system, but the jupyterhub/lab environment for some reason can't see git. I must need to load it into the hub/lab environment. They are built like this:

jupyterhubEnv = pkgs.python311.withPackages (p: with p; [
    jupyterhub
    jupyterhub-systemdspawner
    pip
    ...

It appears you can only load python packages into the underlying environment. Is there a way to load non python packages, such as git?

06:03:30
@data_thrall:matrix.orgdata_thrall joined the room.07:12:52
16 Jan 2024
@benoitdr:matrix.orgbenoitdr

I'm not sure how you are working. Personally, I use nix-shell to package all the development environment (including python modules and non-python packages). Here is an example for jupyter lab :

let
  nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/archive/04220ed6763637e5899980f98d5c8424b1079353.tar.gz";
  pkgs = import nixpkgs { config = {}; overlays = []; };
in
  pkgs.mkShell {
    packages = with pkgs; [
      (python310.withPackages (ps: with ps; [
        numpy
        pandas
        matplotlib
        scipy
        jupyterlab
        ipympl
        jupyterlab-git
      ]))
      git
    ];

  shellHook = ''
    jupyter lab
  '';
  }

Note that unless you use nix-shell --pure, you don't need to add git if it is alrready present at OS level, although it's probability better to add it anyway for portability

12:34:26
17 Jan 2024
@andredornas:matrix.organdredornas joined the room.13:51:45
18 Jan 2024
@fizihcyst:matrix.orgfizihcystHi, is anyone here using julia? I see that recently a pr was merged to nixpkgs to build julia.withPackages similar to python. This is fine for simple scripts, but does anyone have recommendations for using nix with a julia Project.toml/Manifest.toml? Maybe something similar to poetry2nix? I see two julia2nix repos, but had trouble getting them to work. A working example julia project+flake would be helpful.17:08:43
22 Jan 2024
@gkapfham:matrix.orgGregory M. Kapfhammer joined the room.16:59:42
@gkapfham:matrix.orgGregory M. KapfhammerHello, does anyone have a good example of how to get Quarto to work on NixOS? I created this site using Quarto on Arch Linux: https://github.com/gkapfham/www.gregorykapfhammer.com This configuration allows me to use Poetry to manage the project's dependencies. When I am in the poetry shell I can use quarto and it finds all of the project's dependencies in the virtualenv when I run it on Arch Linux. However, when I use NixOS the quarto program installed through nix packages does not seem to pass along the dependencies in the virtual environment. I am glad to share more details. With that said, does anyone have a quick idea as to what I should try next? Thanks!18:27:19
@phiadaarr:matrix.orgphiadaarr joined the room.21:19:10
23 Jan 2024
@bcdarwin:matrix.orgbcdarwin joined the room.22:54:56
25 Jan 2024
@trexd:matrix.orgtrexdWhat are people's thoughts around putting data in the nix store? Only small stuff? Large datasets too? Only in specific situations?16:25:09
@crtified:crtified.meCRTifiedGenerally a nice thing, although painful for larger files. Not directly related to data science, but it was a pain to get vivado (fpga IDE/tool chain) into the store, and that's only like 25GiB worth of data16:34:48
@trexd:matrix.orgtrexdOk thats around the dataset size that I'm dealing with but I can decrompress it before training so fitting it in the store compressed should make things easier. 16:44:30

Show newer messages


Back to Room ListRoom Version: 6