!fXpAvneDgyJuYMZSwO:nixos.org

Nix Data Science

273 Members
57 Servers

Load older messages


SenderMessageTime
11 Jun 2021
@manuelbaerenz:matrix.orgManuel Bärenz joined the room.10:11:13
18 Jun 2021
@evax:matrix.orgevaxHi, I'm trying to use dockerTools and jupyterWith to build an image to use in a jupyterhub instance in kubernetes11:15:21
@evax:matrix.orgevaxmost of it is working except for the terminal - I can't figure out a way to give access to python/ipython with all the selected packages from the terminal11:16:46
@evax:matrix.orgevaxit looks like jupyterWith/jupyterlab come with a default python kernel with a limited package set, and that's what's exposed in the terminal11:18:59
@domenkozar:matrix.orgDomen Kožar evax: in order to have higher chances of getting help, I'd suggest attaching a file that demonstrates how to reproduce the problem 14:49:44
19 Jun 2021
@evax:matrix.orgevaxDownload jupyterlab.nix10:42:04
@evax:matrix.orgevax Domen Kožar: here's an example. You can create a notebook with the "test" environment, and import pandas, but if you open a terminal and run python/ipython, pandas isn't available 10:43:02
@evax:matrix.orgevax
  jupyter = builtins.fetchGit {
    url = https://github.com/tweag/jupyterWith;
    rev = "0940c0d8ff3dbbae8395c138a89a57fd2cdb509c";
  };

  overlays = [
    (import "${jupyter}/nix/python-overlay.nix")
    (import "${jupyter}/nix/overlay.nix")
  ];

  pkgs = import (builtins.fetchTarball {
    url = "https://github.com/nixos/nixpkgs/archive/bad3ccd099ebe9a8aa017bda8500ab02787d90aa.tar.gz";
    sha256 = "07ik6rcfpcvj2zch3mzkxwxjs817iysbvsydkm85f14kn7gnyzp5";
  }) {
    system = "x86_64-linux";
    inherit overlays;
  };

  pythonPackages = ps: with ps; [
    numpy pandas
  ];

  python38 = pkgs.python38.withPackages pythonPackages;

  userEnv = pkgs.symlinkJoin rec {
    name = "user-environment";
    buildInputs = with pkgs; [
      python38
      bashInteractive
      coreutils
      findutils
    ];
    paths = buildInputs;
  };

  iPython = pkgs.jupyterWith.kernels.iPythonWith {
    name = "test";
    python3 = python38;
    packages = pythonPackages;
  };

  jupyterlab = pkgs.jupyterWith.jupyterlabWith {
    kernels = [ iPython ];
    extraPackages = _: [ userEnv ];
  };

  nbUser = "jovyan";

  imageEtc = pkgs.runCommand "image-etc" {} ''
    mkdir -p $out/etc/pam.d
    echo "root:x:0:0::/root:/bin/sh" > $out/etc/passwd
    echo "${nbUser}:x:1000:1000::/home/${nbUser}:${userEnv}/bin/bash" >> $out/etc/passwd
    echo "root:!x:::::::" > $out/etc/shadow
    echo "${nbUser}:!:::::::" >> $out/etc/shadow
    echo "root:x:0:" > $out/etc/group
    echo "${nbUser}:x:1000:" >> $out/etc/group
    echo "root:x::" > $out/etc/gshadow
    echo "${nbUser}:!::" >> $out/etc/gshadow
  '';

  startNotebook = pkgs.writers.writeBash "start-notebook.sh" ''
    /bin/jupyter-lab --ip=0.0.0.0 --no-browser
  '';
in
  pkgs.dockerTools.buildImage {
    name = "jupyter";
    tag = "v0.0.1";
    contents = [
      imageEtc
      userEnv
      jupyterlab
      pkgs.cacert
      pkgs.iana-etc
      pkgs.glibcLocales
      pkgs.bashInteractive
      pkgs.coreutils
      pkgs.neovim
      pkgs.htop
    ];
    config = {
      Env = [
        "LOCALE_ARCHIVE=${pkgs.glibcLocales}/lib/locale/locale-archive"
        "LANG=en_US.UTF-8"
        "LANGUAGE=en_US:en"
        "LC_ALL=en_US.UTF-8"
        "HOME=/home/${nbUser}"
        "SHELL=${userEnv}/bin/bash"
      ];
      ENTRYPOINT = [ "${pkgs.tini}/bin/tini" "-g" "--" ];
      CMD = [ "${startNotebook}" ];
      WorkingDir = "/home/${nbUser}";
      ExposedPorts = {
        "8888" = {};
      };
      User = nbUser;
    };
    extraCommands = ''
      mkdir -m 1777 ./tmp
      mkdir -m 777 -p ./home/${nbUser}
    '';
  }
10:44:39
@evax:matrix.orgevax *
let
  jupyter = builtins.fetchGit {
    url = https://github.com/tweag/jupyterWith;
    rev = "0940c0d8ff3dbbae8395c138a89a57fd2cdb509c";
  };

  overlays = [
    (import "${jupyter}/nix/python-overlay.nix")
    (import "${jupyter}/nix/overlay.nix")
  ];

  pkgs = import (builtins.fetchTarball {
    url = "https://github.com/nixos/nixpkgs/archive/bad3ccd099ebe9a8aa017bda8500ab02787d90aa.tar.gz";
    sha256 = "07ik6rcfpcvj2zch3mzkxwxjs817iysbvsydkm85f14kn7gnyzp5";
  }) {
    system = "x86_64-linux";
    inherit overlays;
  };

  pythonPackages = ps: with ps; [
    numpy pandas
  ];

  python38 = pkgs.python38.withPackages pythonPackages;

  userEnv = pkgs.symlinkJoin rec {
    name = "user-environment";
    buildInputs = with pkgs; [
      python38
      bashInteractive
      coreutils
      findutils
    ];
    paths = buildInputs;
  };

  iPython = pkgs.jupyterWith.kernels.iPythonWith {
    name = "test";
    python3 = python38;
    packages = pythonPackages;
  };

  jupyterlab = pkgs.jupyterWith.jupyterlabWith {
    kernels = [ iPython ];
    extraPackages = _: [ userEnv ];
  };

  nbUser = "jovyan";

  imageEtc = pkgs.runCommand "image-etc" {} ''
    mkdir -p $out/etc/pam.d
    echo "root:x:0:0::/root:/bin/sh" > $out/etc/passwd
    echo "${nbUser}:x:1000:1000::/home/${nbUser}:${userEnv}/bin/bash" >> $out/etc/passwd
    echo "root:!x:::::::" > $out/etc/shadow
    echo "${nbUser}:!:::::::" >> $out/etc/shadow
    echo "root:x:0:" > $out/etc/group
    echo "${nbUser}:x:1000:" >> $out/etc/group
    echo "root:x::" > $out/etc/gshadow
    echo "${nbUser}:!::" >> $out/etc/gshadow
  '';

  startNotebook = pkgs.writers.writeBash "start-notebook.sh" ''
    /bin/jupyter-lab --ip=0.0.0.0 --no-browser
  '';
in
  pkgs.dockerTools.buildImage {
    name = "jupyter";
    tag = "v0.0.1";
    contents = [
      imageEtc
      userEnv
      jupyterlab
      pkgs.cacert
      pkgs.iana-etc
      pkgs.glibcLocales
      pkgs.bashInteractive
      pkgs.coreutils
      pkgs.neovim
      pkgs.htop
    ];
    config = {
      Env = [
        "LOCALE_ARCHIVE=${pkgs.glibcLocales}/lib/locale/locale-archive"
        "LANG=en_US.UTF-8"
        "LANGUAGE=en_US:en"
        "LC_ALL=en_US.UTF-8"
        "HOME=/home/${nbUser}"
        "SHELL=${userEnv}/bin/bash"
      ];
      ENTRYPOINT = [ "${pkgs.tini}/bin/tini" "-g" "--" ];
      CMD = [ "${startNotebook}" ];
      WorkingDir = "/home/${nbUser}";
      ExposedPorts = {
        "8888" = {};
      };
      User = nbUser;
    };
    extraCommands = ''
      mkdir -m 1777 ./tmp
      mkdir -m 777 -p ./home/${nbUser}
    '';
  }
10:46:50
@evax:matrix.orgevax
10:52:10
@evax:matrix.orgevax *
nix build -f jupyterlab.nix
docker load -i ./result
docker run -p 8888:8888 jupyter:v0.0.1
10:53:05
22 Jun 2021
@theophane:hufschmitt.netRegnat joined the room.13:11:42
26 Jun 2021
@grahamc:nixos.org@grahamc:nixos.orgchanged room power levels.01:18:35
@grahamc:nixos.org@grahamc:nixos.org invited @mjolnir:nixos.orgNixOS Moderation Bot.01:23:59
@mjolnir:nixos.orgNixOS Moderation Bot joined the room.01:24:00
27 Jun 2021
@tomberek:matrix.orgtomberek joined the room.04:25:51
30 Jun 2021
@jez:petrichor.meJez (he/him) 🌦️probably a very niche interest, but i just modified the binderhub nix example to demonstrate how to Do It With Flakes if anyone's interested https://tildegit.org/petrichor/binder-nix-flakes15:25:05
@jez:petrichor.meJez (he/him) 🌦️the classic nix example is here https://github.com/binder-examples/nix15:26:29
@kraftnix:matrix.orgkraftnix joined the room.17:35:17
1 Jul 2021
@avx3:matrix.orgavx3 joined the room.15:58:42
2 Jul 2021
@jboy:utwente.ioJohn ✒️ very cool Jez (he/him) 13:15:06
@jboy:utwente.ioJohn ✒️How has your experience been with nix and binder? 13:15:29
@jez:petrichor.meJez (he/him) 🌦️very little so far! i use binder from time to time but i'm no expect16:26:34
@jez:petrichor.meJez (he/him) 🌦️ * very little so far! i use binder from time to time but i'm no expert16:26:36
@jez:petrichor.meJez (he/him) 🌦️binder+flakes was just something i wanted to try once i learned more about nix because the lockfile makes it (in my head) easier to guarantee the precise versions of dependencies for reproducibility16:27:49
@jez:petrichor.meJez (he/him) 🌦️plus a friend of mine is on the binder core team so i occasionally just go and bother them with stupid ideas16:28:24
@jez:petrichor.meJez (he/him) 🌦️but binder & nix seem like a really nice complementary pairing16:28:59
5 Jul 2021
@spacesbot:nixos.devspacesbot - keeps a log of public NixOS channels joined the room.19:21:44
@spacesbot:nixos.devspacesbot - keeps a log of public NixOS channels 19:49:33
6 Jul 2021
@spacesbot:nixos.devspacesbot - keeps a log of public NixOS channels changed their display name from spacesbot to spacesbot - keeps a log of public NixOS channels.22:11:46

Show newer messages


Back to Room ListRoom Version: 6