!fXpAvneDgyJuYMZSwO:nixos.org

Nix Data Science

273 Members
57 Servers

Load older messages


SenderMessageTime
26 Jun 2021
@mjolnir:nixos.orgNixOS Moderation Bot joined the room.01:24:00
27 Jun 2021
@tomberek:matrix.orgtomberek joined the room.04:25:51
30 Jun 2021
@jez:petrichor.meJez (he/him) 🌦️probably a very niche interest, but i just modified the binderhub nix example to demonstrate how to Do It With Flakes if anyone's interested https://tildegit.org/petrichor/binder-nix-flakes15:25:05
@jez:petrichor.meJez (he/him) 🌦️the classic nix example is here https://github.com/binder-examples/nix15:26:29
@kraftnix:matrix.orgkraftnix joined the room.17:35:17
1 Jul 2021
@avx3:matrix.orgavx3 joined the room.15:58:42
2 Jul 2021
@jboy:utwente.ioJohn ✒️ very cool Jez (he/him) 13:15:06
@jboy:utwente.ioJohn ✒️How has your experience been with nix and binder? 13:15:29
@jez:petrichor.meJez (he/him) 🌦️very little so far! i use binder from time to time but i'm no expect16:26:34
@jez:petrichor.meJez (he/him) 🌦️ * very little so far! i use binder from time to time but i'm no expert16:26:36
@jez:petrichor.meJez (he/him) 🌦️binder+flakes was just something i wanted to try once i learned more about nix because the lockfile makes it (in my head) easier to guarantee the precise versions of dependencies for reproducibility16:27:49
@jez:petrichor.meJez (he/him) 🌦️plus a friend of mine is on the binder core team so i occasionally just go and bother them with stupid ideas16:28:24
@jez:petrichor.meJez (he/him) 🌦️but binder & nix seem like a really nice complementary pairing16:28:59
5 Jul 2021
@spacesbot:nixos.devspacesbot - keeps a log of public NixOS channels joined the room.19:21:44
@spacesbot:nixos.devspacesbot - keeps a log of public NixOS channels 19:49:33
6 Jul 2021
@spacesbot:nixos.devspacesbot - keeps a log of public NixOS channels changed their display name from spacesbot to spacesbot - keeps a log of public NixOS channels.22:11:46
9 Jul 2021
@junjihashimoto:matrix.orgjunji hashimoto joined the room.00:19:12
14 Jul 2021
@stephansahm:matrix.orgStephan Sahm joined the room.09:13:14
21 Jul 2021
@schnecfk:ruhr-uni-bochum.deCRTified (old handle) joined the room.10:20:18
23 Jul 2021
Room Avatar Renderer.23:22:41
29 Jul 2021
@dearrude:nitro.chatEbrahim joined the room.19:22:32
3 Aug 2021
@lunik1:lunik.onelunik1 left the room.11:23:58
@lunik1:lunik.onelunik1 joined the room.11:24:06
@lunik1:lunik.onelunik1Any recommendations for data pipeline/toolkits that integrate well with nix?11:31:51
@vk3wtf:matrix.orgjbedoto do what? i use nix with a thin layer on top to manage bioinformatics pipelines12:03:59
@tomberek:matrix.orgtomberek jbedo: do you use bionix? or is it another thing? 13:12:47
@lunik1:lunik.onelunik1Currently I just have a bunch of python scripts I execute in a given order, but I'm looking for something that would help me formalise that order, easily extend/swap out parts of those pipelines, and help with deployment15:01:59
@tomberek:matrix.orgtomberek lunik1: that's a pattern I used Nix/Hydra for. Basically you have a set of "ingress"/"egress" derivations that may be impure (eg: fetch/store from S3) or pure. Then a chain of nix derivations that depend on each other. I defined a function to apply various transformations and map'd them to my list of ingress derivation. It was super nice for iteration, scaling up workers, cached results, experimenting with alternate pipelines. Way better and more productive than something like Airflow. I started to apply content-addressed derivations to them to do short-circuiting as well, it was still in progress for Hydra compatibility. 19:36:25
@lunik1:lunik.onelunik1Damn that sounds awesome, any of this open source?19:37:47
@tomberek:matrix.orgtomberekNo. My plan is to capture the idea, organize it a bit better, and have that be open source. I've heard of a few people re-inventing this a few times, so I want extract out the common portions and perhaps provide a "flow-library" or something to make it easier to put together.19:39:30

Show newer messages


Back to Room ListRoom Version: 6