| 6 Nov 2023 |
 matthewcroughan | So the difference is being able to reproduce something because it was defined correctly in the first place, vs pure experimentation and unreproducible environments that you cannot share with anyone | 18:09:14 |
| matthewcroughan | In reply to @pareto-optimal-dev:matrix.org
Which isn't too hard, and is manageable to figure out... the issue comes when it is "how do I generate a cargo.lock and tell poetry2nix about it". At least it's those kinds of integration issues that usually take time for me. Not hard, just have to learn Nix | 18:09:29 |
| matthewcroughan | And also the build system for Rust. | 18:09:42 |
| matthewcroughan | If you don't like that, you need to complain upstream and ask why they used Rust and Python together. | 18:09:56 |
| matthewcroughan | Alternatively write your own code, or stop using their code. | 18:10:07 |
| matthewcroughan | Do you see now how this has nothing to do with Nix, just software in general? | 18:10:29 |
 pareto-optimal-dev | In reply to @matthewcroughan:defenestrate.it
Not hard, just have to learn Nix Well, I know Nix the language, and I know how to use Nix for Haskell... it's just hard to transfer that knowledge. I guess though learning the build system is a big part of the complexity too. | 18:10:46 |
| matthewcroughan | Some software has fundamental complexity that will not be resolved or masked by any tool if you actually want to reproduce it | 18:11:38 |
| matthewcroughan | The only alternative to reproducing something, is to give someone something you built, an artifact, and that's not correct. | 18:11:51 |
| pareto-optimal-dev | I don't think we disagree about anything really? | 18:12:15 |
| matthewcroughan | So if you want to reproduce something, it's going to be hard. Nix lowers the barrier to entry, but it's a scientific endeavour, which requires research, development, time, effort | 18:12:33 |
| matthewcroughan | Once you have a derivation, nobody can take that away from you, but it can take days, months, years, depends. | 18:13:28 |
| matthewcroughan | https://sourcegraph.com/search?q=context:global+lang:Nix+AND+maturin&patternType=standard&sm=1 | 18:14:19 |
| matthewcroughan | Here's what I did when I couldn't figure out how the maturin hook worked | 18:14:28 |
| matthewcroughan | I went to sourcegraph, and looked through code that already existed that solved the same problem, then copy pasted it | 18:14:42 |
| matthewcroughan | But that person probably spent hours on it | 18:14:58 |
| pareto-optimal-dev | In reply to @matthewcroughan:defenestrate.it
https://sourcegraph.com/search?q=context:global+lang:Nix+AND+maturin&patternType=standard&sm=1 Thanks. I do use sourcegraph for that. If using flakes, I find it useful to also specify path:flake.nix. Yeah, they probably spent a while. | 18:15:43 |
| matthewcroughan | I just built tiktoken anyway | 18:16:01 |
| matthewcroughan | tiktoken = super.tiktoken.overridePythonAttrs
(
old: {
postPatch = ''
ln -s ${./nix/tiktoken/Cargo.lock} Cargo.lock
'';
cargoDeps = pkgs.rustPlatform.importCargoLock {
lockFile = ./nix/tiktoken/Cargo.lock;
};
nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [
pkgs.rustPlatform.cargoSetupHook
pkgs.rustPlatform.maturinBuildHook
];
}
);
| 18:16:23 |
| matthewcroughan | poetry2nix cannot just package things of this complexity for you, yet | 18:16:41 |
| matthewcroughan | suddenly, people start using rust and python together, and they start making new tools to deal with that and build that, it cannot be expected of poetry2nix to automatically deal with this new emerging class of usage | 18:17:28 |
| matthewcroughan | Is there even enough input information in the src to do it? | 18:17:56 |
| matthewcroughan | Another annoying thing seems to be that the Cargo.lock is available if you get it from pypi, but not from github | 18:19:39 |
| pareto-optimal-dev | In reply to @matthewcroughan:defenestrate.it
Another annoying thing seems to be that the Cargo.lock is available if you get it from pypi, but not from github In the issue above they were less resistant to adding it to pypi releases. | 18:20:09 |
| pareto-optimal-dev | In reply to @matthewcroughan:defenestrate.it
tiktoken = super.tiktoken.overridePythonAttrs
(
old: {
postPatch = ''
ln -s ${./nix/tiktoken/Cargo.lock} Cargo.lock
'';
cargoDeps = pkgs.rustPlatform.importCargoLock {
lockFile = ./nix/tiktoken/Cargo.lock;
};
nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [
pkgs.rustPlatform.cargoSetupHook
pkgs.rustPlatform.maturinBuildHook
];
}
);
Do you generate /nix/tiktoken/Cargo.lock yourself? | 18:21:07 |
| matthewcroughan | yes | 18:21:11 |
| matthewcroughan | you go to the source code, you type cargo generate-lockfile, then you have the lock file. | 18:21:23 |
| matthewcroughan | every single time they update the library, you will have to do that. | 18:21:35 |
| matthewcroughan | If you have 5 dependencies that have this problem, you will have to do it 5 times, and you will have to check it in yourself | 18:21:50 |
| matthewcroughan | We chose Nix, not because it is easy, but because it is hard. | 18:22:49 |
