| 6 Nov 2023 |
 matthewcroughan | The only alternative to reproducing something, is to give someone something you built, an artifact, and that's not correct. | 18:11:51 |
 pareto-optimal-dev | I don't think we disagree about anything really? | 18:12:15 |
| matthewcroughan | So if you want to reproduce something, it's going to be hard. Nix lowers the barrier to entry, but it's a scientific endeavour, which requires research, development, time, effort | 18:12:33 |
| matthewcroughan | Once you have a derivation, nobody can take that away from you, but it can take days, months, years, depends. | 18:13:28 |
| matthewcroughan | https://sourcegraph.com/search?q=context:global+lang:Nix+AND+maturin&patternType=standard&sm=1 | 18:14:19 |
| matthewcroughan | Here's what I did when I couldn't figure out how the maturin hook worked | 18:14:28 |
| matthewcroughan | I went to sourcegraph, and looked through code that already existed that solved the same problem, then copy pasted it | 18:14:42 |
| matthewcroughan | But that person probably spent hours on it | 18:14:58 |
| pareto-optimal-dev | In reply to @matthewcroughan:defenestrate.it
https://sourcegraph.com/search?q=context:global+lang:Nix+AND+maturin&patternType=standard&sm=1 Thanks. I do use sourcegraph for that. If using flakes, I find it useful to also specify path:flake.nix. Yeah, they probably spent a while. | 18:15:43 |
| matthewcroughan | I just built tiktoken anyway | 18:16:01 |
| matthewcroughan | tiktoken = super.tiktoken.overridePythonAttrs
(
old: {
postPatch = ''
ln -s ${./nix/tiktoken/Cargo.lock} Cargo.lock
'';
cargoDeps = pkgs.rustPlatform.importCargoLock {
lockFile = ./nix/tiktoken/Cargo.lock;
};
nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [
pkgs.rustPlatform.cargoSetupHook
pkgs.rustPlatform.maturinBuildHook
];
}
);
| 18:16:23 |
| matthewcroughan | poetry2nix cannot just package things of this complexity for you, yet | 18:16:41 |
| matthewcroughan | suddenly, people start using rust and python together, and they start making new tools to deal with that and build that, it cannot be expected of poetry2nix to automatically deal with this new emerging class of usage | 18:17:28 |
| matthewcroughan | Is there even enough input information in the src to do it? | 18:17:56 |
| matthewcroughan | Another annoying thing seems to be that the Cargo.lock is available if you get it from pypi, but not from github | 18:19:39 |
| pareto-optimal-dev | In reply to @matthewcroughan:defenestrate.it
Another annoying thing seems to be that the Cargo.lock is available if you get it from pypi, but not from github In the issue above they were less resistant to adding it to pypi releases. | 18:20:09 |
| pareto-optimal-dev | In reply to @matthewcroughan:defenestrate.it
tiktoken = super.tiktoken.overridePythonAttrs
(
old: {
postPatch = ''
ln -s ${./nix/tiktoken/Cargo.lock} Cargo.lock
'';
cargoDeps = pkgs.rustPlatform.importCargoLock {
lockFile = ./nix/tiktoken/Cargo.lock;
};
nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [
pkgs.rustPlatform.cargoSetupHook
pkgs.rustPlatform.maturinBuildHook
];
}
);
Do you generate /nix/tiktoken/Cargo.lock yourself? | 18:21:07 |
| matthewcroughan | yes | 18:21:11 |
| matthewcroughan | you go to the source code, you type cargo generate-lockfile, then you have the lock file. | 18:21:23 |
| matthewcroughan | every single time they update the library, you will have to do that. | 18:21:35 |
| matthewcroughan | If you have 5 dependencies that have this problem, you will have to do it 5 times, and you will have to check it in yourself | 18:21:50 |
| matthewcroughan | We chose Nix, not because it is easy, but because it is hard. | 18:22:49 |
| matthewcroughan | Although, again, Nix won't be the only tool that has problems dealing with the lack of specification in Python. | 18:23:28 |
| pareto-optimal-dev | This looks interesting for the cargo problem, but then I'll have to learn the flake parts stuff it seems :D
https://github.com/yusdacra/nix-cargo-integration | 18:23:50 |
| matthewcroughan | It is just information. Is there enough information in the repo to reproduce something/ | 18:23:55 |
| matthewcroughan | * It is just information. Is there enough information in the repo to reproduce something? | 18:23:56 |
| matthewcroughan | If there is not enough information in the source code, then even a human will have issues discerning how to reproduce what is inside of it. | 18:24:11 |
| matthewcroughan | It's like OCR, it's error prone because of lack of accuracy | 18:24:40 |
| matthewcroughan | I have an issue now with llama-index :( | 18:30:20 |
| matthewcroughan | llama-index = super.llama-index.overridePythonAttrs
(
old: {
propagatedBuildInputs = (old.propagatedBuildInputs or [ ]) ++ [ self.tiktoken ];
nativeBuildInputs = (old.nativeBuildInputs or [ ]) ++ [ super.poetry ];
}
);
| 18:30:22 |
