| 21 Dec 2024 |
Paul Meyer (katexochen) | In reply to @dnrix:matrix.org Hey all, I recently got annoyed by the lack of caching in incremental Go builds in Nix and decided to just fix it to work the way I thought it should (opt-in similar to ccache). I came up with this: https://github.com/dnr/nix-gocacheprog Let me know how it works for you! Nice! We are working on a similar approach with gocacheprog! https://github.com/adisbladis/gobuild.nix/ | 16:10:41 |
Paul Meyer (katexochen) | Ah, you already saw it. ^^ | 16:11:41 |
| 22 Dec 2024 |
dnr | I did.. that's a more ambitious project and a different focus. If I understand correctly, you end up with one derivation per module and the build caching is effectively at module granularity? I wanted to speed up incremental builds/tests so I wanted file-level granularity, more like ccache. Of course it's at the cost of purity, so it's only useful for development, but for that purpose it should be faster. | 00:54:53 |
dnr | * I did.. that's a more ambitious project and a different focus. If I understand correctly, you end up with one derivation per module and the build caching is effectively at module granularity? I wanted to speed up incremental builds/tests so I wanted package-level granularity, more like ccache. Of course it's at the cost of purity, so it's only useful for development, but for that purpose it should be faster. | 00:58:28 |
adisbladis | I think it's really two projects rolled up into one, with different goals:
-
One is the cache prog bits which you can use with or without the builder infra
I'm very much intending for this to be something that you can use in development shells to load cache either from an impure location, or from the nix store, or a hybrid.
-
The second one is the builder infrastructure
I wouldn't get any value out of a ccache style solution as it requires host-specific config for every host/CI. I want to fundamentally fix the packaging model.
| 04:29:21 |
adisbladis | I think per module granularity is probably a sweet spot, but you could go finer grained if you wanted to. | 04:29:57 |
dnr | That's true, per-module is probably fine for most projects. Per-package (or cache object) came out naturally from the way I shared the cache. | 11:10:45 |
dnr | How do you plan to coordinate writes to a shared impure cache? That's what led me to the client/server model. | 11:11:22 |
dnr | * That's true, per-module is probably fine for most projects. Per-package (or cache object) came out naturally from the way I shared the cache. And I was specifically trying to speed up the edit/build/debug cycle so any tiny improvement is good. | 11:13:06 |
| 23 Dec 2024 |
adisbladis | When I'm saying "impure" I'm talking about development shells (and being able to read cache from the store while writing to an impure location), and for that use case there is no coordination problem as there is only a single uid writing to the cache. | 01:53:10 |
| 26 Dec 2024 |
| pierre joined the room. | 20:16:25 |
| 27 Dec 2024 |
| kdn changed their display name from nazarewk to kdn. | 12:39:14 |
| pinpox changed their display name from pinpox to pinpox [DECT: 7170]. | 15:23:14 |
| 31 Dec 2024 |
| @kaya:catnip.ee changed their profile picture. | 21:49:20 |
| 1 Jan 2025 |
| NixOS Moderation Botchanged room power levels. | 14:26:32 |
| pinpox changed their display name from pinpox [DECT: 7170] to pinpox. | 18:20:17 |
| 5 Jan 2025 |
| gotha joined the room. | 07:12:39 |
| 6 Jan 2025 |
| Frédéric Christ changed their display name from Frédéric Christ 🎄23.12. - 07.01. to Frédéric Christ. | 12:31:55 |
| 8 Jan 2025 |
| vendion joined the room. | 14:37:04 |
vendion | I don't know if this is the right channel for this question, but seeing as how I'm trying to build a Go application with devenv I figure it would be worth a shot.
I have a project that I'm using devenv to manage the development environment, and I have everyithng setup already so I can build the project with devenv build but the resulting binary just gets put in the nix store by default. I'm wanting to use the installPhase hook to put the resulting binary in the project directory but I seem to hit a roadblock in doing so. By default the resulting build artifact goes to a directory like /nix/store/l91035h1q89828h3vd5nzcss483k3k3g-dbctl-0.0.1, but I can't figure out what I need to do in the installPhase of my default.nix to have the resulting build artifact go to a path like ${PROJECT_ROOT}/out.
What I do know from trial and error:
- The PWD during the
buildPhase and installPhase is /build/source
- The
$out variable is set to the path to the Nix store (e.g. /nix/store/l91035h1q89828h3vd5nzcss483k3k3g-dbctl-0.0.1)
$src doesn't seem to work either, as it results in a perms error
If it helps here is my default.nix file:
{ lib
, pkgs
, name
, version
, ...
}:
pkgs.buildGoApplication {
pname = name;
version = version;
src = builtins.path {
path = ./.;
name = "source";
};
doCheck = false;
modules = ./gomod2nix.toml;
# installPhase = ''
# pwd
# mkdir -p $src/out/bin
# mv dbctl out/bin
# '';
meta = with lib; {
description = "Database management tool for RowLogic";
homepage = "";
maintainers = [ ];
};
}
| 15:34:39 |
| 9 Jan 2025 |
diamond (it/its) | In reply to@vendion:matrix.org
I don't know if this is the right channel for this question, but seeing as how I'm trying to build a Go application with devenv I figure it would be worth a shot.
I have a project that I'm using devenv to manage the development environment, and I have everyithng setup already so I can build the project with devenv build but the resulting binary just gets put in the nix store by default. I'm wanting to use the installPhase hook to put the resulting binary in the project directory but I seem to hit a roadblock in doing so. By default the resulting build artifact goes to a directory like /nix/store/l91035h1q89828h3vd5nzcss483k3k3g-dbctl-0.0.1, but I can't figure out what I need to do in the installPhase of my default.nix to have the resulting build artifact go to a path like ${PROJECT_ROOT}/out.
What I do know from trial and error:
- The PWD during the
buildPhase and installPhase is /build/source
- The
$out variable is set to the path to the Nix store (e.g. /nix/store/l91035h1q89828h3vd5nzcss483k3k3g-dbctl-0.0.1)
$src doesn't seem to work either, as it results in a perms error
If it helps here is my default.nix file:
{ lib
, pkgs
, name
, version
, ...
}:
pkgs.buildGoApplication {
pname = name;
version = version;
src = builtins.path {
path = ./.;
name = "source";
};
doCheck = false;
modules = ./gomod2nix.toml;
# installPhase = ''
# pwd
# mkdir -p $src/out/bin
# mv dbctl out/bin
# '';
meta = with lib; {
description = "Database management tool for RowLogic";
homepage = "";
maintainers = [ ];
};
}
installPhase should already both build and move the binary into $out/bin, so you shouldn't be overriding it like that in buildGoApplication | 00:46:45 |
diamond (it/its) | the only case where that wouldn't be true is if you don't have a package main, but it should complain I think | 00:47:13 |
diamond (it/its) | what's the output of tree $out after building everything without overriding any phase? | 00:47:25 |
diamond (it/its) | also I think you're trying to modify the source directory to add the build output? you can't do that | 00:48:02 |
diamond (it/its) | $src is immutable | 00:48:16 |
diamond (it/its) | $src is immutable (practically it's an immutable snapshot of the source code at the point of building) | 00:48:36 |
vendion | In reply to @diamondburned:matrix.org also I think you're trying to modify the source directory to add the build output? you can't do that Yeah, that is exactly what I was trying to do. Okay I'll have to rethink my approach as I was wanting to be able to use a CI pipeline to build the artifacts and use them as downloadable releases. The end users are not using Nix so having to work out deployment strategy | 00:51:12 |
diamond (it/its) | so based on https://devenv.sh/outputs/#defining-outputs, devenv build's functionality is merely to build the packages and print out the outputs to stdout. if you want to make it also put that to a build artifact directory (why?), you might want to use a task to wrap it | 00:51:18 |
diamond (it/its) | In reply to@vendion:matrix.org Yeah, that is exactly what I was trying to do. Okay I'll have to rethink my approach as I was wanting to be able to use a CI pipeline to build the artifacts and use them as downloadable releases. The end users are not using Nix so having to work out deployment strategy you can put the path to a variable then give that to the artifact upload command | 00:51:44 |
diamond (it/its) | are you using GHA? | 00:51:48 |