!QCCCSJHEsTIfozrZxz:nixos.org

Nix + Go

216 Members
Go packaging for and with Nixpkgs. | Be excellent to each other.46 Servers

Load older messages


SenderMessageTime
21 Dec 2024
@katexochen:matrix.orgPaul Meyer (katexochen)
In reply to @dnrix:matrix.org
Hey all, I recently got annoyed by the lack of caching in incremental Go builds in Nix and decided to just fix it to work the way I thought it should (opt-in similar to ccache). I came up with this:
https://github.com/dnr/nix-gocacheprog
Let me know how it works for you!
Nice! We are working on a similar approach with gocacheprog! https://github.com/adisbladis/gobuild.nix/
16:10:41
@katexochen:matrix.orgPaul Meyer (katexochen)Ah, you already saw it. ^^16:11:41
22 Dec 2024
@dnrix:matrix.orgdnrI did.. that's a more ambitious project and a different focus. If I understand correctly, you end up with one derivation per module and the build caching is effectively at module granularity? I wanted to speed up incremental builds/tests so I wanted file-level granularity, more like ccache. Of course it's at the cost of purity, so it's only useful for development, but for that purpose it should be faster.00:54:53
@dnrix:matrix.orgdnr* I did.. that's a more ambitious project and a different focus. If I understand correctly, you end up with one derivation per module and the build caching is effectively at module granularity? I wanted to speed up incremental builds/tests so I wanted package-level granularity, more like ccache. Of course it's at the cost of purity, so it's only useful for development, but for that purpose it should be faster.00:58:28
@adis:blad.isadisbladis

I think it's really two projects rolled up into one, with different goals:

  • One is the cache prog bits which you can use with or without the builder infra

    I'm very much intending for this to be something that you can use in development shells to load cache either from an impure location, or from the nix store, or a hybrid.

  • The second one is the builder infrastructure

    I wouldn't get any value out of a ccache style solution as it requires host-specific config for every host/CI. I want to fundamentally fix the packaging model.

04:29:21
@adis:blad.isadisbladisI think per module granularity is probably a sweet spot, but you could go finer grained if you wanted to.04:29:57
@dnrix:matrix.orgdnrThat's true, per-module is probably fine for most projects. Per-package (or cache object) came out naturally from the way I shared the cache.11:10:45
@dnrix:matrix.orgdnrHow do you plan to coordinate writes to a shared impure cache? That's what led me to the client/server model.11:11:22
@dnrix:matrix.orgdnr* That's true, per-module is probably fine for most projects. Per-package (or cache object) came out naturally from the way I shared the cache. And I was specifically trying to speed up the edit/build/debug cycle so any tiny improvement is good.11:13:06
23 Dec 2024
@adis:blad.isadisbladis When I'm saying "impure" I'm talking about development shells (and being able to read cache from the store while writing to an impure location), and for that use case there is no coordination problem as there is only a single uid writing to the cache. 01:53:10
26 Dec 2024
@pierre:eng42.devpierre joined the room.20:16:25
27 Dec 2024
@nazarewk:matrix.orgkdn changed their display name from nazarewk to kdn.12:39:14
@pinpox:matrix.orgpinpox changed their display name from pinpox to pinpox [DECT: 7170].15:23:14
31 Dec 2024
@kaya:catnip.ee@kaya:catnip.ee changed their profile picture.21:49:20
1 Jan 2025
@mjolnir:nixos.orgNixOS Moderation Botchanged room power levels.14:26:32
@pinpox:matrix.orgpinpox changed their display name from pinpox [DECT: 7170] to pinpox.18:20:17
5 Jan 2025
@gotha:matrix.orggotha joined the room.07:12:39
6 Jan 2025
@frederic:scs.ems.hostFrédéric Christ changed their display name from Frédéric Christ 🎄23.12. - 07.01. to Frédéric Christ.12:31:55
8 Jan 2025
@vendion:matrix.orgvendion joined the room.14:37:04
@vendion:matrix.orgvendion

I don't know if this is the right channel for this question, but seeing as how I'm trying to build a Go application with devenv I figure it would be worth a shot.

I have a project that I'm using devenv to manage the development environment, and I have everyithng setup already so I can build the project with devenv build but the resulting binary just gets put in the nix store by default. I'm wanting to use the installPhase hook to put the resulting binary in the project directory but I seem to hit a roadblock in doing so. By default the resulting build artifact goes to a directory like /nix/store/l91035h1q89828h3vd5nzcss483k3k3g-dbctl-0.0.1, but I can't figure out what I need to do in the installPhase of my default.nix to have the resulting build artifact go to a path like ${PROJECT_ROOT}/out.

What I do know from trial and error:

  • The PWD during the buildPhase and installPhase is /build/source
  • The $out variable is set to the path to the Nix store (e.g. /nix/store/l91035h1q89828h3vd5nzcss483k3k3g-dbctl-0.0.1)
  • $src doesn't seem to work either, as it results in a perms error

If it helps here is my default.nix file:

{ lib
, pkgs
, name
, version
, ...
}:
pkgs.buildGoApplication {
  pname = name;
  version = version;

  src = builtins.path {
    path = ./.;
    name = "source";
  };

  doCheck = false;

  modules = ./gomod2nix.toml;

  # installPhase = ''
  #   pwd
  #   mkdir -p $src/out/bin
  #   mv dbctl out/bin
  # '';

  meta = with lib; {
    description = "Database management tool for RowLogic";
    homepage = "";
    maintainers = [ ];
  };
}
15:34:39
9 Jan 2025
@diamondburned:matrix.orgdiamond (it/its)
In reply to@vendion:matrix.org

I don't know if this is the right channel for this question, but seeing as how I'm trying to build a Go application with devenv I figure it would be worth a shot.

I have a project that I'm using devenv to manage the development environment, and I have everyithng setup already so I can build the project with devenv build but the resulting binary just gets put in the nix store by default. I'm wanting to use the installPhase hook to put the resulting binary in the project directory but I seem to hit a roadblock in doing so. By default the resulting build artifact goes to a directory like /nix/store/l91035h1q89828h3vd5nzcss483k3k3g-dbctl-0.0.1, but I can't figure out what I need to do in the installPhase of my default.nix to have the resulting build artifact go to a path like ${PROJECT_ROOT}/out.

What I do know from trial and error:

  • The PWD during the buildPhase and installPhase is /build/source
  • The $out variable is set to the path to the Nix store (e.g. /nix/store/l91035h1q89828h3vd5nzcss483k3k3g-dbctl-0.0.1)
  • $src doesn't seem to work either, as it results in a perms error

If it helps here is my default.nix file:

{ lib
, pkgs
, name
, version
, ...
}:
pkgs.buildGoApplication {
  pname = name;
  version = version;

  src = builtins.path {
    path = ./.;
    name = "source";
  };

  doCheck = false;

  modules = ./gomod2nix.toml;

  # installPhase = ''
  #   pwd
  #   mkdir -p $src/out/bin
  #   mv dbctl out/bin
  # '';

  meta = with lib; {
    description = "Database management tool for RowLogic";
    homepage = "";
    maintainers = [ ];
  };
}
installPhase should already both build and move the binary into $out/bin, so you shouldn't be overriding it like that in buildGoApplication
00:46:45
@diamondburned:matrix.orgdiamond (it/its) the only case where that wouldn't be true is if you don't have a package main, but it should complain I think 00:47:13
@diamondburned:matrix.orgdiamond (it/its) what's the output of tree $out after building everything without overriding any phase? 00:47:25
@diamondburned:matrix.orgdiamond (it/its)also I think you're trying to modify the source directory to add the build output? you can't do that00:48:02
@diamondburned:matrix.orgdiamond (it/its) $src is immutable 00:48:16
@diamondburned:matrix.orgdiamond (it/its) $src is immutable (practically it's an immutable snapshot of the source code at the point of building) 00:48:36
@vendion:matrix.orgvendion
In reply to @diamondburned:matrix.org
also I think you're trying to modify the source directory to add the build output? you can't do that
Yeah, that is exactly what I was trying to do. Okay I'll have to rethink my approach as I was wanting to be able to use a CI pipeline to build the artifacts and use them as downloadable releases. The end users are not using Nix so having to work out deployment strategy
00:51:12
@diamondburned:matrix.orgdiamond (it/its) so based on https://devenv.sh/outputs/#defining-outputs, devenv build's functionality is merely to build the packages and print out the outputs to stdout. if you want to make it also put that to a build artifact directory (why?), you might want to use a task to wrap it 00:51:18
@diamondburned:matrix.orgdiamond (it/its)
In reply to@vendion:matrix.org
Yeah, that is exactly what I was trying to do. Okay I'll have to rethink my approach as I was wanting to be able to use a CI pipeline to build the artifacts and use them as downloadable releases. The end users are not using Nix so having to work out deployment strategy
you can put the path to a variable then give that to the artifact upload command
00:51:44
@diamondburned:matrix.orgdiamond (it/its)are you using GHA?00:51:48

Show newer messages


Back to Room ListRoom Version: 9