!rWxyQqNqMUDLECdsIf:blad.is

Poetry2nix

322 Members
https://github.com/nix-community/poetry2nix65 Servers

Load older messages


SenderMessageTime
4 Aug 2022
@adis:blad.isadisbladis
In reply to @plato:matrix.org
Do we have a preferred pastebin? I can share the Nix file
I mostly just paste directly in matrix nowadays
10:49:11
@adis:blad.isadisbladisIt's fine10:49:25
@plato:matrix.orgplato
let
  pkgs = import (import ./nix/sources.nix { }).nixpkgs { };
  omniorb_4_2 = with pkgs; stdenv.mkDerivation rec {
    pname = "omniorb";
    version = "4.2.5";

    src = fetchurl {
      url = "mirror://sourceforge/project/omniorb/omniORB/omniORB-${version}/omniORB-${version}.tar.bz2";
      sha256 = "1fvkw3pn9i2312n4k3d4s7892m91jynl8g1v2z0j8k1gzfczjp7h";
    };

    nativeBuildInputs = [ pkg-config ];
    buildInputs = [ python3 ];

    enableParallelBuilding = true;
    hardeningDisable = [ "format" ];
  };
  tango = pkgs.stdenv.mkDerivation rec {
    pname = "tango";
    version = "9.3.5";

    src = pkgs.fetchurl {
      url = "https://gitlab.com/api/v4/projects/24125890/packages/generic/TangoSourceDistribution/${version}/${pname}-${version}.tar.gz";
      sha256 = "1i59023gqm6sk000520y4kamfnfa8xqy9xwsnz5ch22nflgqn9px";
    };

    enableParallelBuilding = true;
    nativeBuildInputs = with pkgs; [ pkg-config jdk ];
    buildInputs = with pkgs; [ zlib omniorb_4_2 zeromq cppzmq ];

    configureFlags = with pkgs; [
      "--enable-java=yes"
      "--enable-dbserver=yes"
      "--with-java=${openjdk8}/bin/java"
      "--with-mysqlclient-lib=${mariadb-connector-c}/lib/mariadb/"
      "--with-mysqlclient-include=${mariadb-connector-c.dev}/include/mariadb"
    ];

    postInstall = ''
      mkdir -p $out/share/sql
      for fn in cppserver/database/*sql; do
        sed -e "s#^source #source $out/share/sql/#" "$fn" > $out/share/sql/$(basename "$fn")
      done
    '';
  };
  poetry2nix-latest-src = pkgs.fetchFromGitHub {
    owner = "nix-community";
    repo = "poetry2nix";
    rev = "1.31.0";
    hash = "sha256-0o856HWRRc5q02+vIhlIW4NpeQUDvCv3CuP1w2rZ+ho=";
  };
  poetry2nix = (import poetry2nix-latest-src { inherit pkgs; poetry = pkgs.poetry; });
  boostPython = pkgs.boost.override {
    enablePython = true;
    python = pkgs.python3;
  };
  poetryOverrides = poetry2nix.overrides.withDefaults (self: super: {
    pytango = super.pytango.overrideAttrs (old: {
      src = pkgs.fetchFromGitLab {
        owner = "tango-controls";
        repo = "pytango";
        rev = "develop";
        hash = "sha256-MC/EWwEN1u6+piv6LgkavM1OF773lLt9j/VO3Tg2/9k=";
      };
      
      # python3 because it needs pyconfig.h
      nativeBuildInputs = with pkgs; [ pkg-config boostPython python3 rsync ];
      buildInputs = with pkgs; [ tango boostPython omniorb_4_2 zeromq cppzmq ];
      propagatedBuildInputs = with super; [ six numpy ];
      makeFlags = [ "prefix=${placeholder "out"}" ];
      postPatch = ''
        sed -ie 's#os.environ.get("BOOST_HEADERS")#"${boostPython.dev}/include"#' setup.py
        sed -ie 's#os.environ.get("BOOST_LIBRARIES")#"${boostPython}"/lib#' setup.py
        sed -ie 's#os.environ.get("BOOST_PYTHON_LIB")#"boost_python310"#' setup.py
        sed -ie 's#boost_python-py#boost_python#' Makefile
        sed -ie 's#CFLAGS := #CFLAGS := -std=c++14 #' Makefile
        mv Makefile Makefile-old
      '';
    });
  });
  pythonPackage = poetry2nix.mkPoetryApplication {
    projectDir = ./.;
    overrides = poetryOverrides;
  };
  pythonEnv = poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    overrides = poetryOverrides;
  };
in
{
  inherit tango;
  inherit pythonPackage;
  inherit boostPython;
}

That's the nix file.

10:49:36
@plato:matrix.orgplato The pyproject.toml just has pytango = "9.3.4" in the dependencies. Super simple 10:50:11
@plato:matrix.orgplato Ah, and disregard the mv Makefile Makefile-old, that's just me debugging. 10:51:01
@plato:matrix.orgplatoAnd the override for pytango is convoluted, but it needs some specific dependencies and builds stuff with boost.python.10:51:23
@adis:blad.isadisbladis
In reply to @plato:matrix.org
let
  pkgs = import (import ./nix/sources.nix { }).nixpkgs { };
  omniorb_4_2 = with pkgs; stdenv.mkDerivation rec {
    pname = "omniorb";
    version = "4.2.5";

    src = fetchurl {
      url = "mirror://sourceforge/project/omniorb/omniORB/omniORB-${version}/omniORB-${version}.tar.bz2";
      sha256 = "1fvkw3pn9i2312n4k3d4s7892m91jynl8g1v2z0j8k1gzfczjp7h";
    };

    nativeBuildInputs = [ pkg-config ];
    buildInputs = [ python3 ];

    enableParallelBuilding = true;
    hardeningDisable = [ "format" ];
  };
  tango = pkgs.stdenv.mkDerivation rec {
    pname = "tango";
    version = "9.3.5";

    src = pkgs.fetchurl {
      url = "https://gitlab.com/api/v4/projects/24125890/packages/generic/TangoSourceDistribution/${version}/${pname}-${version}.tar.gz";
      sha256 = "1i59023gqm6sk000520y4kamfnfa8xqy9xwsnz5ch22nflgqn9px";
    };

    enableParallelBuilding = true;
    nativeBuildInputs = with pkgs; [ pkg-config jdk ];
    buildInputs = with pkgs; [ zlib omniorb_4_2 zeromq cppzmq ];

    configureFlags = with pkgs; [
      "--enable-java=yes"
      "--enable-dbserver=yes"
      "--with-java=${openjdk8}/bin/java"
      "--with-mysqlclient-lib=${mariadb-connector-c}/lib/mariadb/"
      "--with-mysqlclient-include=${mariadb-connector-c.dev}/include/mariadb"
    ];

    postInstall = ''
      mkdir -p $out/share/sql
      for fn in cppserver/database/*sql; do
        sed -e "s#^source #source $out/share/sql/#" "$fn" > $out/share/sql/$(basename "$fn")
      done
    '';
  };
  poetry2nix-latest-src = pkgs.fetchFromGitHub {
    owner = "nix-community";
    repo = "poetry2nix";
    rev = "1.31.0";
    hash = "sha256-0o856HWRRc5q02+vIhlIW4NpeQUDvCv3CuP1w2rZ+ho=";
  };
  poetry2nix = (import poetry2nix-latest-src { inherit pkgs; poetry = pkgs.poetry; });
  boostPython = pkgs.boost.override {
    enablePython = true;
    python = pkgs.python3;
  };
  poetryOverrides = poetry2nix.overrides.withDefaults (self: super: {
    pytango = super.pytango.overrideAttrs (old: {
      src = pkgs.fetchFromGitLab {
        owner = "tango-controls";
        repo = "pytango";
        rev = "develop";
        hash = "sha256-MC/EWwEN1u6+piv6LgkavM1OF773lLt9j/VO3Tg2/9k=";
      };
      
      # python3 because it needs pyconfig.h
      nativeBuildInputs = with pkgs; [ pkg-config boostPython python3 rsync ];
      buildInputs = with pkgs; [ tango boostPython omniorb_4_2 zeromq cppzmq ];
      propagatedBuildInputs = with super; [ six numpy ];
      makeFlags = [ "prefix=${placeholder "out"}" ];
      postPatch = ''
        sed -ie 's#os.environ.get("BOOST_HEADERS")#"${boostPython.dev}/include"#' setup.py
        sed -ie 's#os.environ.get("BOOST_LIBRARIES")#"${boostPython}"/lib#' setup.py
        sed -ie 's#os.environ.get("BOOST_PYTHON_LIB")#"boost_python310"#' setup.py
        sed -ie 's#boost_python-py#boost_python#' Makefile
        sed -ie 's#CFLAGS := #CFLAGS := -std=c++14 #' Makefile
        mv Makefile Makefile-old
      '';
    });
  });
  pythonPackage = poetry2nix.mkPoetryApplication {
    projectDir = ./.;
    overrides = poetryOverrides;
  };
  pythonEnv = poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    overrides = poetryOverrides;
  };
in
{
  inherit tango;
  inherit pythonPackage;
  inherit boostPython;
}

That's the nix file.

There were a bunch of minor issues.

let
  pkgs = import <nixpkgs> {
    overlays = [
      (import ../../overlay.nix)
    ];
  };
  inherit (pkgs) poetry2nix;

  omniorb_4_2 = with pkgs; stdenv.mkDerivation rec {
    pname = "omniorb";
    version = "4.2.5";

    src = fetchurl {
      url = "mirror://sourceforge/project/omniorb/omniORB/omniORB-${version}/omniORB-${version}.tar.bz2";
      sha256 = "1fvkw3pn9i2312n4k3d4s7892m91jynl8g1v2z0j8k1gzfczjp7h";
    };

    nativeBuildInputs = [ pkg-config ];
    buildInputs = [ python3 ];

    enableParallelBuilding = true;
    hardeningDisable = [ "format" ];
  };

  tango = pkgs.stdenv.mkDerivation rec {
    pname = "tango";
    version = "9.3.5";

    src = pkgs.fetchurl {
      url = "https://gitlab.com/api/v4/projects/24125890/packages/generic/TangoSourceDistribution/${version}/${pname}-${version}.tar.gz";
      sha256 = "1i59023gqm6sk000520y4kamfnfa8xqy9xwsnz5ch22nflgqn9px";
    };

    enableParallelBuilding = true;
    nativeBuildInputs = with pkgs; [ pkg-config jdk ];
    buildInputs = with pkgs; [ zlib omniorb_4_2 zeromq cppzmq ];

    configureFlags = with pkgs; [
      "--enable-java=yes"
      "--enable-dbserver=yes"
      "--with-java=${openjdk8}/bin/java"
      "--with-mysqlclient-lib=${mariadb-connector-c}/lib/mariadb/"
      "--with-mysqlclient-include=${mariadb-connector-c.dev}/include/mariadb"
    ];

    postInstall = ''
      mkdir -p $out/share/sql
      for fn in cppserver/database/*sql; do
        sed -e "s#^source #source $out/share/sql/#" "$fn" > $out/share/sql/$(basename "$fn")
      done
    '';
  };

  boostPython = pkgs.boost.override {
    enablePython = true;
    python = pkgs.python3;
  };

  poetryOverrides = poetry2nix.overrides.withDefaults (self: super: {

    pytango = super.pytango.overridePythonAttrs (old: {
      nativeBuildInputs = old.nativeBuildInputs ++ [
        pkgs.pkg-config
        boostPython
        pkgs.rsync
      ];

      buildInputs = old.buildInputs ++ [
        tango
        boostPython
        omniorb_4_2
        pkgs.zeromq
        pkgs.cppzmq
      ];

      propagatedBuildInputs = old.propagatedBuildInputs ++ [
        self.numpy
      ];

      makeFlags = [
        "prefix=${placeholder "out"}"
      ];

      postPatch = let
        v = builtins.elemAt (builtins.split "\\." self.python.version);
        boostPythonLib = v 0 + v 2;
      in ''
        substituteInPlace setup.py \
          --replace 'os.environ.get("BOOST_HEADERS")' '"${boostPython.dev}/include"' \
          --replace 'os.environ.get("BOOST_LIBRARIES")' '"${boostPython}/lib"' \
          --replace 'os.environ.get("BOOST_PYTHON_LIB")' '"boost_python${boostPythonLib}"'

        substituteInPlace Makefile \
          --replace 'boost_python-py' 'boost_python'
      '';
    });

  });

  pythonPackage = poetry2nix.mkPoetryApplication {
    projectDir = ./.;
    overrides = poetryOverrides;
  };

  pythonEnv = poetry2nix.mkPoetryEnv {
    projectDir = ./.;
    overrides = poetryOverrides;
  };

in
{
  inherit tango;
  inherit pythonPackage;
  inherit boostPython;
}

builds fine, I think you can adapt it back without too much issue.

14:07:11
@plato:matrix.orgplato Indeed it works, thanks adisbladis! I suppose it was the adding old. variables? 14:29:30
@adis:blad.isadisbladisYes that was the main thing 14:30:17
@plato:matrix.orgplatoOkay, makes total sense.14:33:52
@plato:matrix.orgplatoThanks for poetry2nix btw.. This alone might make my colleagues use Nix in the end.14:34:17
9 Aug 2022
@gurkan:matrix.orggurkan joined the room.12:24:00
@meain:matrix.orgmeain joined the room.17:10:21
@meain:matrix.orgmeain

Hey everyone, I was trying to package up aw-watcher-mpd using poetry2nix, but I am running into the following issue. Can someone give me a hint as to where to look or what could be wrong? You can find the derivation here ref. The failing CI run can be found here.

error: path '/nix/store/fjm96pf9534sb5f5bxbnpx59xx3dkz0c-source.drv' is not valid

       … while realising the context of path '/nix/store/d529wh3c0086pmc0z40gj2x8gm1sq3x1-source/poetry.lock'

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/pkgs/development/tools/poetry2nix/poetry2nix/lib.nix:70:30:

           69|     );
           70|   readTOML = path: fromTOML (builtins.readFile path);
             |                              ^
           71|

       … while evaluating 'readTOML'

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/pkgs/development/tools/poetry2nix/poetry2nix/lib.nix:70:14:

           69|     );
           70|   readTOML = path: fromTOML (builtins.readFile path);
             |              ^
           71|

       … from call site

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/pkgs/development/tools/poetry2nix/poetry2nix/default.nix:143:20:

          142|
          143|       poetryLock = readTOML poetrylock;
             |                    ^
          144|       lockFiles =

       … while evaluating anonymous lambda

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/pkgs/development/tools/poetry2nix/poetry2nix/default.nix:226:20:

          225|             # Null out any filtered packages, we don't want python.pkgs from nixpkgs
          226|             (self: super: builtins.listToAttrs (builtins.map (x: { name = moduleName x.name; value = null; }) incompatible))
             |                    ^
          227|             # Create poetry2nix layer

       … from call site

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/fixed-points.nix:76:22:

           75|     f: g: final: prev:
           76|       let fApplied = f final prev;
             |                      ^
           77|           prev' = prev // fApplied;

       … while evaluating 'composeExtensions'

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/fixed-points.nix:75:18:

           74|   composeExtensions =
           75|     f: g: final: prev:
             |                  ^
           76|       let fApplied = f final prev;

       … from call site

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/fixed-points.nix:78:22:

           77|           prev' = prev // fApplied;
           78|       in fApplied // g final prev';
             |                      ^
           79|

       … while evaluating 'composeExtensions'

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/fixed-points.nix:75:18:

           74|   composeExtensions =
           75|     f: g: final: prev:
             |                  ^
           76|       let fApplied = f final prev;

       … from call site

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/fixed-points.nix:76:22:

           75|     f: g: final: prev:
           76|       let fApplied = f final prev;
             |                      ^
           77|           prev' = prev // fApplied;

       … while evaluating 'composeExtensions'

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/fixed-points.nix:75:18:

           74|   composeExtensions =
           75|     f: g: final: prev:
             |                  ^
           76|       let fApplied = f final prev;

       … from call site

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/fixed-points.nix:78:22:

           77|           prev' = prev // fApplied;
           78|       in fApplied // g final prev';
             |                      ^
           79|

       … while evaluating 'composeExtensions'

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/fixed-points.nix:75:18:

           74|   composeExtensions =
           75|     f: g: final: prev:
             |                  ^
           76|       let fApplied = f final prev;

       … from call site

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/fixed-points.nix:69:67:

           68|   #
           69|   extends = f: rattrs: self: let super = rattrs self; in super // f self super;
             |                                                                   ^
           70|

       … while evaluating 'extends'

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/fixed-points.nix:69:24:

           68|   #
           69|   extends = f: rattrs: self: let super = rattrs self; in super // f self super;
             |                        ^
           70|

       … from call site

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/customisation.nix:233:14:

          232|       spliced = extra spliced0 // spliced0 // keep self;
          233|       self = f self // {
             |              ^
          234|         newScope = scope: newScope (spliced // scope);

       … while evaluating 'makeScopeWithSplicing'


            1| { lib, poetry2nix, python37, fetchFromGitHub }:
             | ^
            2|

       … from call site

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/customisation.nix:69:16:

           68|     let
           69|       result = f origArgs;
             |                ^
           70|

       … while evaluating 'makeOverridable'

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/customisation.nix:67:24:

           66|   */
           67|   makeOverridable = f: origArgs:
             |                        ^
           68|     let

       … from call site

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/customisation.nix:121:8:

          120|       auto = builtins.intersectAttrs (lib.functionArgs f) autoArgs;
          121|     in makeOverridable f (auto // args);
             |        ^
          122|

       … while evaluating 'callPackageWith'

       at /nix/store/maia4sbq9b5svg2l3yrrbcw5lpbckg2c-source/lib/customisation.nix:117:35:

          116|   */
          117|   callPackageWith = autoArgs: fn: args:
             |                                   ^
          118|     let

       … from call site

       at /home/runner/work/nur-packages/nur-packages/default.nix:41:20:

           40|   pulseaudio-virtualmic = pkgs.callPackage ./pkgs/pulseaudio-virtualmic { };
           41|   aw-watcher-mpd = pkgs.callPackage ./pkgs/aw-watcher-mpd { };
             |                    ^
           42|
17:23:33
11 Aug 2022
@a7p:matrix.orgAlbrecht changed their display name from Albrecht Mühlenschulte to Albrecht.09:58:47
@nazarewk:matrix.orgnazarewk how can i make mkPoetryEnv include dev-dependencies? 11:40:05
@nazarewk:matrix.orgnazarewk erm... i can't install isort with poetry2nix due to error: Missing suitable source/wheel file entry for isort, but it is there https://pypi.org/project/isort/5.10.1/#files 12:49:07
@k900:0upti.meK900 poetry --version? 12:50:10
@nazarewk:matrix.orgnazarewkdoesn't it use some own version of poetry?12:50:21
@k900:0upti.meK900The lockfile generation doesn't12:50:37
@nazarewk:matrix.orgnazarewki'm on 1.1.14 on system12:50:40
@k900:0upti.meK900Is it an old lockfile?12:50:51
@nazarewk:matrix.orgnazarewkyeah, like 2 months12:51:09
@k900:0upti.meK900It's probably broken then, try regenerating it12:51:32
@nazarewk:matrix.orgnazarewk ok, poetry lock helped 12:53:30
13 Aug 2022
@ethanabrooks:matrix.orgEthan Brooks joined the room.18:26:41
@ethanabrooks:matrix.orgEthan BrooksRedacted or Malformed Event18:27:43
@ethanabrooks:matrix.orgEthan BrooksRedacted or Malformed Event18:30:19
@ethanabrooks:matrix.orgEthan BrooksRedacted or Malformed Event18:30:41
14 Aug 2022
@ethanabrooks:matrix.orgEthan Brooks Hello! I am trying to get a package (tensorstore) to build. I am running into an issue where bazelisk.py attempts to write to the home directory. Would it be possible to get some help with this issue? Thank you. 14:07:13

There are no newer messages yet.


Back to Room ListRoom Version: 6