!UUYziobKGGxpovWyAN:nixos.org

Robotnix

248 Members
Build Android (AOSP) using Nix | https://github.com/danielfullmer/robotnix76 Servers

Load older messages


SenderMessageTime
5 Sep 2021
@danielrf:matrix.orgdanielrf My understanding is about --arg matches what samueldr said. The original idea was to have configuration as a top-level argument, so that it matched what NixOS does in nixpkgs/nixos/default.nix. 18:37:50
@danielrf:matrix.orgdanielrf The newer nix command line interface when using flakes does not support --arg on the commandline, you currently have to make a flake.nix and call robotnix.lib.robotnixSystem 18:39:46
@danielrf:matrix.orgdanielrf But if the configuration is short enough (and I'm just testing something out), I like the convenience of specifying the options directly on the command line, as opposed to making a .nix file every time. 18:42:02
@samueldr:matrix.orgsamueldr(off-topic) huh, I didn't know that about flakes... a step backward in UX imho18:42:42
@danielrf:matrix.orgdanielrf kranzes: Hey, that's great! If you're interested in figuring out exactly what is breaking it in the lineageos configuration, you can start commenting out the sections that don't seem needed and testing. Bth, my best guess now actually is the webview configuration overrides we do in the lineageos flavor 18:46:50
@danielrf:matrix.orgdanielrf * kranzes: Hey, that's great! If you're interested in figuring out exactly what is breaking it in the lineageos configuration, you can start commenting out the sections that don't seem needed and testing. Tbh, my best guess now actually is the webview configuration overrides we do in the lineageos flavor 18:46:57
@danielrf:matrix.orgdanielrf samueldr: I have mixed feelings about it as well. Things that should be easy take a bit of boilerplate to make work. Reproducibility/purity benefits outweigh the costs for me though. 18:48:16
@kranzes:matrix.orgkranzesThat would take me ages18:48:18
@kranzes:matrix.orgkranzesevery build is 3 hours18:48:26
@kranzes:matrix.orgkranzesand i don't even know where to start18:48:42
@danielrf:matrix.orgdanielrfup to you if you want to continue tracking it down. The long turnaround time is an unfortunate thing we deal with on android.18:50:21
@danielrf:matrix.orgdanielrfBut I'd probably start with commenting out the webview stuff like so:18:50:38
@danielrf:matrix.orgdanielrf
diff --git a/flavors/lineageos/default.nix b/flavors/lineageos/default.nix
index 1216ca8..5407fc8 100644
--- a/flavors/lineageos/default.nix
+++ b/flavors/lineageos/default.nix
@@ -106,9 +106,9 @@ in mkIf (config.flavor == "lineageos")
         })
       ];
 
-      # LineageOS will sometimes force-push to this repo, and the older revisions are garbage collected.
-      # So we'll just build chromium webview ourselves.
-      "external/chromium-webview".enable = false;
+#      # LineageOS will sometimes force-push to this repo, and the older revisions are garbage collected.
+#      # So we'll just build chromium webview ourselves.
+#      "external/chromium-webview".enable = false;
     }
   ] ++ optionals (deviceMetadata ? "${config.device}") [
     # Device-specific source dirs
@@ -134,16 +134,16 @@ in mkIf (config.flavor == "lineageos")
   source.manifest.url = mkDefault "https://github.com/LineageOS/android.git";
   source.manifest.rev = mkDefault "refs/heads/${LineageOSRelease}";
 
-  # Enable robotnix-built chromium / webview
-  apps.chromium.enable = mkDefault true;
-  webview.chromium.availableByDefault = mkDefault true;
-  webview.chromium.enable = mkDefault true;
-
-  # This is the prebuilt webview apk from LineageOS. Adding this here is only
-  # for convenience if the end-user wants to set `webview.prebuilt.enable = true;`.
-  webview.prebuilt.apk = config.source.dirs."external/chromium-webview".src + "/prebuilt/${config.arch}/webview.apk";
-  webview.prebuilt.availableByDefault = mkDefault true;
-  removedProductPackages = [ "webview" ];
+#  # Enable robotnix-built chromium / webview
+#  apps.chromium.enable = mkDefault true;
+#  webview.chromium.availableByDefault = mkDefault true;
+#  webview.chromium.enable = mkDefault true;
+#
+#  # This is the prebuilt webview apk from LineageOS. Adding this here is only
+#  # for convenience if the end-user wants to set `webview.prebuilt.enable = true;`.
+#  webview.prebuilt.apk = config.source.dirs."external/chromium-webview".src + "/prebuilt/${config.arch}/webview.apk";
+#  webview.prebuilt.availableByDefault = mkDefault true;
+#  removedProductPackages = [ "webview" ];
 
   # Needed by included kernel build for some devices (pioneer at least)
   envPackages = [ pkgs.openssl.dev ] ++ optionals (config.androidVersion == 11) [ pkgs.gcc.cc pkgs.glibc.dev ];
18:50:48
@danielrf:matrix.orgdanielrf * up to you if you want to continue tracking it down. The long build/test cycle time is an unfortunate thing we deal with on android.18:51:33
@jcie74:matrix.orgpie_ samueldr: danielrf https://gist.github.com/cleverca22/240c21dab528666c85a95c4edfdeae58 19:13:40
@jcie74:matrix.orgpie_re: --arg19:13:43
@jcie74:matrix.orgpie_i couldnt remember where to find the relevant part of the source though19:13:53
@samueldr:matrix.orgsamueldrah, I think that's because they float up to the top-level19:15:00
@samueldr:matrix.orgsamueldror something like that19:15:26
@samueldr:matrix.orgsamueldrthough I guess it's less of an issue than could be thought, as I've literally never seen that being an issue19:15:45
@jcie74:matrix.orgpie_i think what happens is that things passed via such get added to a global autoargs variable19:15:51
@jcie74:matrix.orgpie_yeah im not sure why it never appears to be a problem19:16:10
@jcie74:matrix.orgpie_maybe they are only set when an argument isnt otherwise passed or something?19:16:28
@samueldr:matrix.orgsamueldr --arg already requires arguments to be named and present in the attrset pattern argument 19:18:04
@samueldr:matrix.orgsamueldrso it would make sense to have it apply until it can't in deeper nested function calls?19:18:45
@kranzes:matrix.orgkranzes
In reply to @danielrf:matrix.org
diff --git a/flavors/lineageos/default.nix b/flavors/lineageos/default.nix
index 1216ca8..5407fc8 100644
--- a/flavors/lineageos/default.nix
+++ b/flavors/lineageos/default.nix
@@ -106,9 +106,9 @@ in mkIf (config.flavor == "lineageos")
         })
       ];
 
-      # LineageOS will sometimes force-push to this repo, and the older revisions are garbage collected.
-      # So we'll just build chromium webview ourselves.
-      "external/chromium-webview".enable = false;
+#      # LineageOS will sometimes force-push to this repo, and the older revisions are garbage collected.
+#      # So we'll just build chromium webview ourselves.
+#      "external/chromium-webview".enable = false;
     }
   ] ++ optionals (deviceMetadata ? "${config.device}") [
     # Device-specific source dirs
@@ -134,16 +134,16 @@ in mkIf (config.flavor == "lineageos")
   source.manifest.url = mkDefault "https://github.com/LineageOS/android.git";
   source.manifest.rev = mkDefault "refs/heads/${LineageOSRelease}";
 
-  # Enable robotnix-built chromium / webview
-  apps.chromium.enable = mkDefault true;
-  webview.chromium.availableByDefault = mkDefault true;
-  webview.chromium.enable = mkDefault true;
-
-  # This is the prebuilt webview apk from LineageOS. Adding this here is only
-  # for convenience if the end-user wants to set `webview.prebuilt.enable = true;`.
-  webview.prebuilt.apk = config.source.dirs."external/chromium-webview".src + "/prebuilt/${config.arch}/webview.apk";
-  webview.prebuilt.availableByDefault = mkDefault true;
-  removedProductPackages = [ "webview" ];
+#  # Enable robotnix-built chromium / webview
+#  apps.chromium.enable = mkDefault true;
+#  webview.chromium.availableByDefault = mkDefault true;
+#  webview.chromium.enable = mkDefault true;
+#
+#  # This is the prebuilt webview apk from LineageOS. Adding this here is only
+#  # for convenience if the end-user wants to set `webview.prebuilt.enable = true;`.
+#  webview.prebuilt.apk = config.source.dirs."external/chromium-webview".src + "/prebuilt/${config.arch}/webview.apk";
+#  webview.prebuilt.availableByDefault = mkDefault true;
+#  removedProductPackages = [ "webview" ];
 
   # Needed by included kernel build for some devices (pioneer at least)
   envPackages = [ pkgs.openssl.dev ] ++ optionals (config.androidVersion == 11) [ pkgs.gcc.cc pkgs.glibc.dev ];
what does source.manifest do exactly? its still on LineageOS instead of ForkLineageOS?
19:19:44
@danielrf:matrix.orgdanielrf source.manifest is only used if source.evalTimeFetching is enabled (which is an old system that's largely untested these days). Arguably we should remove those options since they aren't being used. 19:22:13
@danielrf:matrix.orgdanielrfSee last section of https://docs.robotnix.org/modules/source.html19:22:36
@kranzes:matrix.orgkranzesI'm not exactly sure what i need to comment out again19:22:55
@kranzes:matrix.orgkranzes
# SPDX-FileCopyrightText: 2020 Daniel Fullmer and robotnix contributors
# SPDX-License-Identifier: MIT

{ config, pkgs, lib, ... }:
let
  inherit (lib)
    optional optionals optionalString optionalAttrs
    elem mapAttrs mapAttrs' nameValuePair filterAttrs
    attrNames getAttrs flatten remove
    mkIf mkMerge mkDefault mkForce
    importJSON toLower hasPrefix;

  androidVersionToLineageBranch = {
    "10" = "lineage-17.1";
    "11" = "lineage-18.1";
  };
  lineageBranchToAndroidVersion = mapAttrs' (name: value: nameValuePair value name) androidVersionToLineageBranch;

  deviceMetadata = lib.importJSON ./device-metadata.json;
  LineageOSRelease = androidVersionToLineageBranch.${builtins.toString config.androidVersion};
  repoDirs = lib.importJSON (./. + "/${LineageOSRelease}/repo.json");
  _deviceDirs = importJSON (./. + "/${LineageOSRelease}/device-dirs.json");
  vendorDirs = importJSON (./. + "/${LineageOSRelease}/vendor-dirs.json");

  # TODO: Condition on soc name?
  dtbReproducibilityFix = ''
    sed -i \
      's/^DTB_OBJS := $(shell find \(.*\))$/DTB_OBJS := $(sort $(shell find \1))/' \
      arch/arm64/boot/Makefile
  '';
  kernelsNeedFix = [
    # Only verified marlin reproducibility is fixed by this, however these other repos have the same issue
    "kernel/asus/sm8150"
    "kernel/bq/msm8953"
    "kernel/essential/msm8998"
    "kernel/google/marlin"
    "kernel/leeco/msm8996"
    "kernel/lge/msm8996"
    "kernel/motorola/msm8996"
    "kernel/motorola/msm8998"
    "kernel/motorola/sdm632"
    "kernel/nubia/msm8998"
    "kernel/oneplus/msm8996"
    "kernel/oneplus/sdm845"
    "kernel/oneplus/sm8150"
    "kernel/razer/msm8998"
    "kernel/samsung/sdm670"
    "kernel/sony/sdm660"
    "kernel/xiaomi/jason"
    "kernel/xiaomi/msm8998"
    "kernel/xiaomi/sdm660"
    "kernel/xiaomi/sdm845"
    "kernel/yandex/sdm660"
    "kernel/zuk/msm8996"
  ];
  # Patch kernels
  patchKernelDir = n: v: v // (optionalAttrs (hasPrefix "kernel/" n) {
    patches = config.kernel.patches;
    postPatch = config.kernel.postPatch
      + optionalString (config.useReproducibilityFixes && (elem n kernelsNeedFix)) ("\n" + dtbReproducibilityFix);
  });
  deviceDirs = mapAttrs patchKernelDir _deviceDirs;

  supportedDevices = attrNames deviceMetadata;

  # TODO: Move this filtering into vanilla/graphene
  filterDirAttrs = dir: filterAttrs (n: v: elem n [ "rev" "sha256" "url" "patches" "postPatch" ]) dir;
  filterDirsAttrs = dirs: mapAttrs (n: v: filterDirAttrs v) dirs;
in
mkIf (config.flavor == "lineageos")
{
  androidVersion =
    let
      defaultBranch = deviceMetadata.${config.device}.branch;
    in
    mkIf (deviceMetadata ? ${config.device}) (mkDefault (lib.toInt lineageBranchToAndroidVersion.${defaultBranch}));

  productNamePrefix = "lineage_"; # product names start with "lineage_"

  buildDateTime = mkDefault 1628634098;

  # LineageOS uses this by default. If your device supports it, I recommend using variant = "user"
  variant = mkDefault "userdebug";

  warnings = optional
    (
      (config.device != null) &&
      !(elem config.device supportedDevices) &&
      (config.deviceFamily != "generic")
    )
    "${config.device} is not an officially-supported device for LineageOS";

  source.dirs = mkMerge ([
    repoDirs

    {
      "vendor/lineage".patches = [
        ./0001-Remove-LineageOS-keys.patch
        (pkgs.substituteAll {
          src = ./0002-bootanimation-Reproducibility-fix.patch;
          inherit (pkgs) imagemagick;
        })
        ./0003-kernel-Set-constant-kernel-timestamp.patch
      ];
      "system/extras".patches = [
        # pkgutil.get_data() not working, probably because we don't use their compiled python
        (pkgs.fetchpatch {
          url = "https://github.com/LineageOS/android_system_extras/commit/7da4b29321eb7ebce9eb9a43d0fbd85d0aa1e870.patch";
          sha256 = "0pv56lypdpsn66s7ffcps5ykyfx0hjkazml89flj7p1px12zjhy1";
          revert = true;
        })
      ];

      # LineageOS will sometimes force-push to this repo, and the older revisions are garbage collected.
      # So we'll just build chromium webview ourselves.
      #      "external/chromium-webview".enable = false;
    }
  ] ++ optionals (deviceMetadata ? "${config.device}") [
    # Device-specific source dirs
    (
      let
        vendor = toLower deviceMetadata.${config.device}.vendor;
        relpathWithDependencies = relpath: [ relpath ] ++ (flatten (map (p: relpathWithDependencies p) deviceDirs.${relpath}.deps));
        relpaths = relpathWithDependencies "device/${vendor}/${config.device}";
        filteredRelpaths = remove (attrNames repoDirs) relpaths; # Remove any repos that we're already including from repo json
      in
      filterDirsAttrs (getAttrs filteredRelpaths deviceDirs)
    )

    # Vendor-specific source dirs
    (
      let
        _vendor = toLower deviceMetadata.${config.device}.vendor;
        vendor = if config.device == "shamu" then "motorola" else _vendor;
        relpath = "vendor/${vendor}";
      in
      filterDirsAttrs (getAttrs [ relpath ] vendorDirs)
    )
  ] ++ optional (config.device == "bacon")
    # Bacon needs vendor/oppo in addition to vendor/oneplus
    # See https://github.com/danielfullmer/robotnix/issues/26
    (filterDirsAttrs (getAttrs [ "vendor/oppo" ] vendorDirs))
  );

  source.manifest.url = mkDefault "https://github.com/LineageOS/android.git";
  source.manifest.rev = mkDefault "refs/heads/${LineageOSRelease}";

  # Enable robotnix-built chromium / webview
  #  apps.chromium.enable = mkDefault true;
  #  webview.chromium.availableByDefault = mkDefault true;
  #  webview.chromium.enable = mkDefault true;

  # This is the prebuilt webview apk from LineageOS. Adding this here is only
  # for convenience if the end-user wants to set `webview.prebuilt.enable = true;`.
  #  webview.prebuilt.apk = config.source.dirs."external/chromium-webview".src + "/prebuilt/${config.arch}/webview.apk";
  #  webview.prebuilt.availableByDefault = mkDefault true;
  #  removedProductPackages = [ "webview" ];

  # Needed by included kernel build for some devices (pioneer at least)
  envPackages = [ pkgs.openssl.dev ] ++ optionals (config.androidVersion == 11) [ pkgs.gcc.cc pkgs.glibc.dev ];

  envVars.RELEASE_TYPE = mkDefault "EXPERIMENTAL"; # Other options are RELEASE NIGHTLY SNAPSHOT EXPERIMENTAL

  # LineageOS flattens all APEX packages: https://review.lineageos.org/c/LineageOS/android_vendor_lineage/+/270212
  signing.apex.enable = false;
  # This environment variable is set in android/build.sh under https://github.com/lineageos-infra/build-config
  envVars.OVERRIDE_TARGET_FLATTEN_APEX = "true";

  # LineageOS needs this additional command line argument to enable
  # backuptool.sh, which runs scripts under /system/addons.d
  otaArgs = [ "--backup=true" ];
}
19:23:10

There are no newer messages yet.


Back to Room ListRoom Version: 6