!coeAONBrWyDJnYMbMi:nixos.org

NixOS System Operations

546 Members
About system administration for running NixOS systems in production. Declaratively manage your operations. | Room recommendations: #networking:nixos.org146 Servers

Load older messages


SenderMessageTime
9 Dec 2024
@adam:robins.wtf@adam:robins.wtfand their docs are hyperlinked00:26:52
@adam:robins.wtf@adam:robins.wtf like clicking on <string> takes you to https://prometheus.io/docs/prometheus/latest/configuration/configuration/#string 00:27:11
@hexa:lossy.networkhexaimage.png
Download image.png
00:27:20
@hexa:lossy.networkhexaimage.png
Download image.png
00:27:39
@hexa:lossy.networkhexathey don't ask for feedback 🤔00:27:47
@adam:robins.wtf@adam:robins.wtfnope00:29:00
@adam:robins.wtf@adam:robins.wtfi clicked no hoping to provide feedback00:30:33
@adam:robins.wtf@adam:robins.wtf

loki clearly has a schema, see!

Dec 08 21:52:50 stink2 systemd[1]: Started Loki Service Daemon.
Dec 08 21:52:50 stink2 loki[26054]: panic: runtime error: index out of range [0] with length 0
Dec 08 21:52:50 stink2 loki[26054]: goroutine 1 [running]:
Dec 08 21:52:50 stink2 loki[26054]: github.com/grafana/loki/v3/pkg/loki.validateSchemaRequirements(0xc000d6c008)
Dec 08 21:52:50 stink2 loki[26054]:         github.com/grafana/loki/v3/pkg/loki/validation.go:32 +0x712
02:55:34
@hexa:lossy.networkhexamaaaaaan02:56:06
@adam:robins.wtf@adam:robins.wtfhelpful error message about what's wrong with the config? nah just panic02:56:10
@hexa:lossy.networkhexa you seem to be holding it wrong02:56:11
@hexa:lossy.networkhexa * you seem to be holding it wrong02:56:17
@adam:robins.wtf@adam:robins.wtfclearly02:56:18
@hexa:lossy.networkhexathe module has config validation baked in02:56:27
@adam:robins.wtf@adam:robins.wtfwell it's passing that then02:56:36
@hexa:lossy.networkhexalmao02:56:39
@hexa:lossy.networkhexaso same shit as rules02:56:42
@hexa:lossy.networkhexagot it02:56:45
@adam:robins.wtf@adam:robins.wtf i was missing the required schema_config.configs apparent 03:02:13
@adam:robins.wtf@adam:robins.wtf * i was missing the required schema_config.configs apparently 03:02:16
@hexa:lossy.networkhexaclearly03:06:09
@adam:robins.wtf@adam:robins.wtfi don't understand their config examples. they're missing required fields??03:18:05
@hexa:lossy.networkhexayou think?03:18:29
@hexa:lossy.networkhexa
  services.loki = {
    enable = true;
    configuration = {
      auth_enabled = false;
      server = {
        http_listen_port = 3100;
      };
      compactor = {
        delete_request_store = "filesystem";
        retention_enabled = true;
        working_directory = "${config.services.loki.dataDir}/compactor";
      };
      ingester = {
        lifecycler = {
          address = "127.0.0.1";
          ring = {
            kvstore.store = "inmemory";
            replication_factor = 1;
          };
          final_sleep = "0s";
        };
        chunk_idle_period = "5m";
        chunk_retain_period = "30s";
      };
      ruler = {
        storage = {
          type = "local";
          local.directory = "/etc/loki/rules/";
        };
        rule_path = "/tmp/loki-rules-scratch";
        alertmanager_url = "http://127.0.0.1:9093";
        ring.kvstore.store = "inmemory";
      };
      schema_config.configs = [ {
        from = "2020-07-01";
        store = "boltdb";
        object_store = "filesystem";
        schema = "v11";
        index = {
          prefix = "index_";
          period = "168h";
        };
      } {
        from = "2024-05-25";
        store = "tsdb";
        object_store = "filesystem";
        schema = "v13";
        index = {
          prefix = "index_";
          period = "24h";
        };
      } ];
      storage_config = {
        boltdb.directory = "${config.services.loki.dataDir}/index";
        tsdb_shipper = {
          active_index_directory = "${config.services.loki.dataDir}/data/2024-05-25/index";
          cache_location = "${config.services.loki.dataDir}/data/2024-05-25/cache";
          cache_ttl = "24h";
        };
        filesystem.directory = "${config.services.loki.dataDir}/chunks";
      };
      limits_config = {
        reject_old_samples = true;
        reject_old_samples_max_age = "168h";
      };
      table_manager = {
        retention_deletes_enabled = false;
        retention_period = "0s";
      };
    };
  };
03:19:11
@adam:robins.wtf@adam:robins.wtf

oh, the nix module

                 # Config validation may fail when using extraFlags = [ "-config.expand-env=true" ].
                 # To work around this, we simply skip it when extraFlags is not empty.
03:31:29
@adam:robins.wtf@adam:robins.wtfi have that enabled for injecting s3 secrets03:31:43
@adam:robins.wtf@adam:robins.wtf
    services.loki = {
      enable = true;
      extraFlags = [ "--config.expand-env=true" ];

      configuration = {
        auth_enabled = false;

        common = {
          instance_interface_names = [ "grid" ];
          ring.kvstore = {
            store = "consul";
            prefix = "loki/collectors/";
          };
        };

        compactor.working_directory = "${config.services.loki.dataDir}/compactor";

        schema_config.configs = [
          {
            from = "2024-01-01";
            store = "tsdb";
            object_store = "s3";
            schema = "v13";
            index = {
              prefix = "index_";
              period = "24h";
            };
          }
        ];

        storage_config = {
          aws = {
            bucketnames = "loki";
            endpoint = site.services.s3;
            region = "garage";
            secret_access_key = "\${S3_SECRET_ACCESS_KEY}";
            access_key_id = "\${S3_ACCESS_KEY_ID}";
            s3forcepathstyle = true;
          };
          tsdb_shipper = {
            active_index_directory = "${config.services.loki.dataDir}/index";
            cache_location = "${config.services.loki.dataDir}/index_cache";
          };
        };

        server = {
          http_listen_address = "127.0.0.1";
          http_listen_port = 12010;
          grpc_listen_address = config.stop.roles.grid.ip.internal.ipv4;
          grpc_listen_port = 12011;
        };
      };

04:29:52
@adam:robins.wtf@adam:robins.wtfin theory that works, but i have nothing to ship logs yet 😆04:35:19
@karlericsson:matrix.org@karlericsson:matrix.org joined the room.07:50:00
@philon:philon.tubephilon ⚡️ joined the room.09:25:45

Show newer messages


Back to Room ListRoom Version: 10