!coeAONBrWyDJnYMbMi:nixos.org

NixOS System Operations

401 Members
About system administration for running NixOS systems in production. Declaratively manage your operations. | Room recommendations: #networking:nixos.org107 Servers

Load older messages


SenderMessageTime
7 Dec 2024
@hexa:lossy.networkhexahuh, ok22:41:45
@hexa:lossy.networkhexathat is kinda the simplicity I was looking for22:42:04
@hexa:lossy.networkhexa
      {unit="postgresql.service"}
        |~ "ALTER DATABASE \\w+ REFRESH COLLATION VERSION"
        | regexp "DATABASE (?<database>[a-zA-Z0-9]+) REFRESH" [5m]
22:42:59
@hexa:lossy.networkhexathat would be one example22:43:03
@hexa:lossy.networkhexa *
      {unit="postgresql.service"}
        |~ "ALTER DATABASE \\w+ REFRESH COLLATION VERSION"
        | regexp "DATABASE (?<database>[a-zA-Z0-9]+) REFRESH"
22:43:13
@k900:0upti.meK900Hm I should steal that 22:43:52
@hexa:lossy.networkhexa😛22:44:28
@hexa:lossy.networkhexanow I wish there was webinterface to browse the state of loki22:52:13
@hexa:lossy.networkhexanot the user facing stuff like grafana, more like what prometheus has22:52:22
@hexa:lossy.networkhexacan't say it is alerting based on that expression22:56:05
@hexa:lossy.networkhexa
[93980] WARNING:  database "template1" has a collation version mismatch
[93980] DETAIL:  The database was created using collation version 2.39, but the operating system provides version 2.40.
[93980] HINT:  Rebuild all objects in this database that use the default collation and run ALTER DATABASE template1 REFRESH COLLATION VERSION, or build PostgreSQL with the right library version.
23:00:16
@hexa:lossy.networkhexa
  rulerConfig = {
    groups = [ {
      name = "postgresql";
      rules = [ {
        alert = "PostgresqlCollationMismatch";
        expr = ''
          {unit="postgresql.service"}
            |~ 'WARNING:  database "[^"]+" has a collation version mismatch'
            | regexp 'database "(?<database>[^"]+)"'
        '';
        labels.severity = "warning";
        annotations.summary = "PostgreSQL database {{ $labels.database }} on {{ $labels.machine }}i reports database collation mismatch";
      } ];
    } ];
  };
23:01:48
@hexa:lossy.networkhexa *
  rulerConfig = {
    groups = [ {
      name = "postgresql";
      rules = [ {
        alert = "PostgresqlCollationMismatch";
        expr = ''
          {unit="postgresql.service"}
            |~ 'WARNING:  database "[^"]+" has a collation version mismatch'
            | regexp 'database "(?<database>[^"]+)"'
        '';
        labels.severity = "warning";
        annotations.summary = "PostgreSQL database {{ $labels.database }} on {{ $labels.machine }}i reports database collation mismatch";
      } ];
    } ];
  };
23:01:52
@hexa:lossy.networkhexaso for one it doesn't like single quotes 😄 23:03:24
@hexa:lossy.networkhexaand doesn't complain about it23:03:28
@hexa:lossy.networkhexa *
  rulerConfig = {
    groups = [ {
      name = "postgresql";
      rules = [ {
        alert = "PostgresqlCollationMismatch";
        expr = ''
          {unit="postgresql.service"}
            |~ "WARNING:  database \"[^\"]+\" has a collation version mismatch"
            | regexp "database \"(?<database>[^\"]+)\""
        '';
        labels.severity = "warning";
        annotations.summary = "PostgreSQL database {{ $labels.database }} on {{ $labels.machine }}i reports database collation mismatch";
      } ];
    } ];
  };
23:04:17
8 Dec 2024
@adam:robins.wtfadamcstephens My favorite, when something silently accepts invalid input 00:22:52
@hexa:lossy.networkhexathe rules are just not as robust ans simple to test as promql00:49:26
@hexa:lossy.networkhexa
# lokitool rules lint ruler.yml 
ERRO[0000] unable parse rules file                       error="could not parse expression for alert 'PostgresqlCollationMismatch' in group 'postgresql': parse error at line 1, col 19: syntax error: unexpected IDENTIFIER" file=ruler.yml
lokitool: error: prepare operation unsuccessful, unable to parse rules files: file read error, try --help
00:50:15
@hexa:lossy.networkhexaI also love how builtins.toJSON puts everything on line 100:50:36
@hexa:lossy.networkhexabecause that is totally for the benefit of dealing with such an error00:50:46
@magic_rb:matrix.redalder.orgmagic_rbKind of wish there was a way to make toJSON spew out pretty printed json00:56:06
@hexa:lossy.networkhexathe loki module should provide an option to accept rules directly and lint them01:22:20
@hexa:lossy.networkhexa
{
  "groups": [
    {
      "name": "postgresql",
      "rules": [
        {
          "alert": "PostgresqlCollationMismatch",
          "annotations": {
            "summary": "PostgreSQL database on {{ $labels.machine }} reports database collations mismatches"
          },
          "expr": "sum by (machine) (counter_over_time({unit=\"postgresql.service\"} |~ \"REFRESH COLLATION VERSION\" [5m])) > 0\n",
          "for": "1m",
          "labels": {
            "severity": "warning"
          }
        }
      ]
    }
  ]
}
01:39:44
@hexa:lossy.networkhexa
ERRO[0000] unable parse rules file                       error="could not parse expression for alert 'PostgresqlCollationMismatch' in group 'postgresql': parse error at line 1, col 19: syntax error: unexpected IDENTIFIER" file=/tmp/rules.yml
01:39:48
@hexa:lossy.networkhexareally, line 1 again?01:39:52
@hexa:lossy.networkhexaalso line 1 when remarshalled to yaml01:40:37
@hexa:lossy.networkhexa
01:41:18
@hexa:lossy.networkhexa *
groups:
- name: postgresql
  rules:
  - alert: PostgresqlCollationMismatch
    annotations:
      summary: PostgreSQL database on {{ $labels.machine }} reports database collations
        mismatches
    expr: "sum by (machine) (counter_over_time({unit=\"postgresql.service\"} |~ \"\
      REFRESH COLLATION VERSION\" [5m])) > 0\n"
    for: 1m
    labels:
      severity: warning
01:41:23
@hexa:lossy.networkhexathat yaml is also the shit01:41:28

Show newer messages


Back to Room ListRoom Version: 10