7 Dec 2024 |
hexa | huh, ok | 22:41:45 |
hexa | that is kinda the simplicity I was looking for | 22:42:04 |
hexa | {unit="postgresql.service"}
|~ "ALTER DATABASE \\w+ REFRESH COLLATION VERSION"
| regexp "DATABASE (?<database>[a-zA-Z0-9]+) REFRESH" [5m]
| 22:42:59 |
hexa | that would be one example | 22:43:03 |
hexa | * {unit="postgresql.service"}
|~ "ALTER DATABASE \\w+ REFRESH COLLATION VERSION"
| regexp "DATABASE (?<database>[a-zA-Z0-9]+) REFRESH"
| 22:43:13 |
K900 | Hm I should steal that | 22:43:52 |
hexa | 😛 | 22:44:28 |
hexa | now I wish there was webinterface to browse the state of loki | 22:52:13 |
hexa | not the user facing stuff like grafana, more like what prometheus has | 22:52:22 |
hexa | can't say it is alerting based on that expression | 22:56:05 |
hexa | [93980] WARNING: database "template1" has a collation version mismatch
[93980] DETAIL: The database was created using collation version 2.39, but the operating system provides version 2.40.
[93980] HINT: Rebuild all objects in this database that use the default collation and run ALTER DATABASE template1 REFRESH COLLATION VERSION, or build PostgreSQL with the right library version.
| 23:00:16 |
hexa | rulerConfig = {
groups = [ {
name = "postgresql";
rules = [ {
alert = "PostgresqlCollationMismatch";
expr = ''
{unit="postgresql.service"}
|~ 'WARNING: database "[^"]+" has a collation version mismatch'
| regexp 'database "(?<database>[^"]+)"'
'';
labels.severity = "warning";
annotations.summary = "PostgreSQL database {{ $labels.database }} on {{ $labels.machine }}i reports database collation mismatch";
} ];
} ];
};
| 23:01:48 |
hexa | * rulerConfig = {
groups = [ {
name = "postgresql";
rules = [ {
alert = "PostgresqlCollationMismatch";
expr = ''
{unit="postgresql.service"}
|~ 'WARNING: database "[^"]+" has a collation version mismatch'
| regexp 'database "(?<database>[^"]+)"'
'';
labels.severity = "warning";
annotations.summary = "PostgreSQL database {{ $labels.database }} on {{ $labels.machine }}i reports database collation mismatch";
} ];
} ];
};
| 23:01:52 |
hexa | so for one it doesn't like single quotes 😄 | 23:03:24 |
hexa | and doesn't complain about it | 23:03:28 |
hexa | * rulerConfig = {
groups = [ {
name = "postgresql";
rules = [ {
alert = "PostgresqlCollationMismatch";
expr = ''
{unit="postgresql.service"}
|~ "WARNING: database \"[^\"]+\" has a collation version mismatch"
| regexp "database \"(?<database>[^\"]+)\""
'';
labels.severity = "warning";
annotations.summary = "PostgreSQL database {{ $labels.database }} on {{ $labels.machine }}i reports database collation mismatch";
} ];
} ];
};
| 23:04:17 |
8 Dec 2024 |
adamcstephens | My favorite, when something silently accepts invalid input | 00:22:52 |
hexa | the rules are just not as robust ans simple to test as promql | 00:49:26 |
hexa | # lokitool rules lint ruler.yml
ERRO[0000] unable parse rules file error="could not parse expression for alert 'PostgresqlCollationMismatch' in group 'postgresql': parse error at line 1, col 19: syntax error: unexpected IDENTIFIER" file=ruler.yml
lokitool: error: prepare operation unsuccessful, unable to parse rules files: file read error, try --help
| 00:50:15 |
hexa | I also love how builtins.toJSON puts everything on line 1 | 00:50:36 |
hexa | because that is totally for the benefit of dealing with such an error | 00:50:46 |
magic_rb | Kind of wish there was a way to make toJSON spew out pretty printed json | 00:56:06 |
hexa | the loki module should provide an option to accept rules directly and lint them | 01:22:20 |
hexa | {
"groups": [
{
"name": "postgresql",
"rules": [
{
"alert": "PostgresqlCollationMismatch",
"annotations": {
"summary": "PostgreSQL database on {{ $labels.machine }} reports database collations mismatches"
},
"expr": "sum by (machine) (counter_over_time({unit=\"postgresql.service\"} |~ \"REFRESH COLLATION VERSION\" [5m])) > 0\n",
"for": "1m",
"labels": {
"severity": "warning"
}
}
]
}
]
}
| 01:39:44 |
hexa | ERRO[0000] unable parse rules file error="could not parse expression for alert 'PostgresqlCollationMismatch' in group 'postgresql': parse error at line 1, col 19: syntax error: unexpected IDENTIFIER" file=/tmp/rules.yml
| 01:39:48 |
hexa | really, line 1 again? | 01:39:52 |
hexa | also line 1 when remarshalled to yaml | 01:40:37 |
hexa |
| 01:41:18 |
hexa | * groups:
- name: postgresql
rules:
- alert: PostgresqlCollationMismatch
annotations:
summary: PostgreSQL database on {{ $labels.machine }} reports database collations
mismatches
expr: "sum by (machine) (counter_over_time({unit=\"postgresql.service\"} |~ \"\
REFRESH COLLATION VERSION\" [5m])) > 0\n"
for: 1m
labels:
severity: warning
| 01:41:23 |
hexa | that yaml is also the shit | 01:41:28 |