| 1 Oct 2023 |
 hexa | where did you find a darwin metric? | 14:46:51 |
 Lily Foster | Actually the metrics are only sent from the central queue manager. So yeah they aren't sending anything | 14:47:18 |
| Lily Foster | (for darwin build info) | 14:47:25 |
| Lily Foster | You're right | 14:47:29 |
|  pbsds joined the room. | 14:49:17 |
 cole-h | In reply to @cafkafk:gitter.im
disk full https://github.com/NixOS/nixpkgs/pull/258395/checks?check_run_id=17292775692 Should be fixed now. | 14:56:40 |
| Lily Foster | Any chance we could run the prometheus agent on darwin cole-h? 🥺 | 14:57:27 |
| cole-h | We do, I guess they're just not scraped | 14:58:43 |
| cole-h | curl -ss http://...:9100/metrics | head
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 3.9458e-05
go_gc_duration_seconds{quantile="0.25"} 5.4583e-05
go_gc_duration_seconds{quantile="0.5"} 6.6707e-05
go_gc_duration_seconds{quantile="0.75"} 9.1292e-05
go_gc_duration_seconds{quantile="1"} 0.000423
go_gc_duration_seconds_sum 251.971410887
go_gc_duration_seconds_count 115795
# HELP go_goroutines Number of goroutines that currently exist.
| 15:00:39 |
| cole-h | OK, they're scraped now. | 15:14:17 |
| hexa | the node_filesystem_free metric that is used in the alerting rule does not exist | 15:31:28 |
| hexa | 
Download image.png | 15:31:52 |
| hexa | this one | 15:31:53 |
| cole-h | 🔥🐶🔥 | 15:32:09 |
| hexa | should probably use avail_bytes 😄 | 15:32:17 |
| hexa | or free_bytes | 15:32:34 |
| hexa | heck, what is the difference? | 15:32:40 |
| cole-h | compression or something maybe? those 2 queries differ on the linux boxes but not the darwin ones | 15:33:21 |
| cole-h | wait nvm it does differ on both but the difference is larger on linux | 15:33:48 |
| cole-h | https://github.com/prometheus/node_exporter/issues/269 🤷 | 15:34:28 |
| cole-h | hexa: I'm officially tasking you: tell me which one I should use, free or avail? 😆 | 15:34:50 |
| hexa | cool. | 15:35:00 |
| hexa | reserved space for root | 15:35:03 |
| Lily Foster | In reply to @hexa:lossy.network
the node_filesystem_free metric that is used in the alerting rule does not exist Does it not have that metric? | 15:35:24 |
| Lily Foster | 
Download image.png | 15:35:26 |
| hexa | let's err on the safe side and use free, because it doesn't include that reserved space? | 15:35:35 |
| cole-h | In reply to @lily:lily.flowers
sent an image. (note that we're talking about the sans-files one) | 15:35:50 |
| hexa | Lily Foster: space vs inode | 15:35:53 |
| cole-h | (that one is inode) | 15:35:56 |
| cole-h | * (that one is inodes) | 15:36:01 |
