i.e. idiomatic would be:

SUBSYSTEM=="net", ACTION=="add", ENV{ID_NET_DRIVER}=="vif|ena|ixgbevf", SYSTEMD_WANTS="policy-routes@.service refresh-policy-routes@.timer"

And then add

[Unit]
Description=Set up policy routes for %I
BindsTo=%i.device
After=%i.device

The script would then get the full sysfs path as an argument (e.g. /sys/devices/pci0000:00/0000:00:05.0/net/ens5) instead of just the device name so the scripts need to be adjusted slightly

In reply to @commiterate:matrix.org

CloudWatch Agent is finally in Nixpkgs: https://github.com/NixOS/nixpkgs/pull/337212

Tracker: https://nixpk.gs/pr-tracker.html?pr=337212

Arian Any concerns with this Fluent Bit module before I try upstreaming it?

https://github.com/commiterate/nix-fluent-bit

Probably going to use it despite the CW Agent work due to the native systemd-journald support and better processing features. That and I'm a bit hesitant now that I've seen the spaghetti under the hood.

Arian Any concerns with this Fluent Bit module before I try adding it to Nixpkgs?

https://github.com/commiterate/nix-fluent-bit

Probably going to use it despite the CW Agent work due to the native systemd-journald support and better processing features. That and I'm a bit hesitant now that I've seen the spaghetti under the hood.

fyi, working on a handful of changes related to AWS and ML:

1. Adding the efa kernel module: https://github.com/NixOS/nixpkgs/pull/360347

2. Adding efa-nv-peermem: https://github.com/NixOS/nixpkgs/pull/360375

3. Adding an updateScript for the out-of-tree ena build and a package bump: https://github.com/NixOS/nixpkgs/pull/360326

I expect a few others before the weekend is over:

• modifying the libfabric drv to support building with efa and HMEM_CUDA

• adding and building libnccl-ofi, plus extending nccl so that it uses it

with all of these in place (minus the ENA part which is independent) it should be possible to support multinode ML training on aws with nixos.

Arian: any ideas on how to expose this in a module and enable it?

EFA supported instances types are here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa.html#efa-instance-types

efa-nv-peermem and the nccl/libfabric stuff is only really needed on p3/p4.*/p5.*/p5e.*/p5en.*