❯ ag eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee result-lib/ --search-binary
result-lib/lib/security/pam_slurm_adopt.la
41:libdir='/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-slurm-23.11.1.1/lib/security'

result-lib/lib/perl5/5.38.2/x86_64-linux-thread-multi/perllocal.pod
7:C<installed into: /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-slurm-23.11.1.1/lib/perl5/site_perl/5.38.2>
29:C<installed into: /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-slurm-23.11.1.1/lib/perl5/site_perl/5.38.2>

Binary file result-lib/lib/libslurm.so.40.0.0 matches.

result-lib/lib/security/pam_slurm.la
41:libdir='/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-slurm-23.11.1.1/lib/security'

Binary file result-lib/lib/slurm/libslurmfull.so matches.

Binary file result-lib/lib/slurm/mpi_pmi2.so matches.

Binary file result-lib/lib/slurm/libslurm_pmi.so matches
❯ strings result-lib/lib/slurm/mpi_pmi2.so | rg eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee
/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-slurm-23.11.1.1/bin/srun

arghhghhhghghghghghggh why

markuskowa

Linking slurm's libpmi2 seems to kind of work at aalto:

❯ ssh triton srun -N3 --mpi=pmi2 singularity exec cpi.sif cpi
srun: job 27525153 queued and waiting for resources
srun: job 27525153 has been allocated resources
...
--------------------------------------------------------------------------
WARNING: There was an error initializing an OpenFabrics device.

  Local host:   csl13
  Local device: mlx5_0
--------------------------------------------------------------------------
--------------------------------------------------------------------------
WARNING: There was an error initializing an OpenFabrics device.

  Local host:   csl2
  Local device: mlx5_0
--------------------------------------------------------------------------
--------------------------------------------------------------------------
WARNING: There was an error initializing an OpenFabrics device.

  Local host:   csl11
  Local device: mlx5_0
--------------------------------------------------------------------------
Process 1 of 3 is on csl11.int.triton.aalto.fi
Process 2 of 3 is on csl13.int.triton.aalto.fi
Process 0 of 3 is on csl2.int.triton.aalto.fi
pi is approximately 3.1415926544231318, Error is 0.0000000008333387
wall clock time = 0.043120