Seed set to 42
Using bfloat16 Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nix/store/h3wgj19ywjsd5ha976k8kajmg1jz7lpw-python3.11-torch-2.3.0/lib/python3.11/site-packages/torch/cuda/__init__.py:184: UserWarning: 
    Found GPU0 NVIDIA GeForce RTX 3080 which is of cuda capability 8.6.
    PyTorch no longer supports this GPU because it is too old.
    The minimum cuda capability supported by this library is 8.9.
    
  warnings.warn(
Files already downloaded and verified
Files already downloaded and verified
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

  | Name      | Type             | Params
-----------------------------------------------
0 | criterion | CrossEntropyLoss | 0     
1 | model     | ViT              | 86.3 M
-----------------------------------------------
86.3 M    Trainable params
0         Non-trainable params
86.3 M    Total params
345.317   Total estimated model params size (MB)
Sanity Checking: |                                                                                                                                                                                                                                                                                    | 0/? [00:00<?, ?it/s]
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
.nix-cuda-test-wrapped 9 <module>
sys.exit(main())

__main__.py 128 main
trainer.fit(

trainer.py 544 fit
call._call_and_handle_interrupt(

call.py 44 _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)

trainer.py 580 _fit_impl
self._run(model, ckpt_path=ckpt_path)

trainer.py 987 _run
results = self._run_stage()

trainer.py 1031 _run_stage
self._run_sanity_check()

trainer.py 1060 _run_sanity_check
val_loop.run()

utilities.py 182 _decorator
return loop_run(self, *args, **kwargs)

evaluation_loop.py 110 run
self.setup_data()

evaluation_loop.py 192 setup_data
length = len(dl) if has_len_all_ranks(dl, trainer.strategy, allow_zero_length) else float("inf")

data.py 103 has_len_all_ranks
if total_length == 0:

RuntimeError:
CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

In reply to @aidalgol:matrix.org

Seed set to 42
Using bfloat16 Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
/nix/store/h3wgj19ywjsd5ha976k8kajmg1jz7lpw-python3.11-torch-2.3.0/lib/python3.11/site-packages/torch/cuda/__init__.py:184: UserWarning: 
    Found GPU0 NVIDIA GeForce RTX 3080 which is of cuda capability 8.6.
    PyTorch no longer supports this GPU because it is too old.
    The minimum cuda capability supported by this library is 8.9.
    
  warnings.warn(
Files already downloaded and verified
Files already downloaded and verified
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

  | Name      | Type             | Params
-----------------------------------------------
0 | criterion | CrossEntropyLoss | 0     
1 | model     | ViT              | 86.3 M
-----------------------------------------------
86.3 M    Trainable params
0         Non-trainable params
86.3 M    Total params
345.317   Total estimated model params size (MB)
Sanity Checking: |                                                                                                                                                                                                                                                                                    | 0/? [00:00<?, ?it/s]
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
.nix-cuda-test-wrapped 9 <module>
sys.exit(main())

__main__.py 128 main
trainer.fit(

trainer.py 544 fit
call._call_and_handle_interrupt(

call.py 44 _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)

trainer.py 580 _fit_impl
self._run(model, ckpt_path=ckpt_path)

trainer.py 987 _run
results = self._run_stage()

trainer.py 1031 _run_stage
self._run_sanity_check()

trainer.py 1060 _run_sanity_check
val_loop.run()

utilities.py 182 _decorator
return loop_run(self, *args, **kwargs)

evaluation_loop.py 110 run
self.setup_data()

evaluation_loop.py 192 setup_data
length = len(dl) if has_len_all_ranks(dl, trainer.strategy, allow_zero_length) else float("inf")

data.py 103 has_len_all_ranks
if total_length == 0:

RuntimeError:
CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

In reply to @aidalgol:matrix.org
An RTX 3080 is too old?!

In reply to @aidalgol:matrix.org
I just cloned https://github.com/ConnorBaker/nix-cuda-test.git, applied the fix described earlier, then ran nix run .#nix-cuda-test

connor (he/him) (UTC-5): Samuel Ainsworth: Madoura:

1 I'd like us to add a more generic alias to this room that would encompass a wider range of topics, rocm at the least. The reasoning is the same: i don't think we need a special room for cuda... Madoura: what do rocm-maintainers think about their presence on matrix?
2 I notice we never brought up the subject of onboarding new people anywhere in public, not even discourse. Wdyt should be done about that?

In reply to @trexd:matrix.org
Could call the room nixos-gpu since that covers rocm too.

To tackle diamond dependencies (among other things), I started making https://github.com/ConnorBaker/dep-tracker

Specify a flake attribute for a package and it’ll grab a copy of all the attributes on the package containing a list of dependencies (the attributes it looks for are here https://github.com/ConnorBaker/dep-tracker/blob/cd8e927c561f3f1ed5c904609654c946d85cf954/packages/dep-tracker/dep_tracker/types.py#L15). It’ll look through those arrays and populate a SQLite database with libraries it finds in those dependencies.

Now, a question: besides recursing and doing the same for every dependency I find (that is, harvesting attributes and updating the database), is there an easier way to get the closure of dependencies without building the package? IIRC nix path-info requires the package be built.

A different question: with that hardcoded list of attributes I inspect, is it possible I’d miss dependencies (and therefore libraries) which are present in the closure?

@someoneserge you have good ideas about finding dependencies — any suggestions? Currently finding what a dependency provides is limited to listing the names of libraries present under lib (https://github.com/ConnorBaker/dep-tracker/blob/cd8e927c561f3f1ed5c904609654c946d85cf954/packages/dep-tracker/dep_tracker/deps.py#L42) and finding out what a library needs is accomplished through patchelf (https://github.com/ConnorBaker/dep-tracker/blob/cd8e927c561f3f1ed5c904609654c946d85cf954/packages/dep-tracker/dep_tracker/deps.py#L56)

IIRC nix path-info requires the package be built.

For deciding which dependencies to retain a runtime reference to

In reply to @tpw_rules:matrix.org
is there some reason we've got an out of date tensorflow build from source?

In reply to @glepage:matrix.org
Because a >200 IQ is necessary to grasp this derivation 😅

In reply to @glepage:matrix.org
Now, if you want to take this challenge and update tensorflow, please go ahead.
For context, there is a stale PR for updating tensorflow to 2.14: https://github.com/NixOS/nixpkgs/pull/272838

In reply to @connorbaker:matrix.org
Maybe it forced the driver to reload lol

In reply to @ss:someonex.net
Have you seen https://fzakaria.com/2023/09/11/quick-insights-using-sqlelf.html?