"No CUDA-capable device is detected" — how to fix it

My GPU is right there and CUDA says there is no device

作者 Jakub Rusinowski

错误信息

RuntimeError: No CUDA-capable device is detected

1. 1

   Confirm the driver sees the GPU

   Start at the bottom of the stack. nvidia-smi talks directly to the driver — if it lists your card, the driver is fine and the problem is higher up (framework/toolkit). If nvidia-smi itself fails or shows nothing, install or update the NVIDIA driver first; nothing above it can work until this does.

   bash复制

   nvidia-smi

   生效了吗？nvidia-smi prints a table naming your GPU, with a driver version and a CUDA Version in its header. That CUDA figure is the newest runtime the driver supports, not a version that is installed.
2. 2

   Install a CUDA-enabled build of your framework

   A plain pip install torch can pull a CPU-only wheel that will never see the GPU. Reinstall the CUDA build that matches your driver. After installing, verify from Python that CUDA is now available.

   bash复制

   # example: CUDA 12.8 build of PyTorch — take the exact line from the selector on pytorch.org
   pip install torch --index-url https://download.pytorch.org/whl/cu128
   
   python -c "import torch; print(torch.cuda.is_available())"

   生效了吗？torch.cuda.is_available() prints True, and torch.version.cuda prints a version no newer than the CUDA Version in the nvidia-smi header. If it is newer, install an older wheel from pytorch.org or update the driver.

   multiple shells — see comments复制

   # bash, zsh or PowerShell — the same command on all of them
   python -c "import torch; print(torch.cuda.is_available(), torch.version.cuda)"

3. 3

   Check CUDA_VISIBLE_DEVICES

   If this variable is set to an empty string or a non-existent index, CUDA hides every GPU. Unset it (or set it to a valid index) and retry.

   bash复制

   echo $CUDA_VISIBLE_DEVICES   # if empty-string or wrong, fix it:
   unset CUDA_VISIBLE_DEVICES

   生效了吗？After unsetting it, torch.cuda.device_count() matches the number of GPUs nvidia-smi lists. On Windows, clearing it in PowerShell lasts for that session only; if it comes back, it is set in your user or system environment variables.

   multiple shells — see comments复制

   # bash
   echo "$CUDA_VISIBLE_DEVICES"
   unset CUDA_VISIBLE_DEVICES
   
   # PowerShell
   $env:CUDA_VISIBLE_DEVICES
   Remove-Item Env:CUDA_VISIBLE_DEVICES

4. 4

   In Docker, pass the GPU through

   Containers don't get the GPU by default. You need the NVIDIA Container Toolkit installed on the host and the --gpus all flag on docker run, otherwise the container sees no CUDA device.

   bash复制

   docker run --gpus all <image>

   生效了吗？docker run --rm --gpus all ubuntu nvidia-smi prints the same GPU table as the host. If it errors instead, the NVIDIA Container Toolkit is missing or not registered with Docker.
5. 5

   Once CUDA works, make sure your model actually fits最常见的修复方法

   When the GPU is finally visible, the next wall people hit is memory — the model being too big for the card. That part is worth getting right up front: check which model and quant fit your VRAM so you don't trade a visibility error for an out-of-memory one.

   生效了吗？With the GPU visible, the model loads fully onto it: in Ollama, the Processor column of ollama ps reads 100% GPU.

   检查哪些模型适配你的硬件once your GPU is detected, see which models fit its VRAM打开显存检查器 →