Ubuntu, CUDA, llama.cpp , nvcc versioning

A user shares their experience resolving CUDA toolkit versioning issues on Ubuntu to enable compute capabilities for newer GPUs like the Blackwell architecture and RTX 5060 Ti. The post highlights that the default apt repository provides outdated CUDA versions, necessitating manual installation of the Debian package from NVIDIA's website.

• The system was upgraded from CUDA 12.0 to version 13.3 to support newer hardware.
• Users must specify the correct path when rebuilding llama.cpp after installing the new toolkit.
• Open-source drivers are recommended over proprietary ones for compute tasks rather than gaming.
• Multiple GPUs of different generations can be used together, though they may run at lower compute capabilities if not configured correctly.

Installing the latest CUDA toolkit from NVIDIA's official site allows users to utilize full compute power on modern graphics cards with local LLM inference tools.