The NVIDIA CUDA Toolkit is a powerful platform designed to perform parallel computing tasks using NVIDIA GPUs. It enables developers to accelerate the development and use of modern ML/AI applications by leveraging the power of GPU computing. In this guide, we will walk you through the process of installing the NVIDIA CUDA Toolkit on a Ubuntu 22.04 Cloud GPU server.

Prerequisites

Before we begin, make sure you have the following prerequisites in place:

  1. Deploy a Ubuntu 22.04 Cloud GPU Server on Shape.host with at least 1/12 NVIDIA GPU and 180GB Storage.
  2. Access the server using SSH.
  3. Create a non-root sudo user account and switch to the new user account. In this guide, we will use the example username shapehost. Replace it with your actual username.

Native Installation

The recommended method to install the NVIDIA CUDA Toolkit is through a native installation using the distribution-dependent release file or an installation script. This method ensures that the CUDA Toolkit is installed without overwriting any NVIDIA driver files. Here’s how you can do it:

  1. Download the latest CUDA Toolkit run file from the NVIDIA website. You can use the following command to download the CUDA Toolkit version 12.0.1:
$ wget https://developer.download.nvidia.com/compute/cuda/12.0.1/local_installers/cuda_12.0.1_525.85.12_linux.run
  1. Execute the installation .run file using the following command:
$ sudo sh cuda_12.0.1_525.85.12_linux.run
  1. Read through the End User License Agreement (EULA) and enter accept when prompted to proceed with the installation.
  2. Deselect the Drivers option in the Installation options list, as the drivers are already available on the Shape.host Cloud GPU servers.
  3. Press Enter to start the installation process.
  4. Once the installation is successful, add the CUDA path to the system PATH and the CUDA Toolkit library path to the LDLIBRARYPATH by running the following commands:
$ echo "export PATH=/usr/local/cuda-12.0/bin${PATH:+:${PATH}}" >> /home/shapehost/.bashrc
$ ech o"export LD_LIBRARY_PATH=/usr/local/cuda-12.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}" >> /home/shapehost/.bashrc
$ source /home/shapehost/.bashrc

Installation using Conda

If you prefer to use Conda for package management, you can install the NVIDIA CUDA Toolkit using Conda with preinstalled Anaconda or Miniconda. Here’s how you can do it:

  1. Make sure you have either Anaconda or Miniconda installed on your server. If not, install the desired version before proceeding.
  2. Activate your Conda virtual environment. For example, if your virtual environment is named env1, use the following command:
$ conda activate env1
  1. Install the CUDA toolkit using the following command:
$ conda install -c "nvidia/label/cuda-12.0.1" cuda -y
  1. After the installation, verify that the CUDA Toolkit is available in your local Conda packages by running the following command:
$ conda list | grep "cuda-toolkit"

Verifying the Installation

Once the NVIDIA CUDA Toolkit is installed on your Ubuntu 22.04 Cloud GPU server, it’s important to verify the installation to ensure everything is working correctly. Here are some verification steps you can perform:

  1. Verify that the NVIDIA GPU drivers are working correctly by running the following command:
$ nvidia-smi
  1. Check the version of the NVIDIA CUDA Compiler (NVCC) package installed with the CUDA Toolkit by running the following command:
$ nvcc --version
  1. Clone the CUDA Samples Git repository, which contains test programs, using the following command:
$ git clone https://github.com/NVIDIA/cuda-samples.git
  1. Switch to the deviceQuery sample program directory:
$ cd cuda-samples/Samples/1_Utilities/deviceQuery
  1. Compile the sample program:
$ make
  1. Run the compiled program:
$ ./deviceQuery

If the CUDA program runs successfully, you should see output similar to the following:

CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 1 CUDA Capable device(s)
Device 0: "NVIDIA A40-1Q"
CUDA Driver Version / Runtime Version 12.0 / 12.0
CUDA Capability Major/Minor version number: 8.6
...
Result = PASS

Congratulations! You have successfully installed the NVIDIA CUDA Toolkit on your Ubuntu 22.04 Cloud GPU server and verified its functionality.

System Compatibility and Requirements

When installing the NVIDIA CUDA Toolkit, it’s important to ensure system compatibility and meet the necessary requirements. Here are the specifications for the CUDA Toolkit version 12.0.1 installed in this guide:

  • GCC Version: 11.3.0
  • NVIDIA graphics driver version: 525.125.06
  • CUDA Driver API Version: 12.0
  • Linux kernel version: 5.15.0-75

Please note that these specifications may vary depending on the CUDA Toolkit version you intend to install. Make sure to verify the system compatibility values specific to your target version.

To verify the compatibility of your system, you can perform the following checks:

  • View the Linux system properties and check for the attached NVIDIA graphics card using the lspci command.
  • Check the installed GCC version using the gcc --version command.
  • Verify the installed NVIDIA GPU drivers using the nvidia-smi command.
  • View the system kernel version using the uname -r command.
  • Check the available Kernel headers using the apt list linux-headers-$(uname -r) command.

By following these steps, you can ensure that your system meets the necessary requirements for the NVIDIA CUDA Toolkit installation.

Conclusion

In this guide, we have walked you through the process of installing the NVIDIA CUDA Toolkit on a Ubuntu 22.04 Cloud GPU server. Whether you choose the native installation method or prefer to use Conda, you can now leverage the power of GPU computing for your machine learning and AI applications. Remember to verify the installation and ensure system compatibility to ensure a smooth development experience.

If you’re looking for reliable and scalable cloud hosting solutions, consider Shape.host. With their Cloud VPS services, you can deploy and manage your Ubuntu 22.04 Cloud GPU server effortlessly. Shape.host provides top-notch support and a secure environment for your GPU-powered applications.