GPU Allocation

To access graphics processing units (GPUs) from Industrial Edge applications, prepare the host system accordingly. This includes installing vendor-specific device drivers and low-level libraries. The following sections describe the required steps for Nvidia GPUs. Support for additional GPUs may be added later.

Nvidia GPUs

First, unpack resource-enablement-nvidia-gpu.tar.gz and install the included Resource Manager plug-in. Unlike the CPU and network isolation plug-ins, the Nvidia GPU plug-in does not provide configuration options.

If the target device operating system does not already include Nvidia GPU drivers, install them manually. Three driver variants exist:

• Closed-source drivers distributed as binaries under a proprietary license by Nvidia.
  Most Linux distributions provide ready-to-use packages. On Debian, for example, the drivers are contained in the package nvidia-driver.
• Open-source drivers published by Nvidia under MIT/GPLv2 license.
  These drivers are still relatively new and not yet part of all Linux distributions.
• Open-source nouveau drivers developed by independent software engineers, partly by reverse-engineering the closed-source drivers.
  These drivers are mainly used for graphics and do not support Nvidia CUDA, which is used, for example, by AI applications to execute neural networks. For this reason, they are not supported by Industrial Edge's Resource Manager.

We recommend Nvidia's open-source drivers. They support CUDA and avoid the licensing restrictions of the proprietary drivers. These restrictions exist because the Linux kernel uses GPLv2. Derived work must also be licensed under GPLv2, which conflicts with the proprietary license of the closed-source drivers.

If the open-source drivers are not shipped with your Linux distribution, download them from GitHub. To compile and install the drivers, follow the instructions in the README file.

NOTICE

Nvidia does not officially support Linux kernels with PREEMPT-RT patch. However, you can override related checks by setting the environment variable IGNORE_PREEMPT_RT_PRESENCE to 1 before compiling the drivers.

To avoid conflicts, make sure that the nouveau driver is disabled. On Debian, add the following lines to /etc/modprobe.d/blacklist-nouveau.conf:

blacklist nouveau
options nouveau modeset=0

After that, regenerate the kernel's initial RAM file system (sudo update-initramfs -u) and reboot the system.

Finally, install CUDA. For a full installation, download a package that matches your system from the Nvidia website and follow the installation instructions. This installs several tools and libraries, including graphics support.

NOTICE

The CUDA package must match the driver version. For example, if you installed version 535.104.05 of the drivers (on a 64-bit x86 Debian 11 system), install cuda-repo-debian11-12-2-local_12.2.2-535.104.05-1_amd64.deb.

Alternatively, install only the components that users of your device actually need. The following table shows a minimal Debian installation for compute applications:

Package Name	Description
libcuda1	Runtime support for CUDA applications
libnvidia-ptxjitcompiler1	JIT compiler generating GPU machine code
libnvidia-nvvm4	Library for JIT link-time optimization
libnvidia-ml1	Library providing a monitoring and management API
libnvidia-allocator1	Library used internally by other driver components
nvidia-smi	NVIDIA System Management Interface

These packages can also be obtained from Nvidia.

NOTICE

As for the closed-source drivers, redistributing CUDA or components thereof may be subject to licensing restrictions (see above).

Carefully check any legal implications and mitigations.

The Nvidia GPU plug-in expects these components in the following directories, which matches the Debian package layout:

Components	Directory	Minimally required set of files
Kernel modules	/lib/modules/<kernel-release>/kernel/drivers/video	nvidia.ko, nvidia-modeset.ko, nvidia-uvm.ko, nvidia-drm.ko
Tools	/usr/lib/nvidia/current	nvidia-smi
Libraries	/usr/lib/x86_64-linux-gnu/nvidia/current	libcuda.so*, libnvidia-ptxjitcompiler.so*, libnvidia-nvvm.so*, libnvidia-ml.so*, libnvidia-allocator.so*

NOTICE

Nvidia provides an installer script which places some of the files in other directories. Therefore, it is recommended to install the components as packages or manually (device drivers).