Run nvidia-smi inside a container and get Failed to initialize NVIDIA driver, or see no cards at all — that’s the first thing that bites you bringing up GPUs on openEuler. The host nvidia-smi works fine, then the cards vanish inside the container. The problem isn’t the driver itself; it’s the layer that maps the GPU devices and driver libraries into the container: nvidia-container-toolkit. openEuler isn’t a first-class NVIDIA-supported distro, so you have to borrow the CentOS 8 repo, and if you get that step wrong the container never sees a card.
Below is our full record of bringing GPUs into Docker on a dual-3060 box, starting from a minimal openEuler install, every command verbatim, with the pitfalls flagged.
Two things about openEuler worth mentioning — not GPU-specific, but they matter for stability: its memory RAS support is solid, letting the kernel and critical processes use high-reliability memory so that a multi-bit memory fault can kill the affected process instead of resetting the whole box; and it ships an ops toolset for online inspection and batch CVE remediation. For a box running GPU training long-term where you don’t want one memory error to take down the whole machine, those two are our actual reasons for picking it.
Environment
CPU: I9-10980XE
Board: MSI X299-PRO
RAM: 32G x2
GPU: GIGABYTE 3060 x2
OS: openEuler 22.03 (LTS-SP1)
Install: minimal
A minimal install means you supply the whole toolchain the driver build needs yourself.
Install dependencies
dnf install kernel-devel dkms make -y
yum install bzip2-devel gmp-devel.x86_64 mpfr-devel.x86_64 libmpc-devel.x86_64 libSM-devel.x86_64 libxml2-devel -y
kernel-devel and dkms are the key ones. The point of dkms is that the driver rebuilds automatically after a kernel upgrade; without it you reinstall the driver by hand every time the kernel moves. Make sure the kernel-devel version matches the running kernel, or the DKMS build fails.
Install the NVIDIA driver
Step one, get the things in the way out of the way: the open-source nouveau driver and IPv6. nouveau holds the card, and the NVIDIA driver won’t come up until it’s disabled. Edit /etc/grub2.cfg or /etc/grub2-efi.cfg — which one depends on your BIOS boot mode (Legacy vs. UEFI) — and add the disable flags at the end of the kernel line:
echo 'Loading openEuler (5.10.0-136.17.0.93.oe2203sp1.x86_64) 22.03 (LTS-SP1)'
linuxefi /vmlinuz-5.10.0-136.17.0.93.oe2203sp1.x86_64 root=UUID=54a9e953-0a9e-45dc-9206-179453b0f341 ro cgroup_disable=files apparmor=0 ipv6.disable=1 rd.driver.blacklist=nouveau nouveau.modeset=0 crashkernel=512M
Here rd.driver.blacklist=nouveau nouveau.modeset=0 are the two flags that disable nouveau; reboot to apply. (As the previous post noted: if you copied nomodeset from somewhere, be careful here — nomodeset blocks the NVIDIA driver too, so it can’t go on this box.)
Step two, run the official .run installer. Silent install, auto-accept the license, skip the X Window check, build the DKMS kernel module:
sh NVIDIA-.run -q -a -s --dkms --no-x-check
If you downloaded the full driver with 32/64-bit libraries, add --no-install-compat32-libs to skip the 32-bit libs — a pure container/compute box doesn’t need them. After this, nvidia-smi on the host should show both cards.
Install Docker and nvidia-container-toolkit
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install docker-ce-19.03.8 docker-ce-cli-19.03.8 containerd.io
Docker here borrows the CentOS repo and installs the pinned 19.03.8 version.
The pitfall is the toolkit: on openEuler you can currently only use the CentOS 8 nvidia-container-toolkit repo. There’s no openEuler source to point at; you have to use centos8:
curl -s -L https://nvidia.github.io/nvidia-docker/centos8/nvidia-docker.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
yum install -y nvidia-container-toolkit
Only after this does the container have the layer that can see the GPU.
Assign a GPU to a container
To check the whole chain works, run nvidia-smi straight inside a container:
docker run -it --gpus='"device=1"' image nvidia-smi
--gpus='"device=1"' hands the second card (numbering starts at 0) to this container alone. To give it several, list them, e.g. --gpus='"device=0,1,2.."'.
If you see the requested card inside the container with the right model and memory, all three layers — driver, Docker, toolkit — are through. If the container is still empty at this point, nine times out of ten the CentOS 8 repo in the toolkit step is wrong; go back and check that the repo in /etc/yum.repos.d/nvidia-container-toolkit.repo points at centos8.