Transcoding on Older NVIDIA Graphics Card

NVIDIA is a well-known graphics card company that received a lot of attention on Linux, benefiting from both official proprietary drivers and an opensource driver named nouveau. Unfortunately, support progressively got phased out for older NVIDIA models such that it is seemingly difficult to use an older NVIDIA card with Linux. There are some legacy drivers provided with distributions such as Debian, but they are still not supported by end-user applications. For instance, Plex states that as of recent versions, Plex requires NVIDIA GPU driver 470.141.03 or higher which makes it impossible to use older NVIDIA cards.

Fortunately, the nouveau open-source driver can be used together with the Video Acceleration API (VA-API) in order to enable hardware transcoding on older NVIDIA cards.

In other to make hardware acceleration via VA-API work for an older NVIDIA, first make sure that the nouveau driver is loaded on startup. To be sure, the driver can be loaded from initramfs by adding the nouveau driver to /etc/initramfs-tools/modules and then the image can be rebuilt by issuing update-initramfs -k all -u, followed by update-grub to boot the new initramfs on startup.

After booting, the nouveau driver should install a framebuffer which should allow larger resolutions. The next step is to install nvidia-vaapi-driver by issuing:

aptitude install nvidia-vaapi-driver

If everything is set up correctly, then the device at /dev/dri/renderD128 should exist along with /dev/dri/card0. If /dev/dri/renderD128 does not exist, then something is not set up correctly.

The next thing to do is to install is the vainfo package. Typically, vainfo polls X11 in order to determine the video drivers that are installed and to list their capabilities. With just the former installed, on an older card, the output of vainfo should suggest that no acceleration is available:

error: can't connect to X server!
libva info: VA-API version 1.17.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nouveau_drv_video.so
libva info: Found init function __vaDriverInit_1_17
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.17 (libva 2.12.0)
vainfo: Driver version: Mesa Gallium driver 22.3.6 for NVAF
vainfo: Supported profile and entrypoints
      VAProfileNone                   : VAEntrypointVideoProc

Now, if the command dmesg is issued, the nouveau driver might state that no firmware could be loaded when issuing vainfo:

nouveau 0000:05:00.0: firmware: failed to load nouveau/nvaf_fuc084 (-2)
nouveau 0000:05:00.0: firmware: failed to load nouveau/nvaf_fuc084 (-2)
nouveau 0000:05:00.0: Direct firmware load for nouveau/nvaf_fuc084 failed with error -2
nouveau 0000:05:00.0: firmware: failed to load nouveau/nvaf_fuc084d (-2)
nouveau 0000:05:00.0: firmware: failed to load nouveau/nvaf_fuc084d (-2)
nouveau 0000:05:00.0: Direct firmware load for nouveau/nvaf_fuc084d failed with error -2
nouveau 0000:05:00.0: msvld: unable to load firmware data
nouveau 0000:05:00.0: msvld: init failed, -19

The problem here is that whilst nouveau is an OpenSource driver that was created by re-engineering the driver itself, the firmware for the graphics card cannot be included in distributions due licensing issues, such that the firmware itself has to be extracted manually from the corresponding driver. In this case the graphics card is an NVIDIA 350M for which there are some instructions on the official freedesktop page. In principle, the process can be summarized by downloading the older and archived official NVIDIA drivers, extracting them and then using a tool to scalp out just the firmware from the drivers. Mirrored here for archival:

mkdir /tmp/nouveau
cd /tmp/nouveau
wget https://raw.github.com/envytools/firmware/master/extract_firmware.py
wget http://us.download.nvidia.com/XFree86/Linux-x86/325.15/NVIDIA-Linux-x86-325.15.run
sh NVIDIA-Linux-x86-325.15.run --extract-only
python extract_firmware.py
mkdir /lib/firmware/nouveau
cp -d nv* vuc-* /lib/firmware/nouveau/

Note that here the driver being downloaded is NVIDIA-Linux-x86-325.15.run matching the 3xx series of the NVIDIA driver, but pretty much any driver version prefixed with 3 for the 3xx could be downloaded. With the firmware copied to /lib/firmware/nouveau, the typical place where Debian stores firmware for various drivers, the vainfo command is ran again.

rror: can't connect to X server!
libva info: VA-API version 1.17.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nouveau_drv_video.so
libva info: Found init function __vaDriverInit_1_17
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.17 (libva 2.12.0)
vainfo: Driver version: Mesa Gallium driver 22.3.6 for NVAF
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc

the important thing to notice here are the encoders such as VAProfileH264Main corresponding to the H264 codec, MPEG2, etc, that now should be working.

Note that given the previous output of vainfo, the output has the following meaning:

VAEntrypointVLD means that the graphics card is able to decode the specified codec on the left,
VAEntrypointEncSlice means that the graphics card is able to encode to the specified codec on the left

The /dev/dri/card0 and /dev/dri/renderD128 devices can be used for transcoding now and, in case Docker is used as a container technology for Plex or Jellyfin, then both devices should be passed to the Plex or Jellyfin container as explained on the Docker workarounds page.