WSL2 and gstreamer using nvenc and nvdec (NVIDIA's hardware acceleration)

[tadam98]

Hi,

The instllation instructions of https://developer.nvidia.com/embedded/dlc/l4t-accelerated-gstreamer-guide-32-1 do not end up with a working use of nvend and nvdec.

1. WSL2 does not yet support USB webcam and the examples are with webcams.
2. The instructions install gstreamer 1.14.5 which is very old. Latest release of gstreamer is 1.18.3. (even versions are stable releases, odd versions are in development. So the latest 1.19.1 is under development and would be released as 1.20.x)
3. Following the recommeneted installation does not install the codecs used in the examples.
4. We need a simple indication whether the hardware acceleration is detected and used at all.
5. NVIDIA has its own FFMPEG https://docs.nvidia.com/video-technologies/video-codec-sdk/ffmpeg-with-nvidia-gpu/index.html yet gstreamer appears to have its own. (a) Forcing gstreamer to use NVIDIA's FFMPEGis not clear. (b) alternatively, making sure that gstreamer's FFMPEG uses NVIDIA's hardware acceleration is not clear. (c) compilation for Ubuntu 18.04 or other is very unclear.

See https://www.collabora.com/news-and-blog/blog/2020/03/19/getting-started-with-gstreamer-gst-build/?quip_approved=0#quip-success-qcom

and https://gitlab.freedesktop.org/gstreamer/gst-build/-/issues/142

Futher input is very welcomed.
Best,
Mickey

[tadam98]

Following the installation instructions I got:

$ gst-inspect-1.0 | grep 264
error: XDG_RUNTIME_DIR not set in the environment.
videoparsersbad:  h264parse: H.264 parser
uvch264:  uvch264mjpgdemux: UVC H264 MJPG Demuxer
uvch264:  uvch264src: UVC H264 Source
x264:  x264enc: x264enc
libav:  avenc_h264_omx: libav OpenMAX IL H.264 video encoder encoder
libav:  avdec_h264: libav H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 decoder
libav:  avmux_ipod: libav iPod H.264 MP4 (MPEG-4 Part 14) muxer
rtp:  rtph264pay: RTP H264 payloader
rtp:  rtph264depay: RTP H264 depayloader
typefindfunctions: video/x-h264: h264, x264, 264

Which one is using the hardware accelaration?

[therealkenc]

WSL2 exposes a virtual GPGPU not a hardware video codec. Legit enough as an ask, side-by-side #938

[tadam98]

It is now 2021 not 2016. And we do expect to use the GPU on WSL2 in the same way we use it on the native Ubuntu. which means that important functions like hardware acceleration must be exposed and supported.

As a minimum, we dont have to waste days just to find out indirectly that there is no intention to support it and it must be mentioned in the main WSL2 and NVIDIA documentation.

[thw0rted]

Just to close the loop here: #4700 said it wasn't supported until #829 "flipped status", which happened last summer. So, is this the issue to follow to get updates on when to expect nvenc to work?

As an aside, I came here hoping to see support vaapi or qsv encoders -- any chance of that?

[therealkenc]

#938 is the new #829. This issue is the new #4700.

[tadam98]

For me the inability to use the hardware accelaration caused, effectively, end of my work with WSL2. We are developing Machine Learning products that generate video and if it is not possble to offload CPU to the Hardware Acceleration - there is no real reason to us WSL2.

[shinctl]

Just killed the use for me too.

[happy2046]

nvidia new driver support nvenc in wsl2 now.
I got obs working when choosing nvenc encoder in wslg.

[TBBle]

@happy2046 It'd be great if you can document the NVIDIA driver version you used. https://docs.nvidia.com/cuda/wsl-user-guide/index.html#changelog doesn't list a change to support nvenc, and the latest version listed there is 470.14, which is also the version offered from https://developer.nvidia.com/cuda/wsl/download

[tadam98]

Indeed. As someone said there are no plans.

[TBBle]

Poking around a bit, a 470.25 driver has apparently been delivered through Windows Update about two weeks ago, but hasn't appeared in NVIDIA's public documentation yet. So perhaps that fixed nvenc support?

Apparently nvenc works from Windows containers with only the DirectX Class GUID (and hence user-mode display driver) exposed, which is similar (or the same) to how WSL2 GPU support works, if I recall correctly.

[happy2046]

I update my driver from windows update.
version:27.21.14.7025 on insider 21370
There is a file named libnvidia-encode.so.1 in System32\lxss\lib and is seems to be the file for nvenc

[tadam98]

Note that in all previous version ffmpeg reported existance of nvenc but when trying to use it ffmpeg have an exception. Did you actually try it ?

Best,
Mickey

[TBBle]

@tadam98 Did the previous versions of the driver include libnvidia-encode.so.1 in the host-driver bundle (System32\lxss\lib)? If so, then clearly it's intended to support nvenc, and the exception was a bug of some kind, and should be reported to NVIDIA.

If not, then it's newly-implemented in 470.25, and my guess would be that ffmpeg was picking up a copy of libnvidia-encode.so.1 from Linux, not from the host-mount, which probably won't work, assuming the host-driver bundle version is modified to work with WSL2 GPU access.

So if you test with NVIDIA driver 470.25, make sure it's picking up the host-mount libnvidia-encode.so.1, not one from the installed Linux OS.

[onomatopellan]

The very first driver with WSL2 support from NVIDIA didn't even have DirectML support, only CUDA and in the following drivers they added more features like DirectML, D3D12, NVENC... etc

Just tried this with my GT 710 (driver 470.25) in WSL2 and looking at Task Manager it truly used the NVENC engine.
ffmpeg -i input.mkv -vcodec h264_nvenc -crf 25 -acodec copy Output.mp4

[therealkenc]

Thanks for the confirm, ono. Available on WSL2 21362+ with a commensurate 5.10.16 kernel. Further issues over at WSLg. /fixed

[ghost]

This bug or feature request originally submitted has been addressed in whole or in part. Related or ongoing bug or feature gaps should be opened as a new issue submission if one does not already exist.

Thank you!

[thw0rted]

@therealkenc I lost the thread on what I was trying to do when I made my previous comment, but it looks like

• VAAPI: /dev/dri not found #4700 was closed and you said above that this is the replacement ticket
• That issue was asking about VAAPI support
• h264_vaapi and hevc_vaapi are listed in your screenshot

Does this mean that VA-API encode acceleration is now supported in ffmpeg under WSL2? (If not, new ticket? Here or over at the WSLg tracker, which currently has no tickets for ffmpeg?)

Also, what about QSV? I don't see any mention of QSV in either place.

[therealkenc]

Does this mean that VA-API encode acceleration is now supported in ffmpeg under WSL2?

What this means is the singular identifiable repro in this issue submission, which was error: XDG_RUNTIME_DIR not set in the environment no longer reproduces on WSL2 21362+ with a commensurate 5.10.16 kernel.

If not, new ticket?

If you can demonstrate a misbehavior in VA-API encode acceleration with a repro, yes, a new ticket, in principle.

Here or over at the WSLg tracker...?

That's a good question. As of this writing we're still tagging (GP)GPU submissions in this tracker. Meaning, to a first order, for practical purposes, NVIDIA and AMD WSL2 kernel layer driver related issues. WSLg is, narrowly speaking, at least arguably about Wayland and Pulse (plus other goodies), or colloquially "GUI apps". Running ffmpeg -vcodec h264_nvenc is strictly speaking unrelated to Wayland. But if it were me I'd post over over in WSLg since for all intents anything going through the freedesktop stack (which for this scenario culminates in libnvidia-encode.so.1) will probably get the best eyeballs over there. But no one would fault a submission here for now either.

[TBBle]

I think VA-API is completely distinct from this ticket, because they are Intel encoding acceleration APIs, and will depend on Intel either providing an appropriate user-mode library to redirect the API into the exported DirectX minidriver (which is what NVidia have done with libnvidia-encode.so.1 bundled in the driver), or to somehow provide a /dev/dri device that does the same, so that the current native VA-API user-mode library works.

That's assuming that va-api is a user-mode shared library like nvenc is; if it's compiled into ffmpeg to call /dev/dri, then only the latter option is open.

Making nvenc work didn't require /dev/dri (and I don't think nvenc on Linux ever used /dev/dri, but I might be wrong about that), so I think this ticket was not really a replacement for #4700.

[tadam98]

Upgraded to windows 11 insider version 21H2 (OS Build 22000.120)
The wsl2 version:

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.5 LTS
Release:	18.04
Codename:	bionic

The nvidia versions:

$nvidia-smi
Driver Version: 471.21       CUDA Version: 11.4
GPU: 2080 TI

# **Note**: update installed 471.41 but the wsl2 page at nvidia points to 471.21 so I downgraded.
# it is the driver installed in the inderlying windows 11.
# https://docs.nvidia.com/cuda/wsl-user-guide/index.html
$ ./BlackScholes #works
[./BlackScholes] - Starting...
GPU Device 0: "Turing" with compute capability 7.5

Initializing data...
...allocating CPU memory for options.
...allocating GPU memory for options.
...generating input data in CPU mem.
...copying input data to GPU mem.
Data init done.
$ docker run --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark
> Windowed mode
> Simulation data stored in video memory
> Single precision floating point simulation
> 1 Devices used for simulation
GPU Device 0: "Turing" with compute capability 7.5

> Compute 7.5 CUDA device: [NVIDIA GeForce RTX 2080 Ti]
69632 bodies, total time for 10 iterations: 117.984 ms
= 410.957 billion interactions per second
= 8219.140 single-precision GFLOP/s at 20 flops per interaction
$ ffmpeg --version
ffmpeg version 4.3.2-0york0~18.04 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)

checks:

gstreamer check:

$ gst-inspect-1.0 | grep 264
typefindfunctions: video/x-h264: h264, x264, 264

ffmpeg check

$ ffmpeg -i girl.mp4 -vcodec h264_nvenc -crf 25 -acodec copy Output.mp4
Unknown encoder 'h264_nvenc'
$ ffmpeg -codecs | grep venc
$

HW acceleration is not detected.
Note: in older releases nvenc wsa detected but gave an exception when attepted to use. This is why the correct check is actually asking ffmpeg to process a file rather than just trusting detection of hw acceleration.

[TBBle]

I suspect

Unknown encoder 'h264_nvenc'

means that ffmpeg binary does not have NVENC support compiled in.

So you'll need to either build your own, using e.g. this quite long process; or use some one else's build, e.g. this PPA.

[tadam98]

Gstreamer too ?

[TBBle]

Yup, I don't see the gstreamer plugin there, but it also wasn't listed in the working example earlier: #6357 (comment)

There's instructions for compiling and installing the nvenc plugin for GStreamer at https://gist.github.com/corenel/a615b6f7eb5b5425aa49343a7b409200. 30 seconds of poking around didn't find a PPA for this plugin, so I'd suggest focussing on getting ffmpeg working first. Then you'll know that the WSL side works, and you can then build the GStreamer plugin and get it working too.

[tadam98]

Procedure for ffmpeg worked find. I removed the default ffmpeg and installed.

$ apt list -a ffmpeg
Listing... Done
ffmpeg/now 7:4.3-2nvenc1~18.04 amd64 [residual-config]
ffmpeg/bionic-updates,bionic-security 7:3.4.8-0ubuntu0.2 amd64 [residual-config]
ffmpeg/bionic 7:3.4.2-2 amd64 [residual-config]

$ sudo apt-get  install ffmpeg
..
..
$ ffmpeg -hide_banner -codecs | grep 264
 DEV.LS h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_crystalhd h264_v4l2m2m h264_vdpau h264_cuvid ) (encoders: libx264 libx264rgb h264_nvenc h264_omx h264_v4l2m2m h264_vaapi nvenc nvenc_h264 )

$ ffmpeg -i girl.mp4 -vcodec h264_nvenc -crf 25 -acodec copy Output.mp4 # works fine now

Procedure for getreamer fails. See [https://gist.github.com/corenel/a615b6f7eb5b5425aa49343a7b409200]

[TBBle]

Good, so the WSL/nvenc setup is working, and so for GStreamer, someone needs to work out the right combination of libraries and checkouts to build under Ubuntu 18.04 with CUDA 11.4.

Your linked gist comment suggests to me that something in that gst-plugins-bad repo needs a newer compiler than comes with Ubuntu 18.04, but I have neither here to check that, so am just guessing. This is probably the wrong place to work it out, since it's distro-specific, and application specific, and this closed bug was tracking the WSL2 plumbing side of the feature, which is working.

[tadam98]

I put it in Gstreamer rather than in wsl2 as indeed it does not belong in wsl2. I would try it with CUDA 10.2 as well. The issue is that the gstreamer does not compile with the provided procedure which is not WSL2 issue.

[rainabba]

I can confirm that in 2022, we finally have this working. I was live-streaming from Coachella ~2016 and so badly wanted nvenc access in Docker, but alas it was WAY too soon. I'm now running a container on Win10 21H2 with WSL2, and nvidia-smi is successfully reporting my GPU!

Instructions