Request to test hardware transcoding

[getroot]

OME now has hardware transcoding capabilities. You can try this feature in the latest master branch. Please refer to the manual below for details.

https://airensoft.gitbook.io/ovenmediaengine/v/master/transcoding/gpu-usage

We welcome all comments, such as bugs, improvements, and issues in the manual.

Your comments on this thread will help a lot in improving OvenMediaEngine.

Thank you for your contribution.

[fcqpl]

Hello,
Propably Oven not seeing my Quadro.

Docker-compose:

  ovenmediaengine:
    image: airensoft/ovenmediaengine:dev
    ports:
      - 443:443
      - 1935:1935
      - 3333:3333
      - 3334:3334
      - 3478:3478
      - 8080:8080
      - 9000:9000
      - 9999:9999/udp
      - 4000-4005:4000-4005/udp
      - 10006-10010:10006-10010/udp
    environment:
      - TZ=Europe/Warsaw
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility,video
    volumes:
      - ./oven_origin_conf:/opt/ovenmediaengine/bin/origin_conf
      - ./ssl:/etc/docker_ssl_cert
    restart: always
    runtime: nvidia
    networks:
      - streaming-net

Server.xml and log:
logs.md
Server.xml.md

When I setup other container with nvidia-docker it's working correctly.

root@oven:/opt/streaming# nvidia-smi
Fri Jul  9 01:11:13 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.27       Driver Version: 465.27       CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA Quadro K...  Off  | 00000000:06:10.0 Off |                  N/A |
| 42%   35C    P0     2W /  39W |    137MiB /  4043MiB |      1%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1059      G   /usr/lib/xorg/Xorg                  2MiB |
|    0   N/A  N/A   2397448      C   ffmpeg                            129MiB |
+-----------------------------------------------------------------------------+

Oven isn't supporting ABR in HLS/DASH/WebRTC?

[fcqpl]

And streams transcoded by software transcoder

<Video>
	<Codec>h264</Codec>
	<Width>1280</Width>
	<Height>720</Height>
	<Bitrate>5000000</Bitrate>
	<Framerate>30</Framerate>
</Video>

it's not playing smoothly with WebRTC in Chrome (91.0.4472.124) and unplayable in Firefox (89.0.2).
With HLS everything OK.

[getroot]

And streams transcoded by software transcoder

<Video>
	<Codec>h264</Codec>
	<Width>1280</Width>
	<Height>720</Height>
	<Bitrate>5000000</Bitrate>
	<Framerate>30</Framerate>
</Video>

it's not playing smoothly with WebRTC in Chrome (91.0.4472.124) and unplayable in Firefox (89.0.2).
With HLS everything OK.

@fcqpl
In my environment, playing with the same settings as yours plays smoothly in both Chrome and Firefox. Maybe your network or CPU isn't fast enough? If your network has high packet loss (check chrome://webrtc-internals), try transport=tcp. If your CPU isn't fast enough, try encoding either h264 or h265 and lower the bitrate.
If neither method solves your problem, please explain in more detail so I can set it up exactly like your environment. (Which encoder is used (obs?) and encoder settings, etc.)

[Keukhan]

@fcqpl

I want to say thank you for your review.

As you checked the attached log, there are missing Hardware Codec related options when building Docker. Rebuild and run the Dockerfile.nv script from the root path of the GitHub source.
For your information, GPU support is not yet a full release and has not registered images with Docker Hub. If you want to use GPU, you need to build the image and use it.

and..

To enable hardware codecs, the HardwareAcceleration option must be enabled as true under OutputProfiles.

    <Applications>
            <Application>
                    <Name>app</Name>
                    <!-- Application type (live/vod) -->
                    <Type>live</Type>
                    <OutputProfiles>
                            <HardwareAcceleration>true</HardwareAcceleration>

                            <OutputProfile>
                                    <Name>profile_1080o</Name>
                                    <OutputStreamName>${OriginStreamName}_o</OutputStreamName>
                                    <Encodes>
                                            <Video>
                                                    ...
                                            </Video>
                                            <Audio>
                                                    ...
                                            </Audio>

Thanks

[fcqpl]

Hello,
Problem with not smoothly playback is on this build:
airensoft/ovenmediaengine:dev
OvenMediaEngine v0.12.1 is started on [35b087cf9405] (Linux x86_64 - 5.4.0-77-generic, #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021)
I refreshed docker image and it's the same:

docker pull airensoft/ovenmediaengine:dev
dev: Pulling from airensoft/ovenmediaengine
4bbfd2c87b75: Already exists
d2e110be24e1: Already exists
889a7173dcfe: Already exists
94daa5dfbe1d: Pull complete
4b0fedf1bc9c: Pull complete
26209b5f574e: Pull complete
Digest: sha256:144af9520ba4c086ed92804c376b1af84c77e2233b058be1d84ca15e477874a3
Status: Downloaded newer image for airensoft/ovenmediaengine:dev
docker.io/airensoft/ovenmediaengine:dev

OvenMediaEngine v0.12.1 is started on [8691d899b8d7] (Linux x86_64 - 5.4.0-77-generic, #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021)

This versions works correctly:
airensoft/ovenmediaengine:0.12.1
OvenMediaEngine v0.12.1 is started on [291d18053bde] (Linux x86_64 - 5.4.0-77-generic, #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021)

https://github.com/AirenSoft/OvenMediaEngine/blob/master/Dockerfile.nv
OvenMediaEngine v0.12.0 [debug] is started on [5db2dec2712c] (Linux x86_64 - 5.4.0-77-generic, #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021)

It's like slowed playback? Here is record:
airensoft/ovenmediaengine:dev v0.12.1 35b087cf9405 https://user-images.githubusercontent.com/46460263/125082838-f8aaf800-e0c7-11eb-9218-21dd3c0bb3d8.mp4 (same on 8691d899b8d7)
airensoft/ovenmediaengine:0.12.1 v0.12.1 291d18053bde https://user-images.githubusercontent.com/46460263/125082831-f6e13480-e0c7-11eb-8c32-5336e82be62c.mp4
But it's topic for another issue.

---

Transcoding on build from Dockerfile.nv propably works correctly for playback by HLS and WebRTC.
OvenMediaEngine v0.12.0 [debug] is started on [5db2dec2712c] (Linux x86_64 - 5.4.0-77-generic, #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021)

root@oven:/opt/streaming# nvidia-smi
Fri Jul  9 15:11:00 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.27       Driver Version: 465.27       CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA Quadro K...  Off  | 00000000:06:10.0 Off |                  N/A |
| 42%   41C    P0     2W /  39W |    293MiB /  4043MiB |      6%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1077      G   /usr/lib/xorg/Xorg                  2MiB |
|    0   N/A  N/A    384342      C   ...ngine/bin/OvenMediaEngine      285MiB |
+-----------------------------------------------------------------------------+
root@oven:/opt/streaming#

[getroot]

@fcqpl
Have you tried playing with ws://host:port/app/stream?transport=tcp to solve the problem of not playing smoothly? I would be very grateful if you could try this and let me know the results. If the problem still occurs, please create a new issue.

[fcqpl]

@fcqpl
Have you tried playing with ws://host:port/app/stream?transport=tcp to solve the problem of not playing smoothly? I would be very grateful if you could try this and let me know the results. If the problem still occurs, please create a new issue.

Yeah, mp4 files is recorded from transport=tcp.
It's local network connected on fiber - there is no packet loss between my PC and OME server.
I'll create another issue.

[fcqpl]

Hi,
Propably HEVC encoder is not working with hardware transcoding.
It transcodes with CPU, from log:

[07-14 00:49:47.593] I [OvenMediaEngine:1] OvenMediaEngine | main.cpp:140  | Trying to create a Transcoder module
[07-14 00:49:47.593] W [OvenMediaEngine:1] Transcoder | transcoder_gpu.cpp:27   | Does not support Intel QuickSync
[07-14 00:49:47.712] I [OvenMediaEngine:1] Transcoder | transcoder_gpu.cpp:47   | Supported NVIDIA CUDA
[...]
x265 [info]: HEVC encoder version 0.0

ffmpeg with hevc_nvenc works correctly with my GPU.

<HardwareAcceleration>true</HardwareAcceleration>
<OutputProfile>
	<Name>bypass_stream</Name>
	<OutputStreamName>${OriginStreamName}_hevc</OutputStreamName>
	<Encodes>
		<Audio>
			<Bypass>true</Bypass>
		</Audio>
		<Video>
			<Codec>h265</Codec>
			<Width>1280</Width>
			<Height>720</Height>
			<Bitrate>2000000</Bitrate>
			<Framerate>30</Framerate>
		</Video>
	</Encodes>
</OutputProfile>

[fcqpl]

And also - there is no support for HW encoding in code on github?
I only see full working code in http://dev.airensoft.com/~pub/test/master.tar.gz archive.

[Keukhan]

@fcqpl

Of course, the github master contains the hardware encoding source code. It's a small mistake. Dockerfile.nv had an invalid OME source path. I modified it and committed it again.

If you recreate the image with the updated Dockerfile, HEVC encoding will work.

Thanks.

[fcqpl]

I'll test HEVC on Tuesday (waiting for new GPU).
My test GPU (Quadro K2200) isn't supporting HEVC. Looks like OME trying to use CUDA and fallback to SW encoder x265.
Maybe add option something like - "When using HW encoder, don't try using SW encoder". For me it's better to don't delivery stream (and show error in player), than have 100% CPU on server (by SW encoding).

[07-18 03:22:17.185] I [OvenMediaEngine:1] OvenMediaEngine | main.cpp:142  | Trying to create a Transcoder module
[07-18 03:22:17.376] I [OvenMediaEngine:1] Transcoder | transcoder_gpu.cpp:47   | Supported NVIDIA CUDA hardware accelerator
[...]
[07-18 03:22:39.179] I [OutboundWorker:94] Publisher | stream.cpp:181  | OVTPublisher Application application has started [SyncFootage(2959730169)] stream
[07-18 03:22:39.223] I [Dech264NV:101] Transcoder | decoder_avc_nv.cpp:196  | [#default#app/SyncFootage(89)] input stream information: [video] h264 (Main 4.2), cuda, 1920x1080 [SAR 0:1 DAR 16:9], 60 fps, 195 kbps, timebase: 1/120, frame_size: 0
[07-18 03:22:39.232] E [Dech264NV:101] Transcoder | encoder_hevc_nv.cpp:36   | Could not find encoder: 173 (hevc)
x265 [info]: HEVC encoder version 0.0
x265 [info]: build info [Linux][GCC 7.5.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64
x265 [info]: Main profile, Level-4 (Main tier)

[getroot]

@fcqpl That's a good idea. I'll add a task to our backlog for adding the <HardwareAcceleration>force</HardwareAcceleration> option.

[fcqpl]

Other GPU supports NVENC H265, but I have same error:

[07-19 23:01:26.414] E [Dech264NV:404] Transcoder | encoder_hevc_nv.cpp:36   | Could not find encoder: 173 (hevc)

ffmpeg with nvenc_hevc works correctly.
And propably software x265 encoder has memory leak. My VM freezes after a while.

[fcqpl]

Here is defined bad codec name: https://github.com/AirenSoft/OvenMediaEngine/blob/master/src/projects/transcoder/codec/encoder/encoder_hevc.cpp
Line 33: h265_nvenc to hevc_nvenc

But after changing i got another error:

[07-19 23:15:57.912] E [AppWorker:96] FFmpeg | third_parties.cpp:111  | [AVFormatContext: 0x7fa7cc000b40] HEVC bitstream malformed, no startcode found
[07-19 23:15:57.913] E [AppWorker:96] Writer | writer.cpp:632  | [0x7fa8240088a8] Could not write the frame: Not yet implemented in FFmpeg,

[Keukhan]

@fcqpl

Thank you very much for your excellent report.

Please let me know what your test environment is. (OS, VM or Docker, Graphics Card)

If you have time, could you activate the Transcoder tag to debug in Logger.xml and send me the full log?

I'm going to reproduce it in the same environment.

Thanks!

[fcqpl]

Hello,
Proxmox VE 7.0-8, VM with Ubuntu 20.04.2 LTS and PCI-E passthrough
In VM installed nvidia-docker2:

• Docker version 20.10.7, build f0df350
• Nvidia driver 465.27 (CUDA 11.3)
• GPU GeForce 1080Ti with unlocked NVENC (https://github.com/keylase/nvidia-patch).
  Everything works correctly with ffmpeg and hevc_nvenc on the same environment (other docker container).

I added to Logger.xml but it doesn't print anything new.
Ofc this files is with replaced h265_nvenc to hevc_nvenc in source code.
OME_hevc_error.md log file
OME_hevc_server.xml.md config file

[Keukhan]

@fcqpl

Thanks for the detailed information.

Based on the log you sent, the problem has been corrected and committed. The problem was caused by improperly formatting the HEVC encoded bitstream. I have confirmed that it works normally with the NVIDIA 1060 graphics card.

Please test again, and if there are additional problems, please report to me.

Thanks :)

[fcqpl]

It's now working! :)

[Keukhan]

Awesome!

[mpisat]

I tested it with QSV. Believe me, it wasn't easy. I compiled it from source code using provided manuals. For me, software transcoding uses 120% CPU (I got 8 core i7-9700k) which is about 1/7th of the total power. Hardware acceleration enabled reduces this to 55%. However, QSV has limits. I managed to do the following:

ffmpeg -re -i bbb_sunflower_1080p_30fps_normal.mp4 -c:v copy -an -f flv rtmp://fr.xxxxxx.com/app/big1
...
ffmpeg -re -i bbb_sunflower_1080p_30fps_normal.mp4 -c:v copy -an -f flv rtmp://fr.xxxxxx.com/app/big10

And transcoded video from big1 or big10 via WebRTC was slow, almost like can't transcode in realtime. I canceled big10 and limited it to 9 transcodes (1080p in, 720p 1200 Kbit/s out) and it was working fine.

I thought QSV had more juice in it? Also, it still manages to use some CPU, which is odd? I mean, it should use fewer CPU cycles when you opt in for QSV. At least this is the case for ffmpeg 4.4.

I wonder if we can somehow fall back to software to max out server capacity by using QSV + Software x264 at the same time? Do we need to create new profiles and send let's say 9 videos from qsv enabled /appqsv/streamnames and 4 or 5 videos from software only /appsoftware/streamnames? Is this even possible with the current config file?

Also, while it uses 55% for one transcode, and 110% cpu for 2 transcodes, it never went above 300% with 10, and it was 330% with 9 transcodes. Which is weird.