[BUG] {High CPU usage while streaming, spams multiple processes}

[yifan-hou]

Describe the bug
When streaming video, sometimes OAK creates a ton of processes and has a very high CPU usage. This affects other time-critical processes running on the same machine.

Below is an example screenshot of htop, when running the rgb_preview example with small modifications:

  camRgb->setPreviewSize(640, 360);
  camRgb->setResolution(
      dai::ColorCameraProperties::SensorResolution::THE_720_P);
  camRgb->setFps(60);

All 32 CPU has ~40% usage, and there are many processes created by the OAK test example.

If I reduce preview size to 300x300, the problem is gone:

There is only one CPU with high usage, which is expected.

The same issue can be reproduced with the rgb_video example, with the same resolution/FPS settings. Note that with rgb_video, even at 300x300, the CPU usage is still high.

If I reduce FPS to 30, the usage per cpu reduces to ~15%, but still many processes are created and all CPU has this usage.

I am using a OAK-D-W camera.

Minimal Reproducible Example
The camera_preview and the camera_video example linked above, with the above mentioned changes.

Expected behavior
There should be only one CPU with high usage, no matter what camera setting I use. Or there should be parameters I can use to control the number of processes/CPU that OAK can occupy. Unfortunately I am yet to find any info about when and how would depthai-core create processes.

Attach system log
log_system_information.json

[moratom]

Thanks for the report @yifan-hou !

@jakaskerl would you mind checking if we can reproduce the issue locally?

[yifan-hou]

Some more info:

• I tried the same code with OAK-1-W camera and got the same high CPU usage.
• I tried the depthai python interface, this time there was only one CPU with high usage. Probably related to how python handles threading?

[yifan-hou]

Some more digging:

Those processes were generated by OpenCV automatically when the computation load is high. I can limit them by calling

    cv::setNumThreads(max_num_threads_i_want);

However I cannot get stable frame rate sometimes (even at 60Hz, where the camera is supposed to support 120Hz).

It seems that the function getCvFrame uses some opencv operations internally at https://github.com/luxonis/depthai-core/blob/main/src/opencv/ImgFrame.cpp , maybe there is room to reduce computation load here?

[moratom]

@yifan-hou could you check other OpenCV versions and let us know what OpenCV version you're using now?

I think this might be OpenCV version related since we would have likely otherwise see this bug already.