frigate requires ffmpeg-full

[PhilippWoelfel]

Describe the bug

The frigate module adds ffmpeg-headless to the systemd service path, but ffmpeg from that package does not compile some filters, such as scale_cuda. As a result, ffmpeg crashes when executed by frigate (on my machine with Nvidia GPU). Replacing ffmpeg-headless with ffmpeg-full fixes the problem.

Steps To Reproduce

Steps to reproduce the behavior:

1. Enable frigate with option hwaccel_args = "preset-nvidia-h264" on a system with Nvidia GPU, and add a camera configuration.
2. Open the web ui for frigate. Observe that there is no camera output. The log file indicates that ffmpeg crashes because the filter scale_cuda is missing. ( ERROR : [AVFilterGraph @ 0x10e3a040] No such filter: 'scale_cuda')

Expected behavior

frigate show the camera, and ffmpeg does not crash.

Additional context

The output of nix-shell -p ffmpeg-headless "ffmpeg -filters" does not contain scale_cuda. But the output of nix-shell -p ffmpeg-full --run "ffmpeg -filters" does.

Also, a simple workaround is

systemd.services.frigate.path = lib.mkBefore [ pkgs.ffmpeg-full ];

If there is a good reason to not use ffmpeg-full by default, maybe there could be a module option, where users can choose their ffmpeg package?

Notify maintainers

@NickCao

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.51, NixOS, 24.05 (Uakari), 24.05.20240918.dbebdd6`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.5`
 - channels(root): `"home-manager, nixos"`
 - nixpkgs: `/nix/store/2bgr2mxzslxfn3cd4ifjw7s68x91dki5-source`

---

Add a 👍 reaction to issues you find important.

[Atemu]

If there is a good reason to not use ffmpeg-full by default, maybe there could be a module option, where users can choose their ffmpeg package?

ffmpeg-full should not be used by default due to its closure size. It's something users should opt in to use if they need niche or advanced features that you wouldn't typically need from ffmpeg.

Adding an option to the module would be the best course of action if needing to customise ffmpeg is a common requirement here. (Please note that I don't know this module, I only maintain ffmpeg.)

Which feature flag provides this functionality you need? withCuda perhaps? Please try ffmpeg-headless.override { withCuda = true; } rather than ffmpeg-full to find out.

We could also look into enabling that feature in regular ffmpeg if you can make a case for it. What we would need to know is whether this is something you'd expect a "normal" ffmpeg package to have and whether it technologically feasible to enable (i.e. no unfree deps and no dramatic closure size).

cc @jopejoe1 @emilazy

[emilazy]

CUDA is non‐free, so we can’t enable it by default. We could condition it on config.cudaSupport if it’s not already, with the side‐effect that users who enable CUDA will have to rebuild FFmpeg and the thousands of packages that depend on it; I’m not sure that trade‐off is worth it.

I agree that a module option is the best solution here.

[Atemu]

withCuda is free as, code-wise, only the ffmpeg support code for CUDA is actually built, no linking against the actual CUDA. It's enabled in ffmpeg-full.

[PhilippWoelfel]

I tested ffmpeg-headless built with cudaSupport. Unfortunately, it does not support this feature (i.e., ffmpeg -filters does not show the filter scale_cuda or any of the other "_cuda" filters that are part of ffmpeg-full).

[Atemu]

Could you figure out which flag it needs then?

[PhilippWoelfel]

Ah, withCudaLLVM does the trick...

[Atemu]

Hm, that might be a bit larger in terms of closure because I believe it pulls in clang/llvm into the runtime closure but I'm not sure.

[Atemu]

Thinking about this some more, I don't think frigate should depend on ffmpeg-full by default. ffmpeg-headless shouldn't have CUDA support either if it means pulling in LLVM.

NVIDIA doesn't play nice with the rest of the industry and requires users to user their own bloated proprietary crap. If Nvidia can't offer a bearable solution here, that's their problem.

If people really want to buy GPUs from such a company, they should be expected to feel the pain of configuring their environment accordingly.

For that to be viable, there should be an option which controls the ffmpeg package used in fragate where the user could then pass ffmpeg-full or a more minimal custom override.

[PhilippWoelfel]

I can think of lots of reasons why users may not have a choice of graphics card, or people may already have NVIDIAs for various reasons.

Anyway, an option to choose the ffmpeg package is a pragmatic solution. The example in the doc could be with ffmpeg-full - this would point users in the right direction.

[Atemu]

As it turns out, cuda-llvm doesn't pull in LLVM into the runtime closure.

Fixed by #354952.