Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transcoder capabilities discovery and validation #2150

Merged
merged 7 commits into from
Feb 7, 2022
Merged

Conversation

cyberj0g
Copy link
Contributor

@cyberj0g cyberj0g commented Dec 23, 2021

What does this pull request do? Explain your changes. (required)
This PR is the last part of #2123 implementation. It will allow Orchestrator to send tasks to correct Transcoder according to requested capabilities (codecs). Also it adds capability test at startup for Nvidia GPUs (which may support only a subset of capabilities).

This PR includes changes of install_ffmpeg.sh discussed here. Corresponding LPMS PR needs to be merged first.

Specific updates (required)

  • add capability test function which is run on node startup
  • add logic to select transcoder according to capabilities in O-T mode
  • add logic to validate capabilities in OT mode

How did you test each of these updates (required)
Ran tests

Does this pull request close any open issues?
#2123

Checklist:

Additional notes:
One missing piece is automatic capability (codec) detection from segment. Currently, if segment is encoded with codec, for which T doesn't support decoding - and decoding capability is not explicitly requested, it will bypass capability-based T selection logic and may fail, even if O has Ts which could decode it. Should go-livepeer has such segment codec check (at which point?), or it's best to handle on MistServer side?

Copy link
Contributor

@oscar-davids oscar-davids left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@cyberj0g
Copy link
Contributor Author

Non-obvious quirk on multi-GPU system I just realized: there's no GPU-level segment scheduling logic, and capability discovery will provide a list of capabilities from less capable GPU, so if user has a rig with 5 x RTX 3090 and 1 GTX 745 to hook up the monitor, no codecs except H.264 would show up in capabilities, unless -nvidia argument excludes latter GPU. It worth documenting.

core/transcoder.go Show resolved Hide resolved
core/transcoder.go Outdated Show resolved Hide resolved
server/ot_rpc.go Outdated Show resolved Hide resolved
core/orchestrator.go Show resolved Hide resolved
cmd/livepeer/livepeer.go Show resolved Hide resolved
…lities, fix formatting, change number of output renditions in Nvidia test. Fix video segment which is used for capabilities test.
Copy link
Contributor Author

@cyberj0g cyberj0g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for review, all comments addressed.

cmd/livepeer/livepeer.go Show resolved Hide resolved
core/transcoder.go Outdated Show resolved Hide resolved
core/transcoder.go Show resolved Hide resolved
server/ot_rpc.go Outdated Show resolved Hide resolved
server/ot_rpc.go Show resolved Hide resolved
Copy link
Contributor

@darkdarkdragon darkdarkdragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after squash/rebase

@cyberj0g cyberj0g merged commit 3ac53ce into master Feb 7, 2022
@yondonfu yondonfu deleted the ip/capabilities branch February 7, 2022 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants