Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix "zoom" effect with some cameras when changing the resolution #10848

Conversation

danxuliu
Copy link
Member

@danxuliu danxuliu commented Nov 8, 2023

This pull request is already ready for review, but it is marked as a draft to prevent merging it; @SystemKeeper will take care of that when/if needed :-)

When the video track resolution is changed the browser may crop and downscale the camera output to satisfy the requested constraints (resizeMode: "crop-and-scale"). However, in some cameras this causes an annoying "zoom" effect when changing between resolutions (it might be caused by the original source being shown for a split second before being cropped and downscaled, which may change the aspect ratio, although the reason is not fully clear). To prevent that, now the browser is asked to provide the camera output as is (although respecting the given constraints), without further adjustments.

Note that the resize mode is not enforced, though. The browser may still crop and scale the camera output if it is not possible to get a native track in the requested resolution range.

The problem (and therefore the steps below) can be reproduced at least with Logitech C920 and integrated Macbook camera, but not with others; nevertheless it should not adversely affect those cameras where the problem did not initially happen (but that should be thoroughly tested ;-) ).

How to test

  • Expose the sentVideoQualityThrottler in the browser console by adding something like the following to src/utils/webtc/index.js:
if (!OCA.Talk) {
	OCA.Talk = {}
}
OCA.Talk.getSentVideoQualityThrottler = () => {
	return sentVideoQualityThrottler
}
  • Rebuild the code
  • Start a call with camera
  • Open the browser console
  • Change between the high and very low resolutions running:
OCA.Talk.getSentVideoQualityThrottler()._videoConstrainer.applyConstraints(4)
OCA.Talk.getSentVideoQualityThrottler()._videoConstrainer.applyConstraints(1)

Result with this pull request

There is no visible "zoom" (although there could be a tone change, which seems to happen due to the lighting adjustment made by the camera when restarting)

Request without this pull request

The image might "jump" when the resolution is adjusted

The actual constraints set in the video track can be checked by running the following in the browser console:

OCA.Talk.SimpleWebRTC.webrtc._videoTrackConstrainer.getOutputTrack().getSettings()

When the video track resolution is changed the browser may crop and
downscale the camera output to satisfy the requested constraints
("resizeMode: crop-and-scale"). However, in some cameras this causes an
annoying "zoom" effect when changing between resolutions (it might be
caused by the original source being shown for a split second before
being cropped and downscaled, which may change the aspect ratio,
although the reason is not fully clear). To prevent that, now the
browser is asked to provide the camera output as is (although respecting
the given constraints), without further adjustments.

Note that the resize mode is not enforced, though. The browser may still
crop and scale the camera output if it is not possible to get a native
track in the requested resolution range.

Signed-off-by: Daniel Calviño Sánchez <danxuliu@gmail.com>
@SystemKeeper SystemKeeper marked this pull request as ready for review November 8, 2023 13:17
@nickvergessen nickvergessen merged commit 0e943a9 into main Nov 8, 2023
@nickvergessen nickvergessen deleted the fix-zoom-effect-with-some-cameras-when-changing-the-resolution branch November 8, 2023 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants