Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow disabling automatic reorientation in ImageDecoder #870

Open
padenot opened this issue Jan 16, 2025 · 3 comments
Open

Allow disabling automatic reorientation in ImageDecoder #870

padenot opened this issue Jan 16, 2025 · 3 comments

Comments

@padenot
Copy link
Collaborator

padenot commented Jan 16, 2025

In PDFs, there are frequently images that do have orientation information in the metadata, but they must not be taken into account during rendering.

https://github.com/mozilla/pdf.js is used all over the web and in some browsers, and takes advantage of ImageDecoder when available, leading to ridiculous performance improvement. They would like to not reorient images, or more accurately, not have to have them reoriented twice (once by the browser engine, and once themselves).

Quoting the PDF spec (https://opensource.adobe.com/dc-acrobat-sdk-docs/pdfstandards/PDF32000_2008.pdf#page=43, bottom of the table):

Parameters that control the decoding process as well as other metadata is embedded within the encoded data
stream using a notation referred to as “markers”. When it defined the use of JPEG images within PostScript data
streams, Adobe System Incorporated defined a particular set of rules pertaining to which markers are to be
recognized, which are to be ignored and which are considered errors. A specific Adobe-defined marker was also
introduced. The exact rules for producing and consuming DCT encoded data within PostScript are provide in
Adobe Technical Note #5116 (reference). PDF DCT Encoding shall exactly follow those rules established by
Adobe for PostScript.

The reference is https://pdfa.org/norm-refs/5116.DCT_Filter.pdf. There is no mention of orientation, and so it has to be ignored for compliance.

I simply propose the following addition to the spec:

partial dict  ImageDecodeOptions {
    boolean ignoreOrientationMetadata = false;
}
@dalecurtis
Copy link
Contributor

dalecurtis commented Jan 16, 2025

I'm not opposed as a simplification, but it seems possible to do this using #351? I.e.,

   let vf = new VideoFrame(rotated_frame, {rotation: 0, flip: false});

This does assume that orientation is always tagged though.

Edit: Oh yeah, there's also image-orientation: none but perhaps that's too big of a stick.

@aosmond
Copy link

aosmond commented Jan 16, 2025

I'm not opposed as a simplification, but it seems possible to do this using #351? I.e.,

   let vf = new VideoFrame(rotated_frame, {rotation: 0, flip: false});

This does assume that orientation is always tagged though.

In Firefox, the image decoding pipeline can handle the reorientation while decoding. Since we don't implement VideoFrame rotation/flip yet, we rely on this to produce a result with the "correct" orientation. When we do implement VideoFrame rotation/flip, this would produce a VideoFrame with rotation 0 and flip false. That is problematic in this case because pdf.js wants to reverse the operation, but doesn't know how without looking into the EXIF data itself manually. It would be nice to retain the optimization for the presumably common case where EXIF orientation is already taken into account. What form that takes however, I don't have strong opinions.

@dalecurtis
Copy link
Contributor

dalecurtis commented Jan 16, 2025

Hmm, Firefox won't be able to pass the wpt with non-tagged behavior:
https://wpt.fyi/results/webcodecs?label=master&label=experimental&aligned&q=image-decoder

We should probably clarify the spec so that tagging is required, since it is necessary to satisfy image-orientation: none

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants