Alpha questions

[jreichel-nvidia]

I've a couple of questions about associated/unassociated alpha. I'm aware of the general explanations in #3267.

1. The supports() method does not seem to recognize "oiio:UnassociatedAlpha" at all except for the null plugin. So there seems to be no programmatic way to detect whether a plugin supports "oiio:UnassociatedAlpha" and I have to figure that out from the code or the documentation.
2. There is a mismatch between code and the documentation. Maybe I'm missing something, but PNG export seems to support "oiio:UnassociatedAlpha" whereas the documentation does not mention it. Same for PSD input, TGA input/output, JPEG-2000 output.
3. The code snippet in the tutorial part of the documentation suggests to check the value of "oiio:UnassociatedAlpha" e.g. after read_image(), suggesting that import operations might set it back to 0 if it was non-zero. Is that possible? If yes, it might be worth pointing that out in the documentation.
4. (Related to the previous question.) For formats that support both cases, e.g. for TIFF, the documentation about the "else" case could be clearer: Quote: "If nonzero, and the file contains unassociated alpha, this will cause the reader to leave alpha unassociated [...]." What if the option is nonzero and the file contains associated alpha? Will alpha be converted to unassociated then or will the request be ignored? (The attribute documented further down seems to provide a way to figure that out, but at least a reference might be useful.)
5. I have/need data with unassociated alpha. I can do the conversion on my own data structures if need be, but is it possibly to convert from/to associated alpha during import/export using OIIO functionality (possibly highly optimized)? Does this require a detour via ImageBuf, or it possible to do that directly with the read/write methods on ImageInput/ImageOutput?
6. I guess the answer is "no" but just for completeness: Is there a way to switch the OIIO API to unassociated alpha and let OIIO do the necessary conversions under the hood (if necessary)? Just dreaming ... ;-)

[lgritz]

1. "oiio:UnassociatedAlpha" is not a documented token for "supports". ("null" seems to support it only because it returns true for supports() no matter what the token.)

   We can add such a token if you think that's helpful. I'm not exactly sure how it would be used given how we intended for the current functionality to work (I'll explain below). What would you think that supports("unassociatedalpha") means: "this format can store unassociated alpha", or "this reader can preserve unassociated alpha", or "this reader can force unassociated alpha"?

2. (& 3, 4) You are right, there are some discrepancies and areas where the docs can be improved. I'm going to create an issue from this discussion and try to make some improvements here.

So let me try to explain more clearly what the current feature is supposed to do (notwithstanding the fact that the docs may be less that crystal clear):

Some image file formats store only associated alpha, some (ick) only unassociated, and some can do either.

OIIO is supposed to convert everything to associated alpha when it reads. So the default behavior when encountering a file with unassociated alpha is to automatically convert to associated by multiplying the color channels by the alpha value. However...

Setting the input open hint "oiio:UnassociatedAlpha" to 1 means:

• If the file contains unassociated alpha,
  • DO NOT premultiply the values, leave it unassociated
  • Set the ImageSpec metadata "oiio:UnassociatedAlpha" to 1 to indicate that the data being passed back to the app is unassociated.
• If the file does not contain unassociated alpha, just proceed as normal. That means:
  • The values returned are premultiplied (as they were in the file)
  • No "oiio:UnassociatedAlpha" metadata is set.

So the way that I imagined this would work is that apps that didn't care about or didn't specifically want unassociated values would do nothing special, and as usual they would always be guaranteed to get associated data no matter what was in the file.

But apps that wanted unassociated alpha when it was available would (a) call open with the configuration hint "oiio:UnassociatedAlpha" = 1 ("give me unassociated if that's what's in the file"), (b) check the ImageSpec returned to see if the data is indeed unassociated (if the metadata has "oiio:UnassociatedAlpha" == 1 -- I mean the image's metadata, not the open configuration hint).

So you can get everything associated, or you can ask that unassociated data stay unassociated and tell you that it's unassociated, but currently there is no provision for forcing the data returned to the app to be unassociated even if it was stored in the file as associated. If you ask for unassociated to be preserved but the metadata tells you that you got associated (which should be because that's what was really in the file), then if you want unassociated you have to do the divide yourself.

I suppose we could alter the convention, so that there is a way to ask for everything to be converted unconditionally to unassociated. I'm not sure how easy that is to do in a central place, and I'm a little hesitant to compel every individual format reader to take on extra complexity to support unassociating alpha, particularly for file formats that disallow it.

[jreichel-nvidia]

Thanks for the explanations!

For input: I'm basically doing what you described in your explanation: requesting "oiio:UnassociatedAlpha" unconditionally, and then check the ImageSpec metadata. Does not really matter to me whether the plugin does not support it at all or whether the file contained associated alpha -- in the end I just need to know whether I still need to unpremultiply or not. Announcing the general capability for input plugins in "supports()" is not really required for that. (Back then I thought every option that is understood is also reported in supports(), but that does not seem to be the case.)

For output: My data is not premultiplied and if possible I would really like to avoid converting the data twice and introducing additional quantization errors (premultiplying on my side, and then unpremultiplying on the OIIO side for formats that requires unassociated alpha). So I guess the semantic I'm looking for is "this format supports unassociated alpha (either exclusively or optionally) and the OIIO plugin will pass it through unchanged(*) if set in the config". This seems to be the semantic that is implemented, just that there is no way to figure out which plugins support this. Contrary to input operations, I can't just set it unconditionally because output plugins that do not support it will then interpret the data that I pass in a different way ... So I have to go through the documentation and hardcode which plugins support it. Announcing that via "supports()" would be really useful.

(*) with respect to premultiplying/unpremultiplying, not float/uint8 conversion etc.

About forcing: this could be very interesting for users that need/have unassociated alpha (basically 6. in my original message), but I understand that this is contrary to the OIIO convention and a change of the current semantic for this option. I have now written code to do the conversions on my side if necessary.