[3.x] Add blend_premul_alpha support to scene shaders.

[jitspoe]

This was previously only supported by 2D shaders, but can be pretty useful in 3D as well. For example, I'm using it for lasers where I don't want to use pure additive blending, because they lose visibility with bright backgrounds, but a normal alpha blend doesn't look good, either, so I make them mostly additive with some slight alpha to ensure visibility on brighter areas.

I'm making this pull request directly into 3.2 because the gles3 renderer was removed and I can't make the pull request in master and have it merged back to 3.2. (Also, can't compile master due to some internal compiler error, so I'll have to figure that out and make a different pull request for master to support vulkan after I figure that out, if this goes through).

Bugsquad edit: This addresses godotengine/godot-proposals#3431 for 3.x.

[jitspoe]

Ah, interesting, compiler settings for the checks are stricter than what was used compiling locally. The blend modes enum is shared for the detail blend mode (which is not impacted by this). Also, seems the pull request took all my changes in that branch, so I need to rebase this, I guess? Still figuring my way around git.

[Calinou]

The 5 commits must be squashed together before this can be merged. See PR workflow for instructions 🙂

[jitspoe]

Ok, I tried to do that, but now I somehow have 2 mystery changelists that ended up in this branch that I didn't make. Time to re-rebase.

[t-mw]

This PR comes in handy for me because I'll be able to apply it to billboarded sprites, so thank you for that. But I also wanted to point out that this will create issues when used with the default spatial shader. For example:

1. Create a MeshInstance and assign a mesh
2. Assign a material with the transparent flag set and convert it to a ShaderMaterial
3. Reduce the alpha of the albedo color -> mesh becomes transparent 👍
4. Replace blend_mix with blend_premul_alpha -> mesh becomes opaque 👎

Multiplying rgb by alpha at https://github.com/godotengine/godot/blob/3.2/drivers/gles3/shaders/scene.glsl#L2181 is an approximate fix for that issue, but still leaves some differences in shading. Also, I'm not sure that 'fix' is compatible with textures with premultiplied alpha, since the rgb components would end up being multiplied twice by alpha.

[jitspoe]

Hmm, this seems to work for me. This is the shader code I ended up with:

shader_type spatial;
render_mode blend_premul_alpha,depth_draw_opaque,cull_back,diffuse_burley,specular_schlick_ggx;
uniform vec4 albedo : hint_color;
uniform sampler2D texture_albedo : hint_albedo;
uniform float specular;
uniform float metallic;
uniform float roughness : hint_range(0,1);
uniform float point_size : hint_range(0,128);
uniform vec3 uv1_scale;
uniform vec3 uv1_offset;
uniform vec3 uv2_scale;
uniform vec3 uv2_offset;


void vertex() {
	UV=UV*uv1_scale.xy+uv1_offset.xy;
}




void fragment() {
	vec2 base_uv = UV;
	vec4 albedo_tex = texture(texture_albedo,base_uv);
	ALBEDO = albedo.rgb * albedo_tex.rgb;
	METALLIC = metallic;
	ROUGHNESS = roughness;
	SPECULAR = specular;
	ALPHA = albedo.a * albedo_tex.a;
}

[t-mw]

I got it a bit wrong, the premultiplied mesh doesn't stay opaque, but there's a difference in shading. Here's a test scene where the materials on each sphere only differ by blend mode:
premultiplied_test.zip

Multiplying rgb by alpha at https://github.com/godotengine/godot/blob/3.2/drivers/gles3/shaders/scene.glsl#L2181 is an approximate fix for that issue, but still leaves some differences in shading. Also, I'm not sure that 'fix' is compatible with textures with premultiplied alpha, since the rgb components would end up being multiplied twice by alpha.

On second thoughts, this is the right fix. The differences in shading would go away once it's only applied to shaders using pre-multiplied blending - doh.

[t-mw]

But I think the albedo component from textures should probably be excluded from being multiplied for the reason mentioned.

Perhaps double-check with someone else before putting in extra work - the lighting model doesn't adapt to other blend modes either, so maybe it's ok like this.

[jitspoe]

The whole point of premul alpha is that you have to premultiply the alpha so you can control the visual behavior (ex: have some sort of hybrid between additive blending and alpha blending). You're suggesting that I multiply the alpha, which would make it just a normal alpha blend.

[t-mw]

Agreed, but the lighting model in the default shader outputs with regular alpha. So you'll get your texture with pre-multiplied alpha rendering correctly, but with potentially weird lighting, unless you go with the unshaded flag. That might be fine for now, since working around it introduces complexity. It's probably best if someone from the engine team reviews the current PR, I'm just another user.

[jitspoe]

Ah, I see what you're saying, now. I was using it for things like lasers, unshaded. Wonder if there's a case where you'd want this lit in practice. In any case, this is better than nothing. We can iterate as needed.

[mortarroad]

Something like this is commonly used for rendering fire + smoke in one pass.
The fire is emissive and added, while the smoke is alpha blended.

This of course causes issues if the material is not unshaded.
For that, we could do the following:
The lighting that has been computed from the ALBEDO, etc. is premultiplied with ALPHA automatically. This should yield exactly the same result, as if the blend mode was blend_mix.
But then, the EMISSION output is added after the premultiplication.

This would allow us to have a material that has an alpha-blended component that is lit, and an emissive component.

A better name for that would maybe be blend_mix_add.

[jitspoe]

This was automatically switched to 3.x, so I don't need to do anything, right?

[akien-mga]

Yeah it's all good, though you might want to rebase on latest 3.x to make sure that it still builds (and possibly that it still works) since the last update was a year ago.

@godotengine/rendering WDYT about the feature? It should be suitable for 3.4 if deemed useful.

[Calinou]

WDYT about the feature? It should be suitable for 3.4 if deemed useful.

Sounds good to me. Not only it's useful for various special effects in particles, it can also be used to avoid issues with alpha edges on 3D sprites.

This should also be forward-ported to master once it's merged.

[jitspoe]

I kind of feel like it's a bug or incomplete feature that blend_premul_alpha exists for 2D, but not 3D, but I went ahead and created a proposal here: godotengine/godot-proposals#3431

[ghost]

You probably also want to add in materials.cpp
BIND_ENUM_CONSTANT(BLEND_MODE_PMALPHA); and
ADD_PROPERTY(PropertyInfo(Variant::INT, "params_blend_mode", PROPERTY_HINT_ENUM, "Mix,Add,Sub,Mul,Premul"), "set_blend_mode", "get_blend_mode");
to make it show up in Spatial Material

Do you have a test project for 3.x ? Seems to compile and the code above seems to work but I don't know exactly what to test for.

[jitspoe]

Ok, finally got around to updating this. I've added the functionality to non-shader spatial materials as well, as suggested by @ghost.

Included a test project here:
test_premul_alpha.zip

Bottom row uses premul alpha, left side is shaded, right unshaded. Version with spatial materials and shaders (identical, but just to test). Mix and additive blending with and without alpha enabled are shown above.

Premul alpha is pretty much how all professional CG/VFX compositing is done, so I think it's a pretty important feature to have. Lets you combine solid and glowy things in one pass.
https://www.youtube.com/watch?v=XobSAXZaKJ8

[mortarroad]

If this situation shouldn't be possible, maybe we can print a warning for this case, so it will get noticed if it ever goes wrong.

[clayjohn]

Its not that it isn't possible, its just that the compiler insists that every case be represented in a switch statement. So this line of code tells the reader that we intentionally choose not to insert any code in the premul alpha case

[Calinou]

Technically, you could use premultiplied alpha for as a detail map blend mode, but it probably doesn't make much sense and wouldn't be used often.

[clayjohn]

Am I right in thinking that this mode requires the user to either use textures with pre-multiplied alpha or pre-multiply manually in the shader? If so, we need to accompany this with a documentation change to clearly say that. I hesitate with features that require either a deep background knowledge and that don't just work when enabled. If shader changes are required I am concerned that we will get a flood of users asking why the feature doesn't just work when enabled.

On that note, docs need to be regenerated and filled in to pass CI how to do so is described here https://docs.godotengine.org/en/stable/community/contributing/updating_the_class_reference.html

[clayjohn]

These are functionally equivalent. In either case source is multiplied by GL_ONE and destination is multiplied by GL_ONE_MINUS_SRC_ALPHA

[jitspoe]

That's what I thought, but I wasn't 100% sure, so I did it like this to be safe.

[clayjohn]

Its not that it isn't possible, its just that the compiler insists that every case be represented in a switch statement. So this line of code tells the reader that we intentionally choose not to insert any code in the premul alpha case

[jitspoe]

Am I right in thinking that this mode requires the user to either use textures with pre-multiplied alpha or pre-multiply manually in the shader?

Correct.

I hesitate with features that require either a deep background knowledge and that don't just work when enabled. If shader changes are required I am concerned that we will get a flood of users asking why the feature doesn't just work when enabled.

This is already a feature of Godot, but it's for 2D only. I'm simply bringing it to 3D for consistency (and it's a very useful feature for FX).

It's just another blend mode, so it should more or less "just work". Not terribly complicated. If people aren't using it with premul alpha textures, it'll look like additive blend mode or mix blend mode (or somewhere in between).

I'll add documentation as well. Also, I need this in Godot 4 for explosions, so I'll probably do another PR for that... I was going to say soon, but whenever that happens. :D

[jitspoe]

Updated PR to add documentation. I was going to go ahead and add support for premul alpha in the detail blending, but then realized that it would consume a bit in the MaterialKey, so I opted not to do that as there are only 5 bits left, and I doubt anybody would actually use this, so better to save them for more practical features.

[jitspoe]

Here's the Godot Engine 4 version: #68548

[fire]

#85609 was merged and I think @lawnjelly we can close as the pr doesn't seem to be enough interest, but it's salvageable.

[lawnjelly]

I do suspect if decisions have (finally 🙂 ) been made and #85609 was merged in 4.x, then that means an easy port or modification of this PR is more easily justifiable, so it might be good to leave this option open imo, unless someone else on rendering has different opinion.

Usually small backports are quite easy to justify.

[jitspoe]

This will probably be difficult to backport since the file structure for rendering stuff completely changed. I imagine it'd be easier to just tweak this CL. Can't remember what all was different between it and the 4.x version offhand.