GLTF: Read material texture "texCoord" property on import

[aaronfranke]

Fixes #80858 and #93884 to the extent which is possible in the glTF code. Supersedes PR #80522.

Godot uses UV1 for the base color, normal, and metallic/roughness. It can also have occlusion and emission on either UV1 or UV2. The glTF spec allows any texture to use any UV map, so not everything in glTF is possible in Godot.

In the current master, the glTF import code does not read this property, and expects that every texture is on UV1. We can do better than that. With this PR, the glTF import code will do its best to consider the UV preferences of material textures, allowing for many more configurations such as color+normal+metallic on a different texCoord, or occlusion/emission on a separate texCoord. When the code encounters a configuration not supported by Godot, it will print a warning.

Here is the test file from #93884. Left is current master, right is with this PR.

Here is the test file from #80858.

This PR also handles the case of exporting materials with emission and/or occlusion on UV2, while previously they always exported as if they were on UV1. The code for exporting is much more straightforward than importing, the only changes are in the _serialize_materials function.

While not all glTF files will have their UV maps perfectly imported into Godot, this should be enough to ensure that exporting from Godot and importing back into Godot will preserve any UV2 map usage.

Also, I renamed some local variables for readability (ex: p -> mesh_prim_dict, sgm -> spec_gloss_ext_dict).

I'm labeling this as an enhancement because support for UV2 in glTF is a new feature, but arguably it's a bug because this is a case of Godot not following the glTF specification.

[fire]

I am in favour of the proposal. Did not technical review yet.

[fire]

@aaronfranke what's a prose version of the algorithm for picking the coordinates?

[aaronfranke]

@fire The prose version is that we want to read what's in the file, except that to fit within Godot's material system, we can only read at most two UVs, and the second UV can only be used for certain things. Therefore, we prefer reading the most important UV maps first to provide the best mapping, starting with the base color / albedo texture, and continuing on to less important UV maps.

[fire]

So like a greedy algorithm that has 2 slots and from the inputs we fit the base color / albedo texture to the first slot and then the others.

[lyuma]

I am not personally in favor of this proposal as is. In short, this creates a few hardcoded rules which impact the mesh depending on a series of conditions, and this logic will never be able to change without breaking compatibility.

Most importantly, I couldn't find the import option to control this. Having an option to control this behavior is an absolute necessity. For example, the change may alter the mesh channels of content already imported into Godot, perhaps using a custom ,tres material.

The thing I am worried about is that this unconditionally changes the order of the UV channels, based on materials which may even be replaced or swapped during the import process.

In my experience, it is common for the UV channels to have semantic meaning, for example TEXCOORD_1 is often used for lightmapping while TEXCOORD_0 is conventionally used for albedo. This flips it around, so that the material attached to a particular mesh determines the UV channels to more closely match with this semantic. While there is some sense in this, it means that the mesh data will be drastically affected by the presence of various settings on the materials.

For example, if the glTF is exported without materials, or perhaps if the user changes some of their material nodes in Blender before export, the order of mesh channels in Godot will be unexpectedly flipped, and the cause will be difficult to diagnose, for example depending on the presence of albedo or emission in blender nodes which may not even be used in Godot.

Still, I might be swayed to be okay with this as an import option.

[aaronfranke]

@lyuma Currently, meshes are plainly broken on import if they are using different UV maps from what you mention is the common convention. I'm not inclined to keep adding more and more import options to preserve broken behavior. I prefer to just have the correct behavior be the only option.

If users are expecting UV maps to be preserved as-is, that is already not the case. Godot doesn't support more than 2 UV maps, so any others are discarded. Furthermore, currently UV2 is always unused on import, and can only be used with custom materials, this PR fixes that.

In my opinion, the primary focus should be to ensure that the glTF's intended appearance is correct in Godot, not in keeping texCoord 0 mapped to Godot's UV1. I don't believe it is worth supporting import of a file where the user explicitly wants Godot to use the wrong UV map. To me it just seems like nonsense behavior that there would be an option that says "yes, import me with the wrong visuals" for the sake of always using the first UV map as the used one. It's better to just always use the UV map that the glTF file says is the correct UV map.

As for a user exporting a glTF without materials and getting different behavior - I don't think it's reasonable to expect the same behavior between different files. Actually, in fact, it may be good that this is the case. This way, if users want the behavior you want, where texCoord 0 is imported as UV1, they can just not export materials. If the user is not putting materials in the glTF file, then they need to specify them in Godot anyway if they want materials.

[huwpascoe]

I agree with what this PR is doing, the GLTF should be visually represented in engine as closely as possible, even if that means there are no guarantees of how data is mapped.

The thing I am worried about is that this unconditionally changes the order of the UV channels, based on materials which may even be replaced or swapped during the import process.

Yes, sometimes you do want explicit control over how things are mapped: #97756.
Perhaps rather than a script extension, it could be fully realized as a property group to override default behavior?

[aaronfranke]

Updated to account for PR #94165, supporting animated UV maps on textures on both import and export to the best extent possible with Godot's material system. I tested UV map animation on the red chair and it works, and also tested the animation pointer test file again just in case and that also works.

[fire]

I would ideally want to see something like #96748 (comment), do you think it's possible for us to do in this or another pr?

[aaronfranke]

@fire There is PR #97756 from @huwpascoe which implements the ability to import custom attributes. That way users can use that system to preserve a specific UV map, separate from the material's UV1/UV2 maps.

[huwpascoe]

Perhaps rather than a script extension, it could be fully realized as a property group to override default behavior?

I've determined that the UI approach is just too much of a mess for what is a technical feature. So I'll be moving forward with the original plan to add the method to DocumentExtension.

How do you want to coordinate this @aaronfranke? This PR is older so I'd probably say get this merged first and then I can add the _remap overrides where needed?

[huwpascoe]

Works as expected, both import and export.

The warning messages are a very nice addition and explains clearly why an import fails.

I couldn't find any problems with the source. 👍

[lyuma]

There was some opposition to this change in the rendering meeting, and I just wanted to comment further on this and also what I think is a better path forward.

First, one of the reasons we are encountering so much trouble handling texcoords is due to limitations on the standard material: basically, the standard material only permits certain texture maps on UV and UV2, and there are no other combinations... Therefore, this change is forced to remap texcoord channels to 0 and 1 so that they can be correctly assigned to the material.

I have some questions how common this issue really is, and it sounds more like making compatibility with marketplace assets than supporting a workflow for game art.

With that, I see two diverging usecases for the asset importer (And accordingly perhaps an import setting or some way to tell Godot which user you are):

1. For game art designed with import to Godot in mind, this content will already be designed to support StandardMaterial and will not want UVs remapped.
2. For marketplace assets or when full compatibility with glTF is desired, rather than conforming to Godot material conventions.

So given that description, for usecase 2, we have the goal of trying to render a glTF asset as accurately as possible, but we are not necessarily required to use StandardMaterial to do so. In this case, it might interesting to create material nodes with VisualShader that best mimics the visual representation in glTF, similar to how Blender works.

The downside of this is it means it will require more complexity to export the material, but it should be possible given that Blender is also able to export material graphs to glTF.

This PR is more like a baking approach where instead of creating a custom material graph, we are reordering UVs or baking textures to fit on the first two UV channels, which will work for some glTF materials but not in all cases. As I said before, I can be swayed if there is an import option, but this approach still feels a little bit messy: it will improve render compatibility with some glTF files but not 100%.

I think it might be worth considering a baking approach where when multiple texcoords are used, Godot can bake them to a single texcoord channel, which can make it look correct (although maybe a bit lower resolution), and this will also improve performance on mobile, which struggles when multiple UVs are in use at once.

And finally, the approach in godotengine/godot-proposals#10892 could be combined with a way for the user to manually select custom channels or UVs. Though there's still a problem that the StandardMaterial is too limited to use CUSTOM for textures.

[tlobig]

as for the question whether this is an unusual request: no. I'm currently trying to establish an art pipeline from Blender to Godot for a VR game where I need lightmaps as SDFGI is very broken for VR :-/ - LightmapGI doesn't have the same quality as what I can do with a Blender addon and requires it's own painful workaround.

I'm a bit stuck as with Blender I get 2 UV maps and Godot doesn't load these properly as standard material:

I approximated what it might look like hand fiddling a visual shader for each of these three objects (disregard overall quality, important thing for me was the shadow of the small cube)

but this is not suitable for an art pipeline. I will see if I can change the Blender side of things to merge everything into the diffuse texture, but this also means I'm less versatile with only one UV map.

so I was quite pumped to see there is a PR fixing MY issue - lo and behold, it does fix it:

Therefore I would love to see this PR or a version of it to make it to 4.4

[lyuma]

@tlobig I'd like to understand more on how the exported file looks and how the blender nodes are set up, to understand better how this change helps.

Since you mention using Blender to create a baked lightmap, is it done through multiplying two diffuse textures? Or is the albedo being baked together with the lighting to a second diffuse texture?

My understanding is with this change, Godot will still only pick a single UV, but it might pick UV2 instead of UV1. Even if it looks better to you, I suspect there is still something it's getting wrong. (Or if not, it should be possible to replicate this on vanilla godot by swapping the order of the UV maps in Blender).

[tlobig]

@lyuma good instincts. I used the lightmap addon "The Lightmapper" (https://github.com/Naxela/The_Lightmapper), and it generates a second UV map automagically and then sets up a texture node using this uv map as input to mix (muliply) the diffuse color. The addon is supposed to be helping setting things up for game engines, so not just for Blender. Yet the standard material is too limited and we would need more involved import options to deal with the problem.

Simply baking (everything to diffuse/albedo) in Blender works, though the result is not as nice as it could be.
That's the TL;DR

Here is a minimal scene, I added the baked result and what the lightmapper generates (combined light (direct+indirect) + AO)
minimal_blender_scene.zip

red cube's color is lost in conversion, bottom cube has no material before the lightmapper does it's thing and the textured cube looses the texture after import to Godot (this PR)

I might be missing a few of the finer points.