GLES2 3D editor and game: meshes and viewports are rendered black (no light?)

[LinuxUserGD]

Godot version:
master (77e8947 )

OS/device including version:
Arch Linux (Antergos Deepin)

Issue description:

Minimal reproduction project:
Any 3D game

[qarmin]

Can't reproduce on Ubuntu.
Are you sure, that you don't change default_env.tres file?

[LinuxUserGD]

It also happens on old games I made (project, not binary) and with new projects with default cubes so it should be a bug in Godot or some missing driver. GLES3 displays everything correctly.

[qarmin]

Do you updated your graphics drivers recently?

[LinuxUserGD]

I installed dxvk today for playing games on wine, not sure if it could cause something like this.

[LinuxUserGD]

Strange thing is that this also happens for me on Windows wine Godot.

[clayjohn]

I can't reproduce either. Ubuntu 16.04 with Intel integrated drivers

[MightyPrinny]

it might be a problem with your project this doesn't happen to me either

[MightyPrinny]

this seems to happen when the sky contribution is 0 and you don't have any lights and the ambient light is black, but it doesn't affect the editor like in your screenshot so that's weird

[Skaruts]

I tried a build a few hours ago and this happened to me as well. Another build from a few days ago didn't.

Windows 7
ATI Radeon HD 4800 series

[Relintai]

This happens to me too, on both windows and linux across 3 different machines.

I haven't had the time to look into it yet. I'll do a regression test later when I'll have some time on my hands, because it only started happening with my most recent build.

[Skaruts]

Also, my viewport to a 3D scene was fullly black in the latest build I tried (the scene contains an Environment with background set (through code) to custom color black).

(I just realized this is because all the objects are rendered black, so they can't be seen against the black background)

[akien-mga]

Any errors in the terminal?

@clayjohn Sounds like redefining round() and co. might be problematic on some hardware in the end.

[Relintai]

No errors for me.

[Relintai]

Found it, it is caused by
47df673

edit: false positive

[akien-mga]

CC @raphael10241024

[clayjohn]

@Relintai Are you sure? None of the code changed in 47df673 should run on a desktop.

[Raphael2048]

Unable to see it both in my mac and windows pc, I'm still trying to fingure out how this happens, it looks confusing.

[ghost]

Yeah, this has also been happening to me. All the 3D meshes in my old GLES2 projects are rendered solid black.

Godot version: 3.2-dev 77e8947
OS: Windows 10 version 1903 x64
Graphics card: NVIDIA GeForce 980 Ti
Graphics driver: NVIDIA driver 430.64

Everything seems to work in 47df673... Not sure that's the culprit. I'll try bisecting in a moment.

[Relintai]

I just figured it out, that this only happens in release_debug, in debug it is not an issue, even on the current master.

On my linux machine reverting to one before the commit I linked fixed the issue, building on that commit, the issue appeared. I'll clean rebuild everything, to double check in a few minutes.

[ghost]

Finished bisecting. 8b4c538 is the culprit!

During my bisecting I discovered that 507e41e renders only ImmediateGeometry lines as black:

It's probably not the commit that broke it, but it's probably worth pointing out regardless.

@Relintai Oh uhh that's really weird. I should probably note that I was building release_debug builds while bisecting, but I don't know what that says.

[clayjohn]

@Relintai @Homer666 Doing a clean build may fix the problem! gles_builder.py doesn't rebuild shaders when only stdlib.glsl has changed. So if you did a partial build after 8b4c538 it may not have triggered a recompile and maybe thats what is causing this.

@LinuxUserGD Can you try a full recompile and see if that solves the issue?

edit: Alternatively, if you want to avoid a full recompile, just add a comment in drivers/gles2/shaders/scene.glsl and that will trigger a shader recompile.

[LinuxUserGD]

@clayjohn Which commit should I choose for the full recompile? I would do git reset --hard, git clean and then scons p=x11 target=release_debug

[clayjohn]

@LinuxUserGD Build from master. Instead of doing a full recompile just force gles_builder to rebuild shaders. You can do that by adding a comment to drivers/gles2/shaders/scene.glsl and drivers/gles2/shaders/canvas.glsl

[LinuxUserGD]

Got some error because I was compiling with lto. Will try a full compile tomorrow (sorry!).

[ghost]

@clayjohn I tried building after using git clean, and also tried building from a freshly-cloned upstream repo. Both don't result in any change, with either my old projects or a new one.

They were release_debug builds, mind you; debug builds have been working fine.

[Relintai]

The black immediate geometry stuff will require more testing. I'll continue tomorrow, with git cleans and scons cache cleans to find the proper culprit. It was 47df673 for me on linux, but without clearing caches, so this might not be the right one. If I find it, I'll open an another issue for that (or pr if I can fix it).

However for the black spatial meshes 8743da9 seems to be the culprit consistently, on both windows and linux.

A few more screenshots, because the original one really doesn't do this justice, it even confused me a little bit first:

Both are windows, release_debug, 64bit builds. Opening the same project.

They are built with vs2017 community, and I cleaned the project and removed scons cache before building them.

This is at 3256fbf:

This is at 8743da9:

The same happens on linux,

I've also tried adding lights, and textures, I get the same black box.

And on GLES3 everything works perfectly.

And no error logs in the console.

Also ImmediateGeometries created ingame are black too. (Haven't tested in-editor)

[clayjohn]

Okay, I can now confirm 3f25dde breaks rendering initially, but only for the lines and grid and then 8743da9 breaks everything else. However, debug builds are fine.

I do not understand why these commits have broken rendering in that way. There are no errors, so the shaders must be compiling fine. @akien-mga We may need to just revert for now and figure something else out...

[Relintai]

I managed to figure out how to get a shader compile error.

The key to it is having a WorldEnvironment in the Scene with fog enabled.

Minimal example project:
Test_bl.zip

Open the project, click on the MeshInstance Node in the scene, and in the mesh property create a new mesh, something that's not an empty ArrayMesh.

I've tried it with multiple builds, it errors every time.

Also, these are the errors I get:

shader_error.txt

[LinuxUserGD]

@Relintai I get some more errors when trying your minimal example project:

 drivers/gles2/shader_gles2.cpp:129 - SceneShaderGLES2: Fragment shader compilation failed:
0:1831(120): error: `light_direction' undeclared
0:1831(102): error: no matching function for call to `dot(vec3, error)'; candidates are:
0:1831(102): error:    float dot(float, float)
0:1831(102): error:    float dot(vec2, vec2)
0:1831(102): error:    float dot(vec3, vec3)
0:1831(102): error:    float dot(vec4, vec4)
0:1831(98): error: no matching function for call to `max(error, float)'; candidates are:
0:1831(98): error:    float max(float, float)
0:1831(98): error:    vec2 max(vec2, float)
0:1831(98): error:    vec3 max(vec3, float)
0:1831(98): error:    vec4 max(vec4, float)
0:1831(98): error:    vec2 max(vec2, vec2)
0:1831(98): error:    vec3 max(vec3, vec3)
0:1831(98): error:    vec4 max(vec4, vec4)
0:1831(98): error:    int max(int, int)
0:1831(98): error:    ivec2 max(ivec2, int)
0:1831(98): error:    ivec3 max(ivec3, int)
0:1831(98): error:    ivec4 max(ivec4, int)
0:1831(98): error:    ivec2 max(ivec2, ivec2)
0:1831(98): error:    ivec3 max(ivec3, ivec3)
0:1831(98): error:    ivec4 max(ivec4, ivec4)
0:1831(94): error: no matching function for call to `pow(error, float)'; candidates are:
0:1831(94): error:    float pow(float, float)
0:1831(94): error:    vec2 pow(vec2, vec2)
0:1831(94): error:    vec3 pow(vec3, vec3)
0:1831(94): error:    vec4 pow(vec4, vec4)
0:1831(69): error: operands to arithmetic operators must be numeric
0:1831(19): error: no matching function for call to `mix(vec3, vec3, error)'; candidates are:
0:1831(19): error:    float mix(float, float, float)
0:1831(19): error:    vec2 mix(vec2, vec2, float)
0:1831(19): error:    vec3 mix(vec3, vec3, float)
0:1831(19): error:    vec4 mix(vec4, vec4, float)
0:1831(19): error:    vec2 mix(vec2, vec2, vec2)
0:1831(19): error:    vec3 mix(vec3, vec3, vec3)
0:1831(19): error:    vec4 mix(vec4, vec4, vec4)
0:1847(35): warning: `fog_color' used uninitialized
0:1847(47): warning: `fog_color' used uninitialized

 drivers/gles2/shader_gles2.cpp:360 - Method/Function Failed, returning: __null
 drivers/gles2/shader_gles2.cpp:88 - Condition ' !version ' is true. returned: false
 drivers/gles2/shader_gles2.h:254 - Condition ' !version ' is true. returned: -1

[Relintai]

After seeing @LinuxUserGD 's error, out of curiosity I put a "#define USE_LIGHTING" into scene.glsl both under the [vertex], and under the [fragment] shader mark, and now it seems to work. Even without a WorldEnvironment.

[Relintai]

I think I found it, in shader_gles2.h the conditionals like USE_LIGHTNING is stored in conditional_version.version as bits.

Before those 2 commits the Conditionals array for the scene shader contained exactly 32 entries.

conditional_version.version is a uint32_t.

And now USE_LIGHTNING's index is 33rd in the array, so it just won't get set anymore.

edit: I've tested it since, by reordering the conditional arrays in the generated scene shader file (I moved the new stuff to the end by hand), doing it fixed both the ImmediateGeometry, and the spatial material issues, so this is the actual problem.

Also I've been trying to change the shader version code to uint64_t since then. It doesn't work yet.

[clayjohn]

@Relintai Great work! Keep us updated!

[clayjohn]

@Relintai I asked Reduz what to do, he explained that we can avoid creating extra code_versions if we use #if defined() instead of #ifdef. Accordingly, we can avoid that 32 conditions limit. Chaosus is making a PR now to fix this.

[Relintai]

Really cool! This solution is soo much simpler and better! Thanks!

[LinuxUserGD]

Thanks! After deleting all the *glsl.gen.h files, everything is working again 👍