Godot crashes or hangs very frequently when trying to bake lightmaps

[jocamar]

Godot version:
3.3.1-stable

OS/device including version:
Windows 10, GeForce RTX2070 MaxQ, i7-10875h @ 2.3Ghz

Issue description:
Godot frequently crashes or hangs when trying to bake lightmaps. When it crashes it just closes the engine and doesn't print anything to the terminal, when it hangs it's always at "Preparing data structures". The scene I'm running is fairly straightforward (the scene I was trying this on at the start is very complex but I started deleting stuff to see if complexity was the issue) with just 4 meshes all marked to be used in baked lighting and with UV2s generated by Godot. I've tried to wait for a long time but the program never continues, it just stays at that state taking up 100% of the CPU.

Weirdly, I got it to a point where if I remove the "use in baked light" checkbox from one of the meshes it almost always works and gets past the "Preparing data structures" stage almost instantly, but if I set the mesh to be used again it always hangs of crashes.

Here are my settings for the baked lightmap:

Steps to reproduce:
I just run created a baked lightmap node, made it envelop my scene and set the settings as the screenshot above and try to run it on this simple scene with 4 meshes. The result is it almost always hangs or crashes.

I don't know if this is a regression or not as I had never used baked lightmaps before this, but it is certainly very annoying as I was hoping to use them to optimize the lighting in my fairly large scene.

[qarmin]

Please post here minimal project, to be sure that we test same scene which crashes for you.

[stilobique]

Godot version:
3.3.1-stable

OS/device including version:
Windows 10, GeForce 1080 i7-3770k@ 3.5Ghz
Linux Manjaro -- Intel® Core™ i7-4790K CPU @ 4.00GHz × 8 -- GeForce GTX 1070/PCIe/SSE2

Switch with the last release, i'm get the issue.
On Windows 10, i'm porte my previous project and get a crash or a freeze to "Prepare data" step.

Try on a desktop computer and a laptop, W10 build 19042.985.
Try with a new Lightmap node and same way. Crash and freeze.
Try a new project with a simple asset import and a node Bake, get a freeze and crash.
Try this project on a linux OS and no crash or issue.

What i can do to give more debug information ?

[JFonS]

@akien-mga looks like we may have re-introduced #45097 with the switch to embree-aarch64?

I will work on switching back to the latest Embree to see if that helps.

@jocamar @stilobique Can you try running on 3.3-stable and see if that works better? It would help us narrow down if it was a regression or a previously unnoticed bug.

[stilobique]

[...]
@jocamar @stilobique Can you try running on 3.3-stable and see if that works better? It would help us narrow down if it was a regression or a previously unnoticed bug.

Yes, it's work perfectly for me with this package.

[akien-mga]

@akien-mga looks like we may have re-introduced #45097 with the switch to embree-aarch64?

Damn :| That's why I always require that we have patch files that fully describe the changes we made to upstream code to make sure we reapply them on update.

This change was lost in the embree-aarch64 update:
https://github.com/godotengine/godot/pull/45380/files#diff-9a5a19d82fa76b17207468d6bfe5f1edd1436ff39c2dea7df240b26d109f6c38R94

[JFonS]

@stilobique Cool, thanks for testing! That means something broke during our switch to embree-aarch64.

@akien-mga From what I see some of our fixes to Embree were not tracked in the patches file, so they got lost during the refactor. I'm working on upgrading to the latest Embree but this time making sure to create a full diff and keep all the fixes.

[akien-mga]

@jocamar @stilobique Could you test either of those binaries to see if they solve the issue?

https://downloads.tuxfamily.org/godotengine/testing/Godot_v3.3.1-stable_win64.exe-fix48822.zip
https://downloads.tuxfamily.org/godotengine/testing/Godot_v3.3.1-stable_win32.exe-fix48822.zip

They're built from 3.3.1-stable + this hotfix:

diff --git a/thirdparty/embree/common/sys/platform.h b/thirdparty/embree/common/sys/platform.h
index 737f14aa6e..e3be41a27a 100644
--- a/thirdparty/embree/common/sys/platform.h
+++ b/thirdparty/embree/common/sys/platform.h
@@ -91,7 +91,7 @@
 #define dll_import
 #endif
 
-#ifdef __WIN32__
+#if defined(__WIN32__) && !defined(__MINGW32__)
 #if !defined(__noinline)
 #define __noinline             __declspec(noinline)
 #endif

It only affects the editor build, so if that works, feel free to use those binaries together with the official 3.3.1-stable export templates, they should work fine.

[jocamar]

That build seems to work, or at least it no longer crashed or hanged and is in the process of baking the shadows for my larger scene. I did get some slowdowns when navigating the scene before I began baking the shadows, but that could just be my impression or that it's a debug build or something. Regardless the main problem seems fixed.

[akien-mga]

The slowdown is likely something that would be reproducible in the first 3.3.1-stable builds, the only difference between the original builds and the ones I linked above is that fix for the MinGW crash: #48822 (comment)

[akien-mga]

I made a test build for all platforms of #48967 which upgrades Embree to 3.13.0, and should also solve this issue: #48967 (comment)

If anyone affected by this issue could give it a try to confirm that it works properly, that would be great :)

[akien-mga]

For users affected by this issue who need a hotfix, I promoted my test builds as an official fix, uploaded here:

https://downloads.tuxfamily.org/godotengine/3.3.1/Godot_v3.3.1-stable_win64.exe-fix48822.zip
https://downloads.tuxfamily.org/godotengine/3.3.1/Godot_v3.3.1-stable_win32.exe-fix48822.zip

They're the exact same version as 3.3.1-stable with #48888 applied as a hotfix, which fixes the crash issue when 3.3.1-stable is built with MinGW. You can use them in production just like the original builds.

[stilobique]

For users affected by this issue who need a hotfix, I promoted my test builds as an official fix, uploaded here:

https://downloads.tuxfamily.org/godotengine/3.3.1/Godot_v3.3.1-stable_win64.exe-fix48822.zip
https://downloads.tuxfamily.org/godotengine/3.3.1/Godot_v3.3.1-stable_win32.exe-fix48822.zip

They're the exact same version as 3.3.1-stable with #48888 applied as a hotfix, which fixes the crash issue when 3.3.1-stable is built with MinGW. You can use them in production just like the original builds.

I'm test with your package and it's ok. It's work. a new official release are planned or this patch is planned with the 3.4 release ? I'm work with a small team, i need to know how to distributed the fix.
Thanks to everyone ;) .

[akien-mga]

a new official release are planned or this patch is planned with the 3.4 release ? I'm work with a small team, i need to know how to distributed the fix.

A fix will be included in 3.3.2 soon, I'm just not sure yet whether it will be the minimal fix in the above build, or a full embree upgrade from #48888.

Until then, you can use this 3.3.1 "fix48822" build in production and share it with other team members who need to bake lightmaps.