Blender crashes during export of animated meshes

[EntitySelf]

Multiple Blender Versions
Blender 2.79b Master (f4dc9f9d68b) 2018-03-22 14:10
blender-2.79.0-git.642c315bae1-windows64

Both FLIP Fluid versions
FLIP FLUID 1.0.4a
FLIP FLUID 9.0.5.0_exp

Window 10 Pro
I9 9900K
NVIDIA 1080 TI
NVIDIA 1060 3G
32GB

Blender keeps crashing when I start baking the simulation, often after exporting animation data. I tried disabling "Generate motion blur vectors", "Enable Whitewater Simulation", disabled almost all other addons, recreated the scene, lowered and increased the resolution and more. But no luck...

Blend: https://entityself.com/files/blender/KEVA%20FLIPb.blend

One time I baked 200 frames but had to pause it, unfortunately, I cannot resume without Blender crashing. Maybe it has something to do with the number of animated objects I have?

I did finish two other projects using FLIP fluid without problems.

[EntitySelf]

More info, I get an EXCEPTION_ACCESS_VIOLATION (using 1.0.4a)

Error Log

[rlguy]

Hi EntitySelf,

Thanks for the report! I am sorry to hear that you are running into problems with this scene. I don't think the issue is that there are too many objects. The largest simulation I have run had around 6000 objects.

I was able to reproduce the crashes consistently and they seem to be caused by trying to access data that does not exist, but I am not sure of the reason for this. They would usually occur for me while exporting the animation or at the end of exporting the animation before simulation begins. Sometimes Blender would crash while I was just scrubbing through the timeline or playing back the rigid body simulation and this would also happen with FLIP Fluids addon disabled.

Sometimes we see .blend files behave like this with rigid body animations when we transfer the .blend file to another system, or save between different Blender versions. Sometimes it seemed that the .blend file was permanently broken so we ended up recreating the scene and no longer ran into issues, but this is a pain and not an ideal workaround.

I was able to get the simulation running more consistently by freeing the rigid body cache and then re-baking the rigid body simulation. Perhaps this may work for you?

Another workaround to create a more stable export could be to bake the simulation to keyframes and combine all of the obstacle pieces into a single mesh object. This would speed up the slow export process also. I am not very experienced in this area of Blender, so I am not sure if this is actually possible.

The most stable way to bake the simulation would be to Bake from the Command Line. I was actually able to get the simulation running with your original .blend file (without re-baking rigid body).

We are actually currently working on restructuring the simulation cache and export cache. Our goal is to create a much faster and more stable export process in the addon. I believe after these changes, this type of export problem may be solved. I am unable to determine when this will make it into a stable version of the addon at the moment.

And just an additional note: I noticed that the 'Hills' obstacle is non-manifold. The object will need to be modified to be manifold/solid to interact with the fluid. Right now, the fluid will just slip through the mesh.

My apologies that I cannot give a certain reason for why this is happening. I'll use this issue to document any other cases of this problem and update if any new information is found.

- Ryan

[Lewiatan45]

I have the exact same problem.
My animation is 11750 frames long and FLIP Fluids starts baking fluid at 8000th frame. I baked 500 frames then I had to stop baking. When I tried to resume baking process Blender crashes in 7 seconds during "Baking in progress... Initializing."

This issue is quite annoying because the estimated time for FLIP Fluid to finish the baking process for 11750 frames is 91 hours.

[rlguy]

Hi Lewiatan45,

Thanks for the report. I'll add a bit of an update for this issue.

If it is a Blender crash during the "Baking in progress... Initializing.", then it is likely caused by an 'Out of Memory' error. Very long animations can take a lot of memory if there are animated obstacles because each animation frame is loaded into RAM at the beginning of the simulation, which is a design flaw in our simulation system. We did not design the fluid simulator to handle very long animations since most fluid effect shots in commercial/vfx are in the range of a few seconds of simulation rather than minute+ long simulations.

The good news is that this will not be a problem in the next version. We are almost finished fixing this design flaw and it should be fully implemented during the upcoming week. The simulator will no longer load all animation at the start of the simulation and will only load the current frame to reduce memory usage for long simulations. The export process will also run quicker and require less mess data (about 2x quicker).

As for stability and crashes that happen during "Exporting Animated Meshes...", this seems to be a bug in Blender and we still need to narrow down what is causing this bug so that it can be reported to the Blender developers.

- Ryan

[Lewiatan45]

Hello Ryan,
Thanks for explaining ins and outs of the problem.
Yes, it does sound like serious design flow. Considering your addon it the only one on the market that allows baking fluids in a reasonable time in Blender, people would use it for longer animations too except for commercials. In my case, the object is being filled oup with dense fluid and then at one point it tips over and spill it on the ground. Due to density, the process is slow and want that fluid to remain on whatever surface it'd end up. I'm not entirely sure how Blender can run out of RAM considering I have 48 GB DDR4 3000Mhz of RAM, maybe Blender cannot use it efficiently.

I am happy to hear that the issue will be fixed next week and fingers crossed.

Lewiatan45

[rlguy]

Hi Lewiatan45,

If it is not an 'Out of Memory' error, it is possible that the crash is from an HDD/SDD timeout error when attempting to write/read the simulation export file. Right now all exported meshes and simulation settings are combined into a single file and if there are many animation frames, this file could grow large enough that it could trigger a timeout when writing/reading the file all at once. We are fixing this by writing many files instead of a single large file.

- Ryan

[EntitySelf]

I keep experiencing EXCEPTION_ACCESS_VIOLATION errors, even from the command line but less often. It also seems to help when I close all 3D views in Blender (might be a coincidence).

Blender 2.79 and 2.8
FLIP FLUID 9.0.5.3

[rlguy]

There are some problems with the Blender viewport that can result in EXCEPTION_ACCESS_VIOLATION when an addon tries to access mesh data. Closing 3d viewports can often help with stability.

An update on this issue: We have completely rewritten the animated mesh export code to workaround these Blender issues as well as fix some flaws in how the simulator handles animated meshes (mentioned in an earlier comment). We have been testing/using this fix for about 5 weeks now and I am very happy with the results of the changes. The changes seem to work around the EXCEPTION_ACCESS_VIOLATION errors and I no longer run into this problem.

I will include these changes in the next version update. At the moment we have not decided when to next update the addon. We are hoping to release the next stable version of the addon around May 15th since this is when the Blender developers will freeze making API changes in Blender 2.80. We may also need to release another experimental version between now and May 15th if there is a breaking API change in Blender 2.80.

- Ryan

[rlguy]

Update: we have decided to release another experimental build soon.

The fixes for the problems detailed in this issue has been testing very well. We will be including these fixes in the next experimental version (9.0.5.4) which will be available near the end of next week.