OSError: [Errno 22] Invalid argument when simulation contains large amount of fluid/particles

[gogobd]

System Information

Blender Version (including hash): 2.79b f4dc9f9
FLIP Fluids Version: 1.0.6 Stable 10-OCT-2019
Operating System: macOS High Sierra 10.13.6
CPU: 3,7 GHz Quad-Core Intel Xenon E5
GFX: AMD FirePro D300 2048 MB
RAM: 16GB 1866 Mhz DDR3

Describe the bug

It's odd, I got errors rendering a scene with 500x500x500 particles. To reproduce I tried the best machine I could use and I tracked the problem down to

398  	def __write_save_state_file_data(file_data_path, data):
399  	    import pdb;pdb.set_trace()
400  ->	    with open(file_data_path, 'wb') as f:
401  	        f.write(data)

(I added a breakpoint.) I can write something to the file (path is ok)

(Pdb) open(file_data_path, 'wb').write(b"hello")
5

But there's an OSError when the data should be written

(Pdb) open(file_data_path, 'wb').write(data)
*** OSError: [Errno 22] Invalid argument

I see that data is large:

(Pdb) p len(data)
3903363120

But my filesystem is ok with large files like that:

$ python
Python 3.7.3 (default, Mar 27 2019, 16:54:48) 
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open('test', 'wb')
>>> f.seek(3903363120)
3903363120
>>> f.write(b".")
1
>>> f.close()
>>> exit()
$ ls -lah test
-rw-r--r--  1 bernhard  2074541850   3.6G Oct 25 13:59 test
$ stat test
16777220 8647011368 -rw-r--r-- 1 bernhard (2074541850 0 3903363121 "Oct 25 13:59:25 2019" "Oct 25 13:59:58 2019" "Oct 25 13:59:58 2019" "Oct 25 13:59:25 2019" 4096 8 0 test
$ 

To Reproduce

I created a .blend file, just adding a 8x8x8 cube and an 8x8x8 cone, made the cube the domain and the cone is fluid. Bake settings Grid Resolution: 500/15, Surface Mesh Subdivisions: 1; I have attached a.blend file (zipped) that can help reproduce the problem: break.blend.zip

Expected Behaviour

Bake goes well

Actual Behaviour

Crashes with OSError after generating the surface mash. Output:

*** Frame Timing Stats ***

Update Obstacle Objects                27.913s     3.2%  ||
Update Liquid Level Set                 0.448s     0.1%  |
Advect Velocity Field                   1.807s     0.2%  |
Save Velocity Field                     5.554s     0.6%  |
Calculate Surface Curvature             0.000s     0.0%  |
Apply Body Forces                       1.390s     0.2%  |
Apply Viscosity                         0.000s     0.0%  |
Solve Pressure System                  12.427s     1.4%  |
Constrain Velocity Fields               3.824s     0.4%  |
Simulate Diffuse Material               0.000s     0.0%  |
Update Sheet Seeding                    0.000s     0.0%  |
Update Marker Particle Velocities       0.000s     0.0%  |
Delete Saved Velocity Field             0.105s     0.0%  |
Advance Marker Particles                0.390s     0.0%  |
Update Fluid Objects                  114.585s    13.0%  ||||||||
Output Simulation Data                 14.207s     1.6%  |
Generate Surface Mesh                 695.672s    79.2%  ||||||||||||||||||||||||||||||||||||||||||||||||

Frame Time:   878.321
Total Time:   878.321

Traceback (most recent call last):
  File "/Users///Library/Application Support/Blender/2.79/scripts/addons/flip_fluids_addon/bake.py", line 1729, in bake
    __run_simulation(fluidsim, data, cache_directory, bakedata)
  File "/Users///Library/Application Support/Blender/2.79/scripts/addons/flip_fluids_addon/bake.py", line 1667, in __run_simulation
    __write_simulation_output(domain, fluidsim, blender_frameno, cache_directory)
  File "/Users///Library/Application Support/Blender/2.79/scripts/addons/flip_fluids_addon/bake.py", line 1628, in __write_simulation_output
    __write_autosave_data(domain_data, cache_directory, fluidsim, frameno)
  File "/Users///Library/Application Support/Blender/2.79/scripts/addons/flip_fluids_addon/bake.py", line 1544, in __write_autosave_data
    __write_save_state_file_data(position_data_path, data)
  File "/Users///Library/Application Support/Blender/2.79/scripts/addons/flip_fluids_addon/bake.py", line 400, in __write_save_state_file_data
    f.write(data)
OSError: [Errno 22] Invalid argument

Screenshots

N/A

[gogobd]

The same problem does not occur at a resolution setting of 320. Maybe it's just a memory issue?

[rlguy]

Hi gogobd,

Thanks for the report. I would guess that this error was caused by an out of memory issue. I was not able to test this file today as I am working from a laptop with 8GB RAM, but can test after the weekend.

A cube shaped domain has quite a lot of voxels. 500x500x500 = 125 million voxels which will require quite a bit of RAM and processing time. An adjustment from 320 -> 500 resolution requires about 4x times the memory. A cone takes up roughly 25% of the domain volume, so that would be about 32 million voxels of fluid. Each fluid voxel requires 8 fluid particles, so in total I estimate about 260 million fluid particles! That is a very large amount of fluid to simulate and when the simulator was designed, I did not have a use-case this large in mind due to the very large simulation times that it would require. At the time I imagined 50 million particles to be an upper limit of what would be reasonable to simulate.

Even if you were able to successfully write the file, you would likely run into errors further on in the simulation. Errors related to too much fluid can start at around 100 million particles and depends on the OS or system setup. The errors are related to a limit in the amount of data we can transfer between the simulation engine and the Blender addon at a time. As well as issues related to handling large numbers within the simulator (integer overflow). This is a design issue and can certainly be fixed in a future release and should not be too difficult to fix.

However, a fix will require changing the structure of the simulation cache and this will break compatibility with caches created in older version of the addon. There is another fix I would like to add that will also break cache compatibility (#462). I would prefer not to break the cache in a minor experimental release so this fix may need to wait until the next full release of a FLIP Fluids version.

- Ryan

[rlguy]

This issue has now been fixed and will be included in the next release (date unknown). This fix will not end up breaking compatibility for older caches.