Unable to download HDF5 file when invoking Tidy3D from AWS Lambda function

[rashidnhm]

Describe the bug
Tidy3D is unable to download HDF5 in intermediate location as directories in AWS Lambda function are read-only by default.

In previous versions of tidy3d, the HDF5 was downloaded directly to the to_file which the user could provide. See Here.

However, in v2.4.0rc2, this behavior is different. The HDF5 file is downloaded to an intermediate location before being copied over to the user provided to_file location. See Here

This causes an error when attempting to do this on an AWS Lambda function since Tidy3D does not have write access to the directory.

To Reproduce
Steps to reproduce the behavior:

• Invoke tidy3d.web.simulation_task.get_simulation_json from any lambda function

Expected behavior
Since the intermediate file is required for decompression, the expected behaviour would be to allow the user to pass the location of the intermediate file as a function parameter, and having a default if the parameter is not set by the user.

The intermediate file location parameter can be provided by the user when invoking this function .. then propagated down by Tidy3d to the other required functions.

Screenshots
Can be provided if needed.

Additional context
Let me know if any further context is required.

[momchil-flex]

Thanks! We are on it!

[momchil-flex]

We are planning to use a tempfile, which we already do in other places of the code, including uploading a task - so it seems to work fine on lambda. I would just suggest that you make sure you have enough temp storage on lambda, by default it seems to be 512MB which should be enough (output data does not go through temp file, only simulation definition files do), but keep in mind to expand if needed.

#1099

[momchil-flex]

Fixed in #1100