add internal construct smatrix

[dmarek-flex]

I find for development work it is handy to have a _construct_smatrix version that accepts an arbitrary batch_data. This allows us to run the simulations once and save/load the batch_data to/from file. I also fixed some docstring links that were broken.

Additional question, @tylerflex is there a simple way to change how the BatchData is created and read from so that it is easier to switch between:

1. Web version that uses Batch.run
2. local backend solver using run_sim
3. loading from file

I find when I am developing I always have some code that is needed to change between these methods for generating BatchData.

[tylerflex]

PR looks good

About the switching between run functions, I think this might be a problem we had recently with the inverse design plugin. I dont know if @momchil-flex has some insight on that. It seems like something we should think about. Off the top of my head I don't have a great idea besides maybe setting some kind of tidy3d.config variable to switch between web / local / etc?

[momchil-flex]

About the switching between run functions, I think this might be a problem we had recently with the inverse design plugin. I dont know if @momchil-flex has some insight on that. It seems like something we should think about. Off the top of my head I don't have a great idea besides maybe setting some kind of tidy3d.config variable to switch between web / local / etc?

Yeah this would certainly be nice. So far I've been doing it by monkeypatching (in on-prem builds). For example you should be able to do something like

# patch_run.py
from tidy3d import web
from tidy3d_backend run import run_sim

def run(simulation, task_name):
    return run_sim(simulation)

web.run = run

Then in your other scripts replace from tidy3d import web with from patch_run import web. Obviously this is a quick and dirty suggestion. But if we support it as a proper config, it would probably still be done through some form of monkeypatching? Just it needs to be considered carefully.

Regarding loading from files: isn't this basically equivalent to implementing local caching? I don't see how else it would work in a convenient way, i.e. we probably don't want to expect the user to pass the simulation file names (if that was the case they might as well just use from_file instead of run). I know @lucas-flexcompute and @yaugenst have been discussing local caching in Tidy3D.

[lucas-flexcompute]

Then in your other scripts replace from tidy3d import web with from patch_run import web.

You don't even need to to that. Just importing patch_run is enough. The underlying web module is the same object. For example:

# File patch_run.py
from tidy3d import web

web.run = lambda *a, **b: print("NOPE")

Just importing it works:

In [1]: import tidy3d as td

In [2]: import patch_run

In [3]: td.web.run("Please?")
NOPE

[weiliangjin2021]

Then in your other scripts replace from tidy3d import web with from patch_run import web.

You don't even need to to that. Just importing patch_run is enough. The underlying web module is the same object. For example:

Nice approach, and it looks you don't even need to write that in a separate patch_run file.

[dmarek-flex]

Thanks for the ideas everyone, seems like there are fairly simple ways to do what I want! +1 for local caching too, that would be useful!