Fix MATReader when loading one Storage and/or zero DC-lines

[danielolsen]

Purpose

Ensure that grids loaded from matfiles are built just like fresh grids, to aid in grid equality comparisons.

What the code is doing

• In abstract_grid.py: add columns to storage_template to match what's in the matfiles.
• In scenario_grid.py:
  • We first check for the iess attribute of mpc, triggering the AttributeError if there's no storage, and if it is there then we check whether it is an int (which a 1x1 array for a single storage gets squeezed to) before trying to check its shape. Previously, a single storage would trigger an AttributeError when trying to access .shape, so we would think there is no storage when there is actually one, so we don't populate the storage table entries as appropriate.
  • Similarly, when there is a single storage device, we need to re-expand the integer entry for each column of the StorageData table, since otherwise they don't get added at all (pandas thinks we're trying to re-assign all the values of that column to the value instead, and it does, but there are zero rows, so it doesn't do anything).
  • We populate the storage genfuel array as well (could be ignored, but this could let us specify different types of storage in the future)
  • When there are no DC lines, we still build an empty dataframe with all the right columns.
• In helpers.py: we expand the columns of the dcline dataframe always, not just when it's nonempty (since now it always has the right column names, even if the rows are empty)

Testing

Together with #362, we can successfully reproduce the grid for Scenario 1712 in the create state, and equality comparison returns True:

>>> from powersimdata.scenario.scenario import Scenario
>>> old_grid = Scenario(1712).state.get_grid()
Transferring ScenarioList.csv from server
100%|#######################################| 235k/235k [00:00<00:00, 1.67Mb/s]
Transferring ExecuteList.csv from server
100%|######################################| 21.0k/21.0k [00:00<00:00, 309kb/s]
SCENARIO: test | new_bus

--> State
analyze
--> Loading grid
Loading bus
Loading plant
Loading heat_rate_curve
Loading gencost_before
Loading gencost_after
Loading branch
Loading sub
Loading bus2sub
--> Loading ct
>>> new_scenario = Scenario('')
>>> new_scenario.state.set_builder(["Texas"])
Reading bus.csv
Reading plant.csv
Reading gencost.csv
Reading branch.csv
Reading dcline.csv
Reading sub.csv
Reading bus2sub.csv
Reading zone.csv
Transferring ScenarioList.csv from server
100%|#######################################| 235k/235k [00:00<00:00, 1.48Mb/s]
--> Summary
# Existing study
test | base | Anchor | Julia
# Available profiles
demand: ercot
hydro: v1 | v2
solar: v2 | v4.1
wind: v1 | v2 | v5.1 | v5
>>> new_scenario.state.builder.change_table.add_bus([{"lat": 30, "lon": -95, "zone_id": 308}])
>>> new_scenario.state.builder.change_table.add_plant([{"type": "wind", "bus_id": 3008161, "Pmax": 400}])
>>> new_scenario.state.builder.change_table.add_branch([{"from_bus_id": 3008160, "to_bus_id": 3008161, "capacity": 300}])
>>> new_scenario.state.builder.change_table.add_storage_capacity({3008161: 100})
>>> old_grid == new_scenario.state.get_grid()
True

Time estimate

15-30 minutes. The code changes themselves are minor, but they are part of a complex chain of objects.

[BainanXia]

mpc.iess.shape[0] works for sure. Does mpc_iess.shape[0] work here, if so, maybe use mpc_iess instead to be consistent.

[danielolsen]

Good call, I think that was my original intention.

[danielolsen]

Done.

[rouille]

I am not not sure I understand the purpose of mpc_iess = mpc.iess. Am I missing something?

[danielolsen]

I wanted to separate the potential AttributeError when mpc doesn't have .iess (no storage) from the potential AttributeError when mpc.iess doesn't have .shape (one storage, so mpc.iess gets squeezed to an int). But with the separate int check before we access .shape, I think this can be simplified.
From:

try:
    # The next line will fail if no iess attribute (index energy storage system)
    mpc_iess = mpc.iess
    # Since we use the 'squeeze_me' param, 1 storage -> an int, not an array
    if isinstance(mpc_iess, int):
        n_storage = 1
    else:
        n_storage = mpc_iess.shape[0]
except AttributeError:
    n_storage = 0

To:

try:
    # The next line will fail if no iess attribute (index energy storage system)
    # Since we use the 'squeeze_me' param, 1 storage -> an int, not an array
    if isinstance(mpc.iess, int):
        n_storage = 1
    else:
        n_storage = mpc.iess.shape[0]
except AttributeError:
    n_storage = 0

[danielolsen]

Or even:

try:
    # The next line will fail if no iess attribute (index energy storage system)
    # Since we use the 'squeeze_me' param, 1 storage -> an int, not an array
    n_storage = 1 if isinstance(mpc.iess, int) else mpc.iess.shape[0]
except AttributeError:
    n_storage = 0

[BainanXia]

All the changes follow our investigation. Thanks for the verification.

[rouille]

Looks good