Issue 450 api option to customize which price and power sensors to ta…

…ke into account for scheduling (#451) Introduce three optional fields to the sensors/(id)/schedules/trigger endpoint to allow further customization of optimization context: - consumption-price-sensor: prices upward deviations from previous zero-consumption commitments - production-price-sensor: prices downward deviations from previous zero-consumption commitments - inflexible-device-sensors: their power forecasts are included in the aggregate load * Add optional API fields to specify which price sensors to use for consumption and feed-in Signed-off-by: F.N. Claessen <felix@seita.nl> * Rename API fields Signed-off-by: F.N. Claessen <felix@seita.nl> * Add optional API field to specify which inflexible device sensors to include in the aggregate power flow Signed-off-by: F.N. Claessen <felix@seita.nl> * Add test data for inflexible device Signed-off-by: F.N. Claessen <felix@seita.nl> * Add test case for scheduling a battery including an inflexible device Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix boundary conditions for including inflexible devices Signed-off-by: F.N. Claessen <felix@seita.nl> * Assume perfect efficiency if no efficiency information is available Signed-off-by: F.N. Claessen <felix@seita.nl> * Switch from random periodic data to a step function, leaving a little headroom with respect to the PV system's nominal capacity Signed-off-by: F.N. Claessen <felix@seita.nl> * Return test data with conftest setup Signed-off-by: F.N. Claessen <felix@seita.nl> * Skip redundant sensor queries in tests Signed-off-by: F.N. Claessen <felix@seita.nl> * Refactor: rename variable for clarity Signed-off-by: F.N. Claessen <felix@seita.nl> * Take into account the nominal capacity of the asset, if given Signed-off-by: F.N. Claessen <felix@seita.nl> * Refactor: rename variable in test Signed-off-by: F.N. Claessen <felix@seita.nl> * Convert consumption/production power sign for our scheduler Signed-off-by: F.N. Claessen <felix@seita.nl> * Split fixtures, set up the building with a flexible device and two inflexible devices Signed-off-by: F.N. Claessen <felix@seita.nl> * Test building solver Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix passing along list of inflexible device sensors for battery scheduling Signed-off-by: F.N. Claessen <felix@seita.nl> * Restrict data used for scheduling to what was known at the time of scheduling Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix tests, given that the values for solar PV and residual demand (defined in the conftest) are measurements rather than forecasts Signed-off-by: F.N. Claessen <felix@seita.nl> * Let test schedule the future instead of the past, which requires forecasts to be present Signed-off-by: F.N. Claessen <felix@seita.nl> * Set up test data for inflexible devices in the Europe/Amsterdam timezone instead of UTC Signed-off-by: F.N. Claessen <felix@seita.nl> * Set up test data for EPEX market prices in the Europe/Amsterdam timezone instead of UTC Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix scheduling tests corresponding to having market prices available in the Europe/Amsterdam timezone instead of UTC Signed-off-by: F.N. Claessen <felix@seita.nl> * black Signed-off-by: F.N. Claessen <felix@seita.nl> * Add comment from earlier commit Signed-off-by: F.N. Claessen <felix@seita.nl> * Clear up inline comment Signed-off-by: F.N. Claessen <felix@seita.nl> * Rename variable internally Signed-off-by: F.N. Claessen <felix@seita.nl> * Rename API field Signed-off-by: F.N. Claessen <felix@seita.nl> * Rename CLI field with future deprecation warning Signed-off-by: F.N. Claessen <felix@seita.nl> * Clarify that the consumption price is being applied to the aggregate consumption of the devices, instead of being applied to the consumption of the single flexible device under consideration. Likewise for production. Signed-off-by: F.N. Claessen <felix@seita.nl> * Clarify description of aggregate power flow Signed-off-by: F.N. Claessen <felix@seita.nl> * Add return type annotation Signed-off-by: F.N. Claessen <felix@seita.nl> * Expand docstring Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix use of wrong exception Signed-off-by: F.N. Claessen <felix@seita.nl> * Refactor: rename util function Signed-off-by: F.N. Claessen <felix@seita.nl> * Changelog entry Signed-off-by: F.N. Claessen <felix@seita.nl> * Implement recommended docstring revision Signed-off-by: F.N. Claessen <felix@seita.nl> Signed-off-by: F.N. Claessen <felix@seita.nl>
FlexMeasures · Aug 24, 2022 · 7ff1480 · 7ff1480
1 parent 8912852
commit 7ff1480
Show file tree

Hide file tree

Showing 19 changed files with 516 additions and 85 deletions.
diff --git a/documentation/changelog.rst b/documentation/changelog.rst
@@ -14,6 +14,7 @@ New features
 * Add CLI command ``flexmeasures jobs show-queues`` [see `PR #455 <http://www.github.com/FlexMeasures/flexmeasures/pull/455>`_]
 * Switched from 12-hour AM/PM to 24-hour clock notation for time series chart axis labels [see `PR #446 <http://www.github.com/FlexMeasures/flexmeasures/pull/446>`_]
 * Get data in a given resolution [see `PR #458 <http://www.github.com/FlexMeasures/flexmeasures/pull/458>`_]
+* New API options to further customize the optimization context for scheduling, including the ability to use different prices for consumption and production (feed-in) [see `PR #451 <http://www.github.com/FlexMeasures/flexmeasures/pull/451>`_]
 
 Bugfixes
 -----------

diff --git a/documentation/index.rst b/documentation/index.rst
@@ -41,7 +41,7 @@ A tiny, but complete example: Let's install FlexMeasures from scratch. Then, usi
     $ flexmeasures db upgrade  # create tables
     $ flexmeasures add toy-account --kind battery  # setup account & a user, a battery (Id 2) and a market (Id 3)
     $ flexmeasures add beliefs --sensor-id 3 --source toy-user prices-tomorrow.csv  # load prices, also possible per API
-    $ flexmeasures add schedule --sensor-id 2 --optimization-context-id 3 \
+    $ flexmeasures add schedule --sensor-id 2 --consumption-price-sensor 3 \
         --start ${TOMORROW}T07:00+01:00 --duration PT12H \
         --soc-at-start 50% --roundtrip-efficiency 90%  # this is also possible per API
     $ flexmeasures show beliefs --sensor-id 2 --start ${TOMORROW}T07:00:00+01:00 --duration PT12H  # also visible per UI, of course

diff --git a/documentation/tut/toy-example-from-scratch.rst b/documentation/tut/toy-example-from-scratch.rst
@@ -21,7 +21,7 @@ Below are the ``flexmeasures`` CLI commands we'll run, and which we'll explain s
     # load prices to optimise the schedule against
     $ flexmeasures add beliefs --sensor-id 3 --source toy-user prices-tomorrow.csv
     # make the schedule
-    $ flexmeasures add schedule --sensor-id 2 --optimization-context-id 3 \
+    $ flexmeasures add schedule --sensor-id 2 --consumption-price-sensor 3 \
         --start ${TOMORROW}T07:00+01:00 --duration PT12H \
         --soc-at-start 50% --roundtrip-efficiency 90%
 
@@ -268,7 +268,7 @@ To keep it short, we'll only ask for a 12-hour window starting at 7am. Finally,
 
 .. code-block:: console
 
-    $ flexmeasures add schedule --sensor-id 2 --optimization-context-id 3 \
+    $ flexmeasures add schedule --sensor-id 2 --consumption-price-sensor 3 \
         --start ${TOMORROW}T07:00+01:00 --duration PT12H \
         --soc-at-start 50% --roundtrip-efficiency 90%
     New schedule is stored.

diff --git a/flexmeasures/api/v1_3/tests/utils.py b/flexmeasures/api/v1_3/tests/utils.py
@@ -29,15 +29,15 @@ def message_for_post_udi_event(
     message = {
         "type": "PostUdiEventRequest",
         "event": "ea1.2018-06.localhost:%s:204:soc",
-        "datetime": "2015-01-01T00:00:00+00:00",
+        "datetime": "2015-01-01T00:00:00+01:00",
         "value": 12.1,
         "unit": "kWh",
     }
     if targets:
         message["event"] = message["event"] + "-with-targets"
-        message["targets"] = [{"value": 25, "datetime": "2015-01-02T23:00:00+00:00"}]
+        message["targets"] = [{"value": 25, "datetime": "2015-01-02T23:00:00+01:00"}]
     if unknown_prices:
         message[
             "datetime"
-        ] = "2040-01-01T00:00:00+00:00"  # We have no beliefs in our test database about 2040 prices
+        ] = "2040-01-01T00:00:00+01:00"  # We have no beliefs in our test database about 2040 prices
     return message
diff --git a/flexmeasures/api/v2_0/tests/test_api_v2_0_assets.py b/flexmeasures/api/v2_0/tests/test_api_v2_0_assets.py
@@ -100,7 +100,7 @@ def test_get_assets(client, add_charging_station_assets, use_owner_id, num_asset
             battery = asset
     assert battery
     assert pd.Timestamp(battery["soc_datetime"]) == pd.Timestamp(
-        "2015-01-01T00:00:00+00:00"
+        "2015-01-01T00:00:00+01:00"
     )
     assert battery["owner_id"] == test_prosumer2_id
     assert battery["capacity_in_mw"] == 2

diff --git a/flexmeasures/api/v3_0/sensors.py b/flexmeasures/api/v3_0/sensors.py
@@ -1,6 +1,6 @@
 from datetime import datetime, timedelta
 import json
-from typing import Optional
+from typing import List, Optional
 
 from flask import current_app
 from flask_classful import FlaskView, route
@@ -221,6 +221,15 @@ def get_data(self, response: dict):
             ),  # todo: allow unit to be set per field, using QuantityField("%", validate=validate.Range(min=0, max=1))
             "targets": fields.List(fields.Nested(TargetSchema), data_key="soc-targets"),
             # todo: add a duration parameter, instead of falling back to FLEXMEASURES_PLANNING_HORIZON
+            "consumption_price_sensor": SensorIdField(
+                data_key="consumption-price-sensor", required=False
+            ),
+            "production_price_sensor": SensorIdField(
+                data_key="production-price-sensor", required=False
+            ),
+            "inflexible_device_sensors": fields.List(
+                SensorIdField, data_key="inflexible-device-sensors", required=False
+            ),
         },
         location="json",
     )
@@ -232,6 +241,9 @@ def trigger_schedule(  # noqa: C901
         unit: str,
         prior: datetime,
         roundtrip_efficiency: Optional[ur.Quantity] = None,
+        consumption_price_sensor: Optional[Sensor] = None,
+        production_price_sensor: Optional[Sensor] = None,
+        inflexible_device_sensors: Optional[List[Sensor]] = None,
         **kwargs,
     ):
         """
@@ -259,6 +271,11 @@ def trigger_schedule(  # noqa: C901
         with a target state of charge of 25 kWh at 4.00pm.
         The minimum and maximum soc are set to 10 and 25 kWh, respectively.
         Roundtrip efficiency for use in scheduling is set to 98%.
+        Aggregate consumption (of all devices within this EMS) should be priced by sensor 9,
+        and aggregate production should be priced by sensor 10,
+        where the aggregate power flow in the EMS is described by the sum over sensors 13, 14 and 15
+        (plus the flexible sensor being optimized, of course).
+        Note that, if forecasts for sensors 13, 14 and 15 are not available, a schedule cannot be computed.
 
         .. code-block:: json
 
@@ -274,7 +291,10 @@ def trigger_schedule(  # noqa: C901
                 ],
                 "soc-min": 10,
                 "soc-max": 25,
-                "roundtrip-efficiency": 0.98
+                "roundtrip-efficiency": 0.98,
+                "consumption-price-sensor": 9,
+                "production-price-sensor": 10,
+                "inflexible-device-sensors": [13, 14, 15]
             }
 
         **Example response**
@@ -396,6 +416,9 @@ def trigger_schedule(  # noqa: C901
             soc_min=soc_min,
             soc_max=soc_max,
             roundtrip_efficiency=roundtrip_efficiency,
+            consumption_price_sensor=consumption_price_sensor,
+            production_price_sensor=production_price_sensor,
+            inflexible_device_sensors=inflexible_device_sensors,
             enqueue=True,
         )
 

diff --git a/flexmeasures/api/v3_0/tests/utils.py b/flexmeasures/api/v3_0/tests/utils.py
@@ -39,16 +39,16 @@ def message_for_post_udi_event(
     targets: bool = False,
 ) -> dict:
     message = {
-        "start": "2015-01-01T00:00:00+00:00",
+        "start": "2015-01-01T00:00:00+01:00",
         "soc-at-start": 12.1,
         "soc-unit": "kWh",
     }
     if targets:
         message["soc-targets"] = [
-            {"value": 25, "datetime": "2015-01-02T23:00:00+00:00"}
+            {"value": 25, "datetime": "2015-01-02T23:00:00+01:00"}
         ]
     if unknown_prices:
         message[
             "start"
-        ] = "2040-01-01T00:00:00+00:00"  # We have no beliefs in our test database about 2040 prices
+        ] = "2040-01-01T00:00:00+01:00"  # We have no beliefs in our test database about 2040 prices
     return message
diff --git a/flexmeasures/cli/data_add.py b/flexmeasures/cli/data_add.py
@@ -15,6 +15,7 @@
 from sqlalchemy.exc import IntegrityError
 from timely_beliefs.sensors.func_store.knowledge_horizons import x_days_ago_at_y_oclock
 import timely_beliefs as tb
+import timely_beliefs.utils as tb_utils
 from workalendar.registry import registry as workalendar_registry
 
 from flexmeasures.data import db
@@ -777,12 +778,26 @@ def create_forecasts(
     required=True,
     help="Create schedule for this sensor. Follow up with the sensor's ID.",
 )
+@click.option(
+    "--consumption-price-sensor",
+    "consumption_price_sensor",
+    type=SensorIdField(),
+    required=False,
+    help="Optimize consumption against this sensor. The sensor typically records an electricity price (e.g. in EUR/kWh), but this field can also be used to optimize against some emission intensity factor (e.g. in kg CO₂ eq./kWh). Follow up with the sensor's ID.",
+)
+@click.option(
+    "--production-price-sensor",
+    "production_price_sensor",
+    type=SensorIdField(),
+    required=False,
+    help="Optimize production against this sensor. Defaults to the consumption price sensor. The sensor typically records an electricity price (e.g. in EUR/kWh), but this field can also be used to optimize against some emission intensity factor (e.g. in kg CO₂ eq./kWh). Follow up with the sensor's ID.",
+)
 @click.option(
     "--optimization-context-id",
     "optimization_context_sensor",
     type=SensorIdField(),
     required=True,
-    help="Optimize against this sensor, which measures a price factor or CO₂ intensity factor. Follow up with the sensor's ID.",
+    help="To be deprecated. Use consumption-price-sensor instead.",
 )
 @click.option(
     "--start",
@@ -847,6 +862,8 @@ def create_forecasts(
 )
 def create_schedule(
     power_sensor: Sensor,
+    consumption_price_sensor: Sensor,
+    production_price_sensor: Sensor,
     optimization_context_sensor: Sensor,
     start: datetime,
     duration: timedelta,
@@ -865,10 +882,20 @@ def create_schedule(
     - only supports datetimes on the hour or a multiple of the sensor resolution thereafter
     """
 
+    # todo: deprecate the 'optimization-context-id' argument in favor of 'consumption-price-sensor' (announced v0.11.0)
+    tb_utils.replace_deprecated_argument(
+        "optimization-context-id",
+        optimization_context_sensor,
+        "consumption-price-sensor",
+        consumption_price_sensor,
+    )
+
     # Parse input
     if not power_sensor.measures_power:
         click.echo(f"Sensor with ID {power_sensor.id} is not a power sensor.")
         raise click.Abort()
+    if production_price_sensor is None:
+        production_price_sensor = consumption_price_sensor
     end = start + duration
     for attribute in ("min_soc_in_mwh", "max_soc_in_mwh"):
         try:
@@ -922,7 +949,8 @@ def create_schedule(
             soc_min=soc_min,
             soc_max=soc_max,
             roundtrip_efficiency=roundtrip_efficiency,
-            price_sensor=optimization_context_sensor,
+            consumption_price_sensor=consumption_price_sensor,
+            production_price_sensor=production_price_sensor,
         )
         if job:
             print(f"New scheduling job {job.id} has been added to the queue.")
@@ -938,7 +966,8 @@ def create_schedule(
             soc_min=soc_min,
             soc_max=soc_max,
             roundtrip_efficiency=roundtrip_efficiency,
-            price_sensor=optimization_context_sensor,
+            consumption_price_sensor=consumption_price_sensor,
+            production_price_sensor=production_price_sensor,
         )
         if success:
             print("New schedule is stored.")

diff --git a/flexmeasures/conftest.py b/flexmeasures/conftest.py
@@ -3,6 +3,7 @@
 from random import random
 from datetime import datetime, timedelta
 from typing import List, Dict
+import pytz
 
 from isodate import parse_duration
 import pandas as pd
@@ -494,14 +495,18 @@ def add_market_prices(db: SQLAlchemy, setup_assets, setup_markets, setup_sources
 
     # one day of test data (one complete sine curve)
     time_slots = pd.date_range(
-        datetime(2015, 1, 1), datetime(2015, 1, 2), freq="1H", closed="left"
+        datetime(2015, 1, 1),
+        datetime(2015, 1, 2),
+        freq="1H",
+        closed="left",
+        tz="Europe/Amsterdam",
     )
     values = [
         random() * (1 + np.sin(x * 2 * np.pi / 24)) for x in range(len(time_slots))
     ]
     day1_beliefs = [
         TimedBelief(
-            event_start=as_server_time(dt),
+            event_start=dt,
             belief_horizon=timedelta(hours=0),
             event_value=val,
             source=setup_sources["Seita"],
@@ -513,12 +518,16 @@ def add_market_prices(db: SQLAlchemy, setup_assets, setup_markets, setup_sources
 
     # another day of test data (8 expensive hours, 8 cheap hours, and again 8 expensive hours)
     time_slots = pd.date_range(
-        datetime(2015, 1, 2), datetime(2015, 1, 3), freq="1H", closed="left"
+        datetime(2015, 1, 2),
+        datetime(2015, 1, 3),
+        freq="1H",
+        closed="left",
+        tz="Europe/Amsterdam",
     )
     values = [100] * 8 + [90] * 8 + [100] * 8
     day2_beliefs = [
         TimedBelief(
-            event_start=as_server_time(dt),
+            event_start=dt,
             belief_horizon=timedelta(hours=0),
             event_value=val,
             source=setup_sources["Seita"],
@@ -573,7 +582,7 @@ def create_test_battery_assets(
         max_soc_in_mwh=5,
         min_soc_in_mwh=0,
         soc_in_mwh=2.5,
-        soc_datetime=as_server_time(datetime(2015, 1, 1)),
+        soc_datetime=pytz.timezone("Europe/Amsterdam").localize(datetime(2015, 1, 1)),
         soc_udi_event_id=203,
         latitude=10,
         longitude=100,
@@ -591,7 +600,7 @@ def create_test_battery_assets(
         max_soc_in_mwh=5,
         min_soc_in_mwh=0,
         soc_in_mwh=2.5,
-        soc_datetime=as_server_time(datetime(2040, 1, 1)),
+        soc_datetime=pytz.timezone("Europe/Amsterdam").localize(datetime(2040, 1, 1)),
         soc_udi_event_id=203,
         latitude=10,
         longitude=100,
@@ -644,7 +653,7 @@ def add_charging_station_assets(
         max_soc_in_mwh=5,
         min_soc_in_mwh=0,
         soc_in_mwh=2.5,
-        soc_datetime=as_server_time(datetime(2015, 1, 1)),
+        soc_datetime=pytz.timezone("Europe/Amsterdam").localize(datetime(2015, 1, 1)),
         soc_udi_event_id=203,
         latitude=10,
         longitude=100,
@@ -662,7 +671,7 @@ def add_charging_station_assets(
         max_soc_in_mwh=5,
         min_soc_in_mwh=0,
         soc_in_mwh=2.5,
-        soc_datetime=as_server_time(datetime(2015, 1, 1)),
+        soc_datetime=pytz.timezone("Europe/Amsterdam").localize(datetime(2015, 1, 1)),
         soc_udi_event_id=203,
         latitude=10,
         longitude=100,