Merge pull request #296 from pypsa-meets-earth/documentation

Enhance Model Documentation

README.md

@@ -1,8 +1,9 @@
 
 
-# PyPSA-Earth-Sec: A Sector-Coupled Open Optimisation Model of the Global Energy System
+# Sector-coupled PyPSA-Earth: A Global Open Source Multi-Energy System Model
 
-## Development Status: Version 0.0.2
+
+## Development Status: Version 0.1
 
 [![Status Linux](https://github.com/pypsa-meets-earth/pypsa-earth-sec/actions/workflows/ci-linux.yaml/badge.svg?branch=main&event=push)](https://github.com/pypsa-meets-earth/pypsa-earth-sec/actions/workflows/ci-linux.yaml)
 [![Status Mac](https://github.com/pypsa-meets-earth/pypsa-earth-sec/actions/workflows/ci-mac.yaml/badge.svg?branch=main&event=push)](https://github.com/pypsa-meets-earth/pypsa-earth-sec/actions/workflows/ci-mac.yaml)
@@ -15,22 +16,27 @@
 [![Discord](https://img.shields.io/discord/911692131440148490?logo=discord)](https://discord.gg/VHH8TCwn)
 [![Google Drive](https://img.shields.io/badge/Google%20Drive-4285F4?style=flat&logo=googledrive&logoColor=white)](https://drive.google.com/drive/folders/1U7fgktbxlaGzWxT2C0-Xv-_ffWCxAKZz)
 
-**Disclaimer: PyPSA-Earth-Sec is still under development.**
-
-The workflow is adaped to work smoothly for the following countries: Morocco, Namibia, Nigeria and Benin. The spatial and temporal resolution of the model are flexible. It's advisable to use more than 3 nodes per country and a timestep not smaller than 3-hours.
+Sector-coupled PyPSA-Earth is the first of its kind: an open-source, sector-coupled energy system model with global coverage. It is designed to facilitate transparent analysis and informed decision-making for a sustainable energy future.
+The model's workflow leverages open datasets to customize the model according to the specific needs of any country or region, delivering complete optimization results that encompass policy, technical, economic, and environmental impacts.
+It is built on top of [PyPSA-Earth](https://github.com/pypsa-meets-earth/pypsa-earth), the two models are soon to be merged into one encompassing the scope of both models.
 
+Our mission with Sector-coupled PyPSA-Earth is to empower stakeholders around the globe with the ability to transparently analyze and plan the transition to a decarbonized, integrated energy system.
+By making this tool openly available, we aim to foster collaboration, innovation, and informed policy-making that leads to sustainable and efficient energy solutions for the future.
 
-Currently, no real sectoral demand data is used for the country inspected, instead, we use dummy data. The collection, compilation and processing of real data is underway.
+The paper describing the model in details is currently under review. The preprint is already available [here](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4743242).
 
+## Key Features
 
-The model now includes the following energy carriers: **electricity**, **hydrogen**, **gas**, *oil** and biomass as well as **carbon** as gas emissions and feedstock for the synthesis of the carriers.
+Sector-coupled PyPSA-Earth model introduces several novel features that set it apart:
 
-The demand sectors covered are: **residential**, **industry**, **land transport**, **aviation**, **shipping**, **services** and agriculture.
+1. **Global Coverage:** It is the first open-source model capable of simulating sector-coupled energy systems on a global scale.
+2. **Adaptable Demand Projection Tool:** The model includes a flexible tool for projecting energy demand, tailored to the specifics of the region or country under study.
+3. **Global Geo-Asset Dataset:** A comprehensive dataset compiled from open sources forms the backbone of the model, providing detailed insights into energy assets worldwide.
+4. **Custom Dataset Integration:** While the model extensively uses open datasets, it also allows for the easy integration of custom datasets, offering flexibility to users who have specific data at their disposal.
 
 The diagram below depicts one representative clustered node showing the combination of carriers and sectors covered in the model as well as the generation and conversion technologies included.
 
-![alt text](https://github.com/pypsa-meets-earth/pypsa-earth-sec/blob/main/docs/pes_v0.0.2.png?raw=true)
-
+![alt text](https://github.com/pypsa-meets-earth/pypsa-earth-sec/blob/documentation/docs/SCPE.png?raw=true)
 
 
 ## Installation

config.default.yaml

@@ -13,7 +13,7 @@ scenario:
   simpl: # only relevant for PyPSA-Eur
   - ""
   clusters: # number of nodes in Europe, any integer between 37 (1 node per country-zone) and several hundred
-  - 74
+  - 10
   planning_horizons: # investment years for myopic and perfect; or costs year for overnight
   - 2030
   ll:
@@ -26,7 +26,7 @@ scenario:
   - "AB"
 
 policy_config:
-  policy: "H2_export_yearly_constraint" #either "H2_export_yearly_constraint", "H2_export_monthly_constraint", "no_res_matching"
+  policy: "no_res_matching" #either "H2_export_yearly_constraint", "H2_export_monthly_constraint", "no_res_matching"
   monthly: # Specify attributes for the monthly constraint
     allowed_excess: 1.0
     reference_case: false # RE electricity is equal to the amount required for additional hydrogen export compared to the 0 export case ("reference_case")
@@ -235,7 +235,8 @@ sector:
   industry_util_factor: 0.7
 
   biomass_transport: true  # biomass transport between nodes
-  solid_biomass_potential: 2 # TWh/a, Potential of whole modelled area
+  biomass_transport_default_cost: 0.1 #EUR/km/MWh
+  solid_biomass_potential: 40 # TWh/a, Potential of whole modelled area
   biogas_potential: 0.5 # TWh/a, Potential of whole modelled area
 
   efficiency_heat_oil_to_elec: 0.9

scripts/add_export.py

@@ -33,8 +33,12 @@ def select_ports(n):
         index_col=None,
         keep_default_na=False,
     ).squeeze()
-    ports = ports[ports.country.isin(countries)]
 
+    ports = ports[ports.country.isin(countries)]
+    if len(ports) < 1:
+        logger.error(
+            "No export ports chosen, please add ports to the file data/export_ports.csv"
+        )
     gadm_level = snakemake.config["sector"]["gadm_level"]
 
     ports["gadm_{}".format(gadm_level)] = ports[["x", "y", "country"]].apply(
@@ -180,12 +184,12 @@ def create_export_profile():
             simpl="",
             clusters="10",
             ll="c1.0",
-            opts="Co2L0.10",
+            opts="Co2L",
             planning_horizons="2030",
-            sopts="300H",
-            discountrate="0.15",
-            demand="DF",
-            h2export="9",
+            sopts="144H",
+            discountrate="0.071",
+            demand="AB",
+            h2export="120",
         )
         sets_path_to_root("pypsa-earth-sec")
 

scripts/prepare_sector_network.py

@@ -1,4 +1,5 @@
 # -*- coding: utf-8 -*-
+import logging
 import os
 import re
 from types import SimpleNamespace
@@ -22,6 +23,8 @@
 )
 from prepare_transport_data import prepare_transport_data
 
+logger = logging.getLogger(__name__)
+
 spatial = SimpleNamespace()
 
 
@@ -77,7 +80,7 @@ def add_generation(n, costs):
         _type_: _description_
     """ """"""
 
-    print("adding electricity generation")
+    logger.info("adding electricity generation")
 
     # Not required, because nodes are already defined in "nodes"
     # nodes = pop_layout.index
@@ -130,8 +133,8 @@ def add_oil(n, costs):
         n.add("Carrier", "oil")
 
     # Set the "co2_emissions" of the carrier "oil" to 0, because the emissions of oil usage taken from the spatial.oil.nodes are accounted seperately (directly linked to the co2 atmosphere bus). Setting the carrier to 0 here avoids double counting. Be aware to link oil emissions to the co2 atmosphere bus.
-    n.carriers.loc["oil", "co2_emissions"] = 0
-    print("co2_emissions of oil set to 0 for testing")  # TODO add logger.info
+    # n.carriers.loc["oil", "co2_emissions"] = 0
+    # print("co2_emissions of oil set to 0 for testing")  # TODO add logger.info
 
     n.madd(
         "Bus",
@@ -481,19 +484,19 @@ def define_spatial(nodes):
 
 
 def add_biomass(n, costs):
-    print("adding biomass")
+    logger.info("adding biomass")
 
     # TODO get biomass potentials dataset and enable spatially resolved potentials
 
     # Get biomass and biogas potentials from config and convert from TWh to MWh
     biomass_pot = snakemake.config["sector"]["solid_biomass_potential"] * 1e6  # MWh
     biogas_pot = snakemake.config["sector"]["biogas_potential"] * 1e6  # MWh
-    print("Biomass and Biogas potential fetched from config")
+    logger.info("Biomass and Biogas potential fetched from config")
 
     # Convert from total to nodal potentials,
     biomass_pot_spatial = biomass_pot / len(spatial.biomass.nodes)
     biogas_pot_spatial = biogas_pot / len(spatial.gas.biogas)
-    print("Biomass potentials spatially resolved equally across all nodes")
+    logger.info("Biomass potentials spatially resolved equally across all nodes")
 
     n.add("Carrier", "biogas")
     n.add("Carrier", "solid biomass")
@@ -574,8 +577,25 @@ def add_biomass(n, costs):
         )
 
         # costs
-        bus0_costs = biomass_transport.bus0.apply(lambda x: transport_costs[x[:2]])
-        bus1_costs = biomass_transport.bus1.apply(lambda x: transport_costs[x[:2]])
+        countries_not_in_index = set(countries) - set(biomass_transport.index)
+        if countries_not_in_index:
+            logger.info(
+                "No transport values found for {0}, using default value of {1}".format(
+                    ", ".join(countries_not_in_index),
+                    snakemake.config["sector"]["biomass_transport_default_cost"],
+                )
+            )
+
+        bus0_costs = biomass_transport.bus0.apply(
+            lambda x: transport_costs.get(
+                x[:2], snakemake.config["sector"]["biomass_transport_default_cost"]
+            )
+        )
+        bus1_costs = biomass_transport.bus1.apply(
+            lambda x: transport_costs.get(
+                x[:2], snakemake.config["sector"]["biomass_transport_default_cost"]
+            )
+        )
         biomass_transport["costs"] = pd.concat([bus0_costs, bus1_costs], axis=1).mean(
             axis=1
         )
@@ -1085,7 +1105,7 @@ def add_shipping(n, costs):
 
 
 def add_industry(n, costs):
-    print("adding industrial demand")
+    logger.info("adding industrial demand")
     # 1e6 to convert TWh to MWh
 
     # industrial_demand.reset_index(inplace=True)
@@ -1397,7 +1417,7 @@ def add_land_transport(n, costs):
     """
     # TODO options?
 
-    print("adding land transport")
+    logger.info("adding land transport")
 
     if options["dynamic_transport"]["enable"] == False:
         fuel_cell_share = get(
@@ -1419,9 +1439,9 @@ def add_land_transport(n, costs):
 
     ice_share = 1 - fuel_cell_share - electric_share
 
-    print("FCEV share", fuel_cell_share)
-    print("EV share", electric_share)
-    print("ICEV share", ice_share)
+    logger.info("FCEV share: {}".format(fuel_cell_share))
+    logger.info("EV share: {}".format(electric_share))
+    logger.info("ICEV share: {}".format(ice_share))
 
     assert ice_share >= 0, "Error, more FCEV and EV share than 1."
 
@@ -1596,11 +1616,11 @@ def create_nodes_for_heat_sector():
     diff = (urban_fraction * central_fraction) - dist_fraction_node
     progress = get(options["district_heating"]["progress"], investment_year)
     dist_fraction_node += diff * progress
-    print(
-        "The current district heating share compared to the maximum",
-        f"possible is increased by a progress factor of\n{progress}",
-        "resulting in a district heating share of",  # "\n{dist_fraction_node}", #TODO fix district heat share
-    )
+    # logger.info(
+    #     "The current district heating share compared to the maximum",
+    #     f"possible is increased by a progress factor of\n{progress}",
+    #     "resulting in a district heating share of",  # "\n{dist_fraction_node}", #TODO fix district heat share
+    # )
 
     return nodes, dist_fraction_node, urban_fraction
 
@@ -1609,7 +1629,7 @@ def add_heat(n, costs):
     # TODO options?
     # TODO pop_layout?
 
-    print("adding heat")
+    logger.info("adding heat")
 
     sectors = ["residential", "services"]
 
@@ -1620,7 +1640,7 @@ def add_heat(n, costs):
     # exogenously reduce space heat demand
     if options["reduce_space_heat_exogenously"]:
         dE = get(options["reduce_space_heat_exogenously_factor"], investment_year)
-        print(f"assumed space heat reduction of {dE*100} %")
+        # print(f"assumed space heat reduction of {dE*100} %")
         for sector in sectors:
             heat_demand[sector + " space"] = (1 - dE) * heat_demand[sector + " space"]
 
@@ -2143,18 +2163,6 @@ def add_residential(n, costs):
         * 1e6
     )
 
-    print("#############################################")
-    print("#############################################")
-    print("#############################################")
-    print(nodes)
-    print("#############################################")
-    print("#############################################")
-    print("#############################################")
-    print(heat_shape)
-    print("#############################################")
-    print("#############################################")
-    print("#############################################")
-
     # TODO make compatible with more counties
     profile_residential = n.loads_t.p_set[nodes] / n.loads_t.p_set[nodes].sum().sum()
 
@@ -2331,13 +2339,13 @@ def add_rail_transport(n, costs):
         snakemake = mock_snakemake(
             "prepare_sector_network",
             simpl="",
-            clusters="56",
-            ll="c1",
+            clusters="10",
+            ll="c1.0",
             opts="Co2L",
             planning_horizons="2030",
-            sopts="2000H",
+            sopts="144H",
             discountrate="0.071",
-            demand="DF",
+            demand="AB",
         )
 
     # Load population layout

test/config.test1.yaml

@@ -240,7 +240,8 @@ sector:
   industry_util_factor: 0.7
 
   biomass_transport: true  # biomass transport between nodes
-  solid_biomass_potential: 2 # TWh/a, Potential of whole modelled area
+  biomass_transport_default_cost: 0.1 #EUR/km/MWh
+  solid_biomass_potential: 10 # TWh/a, Potential of whole modelled area
   biogas_potential: 0.5 # TWh/a, Potential of whole modelled area
   efficiency_heat_oil_to_elec: 0.9
   efficiency_heat_biomass_to_elec: 0.9