From 336e178fe82f752648a0524e91c726913111288d Mon Sep 17 00:00:00 2001 From: ekatef Date: Sat, 11 Feb 2023 16:30:59 +0300 Subject: [PATCH 01/11] Update natura.tiff usage instructions (cherry picked from commit 455f5567343c95c8d744e461579912c0f20696f6) --- doc/tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/tutorial.rst b/doc/tutorial.rst index c08c16d00..a9608e219 100644 --- a/doc/tutorial.rst +++ b/doc/tutorial.rst @@ -185,7 +185,7 @@ A raster file `natura.tiff` is used to store shapes of the protected and reserve .. note:: Skip this recommendation if the region of your interest is within Africa -A pre-built `natura.tiff` is loaded along with other data needed to run a model with `retrieve_databundle_light` rule. Currently this raster is valid for Africa, global `natura.tiff` raster is under development. You may generate the `natura.tiff` for a region of interest using `build_natura_raster` rule which aggregates data on protected areas along the cutout extent. +A pre-built `natura.tiff` is loaded along with other data needed to run a model with `retrieve_databundle_light` rule. This raster file contains data on the on protected areas around the world where areas no (renewable) assets can be installed. The `natura.tiff` raster has now global coverage so you don't need to create it locally. How to validate? ================ From 2f52fa3243889c7b299e4f047806566b96fa1967 Mon Sep 17 00:00:00 2001 From: ekatef Date: Sat, 11 Feb 2023 16:34:35 +0300 Subject: [PATCH 02/11] Update reference to natura.tiff in data description (cherry picked from commit 380df97141fefb3c9e2d2bfebf01725cf88bd933) --- doc/data_workflow.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/data_workflow.rst b/doc/data_workflow.rst index ddcf4a133..64ccc9777 100644 --- a/doc/data_workflow.rst +++ b/doc/data_workflow.rst @@ -63,7 +63,7 @@ These data are used in the `build_renewable_profiles` rule. `GEBCO Date: Sat, 11 Feb 2023 16:58:36 +0300 Subject: [PATCH 03/11] Add a clean-up recommendation (cherry picked from commit e1a926575b1ffdd08aafa705fc4dc5730e469259) --- doc/short_tutorial.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc/short_tutorial.rst b/doc/short_tutorial.rst index cf7047909..4e6d83ecb 100644 --- a/doc/short_tutorial.rst +++ b/doc/short_tutorial.rst @@ -93,7 +93,9 @@ You may use the `clean` rule for making so: .../pypsa-earth (pypsa-earth) % snakemake -j 1 clean -Generally, it's a good idea to repeat the cleaning procedure every time when the underlying data are changed. +Generally, it's a good idea to repeat the cleaning procedure every time when the underlying data are changed to avoid conflicts between run settings corresponding to different scenarios. + +It is also possible to make manual clean-up removing folders "resources", "networks" and "results". Those folders store the intermediate output of the workflow and if you don't need them anymore it is safe to delete them. .. note:: From f3c568007f6dca962cca47e083a4c42d92f9f34f Mon Sep 17 00:00:00 2001 From: ekatef Date: Sat, 11 Feb 2023 16:59:10 +0300 Subject: [PATCH 04/11] Clarify using of build_cutout Add recommendations on temporal extent of the cutout Add headers Fix typos Clarify the description Remove an outdated note Fix grammar Fix typo (cherry picked from commit 79bab6aaf81da3b759888388235163d245708a22) --- doc/data_workflow.rst | 2 +- doc/short_tutorial.rst | 4 ++-- doc/tutorial.rst | 19 ++++++++++++++----- 3 files changed, 17 insertions(+), 8 deletions(-) diff --git a/doc/data_workflow.rst b/doc/data_workflow.rst index 64ccc9777..34366c01f 100644 --- a/doc/data_workflow.rst +++ b/doc/data_workflow.rst @@ -63,7 +63,7 @@ These data are used in the `build_renewable_profiles` rule. `GEBCO `_. -After playing with the tutorial model and before playing with different fucntions, +After playing with the tutorial model and before playing with different functions, it's important to clean-up data in your model folder before to proceed further to avoid data conflicts. You may use the `clean` rule for making so: diff --git a/doc/tutorial.rst b/doc/tutorial.rst index a9608e219..7bf38fe96 100644 --- a/doc/tutorial.rst +++ b/doc/tutorial.rst @@ -171,20 +171,29 @@ In case you are interested in other parts of the world you have to generate a cu These steps are required to use CDS API which allows an automatic file download while executing `build_cutouts` rule. +The `build_cutout` flag should be set `true` to generate the cutout. After the cutout is ready, it's recommended to set `build_cutout` to `false` to avoid overwriting the existing cutout by accident. + +After the first run, if you don't change country and need to increase a considered time span wider than the one you created the cutout with, you may set to false both `retrieve_databundle` and `build_cutout`. + +Spatial extent +^^^^^^^^^^^^^^ + Normally cutout extent is calculated from the shape of the requested region defined by the `countries` parameter in the configuration file `config.yaml`. It could make sense to set the countries list as big as it's feasible when generating a cutout. A considered area can be narrowed anytime when building a specific model by adjusting content of the `countries` list. There is also option to set the cutout extent specifying `x` and `y` values directly. However, these values will overwrite values extracted from the countries shape. Which means that nothing prevents `build_cutout` to extract data which has no relation to the requested countries. Please use direct definition of `x` and `y` only if you really understand what and why you are doing. -The `build_cutout` flag should be set `true` to generate the cutout. After the cutout is ready, it's recommended to set `build_cutout` to `false` to avoid overwriting the existing cutout by accident. +Temporal extent +^^^^^^^^^^^^^^^ + +If you create the cutout for a certain year (let's say 2013) and want to run scenarios for a subset of this year, you don't need to rerun the `build_cutout` as the cutout still contains all the hours of 2013. The workflow will automatically subset the cutout archive to extract data for the particular timeframe of interest. If you instead you want to run the 2014 scenario, then rerun `build_cutout` is needed. + +In case you need model a number of years, a convenient approach may be to create the cutout for the whole period under interest (e.g. 2013-2015) so that you don't need to build any additional cutouts. Note, however, that the disk requirements increase in this case. 3. Build a natura.tiff raster ----------------------------- A raster file `natura.tiff` is used to store shapes of the protected and reserved nature areas. Such landuse restrictions can be taking into account when calculating the renewable potential with `build_renewable_profiles`. -.. note:: - Skip this recommendation if the region of your interest is within Africa - A pre-built `natura.tiff` is loaded along with other data needed to run a model with `retrieve_databundle_light` rule. This raster file contains data on the on protected areas around the world where areas no (renewable) assets can be installed. The `natura.tiff` raster has now global coverage so you don't need to create it locally. How to validate? @@ -238,7 +247,7 @@ The following validation notebooks are worth a look when validating your energy 1. A detailed `network validation `_. -2. Analys of `the installed capacity `_ for the considered area. +2. Analysis of `the installed capacity `_ for the considered area. 3. Validation of `the power demand `_ values and profile. From 75dfdfeb6331f77efd1883481bbacf6042a35601 Mon Sep 17 00:00:00 2001 From: ekatef Date: Tue, 14 Feb 2023 00:09:39 +0300 Subject: [PATCH 05/11] Add clarification regarding cutout resolution --- doc/tutorial.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/tutorial.rst b/doc/tutorial.rst index 7bf38fe96..e9fc0e801 100644 --- a/doc/tutorial.rst +++ b/doc/tutorial.rst @@ -158,6 +158,7 @@ The cutout is the main concept of climate data management in PyPSA ecosystem int .. note:: Skip this recommendation if the region of your interest is within Africa and you are fine with the 2013 weather year +Generally, the spatial and time resolution of the cutout data is determined by parameters of an underlying dataset. That is 30 km x 30 km grid and houtly resolution for ERA5 recommended for usage in PyPSA-Earth. The pre-built cutout for Africa is available for 2013 year and can be loaded directly from zenodo through the rule `retrieve_cutout`. There is also a smaller cutout for Africa built for a two-weeks time span; it is automatically downloaded when retrieving common data with `retrieve_databundle_light`. From 097d730248915981395b043a5fc51c50f63161f9 Mon Sep 17 00:00:00 2001 From: ekatef Date: Tue, 14 Feb 2023 00:09:55 +0300 Subject: [PATCH 06/11] Move the note --- doc/tutorial.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/tutorial.rst b/doc/tutorial.rst index e9fc0e801..3ef416ab5 100644 --- a/doc/tutorial.rst +++ b/doc/tutorial.rst @@ -156,12 +156,13 @@ It could be helpful to keep in mind the following points: The cutout is the main concept of climate data management in PyPSA ecosystem introduced in `atlite `_ package. The cutout is an archive containing a spatio-temporal subset of one or more topology and weather datasets. Since such datasets are typically global and span multiple decades, the Cutout class allows atlite to reduce the scope to a more manageable size. More details about the climate data processing concepts are contained in `JOSS paper `_. -.. note:: - Skip this recommendation if the region of your interest is within Africa and you are fine with the 2013 weather year Generally, the spatial and time resolution of the cutout data is determined by parameters of an underlying dataset. That is 30 km x 30 km grid and houtly resolution for ERA5 recommended for usage in PyPSA-Earth. The pre-built cutout for Africa is available for 2013 year and can be loaded directly from zenodo through the rule `retrieve_cutout`. There is also a smaller cutout for Africa built for a two-weeks time span; it is automatically downloaded when retrieving common data with `retrieve_databundle_light`. +.. note:: + Skip this recommendation if the region of your interest is within Africa and you are fine with the 2013 weather year + In case you are interested in other parts of the world you have to generate a cutout yourself using the `build_cutouts` rule. To run it you will need to: 1. be registered on the `Copernicus Climate Data Store `_; From 5376491b186a64e9a173784e00abd53445487153 Mon Sep 17 00:00:00 2001 From: ekatef Date: Wed, 1 Mar 2023 17:59:34 +0300 Subject: [PATCH 07/11] Add an explanation on the temporal dimension of the cutout --- doc/tutorial.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/doc/tutorial.rst b/doc/tutorial.rst index 3ef416ab5..af82cbfaa 100644 --- a/doc/tutorial.rst +++ b/doc/tutorial.rst @@ -70,6 +70,9 @@ It is advisable to adapt the required range of coordinates to the selection of c dy: 0.3 # cutout resolution # The cutout time is automatically set by the snapshot range. + +Note please that a temporal dimension of the cutout should be consistent with the values set for `snapshots` parameter. A time range of the cutout is determined by the parameters set when building this cutout while the time resolution corresponds to those of the used climate archives. In case of ERA5 dataset used in PyPSA-Earth by default, hourly resolution is implied. + It is also possible to decide which weather data source should be used to calculate potentials and capacity factor time-series for each carrier. For example, we may want to use the ERA-5 dataset for solar and not the default SARAH-2 dataset. @@ -173,7 +176,7 @@ In case you are interested in other parts of the world you have to generate a cu These steps are required to use CDS API which allows an automatic file download while executing `build_cutouts` rule. -The `build_cutout` flag should be set `true` to generate the cutout. After the cutout is ready, it's recommended to set `build_cutout` to `false` to avoid overwriting the existing cutout by accident. +The `build_cutout` flag should be set `true` to generate the cutout. After the cutout is ready, it's recommended to set `build_cutout` to `false` to avoid overwriting the existing cutout by accident. The `snapshots` values set when generating the cutout, will determine the temporal parameters of the cutout. Accessible years which can be used to build a cutout depend on ERA5 data availability. `ERA5 page `_ explains that the data is available from 1950 and updated continuously with about 3 month delay while the data on 1950-1978 should be treated as preliminary as that is a rather recent development. After the first run, if you don't change country and need to increase a considered time span wider than the one you created the cutout with, you may set to false both `retrieve_databundle` and `build_cutout`. From 43088caf4c37849c2625bf826d486064fe4f7c77 Mon Sep 17 00:00:00 2001 From: ekatef Date: Wed, 1 Mar 2023 17:59:44 +0300 Subject: [PATCH 08/11] Fix typo --- doc/tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/tutorial.rst b/doc/tutorial.rst index af82cbfaa..72a4cb542 100644 --- a/doc/tutorial.rst +++ b/doc/tutorial.rst @@ -178,7 +178,7 @@ These steps are required to use CDS API which allows an automatic file download The `build_cutout` flag should be set `true` to generate the cutout. After the cutout is ready, it's recommended to set `build_cutout` to `false` to avoid overwriting the existing cutout by accident. The `snapshots` values set when generating the cutout, will determine the temporal parameters of the cutout. Accessible years which can be used to build a cutout depend on ERA5 data availability. `ERA5 page `_ explains that the data is available from 1950 and updated continuously with about 3 month delay while the data on 1950-1978 should be treated as preliminary as that is a rather recent development. -After the first run, if you don't change country and need to increase a considered time span wider than the one you created the cutout with, you may set to false both `retrieve_databundle` and `build_cutout`. +After the first run, if you don't change country and don't need to increase a considered time span wider than the one you created the cutout with, you may set to false both `retrieve_databundle` and `build_cutout`. Spatial extent ^^^^^^^^^^^^^^ From 0746d9fb361b7177e07dc5fb6cd720b5e6a2b8b0 Mon Sep 17 00:00:00 2001 From: ekatef Date: Wed, 1 Mar 2023 18:32:21 +0300 Subject: [PATCH 09/11] Fix typo --- doc/short_tutorial.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/doc/short_tutorial.rst b/doc/short_tutorial.rst index e2fd59013..79a6814d9 100644 --- a/doc/short_tutorial.rst +++ b/doc/short_tutorial.rst @@ -28,8 +28,6 @@ To do that, you may want to do a reserve copy of your current configuration file In the configuration file `config.yaml` there is a flag `retrieve_databundle` which triggers data loading and a `tutorial` flag which determines that the loaded data belong to the tutorial kit. Currently the tutorial can be run only for Nigeria ("NG"), Benin ("BJ"), Botswana ("BW") and Morocco ("MA"). -: - .. code:: yaml tutorial: true From 5dcc5bd9aab24fac7e772697ea54067cf570c479 Mon Sep 17 00:00:00 2001 From: ekatef Date: Wed, 1 Mar 2023 18:32:41 +0300 Subject: [PATCH 10/11] Add an explanation of the load options --- doc/tutorial.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/doc/tutorial.rst b/doc/tutorial.rst index 72a4cb542..0417338c4 100644 --- a/doc/tutorial.rst +++ b/doc/tutorial.rst @@ -34,6 +34,19 @@ Likewise, the example's temporal scope can be restricted (e.g. to 7 days): end: "2013-03-7" inclusive: "left" # end is not inclusive + +Year-related parameters are also being used when specifying `load_options`: + +.. code:: yaml + + load_options: + ssp: "ssp2-2.6" + weather_year: 2013 + prediction_year: 2030 + scale: 1 + +The `weather_year` value corresponds to the weather data which was used to generate the electricity demand profiles for a selected area while `prediction_year` correspond to the point of a ssp trajectory. The available values for `weather_year` and `prediction_year` can be checked by looking into `pypsa-earth/data/ssp2-2.6` folder. Currently, there are pre-calculated demand data for 2011, 2013, 2018 weather years and for 2030, 2040, 2050, and 2100 scenario prediction years. + It is also possible to allow less or more carbon-dioxide emissions, while defining the current emissions. It is possible to model a net-zero target by setting the `co2limit` to zero: From 1df367046581952c655c6b4cc786b8cf04022ae8 Mon Sep 17 00:00:00 2001 From: ekatef Date: Wed, 1 Mar 2023 18:38:49 +0300 Subject: [PATCH 11/11] Clarify usage on natura.tiff --- doc/tutorial.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/doc/tutorial.rst b/doc/tutorial.rst index 0417338c4..086938de4 100644 --- a/doc/tutorial.rst +++ b/doc/tutorial.rst @@ -207,10 +207,14 @@ If you create the cutout for a certain year (let's say 2013) and want to run sce In case you need model a number of years, a convenient approach may be to create the cutout for the whole period under interest (e.g. 2013-2015) so that you don't need to build any additional cutouts. Note, however, that the disk requirements increase in this case. -3. Build a natura.tiff raster +3. Check natura.tiff raster ----------------------------- -A raster file `natura.tiff` is used to store shapes of the protected and reserved nature areas. Such landuse restrictions can be taking into account when calculating the renewable potential with `build_renewable_profiles`. +A raster file `natura.tiff` is used to store shapes of the protected and reserved nature areas. Such landuse restrictions can be taking into account when calculating the renewable potential with `build_renewable_profiles` by switching-on `natura` option: + +.. code:: bash + + natura: true A pre-built `natura.tiff` is loaded along with other data needed to run a model with `retrieve_databundle_light` rule. This raster file contains data on the on protected areas around the world where areas no (renewable) assets can be installed. The `natura.tiff` raster has now global coverage so you don't need to create it locally.