diff --git a/INSTALL.md b/INSTALL.md new file mode 100644 index 0000000000..3c596a5a2f --- /dev/null +++ b/INSTALL.md @@ -0,0 +1,128 @@ +For more detailed information, please visit our [documentation](https://ugr-sail.github.io/sinergym/compilation/main/index.html). + +## Docker container (recommended) + +We include a **Dockerfile** for installing all dependencies and setting +up the image for running *Sinergym*. + +By default, Dockerfile will do `pip install -e .[extras]`, if you want to install a different setup, you will have to do in root repository: + +```sh + $ docker build -t --build-arg SINERGYM_EXTRAS=[] . +``` + +For example, if you want a container with only documentation libraries and testing: + +```sh + $ docker build -t example1/sinergym:latest --build-arg SINERGYM_EXTRAS=[doc,test] . +``` + +On the other hand, if you don't want any extra library, it's necessary to write an empty value like this: + +```sh + $ docker build -t example1/sinergym:latest --build-arg SINERGYM_EXTRAS= . +``` + +:pencil: You can install directly our container from `Docker Hub repository `__, all releases of this project are there. + +:pencil: If you use [Visual Studio Code](https://code.visualstudio.com/), by simply opening the root directory and clicking on the pop-up button *Reopen in container*, all the dependencies will be installed automatically and you will be able to run *Sinergym* in an isolated environment. For more information about how to use this functionality, check [VSCode Containers extension documentation](https://code.visualstudio.com/docs/remote/containers). + +## Manual installation + +To install *Sinergym* manually instead of through the container (not recommended), follow these steps: + +#### 1. Configure Python environment + +- First, clone this repository: + +```sh + $ git clone https://github.com/ugr-sail/sinergym.git + $ cd sinergym +``` + +- Then, it is recommended to create a **virtual environment**. You can do so by: + +```sh + $ sudo apt-get install python-virtualenv virtualenv + $ virtualenv env_sinergym --python=python3.10 + $ source env_sinergym/bin/activate + $ pip install -e .[extras] +``` + +- There are other alternatives like **conda environments** (recommended). Conda is very comfortable to use and we have a file to configure it automatically: + +```sh + $ cd sinergym + $ conda env create -f python_environment.yml + $ conda activate sinergym +``` + +Sinergym has been updating the compatibility with different components, here it is a summary about important versions support: + +| **Sinergym version** | **Ubuntu version** | **Python version** | **EnergyPlus version** | +| -------------------- | ------------------ | ------------------ | ---------------------- | +| **0.0** | 18.04 LTS | 3.6 | 8.3.0 | +| **1.1.0** | 18.04 LTS | 3.6 | **9.5.0** | +| **1.7.0** | 18.04 LTS | **3.9** | 9.5.0 | +| **1.9.5** | **22.04 LTS** | **3.10** | 9.5.0 | + +- Now, we have a correct python version with required modules to run *Sinergym*. Let's continue with the rest of the programs that are needed outside of Python to run the simulations: + +#### 2. Install EnergyPlus 9.5.0 + +Install EnergyPlus. Currently it has been update compatibility to 9.5.0 and it has +been tested, but code may also work with other versions. Other combination may works, but they don't have been tested. + +Follow the instructions [here](https://energyplus.net/downloads) and +install it for Linux (only Ubuntu is supported). Choose any location +to install the software. Once installed, a folder called +`Energyplus-9-5-0` should appear in the selected location. + +#### 3. Install BCVTB software + +Follow the instructions +[here](https://simulationresearch.lbl.gov/bcvtb/Download) for +installing BCVTB software. Another option is to copy the `bcvtb` +folder from [this repository](https://github.com/zhangzhizza/Gym-Eplus/tree/master/eplus_env/envs) + +#### 4. Set environment variables + +Two environment variables must be set: `EPLUS_PATH` and +`BCVTB_PATH`, with the locations where EnergyPlus and BCVTB are +installed respectively. + +## About Sinergym package + +As we have told you in section *Manual Installation*, Python environment can be set up using *python_environment.yml* with *conda*. +However, we can make an installation using the Github repository itself: + +```sh + $ cd sinergym + $ pip install -e . +``` + +Extra libraries can be installed by typing ``pip install -e .[extras]``. +*extras* include all optional libraries which have been considered in this project such as +testing, visualization, Deep Reinforcement Learning, monitoring , etc. +It's possible to select a subset of these libraries instead of 'extras' tag in which we select all optional libraries, for example: + +```sh + $ pip install -e .[test,doc] +``` + +In order to check all our tag list, visit `setup.py `__ in *Sinergym* root repository. In any case, they are not a requirement of the package. + +You can also install from `oficial pypi repository `__ with last stable version by default: + +```sh + $ pip install sinergym[extras] +``` + +## Check Installation + +This project is automatically supervised using tests developed specifically for it. If you want to check *Sinergym* has been installed successfully, run next command: + +```sh +$ pytest tests/ -vv +``` +Anyway, every time *Sinergym* repository is updated, the tests will run automatically in a remote container using the Dockerfile to build it. `Github Action `__ will do that job (see our documentation for more information). \ No newline at end of file diff --git a/README.md b/README.md index ab7b30a9a9..aea2338e59 100644 --- a/README.md +++ b/README.md @@ -51,11 +51,10 @@ The goal of this project is to create an environment following [Gymnasium interface](https://gymnasium.farama.org/), for wrapping simulation engines for building control using **deep reinforcement learning**. -Please, help us to improve by **reporting your questions and issues** [here](https://github.com/ugr-sail/sinergym/issues). It is easy, just 2 clicks using our issue templates (questions, bugs, improvements, etc.). More detailed info on how to report issues [here](https://docs.github.com/en/issues/tracking-your-work-with-issues/creating-an-issue). +Please, help us to improve by **reporting your questions and issues** [here](https://github.com/ugr-sail/sinergym/issues). It is easy, just 2 clicks using our issue templates (questions, bugs, improvements, etc.). More detailed info on how to report issues [here](https://docs.github.com/en/issues/tracking-your-work-with-issues/creating-an-issue). Don't forget to take a look at [CONTRIBUTING.md](https://github.com/ugr-sail/sinergym/blob/main/CONTRIBUTING.md) if you're thinking about contributing to Sinergym. The main functionalities of *Sinergym* are the following : - - **Include different simulation engines**. Communication between Python and [EnergyPlus](https://energyplus.net/) is established using [BCVTB](https://simulationresearch.lbl.gov/bcvtb/FrontPage) middleware. @@ -106,16 +105,15 @@ The main functionalities of *Sinergym* are the following : with deep reinforcement learning algorithms. This tool can be used with any other DRL library that supports the * Gymnasium* interface as well. -- **Google Cloud Integration**. Whether you have a Google Cloud account and you want to - use your infrastructure with *Sinergym*, we tell you some details about how doing it. - -- **Mlflow tracking server**. [Mlflow](https://mlflow.org/) is an open source platform for the machine - learning lifecycle. This can be used with Google Cloud remote server (if you have Google Cloud account) - or using local store. This will help you to manage and store your runs and artifacts generated in an orderly - manner. +- **Weights & Biases tracking and visualization**. One of Sinergym's objectives is to automate + and facilitate the training, reproducibility and comparison of agents in simulation-based + building control problems, managing and monitoring model lifecycle from training to deployment. [WandB](https://wandb.ai/site) + is an open-source platform for the machine learning lifecycle helping us with this issue. + It lets us register experiments hyperparameters, visualize data recorded in real-time, + and store artifacts with experiment outputs and best obtained models. -- **Data Visualization**. Using *Sinergym* logger or Tensorboard server to visualize training and evaluation information - in real-time. +- **Google Cloud Integration**. Whether you have a Google Cloud account and you want to + use your infrastructure with *Sinergym*, we tell you some details about how to do it. - **Notebooks examples**. *Sinergym* develops code in notebook format with the purpose of offering use cases to the users in order to help them become familiar with the tool. They are constantly updated, along with the updates @@ -126,7 +124,7 @@ The main functionalities of *Sinergym* are the following : - Many more! -_This is a work in progress project. Stay tuned for upcoming releases._ +_This is a project in active development. Stay tuned for upcoming releases._


@@ -138,135 +136,7 @@ If you would like to see a complete and updated list of our available environmen ## Installation -For more detailed information, please visit our [documentation](https://ugr-sail.github.io/sinergym/compilation/main/index.html). - -### Docker container - -We include a **Dockerfile** for installing all dependencies and setting -up the image for running *Sinergym*. - -By default, Dockerfile will do `pip install -e .[extras]`, if you want to install a different setup, you will have to do in root repository: - -```sh - $ docker build -t --build-arg SINERGYM_EXTRAS=[] . -``` - -For example, if you want a container with only documentation libraries and testing: - -```sh - $ docker build -t example1/sinergym:latest --build-arg SINERGYM_EXTRAS=[doc,test] . -``` - -On the other hand, if you don't want any extra library, it's necessary to write an empty value like this: - -```sh - $ docker build -t example1/sinergym:latest --build-arg SINERGYM_EXTRAS= . -``` - -:pencil: You can install directly our container from `Docker Hub repository `__, all releases of this project are there. - -:pencil: If you use [Visual Studio Code](https://code.visualstudio.com/), by simply opening the root directory and clicking on the pop-up button *Reopen in container*, all the dependencies will be installed automatically and you will be able to run *Sinergym* in an isolated environment. For more information about how to use this functionality, check [VSCode Containers extension documentation](https://code.visualstudio.com/docs/remote/containers). - -### Manual installation - -To install *Sinergym* manually instead of through the container (not recommended), follow these steps: - -#### 1. Configure Python environment - -- First, clone this repository: - -```sh - $ git clone https://github.com/ugr-sail/sinergym.git - $ cd sinergym -``` - -- Then, it is recommended to create a **virtual environment**. You can do so by: - -```sh - $ sudo apt-get install python-virtualenv virtualenv - $ virtualenv env_sinergym --python=python3.10 - $ source env_sinergym/bin/activate - $ pip install -e .[extras] -``` - -- There are other alternatives like **conda environments** (recommended). Conda is very comfortable to use and we have a file to configure it automatically: - -```sh - $ cd sinergym - $ conda env create -f python_environment.yml - $ conda activate sinergym -``` - -Sinergym has been updating the compatibility with different components, here it is a summary about important versions support: - -| **Sinergym version** | **Ubuntu version** | **Python version** | **EnergyPlus version** | -| -------------------- | ------------------ | ------------------ | ---------------------- | -| **0.0** | 18.04 LTS | 3.6 | 8.3.0 | -| **1.1.0** | 18.04 LTS | 3.6 | **9.5.0** | -| **1.7.0** | 18.04 LTS | **3.9** | 9.5.0 | -| **1.9.5** | **22.04 LTS** | **3.10** | 9.5.0 | - -- Now, we have a correct python version with required modules to run *Sinergym*. Let's continue with the rest of the programs that are needed outside of Python to run the simulations: - -#### 2. Install EnergyPlus 9.5.0 - -Install EnergyPlus. Currently it has been update compatibility to 9.5.0 and it has -been tested, but code may also work with other versions. Other combination may works, but they don't have been tested. - -Follow the instructions [here](https://energyplus.net/downloads) and -install it for Linux (only Ubuntu is supported). Choose any location -to install the software. Once installed, a folder called -`Energyplus-9-5-0` should appear in the selected location. - -#### 3. Install BCVTB software - -Follow the instructions -[here](https://simulationresearch.lbl.gov/bcvtb/Download) for -installing BCVTB software. Another option is to copy the `bcvtb` -folder from [this repository](https://github.com/zhangzhizza/Gym-Eplus/tree/master/eplus_env/envs) - -#### 4. Set environment variables - -Two environment variables must be set: `EPLUS_PATH` and -`BCVTB_PATH`, with the locations where EnergyPlus and BCVTB are -installed respectively. - - -## About Sinergym package - -As we have told you in section *Manual Installation*, Python environment can be set up using *python_environment.yml* with *conda*. -However, we can make an installation using the Github repository itself: - -```sh - $ cd sinergym - $ pip install -e . -``` - -Extra libraries can be installed by typing ``pip install -e .[extras]``. -*extras* include all optional libraries which have been considered in this project such as -testing, visualization, Deep Reinforcement Learning, monitoring , etc. -It's possible to select a subset of these libraries instead of 'extras' tag in which we select all optional libraries, for example: - -```sh - $ pip install -e .[test,doc] -``` - -In order to check all our tag list, visit `setup.py `__ in *Sinergym* root repository. In any case, they are not a requirement of the package. - -You can also install from `oficial pypi repository `__ with last stable version by default: - -```sh - $ pip install sinergym[extras] -``` - -## Check Installation - -This project is automatically supervised using tests developed specifically for it. If you want to check *Sinergym* has been installed successfully, run next command: - -```sh -$ pytest tests/ -vv -``` -Anyway, every time *Sinergym* repository is updated, the tests will run automatically in a remote container using the Dockerfile to build it. `Github Action `__ will do that job (see our documentation for more information). +Please, visit [INSTALL.md](https://github.com/ugr-sail/sinergym/blob/main/INSTALL.md) for more information about Sinergym installation. ## Usage example @@ -298,10 +168,24 @@ Notice that a folder will be created in the working directory after creating the ## Google Cloud Platform support -Cloud Computing - For more information about this functionality, please, visit our documentation [here](https://ugr-sail.github.io/sinergym/compilation/main/pages/gcloudAPI.html#sinergym-with-google-cloud). +## Projects using Sinergym + +The following are some of the projects benefiting from the advantages of Sinergym: + +- [Demosthen/ActiveRL](https://github.com/Demosthen/ActiveRL) +- [VectorInstitute/HV-Ai-C](https://github.com/VectorInstitute/HV-Ai-C) +- [rdnfn/beobench](https://github.com/rdnfn/beobench) + +:pencil: If you want to appear in this list, do not hesitate to send us a PR and include the following badge in your repository: + +

+ + + +

+ ## Citing Sinergym If you use *Sinergym* in your work, please cite our [paper](https://dl.acm.org/doi/abs/10.1145/3486611.3488729): diff --git a/docs/source/_templates/sinergym.html b/docs/source/_templates/sinergym.html new file mode 100644 index 0000000000..94598aa115 --- /dev/null +++ b/docs/source/_templates/sinergym.html @@ -0,0 +1,6 @@ +

+ + + +

\ No newline at end of file diff --git a/docs/source/index.rst b/docs/source/index.rst index 99b2d3aaae..65a35b890d 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -12,6 +12,21 @@ You can check `issues `__ in the re If you want to contribute, please read `CONTRIBUTING.md `__ first. +######################## +Projects using Sinergym +######################## + +The following are some of the projects benefiting from the advantages of Sinergym: + +- `Demosthen/ActiveRL `__ +- `VectorInstitute/HV-Ai-C `__ +- `rdnfn/beobench `__ + +If you want to appear in this list, do not hesitate to send us a PR and include the following badge in your repository: + +.. raw:: html + :file: ./_templates/sinergym.html + ############ Examples ############ diff --git a/docs/source/pages/installation.rst b/docs/source/pages/installation.rst index eb71691920..030671593d 100644 --- a/docs/source/pages/installation.rst +++ b/docs/source/pages/installation.rst @@ -184,6 +184,6 @@ Cloud Computing You can run your experiments in the Cloud too. We are using `Google Cloud `__ in order to make it possible. Our team aim to set up an account in which execute our *Sinergym* container -with **remote storage** and **mlflow tracking**. +with **remote storage** and **WandB tracking**. For more detail about installation and getting Google Cloud SDK ready to run your experiments, visit our section :ref:`Preparing Google Cloud`. \ No newline at end of file diff --git a/docs/source/pages/introduction.rst b/docs/source/pages/introduction.rst index fcc3724312..9838f28992 100644 --- a/docs/source/pages/introduction.rst +++ b/docs/source/pages/introduction.rst @@ -72,13 +72,12 @@ The main functionalities of *Sinergym* are the following: - **Google Cloud Integration**. Whether you have a Google Cloud account and you want to use your infrastructure with *Sinergym*, we tell you some details about how doing it. -- **Mlflow tracking server**. `Mlflow `__ is an open source platform for the machine - learning lifecycle. This can be used with Google Cloud remote server (if you have Google Cloud account) - or using local store. This will help you to manage and store your runs and artifacts generated in an orderly - manner. - -- **Data Visualization**. Using *Sinergym* logger or Tensorboard server to visualize training and evaluation information - in real-time. +- **Weights & Biases tracking and visualization**. One of Sinergym's objectives is to automate + and facilitate the training, reproducibility and comparison of agents in simulation-based + building control problems, managing and monitoring model lifecycle from training to deployment. `WandB `__ + is an open-source platform for the machine learning lifecycle helping us with this issue. + It lets us register experiments hyperparameters, visualize data recorded in real-time, + and store artifacts with experiment outputs and best obtained models. - **Notebooks examples**. *Sinergym* develops code in notebook format with the purpose of offering use cases to the users in order to help them become familiar with the tool. They are constantly updated, along with the updates diff --git a/mlflowbuild.sh b/mlflowbuild.sh deleted file mode 100644 index 574ef97b38..0000000000 --- a/mlflowbuild.sh +++ /dev/null @@ -1,88 +0,0 @@ -#!/bin/bash - -# This scrip is used to build a mlflow server in Google Cloud, it is important -# to set up account previously. - -# Please, visit our documentation here --> https://ugr-sail.github.io/sinergymcompilationhtml/index.html - -# Step 0 - Store all parameters - -PROJECT_ID=$1 -BUCKET_NAME=$2 -ZONE=$3 -REGION=${ZONE::-2} -DB_ROOT_PASSWORD=$4 -MACHINE_TYPE=e2-medium -MLFLOW_IMAGE=kaysush/mlflow:1.14.1 -CLOUD_SQL_PROXY_IMAGE=gcr.io/cloudsql-docker/gce-proxy:1.19.1 -MYSQL_INSTANCE=${PROJECT_ID}:${REGION}:mlflow-backend - -# Step 1 - Service account for mlflow service -echo "Creating Service account for mlflow service [mlflow-tracking-sa]..." -gcloud iam service-accounts create mlflow-tracking-sa --description="Service Account to run the MLFLow tracking server" --display-name="MLFlow tracking SA" - -# Step 2 - Artifact used by mlflow to store all runs information -echo "Creating Back-end artifact bucket [$BUCKET_NAME]..." -gsutil mb -l europe-west1 gs://$BUCKET_NAME - -# Step 3 - CLoud SQL, instance with SQL and "mlflow" database inner -echo "Creating sql instance with mlflow database [mlflow-backend]..." -gcloud sql instances create mlflow-backend --tier=db-f1-micro --region=${REGION} --root-password=${DB_ROOT_PASSWORD} --storage-type=SSD -gcloud sql databases create mlflow --instance=mlflow-backend - -# Step 4 - IAM: Provisioning service account privileges in order to manipulate bucket and back-end -echo "Creating service account privileges to use Back-end [roles/cloudsql.editor]..." -gsutil iam ch "serviceAccount:mlflow-tracking-sa@${PROJECT_ID}.iam.gserviceaccount.com:roles/storage.admin" gs://${BUCKET_NAME} -gcloud projects add-iam-policy-binding ${PROJECT_ID} --member="serviceAccount:mlflow-tracking-sa@${PROJECT_ID}.iam.gserviceaccount.com" --role=roles/cloudsql.editor - -# Step 5 - Creating start_mlflow_tracking.sh to initialize instance -echo "Creating start_mlflow_tracking.sh to initialize instance..." -cat <./start_mlflow_tracking.sh -echo "Starting Cloud SQL Proxy'" -docker run -d --name mysql --net host -p 3306:3306 $CLOUD_SQL_PROXY_IMAGE /cloud_sql_proxy -instances $MYSQL_INSTANCE=tcp:0.0.0.0:3306 - -echo "Starting mlflow-tracking server" -docker run -d --name mlflow-tracking --net host -p 5000:5000 $MLFLOW_IMAGE mlflow server --backend-store-uri mysql+pymysql://root:${DB_ROOT_PASSWORD}@localhost/mlflow --default-artifact-root gs://${BUCKET_NAME}/mlflow_artifacts/ --host 0.0.0.0 - -echo "Altering IPTables" -iptables -A INPUT -p tcp --dport 5000 -j ACCEPT -EOF - -# Step 6 - Uploading start script and deleting from local -echo "Uploading start_mlflow_tracking.sh at gs://${BUCKET_NAME}/scripts/start_mlflow_tracking.sh..." -gsutil cp ./start_mlflow_tracking.sh gs://${BUCKET_NAME}/scripts/start_mlflow_tracking.sh -echo "Deleting temporal local script [start_mlflow_tracking.sh]" -rm ./start_mlflow_tracking.sh - -#Step 7 - creating static external ip for mlflow server -echo "Creating static external ip for mlflow-tracking-server [mlflow-ip]" -gcloud compute addresses create mlflow-ip \ - --region europe-west1 - -# Step 8 - Compute Instance -echo "Deploying remote server [mlflow-tracking-server]..." -gcloud compute --project=$PROJECT_ID instances create mlflow-tracking-server \ - --zone=$ZONE \ - --machine-type=$MACHINE_TYPE \ - --subnet=default \ - --network-tier=PREMIUM \ - --metadata=startup-script-url=gs://${BUCKET_NAME}/scripts/start_mlflow_tracking.sh \ - --maintenance-policy=MIGRATE \ - --service-account=mlflow-tracking-sa@${PROJECT_ID}.iam.gserviceaccount.com \ - --scopes=https://www.googleapis.com/auth/cloud-platform \ - --tags=mlflow-tracking-server \ - --image=cos-77-12371-1109-0 \ - --image-project=cos-cloud \ - --boot-disk-size=10GB \ - --boot-disk-type=pd-balanced \ - --boot-disk-device-name=mlflow-tracking-server \ - --no-shielded-secure-boot \ - --shielded-vtpm \ - --shielded-integrity-monitoring \ - --reservation-affinity=any \ - --address $(gcloud compute addresses describe mlflow-ip --format='get(address)') - -# Step 8 - Firewall -echo "Creating firewall rules [allow-mlflow-tracking]..." -gcloud compute firewall-rules create allow-mlflow-tracking --network default --priority 1000 --direction ingress --action allow --target-tags mlflow-tracking-server --source-ranges 0.0.0.0/0 --rules tcp:5000 --enable-logging - diff --git a/sinergym/version.txt b/sinergym/version.txt index f90b1afc08..0bee604df7 100644 --- a/sinergym/version.txt +++ b/sinergym/version.txt @@ -1 +1 @@ -2.3.2 +2.3.3