BoMoDT: Bologna Mobility Digital Twin

BoMoDT is the Mobility Digital Twin (MoDT) platform designed for the Italian City of Bologna (Bo). Developed following a Model-Driven Architecture (MDA) approach and using automatized model-to-model transformations MoDT-M2M-MTT tool, BoMoDT is based on open-source data retrieved from the Municipality of Bologna and open-source technologies, i.e., FIWARE for data management and data interoperability and Eclipse Simulator of Urban MObility (SUMO) for traffic conditions modeling and simulations.

The BoMoDT platform provides:

1. Digital representation and state simulation to model the structural and behavioral aspects of Bologna's infrastructure and simulate traffic scenarios.

   • Eclipse SUMO is utilized for traffic modeling and simulation.

2. Bidirectional synchronization to enable data flow from physical sensors to the Digital Twin and feedback from the Digital Twin to physical traffic light systems.

   • FIWARE Generic Enablers (GEs) facilitate data exchange and management between the real system and its Digital Twin, ensuring semantic data interoperability.

3. Traffic monitoring to observe current traffic flow and stay updated on the state of Bologna City as well as simulation results based on dynamically generated scenarios.

   • A Django WebApp, combined with a Grafana dashboard, is implemented to:
     (i) monitor the state of context entities modeled with FIWARE Smart Data Models,
     (ii) track traffic flow patterns using the Grafana Dashboard, and
     (iii) visualize simulation results after the simulation is completed.

BoMoDT Architecture Overview

The architecture of the BoMoDT platform is depicted in Figure 1. This diagram offers a simplified overview of the platform's design to facilitate better understanding. Detailed information about each component can be found in their respective folders. The BoMoDT platform incorporates open-source data from the following sources:

• Bologna Open Data: Real traffic data from Bologna are collected. For more information, refer to the data README
• OpenStreetMap: The platform retrieves Bologna's road network layout and additional infrastructure details, such as induction loops and traffic lights. For more information, refer to the OSM README

Figure 1: BoMoDT Architecture.

The modules involved in BoMoDT are as follows:

1. Bologna Mobility Virtual Environment (MVENV): This module was introduced because direct access to Bologna's real infrastructure is not feasible. As a result, BoMoDT incorporates an emulator of Bologna's traffic data streams capable of executing control commands for adaptive traffic light management. As a result, BoMoDT incorporates an emulator to ensure the presence of a continuously running physical counterpart in a Digital Twin context. Further details at BOLOGNA MVENV README

2. FIWARE Orion-LD Context Broker: This is the mandatory component in any "powered by FIWARE" smart solution. It is responsible for managing context entities in compliance with the Next Generation Service Interface (NGSI) protocol in Linked Data (LD) version. The current context entities' state is stored in a Mongo database. Further details at FIWARE VENV README

3. FIWARE IoT Agent - JSON: This is the Internet of Things Agent (IOTA) for JSON-based protocols (with AMQP, HTTP, and MQTT transports). This IoT Agent acts as a bridge between JSON and the NGSI interface of the context broker, converting device-specific protocols into the NGSI standard. Devices sending measurements or receiving commands must be registered beforehand. Further details at FIWARE VENV README

4. FIWARE QuantumLeap: This is a time-based data-persistence Generic Enabler subscribed to context updates, for storing and querying time-series data in Timescale. QuantumLeap addresses the inherent limitation of the Context Broker, which only stores the current state. Further details at FIWARE VENV README

5. Eclipse SUMO: This is microscopic multi-modal traffic simulator that modeling individual road users, including cars, buses, and pedestrians, enabling detailed analysis of traffic phenomena like congestion and emissions. Simulation data inputs and outputs are locally stored. Further details at SUMO README

6. The Digital Shadow Manager, Planner & Scenario Generator, and Digital Twin Manager are key Python modules available in libraries. These modules are responsible for generating digital shadows (temporal data traces), creating and executing simulation scenarios, planning actions to be performed in the physical system, and orchestrating the overall Digital Twin system, respectively.

7. Django WebApp & Grafana Dashboard provides a user interface for monitoring (i) context entities modeled with FIWARE Smart Data Models, real-time traffic flow patterns, and simulation results.

Device Compatibility

The BoMoDT platform is deployed and tested on a Windows device. Its compatibility with Linux and macOS devices depends on the underlying technologies supporting the platform. While FIWARE is fully containerized and can potentially be adapted to other environments, the Eclipse SUMO compatibility should be verified by consulting the Eclipse Documentation.

Figure 2: BoMoDT UML Deployment Diagram.

BoMoDT consists of three execution environments:

1. Docker Environment provides a containerized infrastructure for deploying FIWARE components, including IoT Agent JSON, Orion-LD, and QuantumLeap, along with MongoDB for current data storage and TimescaleDB for historical data storage. Additionally, Grafana is containerized and available for independent visualization and monitoring, separate from the Django WebApp.

2. Eclipse Simulator Engine is responsible for performing urban mobility simulations.

3. Python Virtual Environment hosts the Python modules generated through the MDA-based approach, alongside additional modules developed to implement and execute the Bologna MVENV.

BoMoDT Repository Structure

The repository is structured to support the BoMoDT platform, providing modules for data handling, traffic
simulation, Digital Twin modeling, and backend management. Below is the detailed structure of the repository:

├── BoMoDT
│   ├── data
│   │   ├── digitalshadow               # Contains the digital shadows used and updated in the platform.
│   │   ├── preprocessing
│   │   │   ├── preprocessingSetup.py   # Python script to handle data preprocessing.
│   │   │   └── README.md               # Documentation for preprocessing steps and files.
│   │   ├── realworlddata
│   │   │   ├── mvenvdata               # Traffic data used by the Bologna MVENV.
│   │   │   ├── opendata                # Traffic data retrieved by the Bologna Open Data.
│   │   └── README.md                   # Documentation for the folder, including details and potential updates 
│   │                                   # if the real dataset is modified.
│   │
│   ├── fiwareenv
│   │   ├── .env                        # Environment variables file for configuring FIWARE components.
│   │   ├── docker-compose.yml          # Docker Compose file for deploying FIWARE components.
│   │   └── README.md                   # Documentation for setting up the FIWARE environment.
│   │
│   ├── libraries
│   │   ├── classes
│   │   │   ├── Agent.py                # Manages interaction with FIWARE IoT Agent JSON.
│   │   │   ├── Broker.py               # Manages interaction with FIWARE Context Broker Orion-LD.
│   │   │   ├── DataManager.py          # Handles data storage and retrieval processes.
│   │   │   ├── DigitalShadowManager.py # Manages digital shadows for the simulation.
│   │   │   ├── DigitalTwinManager.py   # Orchestrates the Digital Twin system.
│   │   │   ├── Planner.py              # Implements planning actions and scenario generation.
│   │   │   ├── SubscriptionManager.py  # Manages subscriptions through FIWARE QuantumLeap.
│   │   │   └── SumoSimulator.py        # Facilitates interaction with the SUMO Simulator.
│   │
│   ├── utils
│   │   ├── generalUtils.py             # Provides generic utility functions used across the project.
│   │   ├── preprocessingUtils.py       # Implements utilities for data preprocessing and transformation.
│   │   └── constants.py                # Stores constants such as configurations and file paths.
│   │
│   ├── mobilityvenv
│   │   ├── MobilityVirtualEnvironment.py # Defines the Mobility Virtual Environment.
│   │   ├── PhysicalSystemConnector.py    # Defines the connector within a physical system made of sensors and actuators.
│   │
│   ├── sumoenv
│   │   ├── standalone
│   │   │   └── (configuration files for standalone execution of Eclipse SUMO).
│   │   ├── static
│   │   │   └── (static files used for simulation).
│   │   ├── README.md                   # Documentation for configuring and executing the SUMO environment.
│   │   └── run.sumocfg                 # SUMO configuration file for executing traffic simulations.
│   │
│   ├── udtBackEnd
│   │   ├── udtApp                      # Contains application logic for the webApp backend.
│   │   ├── udtBackEnd                  # Handles backend functionality for the Digital Twin WebApp.
│   │   ├── db.sqlite3                  # SQLite database file storing application data.
│   │   ├── manage.py                   # Entry point for managing the WebApp backend.
│
├── LICENSE                             # Licensing information for the repository.
├── main.py                             # Main script for executing the BoMoDT platform.
├── README.md                           # Documentation for understanding and using the repository.
├── requirements.txt                    # Lists all Python dependencies required for the project.
├── setup.bat                           # Batch script for setting up the environment on Windows.

BoMoDT Platform Execution

The BoMoDT platform can be executed either manually via the command line or by using the provided bash script (for Windows OS) included in the repository. Below are the detailed steps to deploy and execute the BoMoDT platform.

Prerequisites

Before executing the BoMoDT platform, ensure the following prerequisites are installed and properly configured on your system:

• Python >= 3.12: Required to execute the core scripts and manage the environment. Install the latest version from Python.org.
• Eclipse SUMO 1.19: Necessary for traffic modeling and simulation. Download and install from [Eclipse SUMO] (https://www.eclipse.org/sumo/).
• Docker and Docker Compose: Required to deploy the FIWARE environment and associated components. Install the latest versions from Docker and Docker Compose.
• Git: Needed to clone the repository. Install Git from Git SCM.
• System Requirements: Sufficient disk space and computational power to run Docker containers, execute Eclipse simulator and store simulation results.

Ensure all the above dependencies are installed and available in your system's PATH before proceeding with the setup and execution of the platform.

---

Deployment and Execution from Command Line

1. Clone the repository to your local machine:

   git clone repository-url

2. Navigate to the repository directory:

   cd /path/to/repository/BoMoDT

FIWARE Environment Setup

After cloning the repository, the FIWARE environment must be activated to enable required components.

1. Navigate to the fiwareenv directory:

   cd /path/to/repository/BoMoDT/fiwareenv

Ensure that the following files are present in the fiwareenv directory: docker-compose.yml and .env

2. Deploy the required containers for Orion-LD, IoT Agent JSON, QuantumLeap, MongoDB, TimescaleDB, and Grafana by running the following command:

   docker-compose up -d

Further details are provided in the folder README.md.

MVENV and Python Environment Setup

Before running main.py script, a Python Virtual Environment must be activated. The environment can be created using the provided requirements.txt file.

1. Create and activate a Python virtual environment in repository directory:
   1. If virtualenv is not installed in your local machine, install it by running:

   python3 -m pip install virtualenv

   2. Create a Python virtual environment:

   virtualenv venv

   3. Activate the virtual environment:

   source venv/bin/activate

2. Once the Python virtual environment has been created and activated, install the required Python packages using the requirements.txt:

   pip3 install -r requirements.txt

3. Run the main.py script:

   python3 main.py

Note: Running this command will activate the Bologna MVENV, initiating data transmission to the Digital Twin through FIWARE GEs. The simulation scenarios are automatically configured and executed in SUMO during this process.

Django WebApp and Grafana Dashboard

1. Navigate to webApp backend repository:

   cd path/to/repository/BoMoDT/udtBackEnd

2. Run the command:

   python3 manage.py runserver

Eclipse SUMO Environment Setup

The Eclipse SUMO Simulator is utilized during the execution of the BoMoDT platform for modeling and simulating traffic scenarios. Additionally, Eclipse SUMO can be executed independently using the standalone configuration provided in the folder sumoenv/standalone.

For detailed instructions on setting up and executing Eclipse SUMO, refer to the README.md file located in the sumoenv folder.

---

Deployment and Execution for Windows OS

The setup.bat script is provided for setting up the environment and running the application.

Prerequisites

• Docker: Ensure Docker is running before executing the script. If Docker is not active, the script will terminate.
• Python Virtual Environment (venv): A Python virtual environment must be created beforehand, either using the
  steps outlined earlier or by setting it up in your preferred IDE (e.g., PyCharm, VSCode, etc.). The script will abort if no virtual environment is detected.

Script Workflow

The setup.bat script performs the following steps:

1. Creates the required Docker containers.
2. Activates the Python virtual environment.
3. Verifies that the virtual environment is correctly activated.
4. Runs the Django web application.
5. Executes the main.py script to initialize and run the MVENV.

Run the script by executing:

setup.bat

or double clicking on setup.bat file.