This repository contains the code and data for our research project on the Decision Transformer (DT) as a booster for other learners in a reinforcement learning setting. The project investigates if the offline, batch learning capabilities of DT can be applied on other agents to generalize to more optimal actions.
Reinforcement Learning (RL) is a powerful paradigm for developing agents that can learn to make decisions in complex environments. However, RL algorithms can be slow and sample inefficient when learning through online interaction with the environment. The Decision Transformer (DT) offers a promising alternative by learning offline from a dataset of trajectories. DT has shown impressive results in several domains, including Atari games, but it requires access to an expert dataset of trajectories.
In this research project, we investigate whether the DT can be used as a booster for other learners in a reinforcement learning setting. Specifically, we aim to collect and use two datasets of trajectories, one containing random trajectories and the other containing expert trajectories generated by a Q-learning agent. We will then train the DT on these datasets and compare its performance with that of a Q-learning agent and a more sophisticated expert learner, the Deep Q-Network (DQN).
Can the DT effectively learn from the expert dataset of trajectories to outperform other learners? How do the hyperparameters of the DT affect its performance, and can we identify optimal hyperparameters? Can the DT be used as a booster for other learners in a reinforcement learning setting, and does it provide a performance gain over the baseline learners?
- Clone our repository with
git clone git@github.com:haraldger/DRL-DecisionTransformer.git(preferred) or withgit clone https://github.com/haraldger/DRL-DecisionTransformer.gitto a desired directory. Navigate to the directory:cd DRL-DecisionTransformer. - (Recommended) Create a virtual environment for package management and separation. In the project directory
python -m venv ./venv
source venv/bin/activate
To activate the venv on Windows:
venv\scripts\activate.bat
- Make sure pip is updated:
pip install --upgrade pip - Install dependencies from requirements file:
pip install -r requirements.txt - Launch the random agent (basic) with:
python main.py
To add dependencies, follow these steps:
- Make sure you are working on a virtual environment, with only the dependencies of the project installed.
- To list installed packages:
pip freeze - When the new required package is installed through pip, run:
pip freeze > requirements.txt.