From a3425de56a2a5d29f2504307a641b52eba13e563 Mon Sep 17 00:00:00 2001 From: J K Terry Date: Thu, 23 Sep 2021 17:02:09 -0400 Subject: [PATCH] new environments --- docs/environments.md | 329 ++++++++++++++++++++++--------------------- docs/misc.md | 7 - 2 files changed, 168 insertions(+), 168 deletions(-) delete mode 100644 docs/misc.md diff --git a/docs/environments.md b/docs/environments.md index 69a7cf90367..3c732b259e7 100644 --- a/docs/environments.md +++ b/docs/environments.md @@ -1,317 +1,324 @@ -# Third Pary Environments +# Third Party Environments -### Procgen +# Video Games environments + +## Procgen + +https://github.com/openai/procgen 16 simple-to-use procedurally-generated gym environments which provide a direct measure of how quickly a reinforcement learning agent learns generalizable skills. The environments run at high speed (thousands of steps per second) on a single core. -Learn more here: https://github.com/openai/procgen +## SlimeVolleyGym: A simple environment for single and multi-agent reinforcement learning -## Third Party Environments +https://github.com/hardmaru/slimevolleygym -The gym comes prepackaged with many many environments. It's this common API around many environments that makes Gym so great. Here we will list additional environments that do not come prepacked with the gym. Submit another to this list via a pull-request. +A simple environment for benchmarking single and multi-agent reinforcement learning algorithms on a clone of Slime Volleyball game. Only dependencies are gym and numpy. Both state and pixel observation environments are available. The motivation of this environment is to easily enable trained agents to play against each other, and also facilitate the training of agents directly in a multi-agent setting, thus adding an extra dimension for evaluating an agent's performance. -### gym-algorithmic +## stable-retro -These are a variety of algorithmic tasks, such as learning to copy a sequence, present in Gym prior to Gym 0.20.0. +https://github.com/MatPoliquin/stable-retro -Learn more here: https://github.com/Rohan138/gym-algorithmic +Supported fork of gym-retro with additional games, states, scenarios, etc. Open to PRs of additional games, features and plateforms since gym-retro is no longer maintained -### gym-legacy-toytext +## gym-derk: GPU accelerated MOBA environment -These are the unused toy-text environments present in Gym prior to Gym 0.20.0. +https://gym.derkgame.com -Learn more here: https://github.com/Rohan138/gym-legacy-toytext +This is a 3v3 MOBA environment where you train creatures to fight each other. It runs entirely on the GPU so you can easily have hundreds of instances running in parallel. There are around 15 items for the creatures, 60 "senses", 5 actions, and roughly 23 tweakable rewards. It's also possible to benchmark an agent against other agents online. It's available for free for training for personal use, and otherwise costs money; see licensing details on the website -### gym-spoof +## Unity ML Agents -Spoof, otherwise known as "The 3-coin game", is a multi-agent (2 player), imperfect-information, zero-sum game. +https://github.com/Unity-Technologies/ml-agents + +Gym wrappers for arbitrary and premade environments with the Unity game engine. -Learn more here: https://github.com/MouseAndKeyboard/gym-spoof +# Robotics Environments -Platforms: Windows, Mac, Linux +## PyBullet Robotics Environments -### PyBullet Robotics Environments +Docs: https://docs.google.com/document/d/10sXEhzFRSnvFcl3XxNGhnD4N2SedqwdAvK3dsihxVUA/edit#heading=h.wz5to0x8kqmr 3D physics environments like the Mujoco environments but uses the Bullet physics engine and does not require a commercial license. Works on Mac/Linux/Windows. -Learn more here: https://docs.google.com/document/d/10sXEhzFRSnvFcl3XxNGhnD4N2SedqwdAvK3dsihxVUA/edit#heading=h.wz5to0x8kqmr +## gym-gazebo -### Obstacle Tower +https://github.com/erlerobot/gym-gazebo/ -3D procedurally generated tower where you have to climb to the highest level possible +gym-gazebo presents an extension of the initial OpenAI gym for robotics using ROS and Gazebo, an advanced 3D modeling and +rendering tool. -Learn more here: https://github.com/Unity-Technologies/obstacle-tower-env +## GymFC: A flight control tuning and training framework -Platforms: Windows, Mac, Linux +https://github.com/wil3/gymfc/ -### PGE: Parallel Game Engine +GymFC is a modular framework for synthesizing neuro-flight controllers. The +architecture integrates digital twinning concepts to provide seamless transfer +of trained policies to hardware. The OpenAI environment has been used to +generate policies for the worlds first open source neural network flight +control firmware [Neuroflight](https://github.com/wil3/neuroflight). -PGE is a FOSS 3D engine for AI simulations, and can interoperate with the Gym. Contains environments with modern 3D graphics, and uses Bullet for physics. +## gym-jiminy: training Robots in Jiminy -Learn more here: https://github.com/222464/PGE +https://github.com/Wandercraft/jiminy -### gym-inventory: Inventory Control Environments +gym-jiminy presents an extension of the initial OpenAI gym for robotics using Jiminy, an extremely fast and light weight simulator for poly-articulated systems using Pinocchio for physics evaluation and Meshcat for web-based 3D rendering. -gym-inventory is a single agent domain featuring discrete state and action spaces that an AI agent might encounter in inventory control problems. +## gym-goddard: Goddard's Rocket Problem -Learn more here: https://github.com/paulhendricks/gym-inventory +https://github.com/osannolik/gym-goddard -### gym-gazebo: training Robots in Gazebo +An environment for simulating the classical optimal control problem where the thrust of a vertically ascending rocket shall be determined such that it reaches the maximum possible altitude, while being subject to varying aerodynamic drag, gravity and mass. -gym-gazebo presents an extension of the initial OpenAI gym for robotics using ROS and Gazebo, an advanced 3D modeling and -rendering tool. +## gym-pybullet-drones -Learn more here: https://github.com/erlerobot/gym-gazebo/ +https://github.com/JacopoPan/gym-pybullet-drones -### gym-maze: 2D maze environment -A simple 2D maze environment where an agent finds its way from the start position to the goal. +A simple environment using [PyBullet](https://github.com/bulletphysics/bullet3) to simulate the dynamics of a [Bitcraze Crazyflie 2.x](https://www.bitcraze.io/documentation/hardware/crazyflie_2_1/crazyflie_2_1-datasheet.pdf) nanoquadrotor -Learn more here: https://github.com/tuzzer/gym-maze/ +## Offworld-gym -### osim-rl: Musculoskeletal Models in OpenSim +https://github.com/offworld-projects/offworld-gym -A human musculoskeletal model and a physics-based simulation environment where you can synthesize physically and physiologically accurate motion. One of the environments built in this framework is a competition environment for a NIPS 2017 challenge. +Gym environments that let you control physics robotics in a laboratory via the internet. -Learn more here: https://github.com/stanfordnmbl/osim-rl +## MarsExplorer: Deep Reinforcement Learning for Extraterrestrial Exploration -### gym-minigrid: Minimalistic Gridworld Environment +https://github.com/dimikout3/MarsExplorer -A minimalistic gridworld environment. Seeks to minimize software dependencies, be easy to extend and deliver good performance for faster training. +Mars Explorer is an openai-gym compatible environment designed and developed as an initial endeavor to bridge the gap between powerful Deep Reinforcement Learning methodologies and the problem of exploration/coverage of an unknown terrain. -Learn more here: https://github.com/maximecb/gym-minigrid +## robo-gym: Environments for Real and Simulated Robots -### gym-miniworld: Minimalistic 3D Interior Environment Simulator +https://github.com/jr-robotics/robo-gym -MiniWorld is a minimalistic 3D interior environment simulator for reinforcement learning & robotics research. It can be used to simulate environments with rooms, doors, hallways and various objects (eg: office and home environments, mazes). MiniWorld can be seen as an alternative to VizDoom or DMLab. It is written 100% in Python and designed to be easily modified or extended. +robo-gym provides a collection of reinforcement learning environments involving robotic tasks applicable in both simulation and real world robotics. -Learn more here: https://github.com/maximecb/gym-miniworld +# Classic Environments (board, card, etc. games) -### gym-sokoban: 2D Transportation Puzzles +## gym-spoof -The environment consists of transportation puzzles in which the player's goal is to push all boxes on the warehouse's storage locations. -The advantage of the environment is that it generates a new random level every time it is initialized or reset, which prevents over fitting to predefined levels. +https://github.com/MouseAndKeyboard/gym-spoof -Learn more here: https://github.com/mpSchrader/gym-sokoban +Spoof, otherwise known as "The 3-coin game", is a multi-agent (2 player), imperfect-information, zero-sum game. -### gym-duckietown: Lane-Following Simulator for Duckietown +## gym-xiangqi: Xiangqi - The Chinese Chess Game -A lane-following simulator built for the [Duckietown](http://duckietown.org/) project (small-scale self-driving car course). +https://github.com/tanliyon/gym-xiangqi -Learn more here: https://github.com/duckietown/gym-duckietown +A reinforcement learning environment of Xiangqi, the Chinese Chess game. -### GymFC: A flight control tuning and training framework +## gym-abalone: A two-player abstract strategy board game -GymFC is a modular framework for synthesizing neuro-flight controllers. The -architecture integrates digital twinning concepts to provide seamless transfer -of trained policies to hardware. The OpenAI environment has been used to -generate policies for the worlds first open source neural network flight -control firmware [Neuroflight](https://github.com/wil3/neuroflight). +https://github.com/towzeur/gym-abalone -Learn more here: https://github.com/wil3/gymfc/ - -### gym-anytrading: Environments for trading markets +An implementation of the board game Abalone. -AnyTrading is a collection of OpenAI Gym environments for reinforcement learning-based trading algorithms with a great focus on simplicity, flexibility, and comprehensiveness. +## RubiksCubeGym -Learn more here: https://github.com/AminHP/gym-anytrading +https://github.com/DoubleGremlin181/RubiksCubeGym -### gym-mtsim: Real-world simulator/environment for trading markets +The RubiksCubeGym package provides environments for twisty puzzles with multiple reward functions to help simluate the methods used by humans. -MtSim is a general-purpose, flexible, and easy-to-use simulator alongside an OpenAI Gym trading environment for MetaTrader 5 trading platform. +# Autonomous Driving and Traffic Control Environments -Learn more here: https://github.com/AminHP/gym-mtsim +## gym-duckietown -### GymGo: The Board Game Go +https://github.com/duckietown/gym-duckietown -An implementation of the board game Go +A lane-following simulator built for the [Duckietown](http://duckietown.org/) project (small-scale self-driving car course). -Learn more here: https://github.com/aigagror/GymGo +## gym-electric-motor: Intelligent control of electric drives -### gym-electric-motor: Intelligent control of electric drives +https://github.com/upb-lea/gym-electric-motor An environment for simulating a wide variety of electric drives taking into account different types of electric motors and converters. Control schemes can be continuous, yielding a voltage duty cycle, or discrete, determining converter switching states directly. -Learn more here: https://github.com/upb-lea/gym-electric-motor +## highway-env: Tactical Decision-Making for Autonomous Driving -### NASGym: gym environment for Neural Architecture Search (NAS) +https://github.com/eleurent/highway-env -The environment is fully-compatible with the OpenAI baselines and exposes a NAS environment following the Neural Structure Code of [BlockQNN: Efficient Block-wise Neural Network Architecture Generation](https://arxiv.org/abs/1808.05584). Under this setting, a Neural Network (i.e. the state for the reinforcement learning agent) is modeled as a list of NSCs, an action is the addition of a layer to the network, and the reward is the accuracy after the early-stop training. The datasets considered so far are the CIFAR-10 dataset (available by default) and the meta-dataset (has to be manually downloaded as specified in [this repository](https://github.com/gomerudo/meta-dataset)). +An environment for behavioural planning in autonomous driving, with an emphasis on high-level perception and decision rather than low-level sensing and control. The difficulty of the task lies in understanding the social interactions with other drivers, whose behaviours are uncertain. Several scenes are proposed, such as highway, merge, intersection and roundabout. -Learn more here: https://github.com/gomerudo/nas-env +## gym-carla: Gym Wrapper for CARLA Driving Simulator -### gym-jiminy: training Robots in Jiminy +https://github.com/cjy1992/gym-carla -gym-jiminy presents an extension of the initial OpenAI gym for robotics using Jiminy, an extremely fast and light weight simulator for poly-articulated systems using Pinocchio for physics evaluation and Meshcat for web-based 3D rendering. +gym-carla provides a gym wrapper for the [CARLA simulator](http://carla.org/), which is a realistic 3D simulator for autonomous driving research. The environment includes a virtual city with several surrounding vehicles running around. Multiple source of observations are provided for the ego vehicle, such as front-view camera image, lidar point cloud image, and birdeye view semantic mask. Several applications have been developed based on this wrapper, such as deep reinforcement learning for end-to-end autonomous driving. -Learn more here: https://github.com/Wandercraft/jiminy +## sumo-rl -### highway-env: Tactical Decision-Making for Autonomous Driving +https://github.com/LucasAlegre/sumo-rl -An environment for behavioural planning in autonomous driving, with an emphasis on high-level perception and decision rather than low-level sensing and control. The difficulty of the task lies in understanding the social interactions with other drivers, whose behaviours are uncertain. Several scenes are proposed, such as highway, merge, intersection and roundabout. +Gym wrapper for various environments in the Sumo traffic simulator -Learn more here: https://github.com/eleurent/highway-env +# Other Environments -### gym-carla: Gym Wrapper for CARLA Driving Simulator +## gym-algorithmic -gym-carla provides a gym wrapper for the [CARLA simulator](http://carla.org/), which is a realistic 3D simulator for autonomous driving research. The environment includes a virtual city with several surrounding vehicles running around. Multiple source of observations are provided for the ego vehicle, such as front-view camera image, lidar point cloud image, and birdeye view semantic mask. Several applications have been developed based on this wrapper, such as deep reinforcement learning for end-to-end autonomous driving. +https://github.com/Rohan138/gym-algorithmic -Learn more here: https://github.com/cjy1992/gym-carla +These are a variety of algorithmic tasks, such as learning to copy a sequence, present in Gym prior to Gym 0.20.0. -### openmodelica-microgrid-gym: Intelligent control of microgrids +## gym-legacy-toytext -The OpenModelica Microgrid Gym (OMG) package is a software toolbox for the simulation and control optimization of microgrids based on energy conversion by power electronic converters. +https://github.com/Rohan138/gym-legacy-toytext -Learn more here: https://github.com/upb-lea/openmodelica-microgrid-gym +These are the unused toy-text environments present in Gym prior to Gym 0.20.0. -### RubiksCubeGym: OpenAI Gym environments for various twisty puzzles +## Obstacle Tower -The RubiksCubeGym package provides environments for twisty puzzles with multiple reward functions to help simluate the methods used by humans. +https://github.com/Unity-Technologies/obstacle-tower-env -Learn more here: https://github.com/DoubleGremlin181/RubiksCubeGym +3D procedurally generated tower where you have to climb to the highest level possible -### SlimeVolleyGym: A simple environment for single and multi-agent reinforcement learning +## PGE: Parallel Game Engine -A simple environment for benchmarking single and multi-agent reinforcement learning algorithms on a clone of Slime Volleyball game. Only dependencies are gym and numpy. Both state and pixel observation environments are available. The motivation of this environment is to easily enable trained agents to play against each other, and also facilitate the training of agents directly in a multi-agent setting, thus adding an extra dimension for evaluating an agent's performance. +https://github.com/222464/PGE -Learn more here: https://github.com/hardmaru/slimevolleygym +PGE is a FOSS 3D engine for AI simulations, and can interoperate with the Gym. Contains environments with modern 3D graphics, and uses Bullet for physics. -### Gridworld: A simple 2D grid environment +## gym-inventory: Inventory Control Environments -The Gridworld package provides grid-based environments to help simulate the results for model-based reinforcement learning algorithms. Initial release supports single agent system only. Some features in this version of software have become obsolete. New features are being added in the software like windygrid environment. +https://github.com/paulhendricks/gym-inventory -Learn more here: https://github.com/addy1997/Gridworld +gym-inventory is a single agent domain featuring discrete state and action spaces that an AI agent might encounter in inventory control problems. -### gym-goddard: Goddard's Rocket Problem +## gym-maze -An environment for simulating the classical optimal control problem where the thrust of a vertically ascending rocket shall be determined such that it reaches the maximum possible altitude, while being subject to varying aerodynamic drag, gravity and mass. +https://github.com/tuzzer/gym-maze/ -Learn more here: https://github.com/osannolik/gym-goddard +A simple 2D maze environment where an agent finds its way from the start position to the goal. -### gym-pybullet-drones: Learning Quadcopter Control +## osim-rl: Musculoskeletal Models in OpenSim -A simple environment using [PyBullet](https://github.com/bulletphysics/bullet3) to simulate the dynamics of a [Bitcraze Crazyflie 2.x](https://www.bitcraze.io/documentation/hardware/crazyflie_2_1/crazyflie_2_1-datasheet.pdf) nanoquadrotor +https://github.com/stanfordnmbl/osim-rl -Learn more here: https://github.com/JacopoPan/gym-pybullet-drones +A human musculoskeletal model and a physics-based simulation environment where you can synthesize physically and physiologically accurate motion. One of the environments built in this framework is a competition environment for a NIPS 2017 challenge. -### gym-derk: GPU accelerated MOBA environment +## gym-miniworld: Minimalistic 3D Interior Environment Simulator -This is a 3v3 MOBA environment where you train creatures to figth each other. It runs entirely on the GPU so you can easily have hundreds of instances running in parallel. There are around 15 items for the creatures, 60 "senses", 5 actions, and ~23 tweakable rewards. It's also possible to benchmark an agent against other agents online. It's available for free for training for personal use, and otherwise costs money; see licensing details on the website. +https://github.com/maximecb/gym-miniworld -More here: https://gym.derkgame.com +MiniWorld is a minimalistic 3D interior environment simulator for reinforcement learning & robotics research. It can be used to simulate environments with rooms, doors, hallways and various objects (eg: office and home environments, mazes). MiniWorld can be seen as an alternative to VizDoom or DMLab. It is written 100% in Python and designed to be easily modified or extended. -### gym-abalone: A two-player abstract strategy board game +## gym-sokoban: 2D Transportation Puzzles -An implementation of the board game Abalone. +https://github.com/mpSchrader/gym-sokoban -Learn more here: https://github.com/towzeur/gym-abalone +The environment consists of transportation puzzles in which the player's goal is to push all boxes on the warehouse's storage locations. +The advantage of the environment is that it generates a new random level every time it is initialized or reset, which prevents over fitting to predefined levels. -### gym-adserver: Environment for online advertising +## gym-anytrading: Environments for trading markets -An environment that implements a typical [multi-armed bandit scenario](https://en.wikipedia.org/wiki/Multi-armed_bandit) where an [ad server](https://en.wikipedia.org/wiki/Ad_serving) must select the best advertisement to be displayed in a web page. Some example agents are included: Random, epsilon-Greedy, Softmax, and UCB1. +https://github.com/AminHP/gym-anytrading -Learn more here: https://github.com/falox/gym-adserver +AnyTrading is a collection of OpenAI Gym environments for reinforcement learning-based trading algorithms with a great focus on simplicity, flexibility, and comprehensiveness. -### gym-autokey: Automated rule-based deductive program verification +## gym-mtsim: Real-world simulator/environment for trading markets -An environment for automated rule-based deductive program verification in the KeY verification system. +https://github.com/AminHP/gym-mtsim -Learn more here: https://github.com/Flunzmas/gym-autokey +MtSim is a general-purpose, flexible, and easy-to-use simulator alongside an OpenAI Gym trading environment for MetaTrader 5 trading platform. -### gym-riverswim: A hard-exploration environment +## NASGym: gym environment for Neural Architecture Search (NAS) -A simple environment for benchmarking reinforcement learning exploration techniques in a simplified setting. +https://github.com/gomerudo/nas-env -Learn more here: https://github.com/erfanMhi/gym-riverswim +The environment is fully-compatible with the OpenAI baselines and exposes a NAS environment following the Neural Structure Code of [BlockQNN: Efficient Block-wise Neural Network Architecture Generation](https://arxiv.org/abs/1808.05584). Under this setting, a Neural Network (i.e. the state for the reinforcement learning agent) is modeled as a list of NSCs, an action is the addition of a layer to the network, and the reward is the accuracy after the early-stop training. The datasets considered so far are the CIFAR-10 dataset (available by default) and the meta-dataset (has to be manually downloaded as specified in [this repository](https://github.com/gomerudo/meta-dataset)). -### gym-ccc: Continuous classic control environments +## openmodelica-microgrid-gym: Intelligent control of microgrids -Environments that extend gym's classic control and add more problems. -These environments have features useful for non-RL controllers. +https://github.com/upb-lea/openmodelica-microgrid-gym -The main highlights are: -1) non normalized observation corresponding directly to the dynamical state -2) normalized observation with dynamical state captured in `info['state']` -3) action spaces are continuous -4) system parameters (mass, length, etc.) can be specified -5) reset function (to specify initial conditions) can be specified. +The OpenModelica Microgrid Gym (OMG) package is a software toolbox for the simulation and control optimization of microgrids based on energy conversion by power electronic converters. -Learn more here: https://github.com/acxz/gym-ccc +## Gridworld: A simple 2D grid environment -### NLPGym: A toolkit to develop RL agents to solve NLP tasks +https://github.com/addy1997/Gridworld -[NLPGym](https://arxiv.org/pdf/2011.08272v1.pdf) provides interactive environments for standard NLP tasks such as sequence tagging, question answering, and sequence classification. Users can easily customize the tasks with their own datasets, observations, featurizers and reward functions. +The Gridworld package provides grid-based environments to help simulate the results for model-based reinforcement learning algorithms. Initial release supports single agent system only. Some features in this version of software have become obsolete. New features are being added in the software like windygrid environment. -Learn more here: https://github.com/rajcscw/nlp-gym +## gym-adserver: Environment for online advertising -### math-prog-synth-env +https://github.com/falox/gym-adserver -In our paper "A Reinforcement Learning Environment for Mathematical Reasoning via Program Synthesis" we convert the DeepMind Mathematics Dataset into an RL environment based around program synthesis. +An environment that implements a typical [multi-armed bandit scenario](https://en.wikipedia.org/wiki/Multi-armed_bandit) where an [ad server](https://en.wikipedia.org/wiki/Ad_serving) must select the best advertisement to be displayed in a web page. Some example agents are included: Random, epsilon-Greedy, Softmax, and UCB1. -Learn more here: https://github.com/JohnnyYeeee/math_prog_synth_env , https://arxiv.org/abs/2107.07373 +## gym-autokey: Automated rule-based deductive program verification -### VirtualTaobao: Environment of online recommendation +https://github.com/Flunzmas/gym-autokey -An environment for online recommendation, where customers are learned from Taobao.com, one of the world's largest e-commerce platform. +An environment for automated rule-based deductive program verification in the KeY verification system. -Learn more here: https://github.com/eyounx/VirtualTaobao/ +## gym-riverswim: A hard-exploration environment -### gym-recsys: Customizable RecSys Simulator for OpenAI Gym +https://github.com/erfanMhi/gym-riverswim -This package describes an OpenAI Gym interface for creating a simulation environment of reinforcement learning-based recommender systems (RL-RecSys). The design strives for simple and flexible APIs to support novel research. +A simple environment for benchmarking reinforcement learning exploration techniques in a simplified setting. -Learn more here: https://github.com/zuoxingdong/gym-recsys +## gym-ccc: Continuous classic control environments -### QASGym: gym environment for Quantum Architecture Search (QAS) +https://github.com/acxz/gym-ccc -This a list of environments for quantum architecture search following the description in [Quantum Architecture Search via Deep Reinforcement Learning](https://arxiv.org/abs/2104.07715). The agent design the quantum circuit by taking actions in the environment. Each action corresponds to a gate applied on some wires. The goal is to build a circuit U such that generates the target n-qubit quantum state that belongs to the environment and hidden from the agent. The circuits are built using [Google QuantumAI Cirq](https://quantumai.google/cirq). +Environments that extend gym's classic control and add many new features. -Learn more here: https://github.com/qdevpsi3/quantum-arch-search +## NLPGym: A toolkit to develop RL agents to solve NLP tasks -### robo-gym: Environments for Real and Simulated Robots +https://github.com/rajcscw/nlp-gym -robo-gym provides a collection of reinforcement learning environments involving robotic tasks applicable in both simulation and real world robotics. +[NLPGym](https://arxiv.org/pdf/2011.08272v1.pdf) provides interactive environments for standard NLP tasks such as sequence tagging, question answering, and sequence classification. Users can easily customize the tasks with their own datasets, observations, featurizers and reward functions. -Learn more here: https://github.com/jr-robotics/robo-gym +## math-prog-synth-env -### gym-xiangqi: Xiangqi - The Chinese Chess Game +https://github.com/JohnnyYeeee/math_prog_synth_env -A reinforcement learning environment of Xiangqi, the Chinese Chess game. +In our paper "A Reinforcement Learning Environment for Mathematical Reasoning via Program Synthesis" we convert the DeepMind Mathematics Dataset into an RL environment based around program synthesis.https://arxiv.org/abs/2107.07373 -Learn more here: https://github.com/tanliyon/gym-xiangqi +## VirtualTaobao: Environment of online recommendation -### anomalous_rl_envs: Gym environments with anomaly injection +https://github.com/eyounx/VirtualTaobao/ -A set of environments from control tasks: Acrobot, CartPole, and LunarLander with various types of anomalies injected into them. It could be very useful to study the behavior and robustness of a policy. +An environment for online recommendation, where customers are learned from Taobao.com, one of the world's largest e-commerce platform. -Learn more here: https://github.com/modanesh/anomalous_rl_envs +## gym-recsys: Customizable RecSys Simulator -### stable-retro +https://github.com/zuoxingdong/gym-recsys -Supported fork of gym-retro with additional games, states, scenarios, etc. Open to PRs of additional games, features and plateforms since gym-retro is no longer maintained +This package describes an OpenAI Gym interface for creating a simulation environment of reinforcement learning-based recommender systems (RL-RecSys). The design strives for simple and flexible APIs to support novel research. -https://github.com/MatPoliquin/stable-retro +## QASGym: gym environment for Quantum Architecture Search (QAS) + +https://github.com/qdevpsi3/quantum-arch-search + +This a list of environments for quantum architecture search following the description in [Quantum Architecture Search via Deep Reinforcement Learning](https://arxiv.org/abs/2104.07715). The agent design the quantum circuit by taking actions in the environment. Each action corresponds to a gate applied on some wires. The goal is to build a circuit U such that generates the target n-qubit quantum state that belongs to the environment and hidden from the agent. The circuits are built using [Google QuantumAI Cirq](https://quantumai.google/cirq). + +## anomalous_rl_envs: Gym environments with anomaly injection -### CompilerGym +https://github.com/modanesh/anomalous_rl_envs + +A set of environments from control tasks: Acrobot, CartPole, and LunarLander with various types of anomalies injected into them. It could be very useful to study the behavior and robustness of a policy. + +## CompilerGym + +https://github.com/facebookresearch/CompilerGym Reinforcement learning environments for compiler optimization tasks, such as LLVM phase ordering, GCC flag tuning, and CUDA loop nest code generation. -Learn more here: https://github.com/facebookresearch/CompilerGym +## LongiControl -### LongiControl +https://github.com/dynamik1703/gym_longicontrol An environment for the stochastic longitudinal control of an electric vehicle. It is intended to be a descriptive and comprehensible example for a continuous real-world problem within the field of autonomous driving. -Learn more here: https://github.com/dynamik1703/gym_longicontrol +## safe-control-gym -### safe-control-gym +https://github.com/utiasDSL/safe-control-gym PyBullet-based CartPole and Quadrotor environments—with [CasADi](https://web.casadi.org) (symbolic) *a priori* dynamics and constraints—for learning-based control and model-based reinforcement learning. -Learn more here: https://github.com/utiasDSL/safe-control-gym - -### MarsExplorer: Deep Reinforcement Learning for Extraterrestrial Exploration +## gym-games -Mars Explorer is an openai-gym compatible environment designed and developed as an initial endeavor to bridge the gap between powerful Deep Reinforcement Learning methodologies and the problem of exploration/coverage of an unknown terrain. +https://github.com/qlan3/gym-games -Learn more here: https://github.com/dimikout3/MarsExplorer +Gym implementations of the MinAtar games, various PyGame Learning Environment games, and various custom exploration games diff --git a/docs/misc.md b/docs/misc.md deleted file mode 100644 index abdcaef6b49..00000000000 --- a/docs/misc.md +++ /dev/null @@ -1,7 +0,0 @@ -# Miscellaneous - -Here we have a bunch of tools, libs, apis, tutorials, resources, etc. provided by the community to add value to the gym ecosystem. - -## OpenAIGym.jl - -Convenience wrapper of the OpenAI Gym for the Julia language [/tbreloff/OpenAIGym.jl](https://github.com/tbreloff/OpenAIGym.jl) \ No newline at end of file