[Bug Report] Import of custom environments is broken in 0.22

[vadim0x60]

Describe the bug
In gym 0.21 there is a useful feature for loading custom environments. One can call

import gym
gym.make('module:Env')

And gym will import the module before trying to make Env. In gym 0.22 it got broken.

Code example

Make a file named test.py:

print('File loaded!')

import gym

class DummyEnv(gym.Env):
    action_space = gym.spaces.Discrete(2)
    observation_space = gym.spaces.Discrete(2)

    def reset(self):
        obs = observation_space.sample()
        return obs

    def step(self, action):
        obs = observation_space.sample()
        r = 0
        done = False
        return obs, r, done, {}

gym.register('Dummy-v0', entry_point='test:DummyEnv')

then, in python console or in another file, run gym.make('test:Dummy-v0'). If you have gym 0.21.0 installed, you get

File loaded!
<OrderEnforcing<DummyEnv<Dummy-v0>>>

With gym 0.22.0 it's

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/320012346/.cache/pypoetry/virtualenvs/cleanrl-9SW_WqcQ-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 676, in make
    return registry.make(id, **kwargs)
  File "/home/320012346/.cache/pypoetry/virtualenvs/cleanrl-9SW_WqcQ-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 490, in make
    versions = self.env_specs.versions(namespace, name)
  File "/home/320012346/.cache/pypoetry/virtualenvs/cleanrl-9SW_WqcQ-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 220, in versions
    self._assert_name_exists(namespace, name)
  File "/home/320012346/.cache/pypoetry/virtualenvs/cleanrl-9SW_WqcQ-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 297, in _assert_name_exists
    raise error.NameNotFound(message)
gym.error.NameNotFound: Environment `test:Dummy` doesn't exist.

System Info

Python 3.9.12, installing gym 0.21.0 and 0.22.0 via poetry on Ubuntu 18.04.6 LTS, 4.15.0-176-generic

Checklist

• I have checked that there is no similar issue in the repo (required)

[pseudo-rnd-thoughts]

You need to register the environment before making an environment
https://www.gymlibrary.ml/content/environment_creation/#registering-envs

In 0.22.0, there was a large modification to the make function which is probably what caused this issue.
If you want, you can initialise the environment without make and add the wrappers yourself (TimeLimit and OrderEnforcing).

[vadim0x60]

You need to register the environment before making an environment

Last line of test.py

gym.register('Dummy-v0', entry_point='test:DummyEnv')

[pseudo-rnd-thoughts]

Sorry, I didnt see that final line, I should read the code more closely
I believe the issue is that you need to run gym.make('Dummy-v0') not gym.make('test:Dummy-v0')
This works with 0.21, 0.22 and 0.23 and had no issues.

[Markus28]

I can reproduce this behavior and it seems like a serious bug.

@RedTachyon could you explain how the registration of custom environments is supposed to work and which module has to be imported to make the custom environment? When I wrote the documentation I also faced this issue. Therefore, I wrote that we need to import the custom module before calling gym.make but this is probably not how it is supposed to work.

@pseudo-rnd-thoughts I think the point is that in 0.22 you need to import test before calling gym.make for things to work and that's not the case in 0.21

[RedTachyon]

First time I'm hearing of the pattern gym.make('module:Env'), was this documented somewhere or mentioned in some tutorial or something?

[Markus28]

I doubt that there is any real documentation of this (because there hasn't been any real documentation at all so far). However, the template of the environment creation page in gym-docs used this pattern at the very bottom of the page:

https://github.com/Farama-Foundation/gym-docs/blob/b37fa0e97dc325b7ea96b81c89c026cfb53cc09f/docs/pages/environment_creation/index.md

It didn't work for me so I changed it to something that worked with 0.22.

The template page appeared in the second commit in gym-docs (by Jordan). It was titled "move over docs". Idk where it came from.

[RedTachyon]

Ok, I took a closer look at the code, I think I know what happened.

The problem is that you used to be able to write, for example:

import gym
gym.make('pybullet_envs:HopperBulletEnv-v0')

and this will work, because gym.make will import pybullet_envs under the hood (pybullet_envs is just an example of a library that you can install, and which will register some envs when you import it)

This happens due to the load(...) function in gym/envs/registration.py which imports whatever is before the colon.

Between 0.21 and 0.22, there was a pretty large change to the registration mechanism, which made environment creation a bit more precise - it uses all this fancy logic to check if the environment exists, if there's some typos etc. It seems to have forgotten this specific pattern, which is the unfortunate reality of maintaining a previously abandoned project.

This same registration mechanism happens to be completely erased and revamped now for the upcoming 0.24 release, and integrating this feature into it would be fairly simple with the new approach if there's interest in it. I agree it's actually a pretty useful thing, but I'm treating it as more of a "new feature request" at this point, since it was a niche feature that accidentally disappeared a few releases ago.

[vadim0x60]

Thanks for looking into it! If this is now a "new feature request", let me quickly make a case for this feature.

It's most useful for programs that accept the name of the environment as a command line argument, environment variable, etc. - basically some form of input. So,

yet_another_train ppo CartPole-v0

can become

yet_another_train ppo module:Custom-v0

The alternative is modifying the source code of yet_another_train, an arbitrarily complicated library that you didn't write. Some libraries even include an instruction of where to edit their source code for this purpose.

As far as I understand, the newly implemented plugin system addresses the same issue or at least overlaps a lot. But, again, as far as I understand, the plugin protocol needs to be implemented by the author of the environment. module:Env syntax, on the other hand, lets me just magically connect someone's training library that doesn't put a lot of thought into supporting custom environments to a custom environment that doesn't put a lot of thought into compatibility with training libraries.

[pseudo-rnd-thoughts]

@RedTachyon Are you happy to work on this new feature?

[RedTachyon]

I don't really have the bandwidth to do it properly at the moment, if someone else wants to work on it now, that'd be great. Otherwise I can come back to it in some time. I can also offer pretty specific guidance on how I'd see this actually being implemented - it shouldn't be too complex overall, and most of the work would be writing tests and documentation (which is the bottleneck for me right now)

[arjun-kg]

I can work on this if that's okay. I'll maybe check with @RedTachyon if I run into any complications.

[lasri049]

Hi, can you please tell me if there is any update on this?

[Markus28]

It should work properly in the more recent releases of Gym (maybe try 0.25)