pybullet HumanoidBulletEnv-v0 score is not reasonable

[zhan0903]

Hi, I use the Openai Spinningup td3 algorithm to test pybullet's HumanoidBulletEnv-v0 environment, but got the test score around 1600 even from the beginning which is not reasonable (td3 should not work in this benchmark), why is it like this? Does anyone have similar results? Thank you.

[erwincoumans]

Can you share the learning curve and a video of the resulting policy? Also, steps to reproduce it, in detail (pip installs, apt-get installs etc)

[zhan0903]

Can you share the learning curve and a video of the resulting policy? Also, steps to reproduce it, in detail (pip installs, apt-get installs etc)

Hi, I install the pybullet through pip install pybullet, and use the Openai td3's default setting for the test. Below is its test result(average over ten times test) from seed 4.

Every seed(0-4) has similar testing results and no significant change during the whole training process(the test results stay around 1600).

[erwincoumans]

Do you have a video of the resulting policy?

I am not familiar with Openai td3. How do you exactly install it? What version of Tensorflow, gym etc?

[zhan0903]

Do you have a video of the resulting policy?

I am not familiar with Openai td3. How do you exactly install it? What version of Tensorflow, gym etc?

Hi, TensorFlow version is 1.14.0, gym version:0.14.0, I install it just flow its official instruction on its website openai spinningup. I tried to run the resulting policy in humanoid env, and the policy failed.

[zhan0903]

Do you have a video of the resulting policy?

I am not familiar with Openai td3. How do you exactly install it? What version of Tensorflow, gym etc?

By the way, how to save the simulation to a video? I can run it on my computer(for humanoid env, the policy seems failed), but don't know how to save it to a video. I tried to use gym.wrappers.Monitor to save it but occurred this error "TypeError: 'module' object is not callable" .

[brokenBrain]

I'm having the same issue. In my case, I'm instantiating the environment twice. The first environment seems to be behaving normally. But the second environment is misbehaving in exactly the above manner.

[erwincoumans]

I haven't trained humanoid myself. Did you check out https://github.com/hill-a/stable-baselines/releases
and https://github.com/araffin/rl-baselines-zoo
It has a trained policy using ppo2 and td3.

[forcecore]

The issue is real and it is in pybullet-gym implementation.
The fix is here: benelot/pybullet-gym#61