[RLlib] Enable eager_tracing=True by default. #36556
Conversation
| self.assertGreaterEqual(coeff, 0.0001) | ||
| else: | ||
| self.assertLessEqual(coeff, 0.01) | ||
| self.assertGreaterEqual(coeff, 0.001) |
There was a problem hiding this comment.
Can we redesign this test a little bit?
It could be simpler for example by doing:
entropy_coeff_schedule=[[0, 0.1], [200, 0.001], [600, 0.0001]]
Also, _step_n_times() should be "step_until_n_steps_reached()".
We should then be able to reuse this with entropy coefficient tests for other algorithms if so desired.
The "~100 timesteps" thing can easily change per algorithm or when something else in the algorithm under test changes that has nothing to do with the coefficient schedule.
There was a problem hiding this comment.
I need to fix this, yes. I think b/c eager tracing is much faster, the async sampling also runs faster in the background.
I will add a proper check here to make sure this test performs the right checks based on the actual timesteps sampled.
| and not exp["config"].get("eager_tracing") is False | ||
| ): | ||
|
|
||
| exp["config"]["eager_tracing"] = True |
There was a problem hiding this comment.
Can we actually make this so that when local mode is True for the regression tests script (we use this only for debugging, right?), eager tracing will be False?
There was a problem hiding this comment.
I think in almost all cases we are happy that eager tracing is disabled when using local mode here?
There was a problem hiding this comment.
True, but I feel like then you should just set it in your config, no? It's not good to change stuff b/c we assume something w/o the user being in control. I have run into this issue several times while debugging, thinking that eager tracing was True (I wanted to debug a bug that only happened for eager_tracing=True), when it wasn't b/c I was also using local mode. It took me a while to find out that RLlib had sneakily changed my config :)
Signed-off-by: e428265 <arvind.chandramouli@lmco.com>
Enable eager_tracing=True by default.
When running Algorithms with framework=="tf2", it is considerably slower to do so with the old default eager_tracing=False setting. This PR switches the default setting from eager_tracing=False to True, RLlib-wide.
Why are these changes needed?
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.