[RLlib] DatasetReader action normalization

[charlesjsun]

Why are these changes needed?

DatasetReader didn't normalize actions when actions_in_input_normalized was False, which meant CRR was using unnormalized actions during training.

Added normalization to DatasetReader (same as JsonReader)

Related issue number

Closes #27335

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[avnishn]

this looks good to me. I think one thing is that if there isn't a test for this already, we should make the test you added apply not just to offline datasets but also to other regular environments.

If you think im wrong lmk :)

[avnishn]

nice

[charlesjsun]

this looks good to me. I think one thing is that if there isn't a test for this already, we should make the test you added apply not just to offline datasets but also to other regular environments.

If you think im wrong lmk :)

This PR doesn't apply in the online case because this just normalizes actions from dataset (i.e. DatasetReader). Online action normalization is handled elsewhere (somewhere in env runner).

[avnishn]

The only other thing I would test if I were you is that if you set any other flag combination of actions_in_input_normalized and normalize_actions that the actions won't be normalized by dataset if they aren't supposed to be

[charlesjsun]

The only other thing I would test if I were you is that if you set any other flag combination of actions_in_input_normalized and normalize_actions that the actions won't be normalized by dataset if they aren't supposed to be

Addressed! Added all combinations of actions_in_input_normalized and normalize_actions