Pin pydantic to < 2.0.0

[alanwguo]

2.0.0 brings breaking changes

Cherry-pick of #37000

Here's a failed CI run: https://buildkite.com/ray-project/oss-ci-build-pr/builds/27338#01890d26-94ff-42f2-80be-c2ac0a86e8d3/447-762

Why are these changes needed?

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[bveeramani]

@can-anyscale for my own understanding, when do we need to update versions in the BYOD files?

[can-anyscale]

@bveeramani BYOD is only for release test dependencies, and pydantic is not a part of it so we don't need to update BYOD dependencies in this case. Generally, DevProd can handle the upgrade of BYOD dependencies, but test owners can update them at anytime as well.