add functions to inspect model and optimizer status to trainer.py

[CKeibel]

What does this PR do?

Add functions get_num_trainable_parameters to return the number of parameters which require grad, get_learning_rates to return the learning rates of parameter groups and get_optimizer_group to return optimizer groups for parameters.

Fixes #29016

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts @muellerzr

[muellerzr]

Very nice PR! Any chance you could add some basic tests in tests/trainer/test_trainer.py for these?

[CKeibel]

Very nice PR! Any chance you could add some basic tests in tests/trainer/test_trainer.py for these?

Of course, I wasn't sure where to add them, but now I see that I missed that test_trainer.py exists. I will add the tests later. :)

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[muellerzr]

Great, thanks! I'll give a ✅ after, great job and very clean implementation :)

[amyeroberts]

Nice - thanks for adding!

+1 for adding tests. Once those are done, you can ping for a final review and we can merge 🤗

[muellerzr]

Let's perhaps make this two linear layers, one that's frozen, one that's not, so this way trainable checks if we have different optimizer groups? :)

[CKeibel]

Yes, you're right, that would definitely be better :)

[amyeroberts]

Nice tests! Thanks for adding!

Just a small nit on the torch condition - then we're good to merge!

[muellerzr]

Thanks! Looks good to me as well, let's fix that spacing though 😉

[CKeibel]

Is there anything else to do? :) @amyeroberts @muellerzr

[amyeroberts]

@CKeibel Nope - merging now!

[CKeibel]

Excellent thanks, then I'll start looking for a new issue to work on. :)

[amyeroberts]

Looking forward to the future PRs :) Thanks again for contributing to improving the library!