Finish switching to PyO3 0.21's native datetime support, now available with abi3, once Python 3.9 is the minimum version

[itamarst]

Description

In Python 3.9 there's now a timezone database available. That means PyO3 can have native timezone conversion between chrono_tz::Tz and Python timezone objects.

Currently Polars has its own custom conversion code, which is likely slower than PyO3's since it's in Python. However, until Python 3.9 is the minimum supported version, this change can't happen. I.e. this should be revisited in November 2024 or later.

(See also #16158, where most of the relevant work was done.)

[itamarst]

I will try to do this.

[itamarst]

Another place that should be updated maybe is conversion/any_value.rs in polars-python crate, which refers to some limitation in abi3-py37.

[mcrumiller]

Is this related to #20111?

[itamarst]

It's not a requirement at least, it's been possible to implement this since Polars dropped 3.8 support (a month or two ago?).

[bschoenmaeckers]

I'm interested to pick this up if nobody minds.

[itamarst]

I was gonna start on this, unless you've already started @bschoenmaeckers ?

[bschoenmaeckers]

I’ve started the work on this but was not able to continue because of pyo3 wrongly converting ambiguous datetimes. This is fixed in the next version of pyo3 so we have to wait until then.