feat(python): add include_index option on init from pandas frames

[alexander-beedie]

Closes #6763.

Allows for optional loading of non-default pandas frame indexes as columns; defaults to False to ensure no changes in behaviour - we can revisit that default later, if requested/desired.

Setup

from datetime import datetime
import pandas as pd
import polars as pl

pdf = pd.DataFrame(
  {
    "dtm": [datetime(2023,1,1), datetime(2023,1,2)],
    "val": [100, 200],
    "misc": ["x", "y"],
  }
).set_index( ["dtm"] )

#             val misc
# dtm                 
# 2023-01-01  100    x
# 2023-01-02  200    y

Before (caller has to manually reset the index if they want to load it)

pl.from_pandas( pdf )

# shape: (2, 2)
# ┌─────┬──────┐
# │ val ┆ misc │
# │ --- ┆ ---  │
# │ i64 ┆ str  │
# ╞═════╪══════╡
# │ 100 ┆ x    │
# │ 200 ┆ y    │
# └─────┴──────┘

After (new param allows for easy/optimised index load)

pl.from_pandas( pdf, include_index=True )

# shape: (2, 3)
# ┌─────────────────────┬─────┬──────┐
# │ dtm                 ┆ val ┆ misc │
# │ ---                 ┆ --- ┆ ---  │
# │ datetime[ns]        ┆ i64 ┆ str  │
# ╞═════════════════════╪═════╪══════╡
# │ 2023-01-01 00:00:00 ┆ 100 ┆ x    │
# │ 2023-01-02 00:00:00 ┆ 200 ┆ y    │
# └─────────────────────┴─────┴──────┘

Update

Using this method is now optimal in comparison to the caller using reset_index, as we're able to avoid the pandas-side copy that would trigger.

[ritchie46]

Nice! Shall we immediately make it keyword only?

[alexander-beedie]

Nice! Shall we immediately make it keyword only?

Sure, let's do it; it is this season's on-trend coding style... :)
Will update that along with tweaking the index reset to guarantee no unnecessary pandas-side copies.

[alexander-beedie]

@ritchie46: done - include_index is now kwarg-only, and the index load has been memory-optimised 👍

[ritchie46]

Thanks everybody! For the good reviews and solutions!