[SPARK-40500][PS] Deprecate iteritems in DataFrame and Seriese

[zhengruifeng]

What changes were proposed in this pull request?

1. Use pd.items instead of pd.iteritems
2. Deprecate ps.iteritems

Why are the changes needed?

pd.iteritems is deprecated in 1.5

before:

In [4]: import pyspark.pandas as ps

In [5]: ps.Series([3, 4, 1, 1, 5])
/Users/ruifeng.zheng/Dev/spark/python/pyspark/pandas/internal.py:1573: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.
  fields = [
/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/pandas/conversion.py:486: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.
  for column, series in pdf.iteritems():
                                                                                
0    3
1    4
2    1
3    1
4    5
dtype: int64

after:

In [1]: import pyspark.pandas as ps

In [2]: ps.Series([3, 4, 1, 1, 5])
                                                                                
0    3
1    4
2    1
3    1
4    5
dtype: int64

Does this PR introduce any user-facing change?

Eliminate iteritems warnings

How was this patch tested?

existing UT

[srowen]

Seems OK, but there also seem to be more usages in the code, like in frame.py and test_dataframe.py - are those usages the same, also deprecated?

[zhengruifeng]

remaining iteritemss in frame.py and test_dataframe.py are the definition and tests of PS's iteritems itself, so I think we should not modify them.

as to the deprecation of PS's iteritems, I think we can deprecate them now, WDYT @itholic @HyukjinKwon @Yikun

[HyukjinKwon]

Yeah let's match w/ pandas

[Yikun]

as to the deprecation of PS's iteritems, I think we can deprecate them now

Agree. And items are also supported by _minimum_pandas_version (v1.0.5), so it's a safe change.

>>> pd.__version__
'1.0.5'
>>> pd.Series([3, 4, 1, 1, 5]).iteritems()
<zip object at 0x7f2087983280>
>>> pd.Series([3, 4, 1, 1, 5]).items()
<zip object at 0x7f2087afcb80>

so LGTM

[itholic]

Then maybe should we also add .. deprecated:: 3.4.0 into the docstring of DataFrame.iteritems and Series.iteritems ??

Otherwise, LGTM.

[HyukjinKwon]

Merged to master.

[zhengruifeng]

Thank you all!