BUG: Cast ExtensionArray to numpy ndarray before plot

[sighingnow]

Cast ExtensionArray(eg. IntegerArray) to np.ndarray before send to matplotlib to plot, fixes #25587.

• closes plotting dtype "UInt32" fails #25587
• tests added / passed
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

[codecov]

Codecov Report

Merging #25590 into master will increase coverage by <.01%.
The diff coverage is 80%.

@@            Coverage Diff             @@
##           master   #25590      +/-   ##
==========================================
+ Coverage   91.26%   91.26%   +<.01%     
==========================================
  Files         173      173              
  Lines       52966    52971       +5     
==========================================
+ Hits        48337    48343       +6     
+ Misses       4629     4628       -1

Flag	Coverage Δ
#multiple	89.83% <80%> (ø)	⬆️
#single	41.7% <20%> (-0.01%)	⬇️

Impacted Files	Coverage Δ
pandas/plotting/_core.py	83.64% <80%> (+0.05%)	⬆️
pandas/util/testing.py	87.66% <0%> (+0.09%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 74a9ae3...b3445fc. Read the comment docs.

[codecov]

Codecov Report

Merging #25590 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #25590      +/-   ##
==========================================
+ Coverage   91.25%   91.25%   +<.01%     
==========================================
  Files         172      172              
  Lines       52973    52976       +3     
==========================================
+ Hits        48338    48341       +3     
  Misses       4635     4635

Flag	Coverage Δ
#multiple	89.82% <100%> (ø)	⬆️
#single	41.74% <0%> (ø)	⬆️

Impacted Files	Coverage Δ
pandas/plotting/_core.py	83.64% <100%> (+0.1%)	⬆️
pandas/util/testing.py	88.98% <0%> (-0.1%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 998e1de...3afc9eb. Read the comment docs.

[TomAugspurger]

Are there any EAs we wouldn't want a regular np.asarary for? IOW, do we need to dispatch to the EA for plotting specifically?

[TomAugspurger]

Yeah, ignoring those plots for now seems fine.

…

On Fri, Mar 8, 2019 at 8:43 PM Tao He ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In pandas/plotting/_core.py <#25590 (comment)>: > @@ -574,6 +576,13 @@ def _get_xticks(self, convert_period=False): @classmethod def _plot(cls, ax, x, y, style=None, is_errorbar=False, **kwds): + # GH25587: cast ExtensionArray of pandas (IntegerArray, etc.) to After move the conversion to _compute_plot_data, kde and area plots still don't work because we cast IntegerArray to nd.array(dtype=object), but kde and area needs dtype that can be used to ufunc add of numpy. Could we ignore the tests for kde and area and open a new issue to track that trouble? The traceback of `kde` File "D:\Open\pandas\pandas\plotting\_core.py", line 1474, in _plot gkde = gaussian_kde(y, bw_method=bw_method) File "D:\Program\Conda\lib\site-packages\scipy\stats\kde.py", line 208, in __init__ self.set_bandwidth(bw_method=bw_method) File "D:\Program\Conda\lib\site-packages\scipy\stats\kde.py", line 540, in set_bandwidth self._compute_covariance() File "D:\Program\Conda\lib\site-packages\scipy\stats\kde.py", line 551, in _compute_covariance aweights=self.weights)) File "D:\Program\Conda\lib\site-packages\numpy\lib\function_base.py", line 2427, in cov avg, w_sum = average(X, axis=1, weights=w, returned=True) File "D:\Program\Conda\lib\site-packages\numpy\lib\function_base.py", line 419, in average scl = wgt.sum(axis=axis, dtype=result_dtype) File "D:\Program\Conda\lib\site-packages\numpy\core\_methods.py", line 36, in _sum return umr_sum(a, axis, dtype, out, keepdims, initial) TypeError: No loop matching the specified signature and casting was found for ufunc add The traceback of `area` File "D:\Open\pandas\pandas\plotting\_core.py", line 252, in generate self._make_plot() File "D:\Open\pandas\pandas\plotting\_core.py", line 1000, in _make_plot **kwds) File "D:\Open\pandas\pandas\plotting\_core.py", line 1151, in _plot cls._update_stacker(ax, stacking_id, y) File "D:\Open\pandas\pandas\plotting\_core.py", line 1083, in _update_stacker ax._stacker_pos_prior[stacking_id] += values TypeError: ufunc 'add' output (typecode 'O') could not be coerced to provided output parameter (typecode 'd') according to the casting rule ''same_kind'' — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#25590 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIvIcdLkUY6mBqeRC7WEZe7yGwcLMks5vUx_RgaJpZM4bjP6l> .

[sighingnow]

Fix the lint error and add more tests for different kinds of plotting.

@TomAugspurger I'm wondering how should I resolve the conflicts between this PR and master? Merge or rebase and push with --force?

Thanks in advance!

[teto]

I don't use kde and area plots so I am fine with the current PR. Hopefully it can make it before the next release.

[TomAugspurger]

Something like `df = df.assign(**dict(zip(df.columns, values)))` should work.

…

On Wed, Mar 13, 2019 at 10:06 PM Tao He ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In pandas/plotting/_core.py <#25590 (comment)>: > @@ -365,6 +366,12 @@ def _compute_plot_data(self): if is_empty: raise TypeError('no numeric data to plot') + # GH25587: cast ExtensionArray of pandas (IntegerArray, etc.) to .assign still doesn't work (if I don‘’t miss something). .assign requires kwargs, which is a dict of {str. callable or Series}, however in many dataframes the column label may not be a string (like pd.DataFrame([1,2,3])). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#25590 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIlT3NBvXCkZ6r-N0DjPezf1VXDvgks5vWby4gaJpZM4bjP6l> .

[sighingnow]

@TomAugspurger thanks for the suggestion, but we would still get TypeError: assign() keywords must be strings.

[TomAugspurger]

Huh, sorry thought that worked. You should be able to `.copy()` before doing the updates. That would avoid any settingwithcopy warnings.

…

On Thu, Mar 14, 2019 at 6:45 AM Tao He ***@***.***> wrote: @TomAugspurger <https://github.com/TomAugspurger> thanks for the suggestion, but we would still get TypeError: assign() keywords must be strings. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#25590 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIs6KU6djbRJgW9hN1Yy8N3zFK1taks5vWjZVgaJpZM4bjP6l> .

[TomAugspurger]

Thanks @sighingnow!