handle nan values in DataFrame.update when overwrite=False #15593

[pcluo]

closes #15593

[jreback]

can you add some tests from the issue in pandas/tests/frame//test_combine_concat.py

[jreback]

and a whatsnew note (0.20.0) in the bug fixes section

[mayukh18]

I think it should be

if isnull(that).all():
    continue

The line of self[col] = expressions.where(mask, this, that, raise_on_error=True) should be skipped on the absence of values in that i.e. column of the other dataframe whereas mask is computed on this or the column values of base dataframe for overwrite=False.

The committed solution works but is not correct I feel.

[codecov-io]

Codecov Report

Merging #15617 into master will decrease coverage by -0.03%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #15617      +/-   ##
==========================================
- Coverage   91.02%   90.99%   -0.03%     
==========================================
  Files         143      143              
  Lines       49301    49301              
==========================================
- Hits        44875    44863      -12     
- Misses       4426     4438      +12

Impacted Files	Coverage Δ
pandas/core/frame.py	97.87% <100%> (-0.1%)	❌
pandas/io/gbq.py	25% <0%> (-58.34%)	❌
pandas/tools/merge.py	91.76% <0%> (-0.35%)	❌
pandas/tseries/index.py	95.29% <0%> (-0.1%)	❌
pandas/core/generic.py	96.25% <0%> (ø)	✅

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3d69988...02ebf34. Read the comment docs.

[jreback]

closing as stale, if you'd like to update and add some tests pls do.

[ngnikhilgoyal]

I still am receiving the invalid type promotion error in pandas 0.19.2. when using overwrite = False and both old df and new df contain NaN values for same indices. :-(

[pcluo]

would like to re-open and add a short-term fix for this. just created a pull request #16430. @jreback please advise. long-time user of pandas but first time contributing. thx.

[jreback]

hmm for some reason not allowing to reopen can just submit another PR (you can use commits from this one) just submit it with a new branch