Skip to content

Commit

Permalink
BUG/DEPR: combine dtype fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
sinhrks committed Aug 12, 2016
1 parent 4a80521 commit 2046cb5
Show file tree
Hide file tree
Showing 7 changed files with 410 additions and 227 deletions.
2 changes: 2 additions & 0 deletions doc/source/whatsnew/v0.19.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -788,6 +788,7 @@ Deprecations
- ``pandas.tseries.frequencies.get_standard_freq`` is deprecated. Use ``pandas.tseries.frequencies.to_offset(freq).rule_code`` instead. (:issue:`13874`)
- ``pandas.tseries.frequencies.to_offset``'s ``freqstr`` keyword is deprecated in favor of ``freq``. (:issue:`13874`)


.. _whatsnew_0190.prior_deprecations:

Removal of prior version deprecations/changes
Expand Down Expand Up @@ -939,6 +940,7 @@ Bug Fixes


- Bug in ``Series`` comparison operators when dealing with zero dim NumPy arrays (:issue:`13006`)
- Bug in ``.combine_first`` may return incorrect ``dtype`` (:issue:`7630`, :issue:`10567`)
- Bug in ``groupby`` where ``apply`` returns different result depending on whether first result is ``None`` or not (:issue:`12824`)
- Bug in ``groupby(..).nth()`` where the group key is included inconsistently if called after ``.head()/.tail()`` (:issue:`12839`)
- Bug in ``.to_html``, ``.to_latex`` and ``.to_string`` silently ignore custom datetime formatter passed through the ``formatters`` key word (:issue:`10690`)
Expand Down
26 changes: 18 additions & 8 deletions pandas/core/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,20 +31,22 @@
_possibly_downcast_to_dtype,
_invalidate_string_dtypes,
_coerce_to_dtypes,
_maybe_upcast_putmask)
_maybe_upcast_putmask,
_find_common_type)
from pandas.types.common import (is_categorical_dtype,
is_object_dtype,
is_extension_type,
is_datetimetz,
is_datetime64_dtype,
is_datetime64tz_dtype,
is_bool_dtype,
is_integer_dtype,
is_float_dtype,
is_integer,
is_scalar,
is_dtype_equal,
needs_i8_conversion,
_get_dtype_from_object,
_lcd_dtypes,
_ensure_float,
_ensure_float64,
_ensure_int64,
Expand Down Expand Up @@ -3700,17 +3702,20 @@ def combine(self, other, func, fill_value=None, overwrite=True):
otherSeries[other_mask] = fill_value

# if we have different dtypes, possibily promote
new_dtype = this_dtype
if this_dtype != other_dtype:
new_dtype = _lcd_dtypes(this_dtype, other_dtype)
series = series.astype(new_dtype)
if notnull(series).all():
new_dtype = this_dtype
otherSeries = otherSeries.astype(new_dtype)
else:
new_dtype = _find_common_type([this_dtype, other_dtype])
if not is_dtype_equal(this_dtype, new_dtype):
series = series.astype(new_dtype)
if not is_dtype_equal(other_dtype, new_dtype):
otherSeries = otherSeries.astype(new_dtype)

# see if we need to be represented as i8 (datetimelike)
# try to keep us at this dtype
needs_i8_conversion_i = needs_i8_conversion(new_dtype)
if needs_i8_conversion_i:
this_dtype = new_dtype
arr = func(series, otherSeries, True)
else:
arr = func(series, otherSeries)
Expand All @@ -3721,7 +3726,12 @@ def combine(self, other, func, fill_value=None, overwrite=True):

# try to downcast back to the original dtype
if needs_i8_conversion_i:
arr = _possibly_cast_to_datetime(arr, this_dtype)
# ToDo: This conversion should be handled in
# _possibly_cast_to_datetime but the change affects lot...
if is_datetime64tz_dtype(new_dtype):
arr = DatetimeIndex._simple_new(arr, tz=new_dtype.tz)
else:
arr = _possibly_cast_to_datetime(arr, new_dtype)
else:
arr = _possibly_downcast_to_dtype(arr, this_dtype)

Expand Down
Loading

0 comments on commit 2046cb5

Please sign in to comment.