tslibs TODO list

[jbrockmendel]

Misc:

• Timestamp.replace should support fold kwarg. ATM the kwarg exists but is ignored.
• Timestamp constructor should accept nanosecond kwarg (ENH: Timestamp constructor does not accept nanosecond arg #18898) (closed by ENH: Allow Timestamp to accept Nanosecond argument #19889)

Refactoring:

• move _nat_strings and _checknull_with_nat to a single canonical location
• put OutOfBoundsDatetimeError in a dependency-free util-like module. (Implement npy_dtime.pyx #17805)
• Possible redundancy between parse_datetime_string and parse_time_string update not redundant.
• De-privatize names in tslibs.parsing
• cython version of compat for string_types, StringIO
• Don't import DateParseError in tslib
• de-privatize nattype._checknull_with_nat (closed by Cross off a few tslibs-TODOs #18443)

Check if these need to be exposed or (in some cases) need to exist:

• _Period._comparables (closed by Cross off a few tslibs-TODOs #18443)
• Timedelta._validate_ops_compat
• Timedelta._binary_op_method_timedeltalike
• Timedelta._op_unary_method
• Timedelta._roundupdate Won't fix/correct-as-is
• Timestamp._roundupdate Won't fix/correct-as-is
• _Timestamp._get_fieldupdate Won't fix/correct-as-is
• _Timestamp._get_named_field
• _Timestamp._get_start_end_fieldupdate Won't fix/correct-as-is
• lib.arrmap (closed by Cross off a few tslibs-TODOs #18443)
• _libs.src.numpy_helper.infer_type

TZ Conversion:

• tslib._localize_tso appears to make a handful of unnecessary calls, see localize_tso unnecessary calls? #17944
• Is it the case that tslib._localize_tso argument obj will always have obj.tzinfo == None? This is the case in all existing usages.
• Inconsistency between tz_convert and tz_convert_single Inconsistency in tz_convert_single? #17734

Offsets:

• normalize attr should be relevant for __eq__ BUG: offsets normalize and __eq__ #17689 (closed by fix DateOffset eq to depend on normalize attr #21404)
• update docstrings
• Track down meaning behind comment # I suspect this is wrong for *all* of them. in BQuarterBegin.

Timestamp Methods (moved from #17876)-

• Timestamp classmethods should return cls, not hard-coded Timestamp
• ... if and only if is_timestamp is updated to recognized subclasses.
• NaT / timedelta should behave like NaT / Timedelta nat division by timedelta #17955
• _assert_tzawareness_compat doesn't actually return the declared return typeupdate but declaring it as void leads to build-time error because that mis-matches the except? -1
• Add timestamp method+test; closes #17329 #17906 implements Timestamp.timestamp as round(self.value / 1e9, 6) for consistency with datetime method. Are we sure we want to drop nanosecond information? Might it be better to update utcfromtimestamp for round-trip internal consistency?
• Timestamp.replace should handle freq Timestamp.replace should take freq kwarg #18544
• Timestamp.replace should handle DST gracefully Timestamp.replace should handle DST transition gracefully #18319 (closed by handle DST appropriately in Timestamp.replace #18618)

Follow-Up Cleanup:

• Fix camelCase naming in offsets
• docstrings for Timestamp.utcnow, utcfromtimestamp, fromtimestamp, combine
• ci/lint.sh stricter with pyx files
• simplify/correct NaT docstrings
• de-privatize _get_rule_month
• Possibly remove unique_deltas, asv etc Tslibs resolution #18034 (comment) update unique_deltas does appear to be more performant than the alternatives.
• add types, docstrings, de-privatize in some cases in tslibs.frequencies Move frequencies functions to cython #17746 (comment), Move frequencies functions to cython #17746 (comment)
• docstring for tslibs.resolution.resolution
• consider renaming is_null_datetimelike --> checknull_datetimelike (implement libmissing; untangles _libs dependencies #18357 (comment)) (2021-12-07 is_null_datetimelike has been removed)
• check for unit already PANDAS_FR_ns in conversion.ensure_datetime64ns
• Remove #noqad unused imports

Decision Making

• Have NaT not subclass datetime?
• Wrap datetime.date with Timestamp? Or possible treat it as Period(..., freq="D")? COMPAT: datetime.date comparisons with datetime64[ns] #17965
• Make Resolution singleton or use instance or [...]. See Tslibs resolution #18034.
• Consider making Timestamp.replace raise instead of returning an imaginary time (handle DST appropriately in Timestamp.replace #18618 (comment))

Issues:

• check if BUG: to_timedelta overflows without raising in some very particular cases #17037 is handled by BUG: overflow on Timedelta construction & arithmetic now raises #17640 Update It is not.

Questions:

• Shouldn't pd.Period('2017').day be None instead of 31? high-freq properties of low-freq Periods #18378

Unsorted

• Build warnings for timedeltastruct implement libmissing; untangles _libs dependencies #18357 (comment) (closed by Cross off a few tslibs-TODOs #18443)
• Ween off of dateutil internals, specifically import of _timelex in tslibs.parsing.
• Period tests for __sub__ with datetimelike check for datetime+period addition #18524 (comment)
• Fix missing tz arg in WIP: Pass missing tz arg in _parsed_string_to_bounds #18471.
• Make sure we aren't comparing tzinfo1 == tzinfo2 as that may be misleading for pytz
• investigate why an "extra layer" is needed in np_datetime.cstring_to_dts
• docstrings for np_datetime._string_to_dts, cstring_to_dts
• de-privatize frequencies._MONTH_NUMBERS etc, mismatch _MONTHS vs MONTHS

Low Priority:

• Set seen_datetime consistently in array_to_datetime array_to_datetime set seen_datetime consistently #18554

[jreback]

as an FYI, am happy to have you make 1 issue that you create (e.g. this one), that you can then put ToDO items and then check them off.

[jbrockmendel]

FYI most of these are solved locally and are slowly getting pushed back upstream. The main ones on which I need input are a) the tz_convert vs tz_convert_single issue (see discussion in #17734), b) the question about obj.tzinfo in tslib._localize_tso. and c) the BQuarterBegin issue.

[jreback]

for _nat_strings, I think tslibs.parsing is defining these as well as tslibs.nattype

[WillAyd]

@jbrockmendel for the pyx LINT to-do do you need help updating the pyx files? From what I can tell most of the errors there would be simple like E225 missing whitespace operator

Relatively minor but having worked on some pyx files recently including those in lint-diff in the Makefile would be nice. Very clean and easy if we don't need to select a bunch of codes as we currently do in ci/lint.sh

Happy to knock out some of those pyx files if you need help. Let me know.

[jbrockmendel]

I think the E225 errors are for cython pointer arguments &dts which the linter thinks are typos. Do you know of a cython-aware linter? Most times I try to run flake8 on cython it hits a syntax error pretty early and then chokes.

[WillAyd]

My mistake - I just saw all of those errors and assumed they were legitimate, but now I see the caveat. I'm not aware of a cython-aware linter and doubt there is one.

[WillAyd]

Can't claim to have done this before but one of the answers in the below SO article makes me think there's a way to modify setup.cfg to have different matching criteria just for .pyx files, which at the very least could make it easier to consistently apply LINTing to the project whether its called from the Makefile or from ci/lint.sh. Worth a look if there's an appetite to keep that consistent

https://stackoverflow.com/questions/25535599/options-for-linting-cython-code

[jbrockmendel]

Everything actionable has either been addressed or has its own issue. closing.