fix column cleaning and general cleanup

[shouples]

• enforce datatypes after sampling
• remove unnecessary column cleaning when comparing dataframes
• remove lingering callout code
• more adjustments to debug logging
• add tests for formatting dataframes with missing values (None, np.nan, pd.NA)

known issue:

• conversion to original dtypes after resampling/filtering may not be successful on some types (pd.Period/pd.Interval) since pandas can't parse a stringified list, so we might need to keep track of some intermediate normalized set of dtypes instead of the "original" dtypes

[willingc]

Thanks for the updated tests. I had a couple of questions but those shouldn't block merge.

[willingc]

Curious. What is the use case for this utility?

[shouples]

This was previously for the blueprintjs callout to the user after a resample request that returned instructions to pass the filtered subset to a new variable, like:

new_df = my_awesome_df.query( < pandas query string >, engine="python")

But it's going to be passed in the metadata to the frontend to let DEX handle displaying the query syntax dynamically when the filters change, instead of just on (backend) resample requests. I'd like to have this handled in the dataframe caching branch for easier access.

[willingc]

Why is the type unknown for these?

[shouples]

Mostly as a catch-all for if we run into objects that aren't JSON serializable (custom classes/objects) that we haven't handled in an explicit way earlier, to avoid breaking the pandas build_json_schema() call. Definitely open to suggestions here!