Fixed date parsing while fetching entries for task_status

[muhammed-ajmal]

Found issues while parsing the last_updated date from the task_status table.

• The fix includes parsing the last_updated date without microseconds as ckan APIs are not promising the date will always include microseconds.

Error logs

Traceback (most recent call last):
  File "/usr/lib/ckan/odp/src/ckanext-datastore-refresh/ckanext/datastore_refresh/cli.py", line 56, in dataset
    _submit_resource(pkg_dict, res, context)
  File "/usr/lib/ckan/odp/src/ckanext-datastore-refresh/ckanext/datastore_refresh/cli.py", line 117, in _submit_resource
    success = tk.get_action("xloader_submit")(context, data_dict)
  File "/usr/lib/ckan/odp/src/ckan/ckan/logic/__init__.py", line 504, in wrapped
    result = _action(context, data_dict, **kw)
  File "/usr/lib/ckan/odp/src/ckanext-xloader/ckanext/xloader/action.py", line 105, in xloader_submit
    updated = datetime.datetime.strptime(
  File "/opt/pyenv/versions/3.8.9/lib/python3.8/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/opt/pyenv/versions/3.8.9/lib/python3.8/_strptime.py", line 349, in _strptime
    raise ValueError("time data %r does not match format %r" %
ValueError: time data '2022-06-30T19:48:06' does not match format '%Y-%m-%dT%H:%M:%S.%f'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/ckan/odp/bin/ckan", line 33, in <module>
    sys.exit(load_entry_point('ckan', 'console_scripts', 'ckan')())
  File "/usr/lib/ckan/odp/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/ckan/odp/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/lib/ckan/odp/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/ckan/odp/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/ckan/odp/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/ckan/odp/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/ckan/odp/src/ckanext-datastore-refresh/ckanext/datastore_refresh/cli.py", line 58, in dataset
    click.secho(e, fg="red")
  File "/usr/lib/ckan/odp/lib/python3.8/site-packages/click/termui.py", line 547, in secho
    message = style(message, **styles)
  File "/usr/lib/ckan/odp/lib/python3.8/site-packages/click/termui.py", line 519, in style
    return "".join(bits)
TypeError: sequence item 1: expected str instance, ValueError found

[smotornyuk]

This solution looks overcomplicated. Try using datetime.fromisoformat instead.
datetime.fromisoformat won't work in pre-3.7 python. +1 to @ThrawnCA suggestion bellow

[ThrawnCA]

These are both ISO formats. Perhaps dateutil.parser.isoparse would be simpler?

[muhammed-ajmal]

updated with that, do we need to add this inside a try-except block?

[ThrawnCA]

This isn't isoparse...

[muhammed-ajmal]

sorry, please check the latest commit.

ckanext-xloader/ckanext/xloader/action.py

Line 103 in 96164ba

updated = parse_iso_date(existing_task['last_updated'])