column for date mod updated

[anxcon]

alot of times ckan gets a new mod and i don't notice it, be it complete new mod, or recently updated to the current ksp version

if there was a column for date last updated (as in version of mod added or changed, not ckan scan updating full db) i could sort by date :) see whats new

[janbrohl]

Currently there are no dates in the underlying ckan-metadata at all (also no helpful ones in the master.zip) so this propably depends on #1116 or on some git-magic (meta-metadata?)

[anxcon]

even if the date database is started at current date would be fine, still helps to to sort newest since then :)

[pjf]

Lots of people like this, so I'm sticking a bunch of higher priority labels on it.

[politas]

When adding this, time zones will need to be considered. It is probably easiest to use UTC Date/time in the metadata, and then look at converting to local time zone in the client.

[funkcanna]

This would be very useful. Im currently checking the 1.1 updates but as more get added its harder to keep track of the new ones.

[PyroSA]

This would be awesome addition, I've limited myself to checking the new list after every refresh, as that was the only feasible way of seeing what's fresh. Suspected the lack of backing data would be exist, but even a local date would suffice for my needs.

[techman83]

I've given this a lot of thought as we have a much more flexible architecture since #2789

These are my dot point thoughts:

• Firstly this needs to be discussed and added to the Spec, ie
  • Field assumed to be UTC
  • Full date / time
  • ISO 8601 compliant (2019-11-07T01:29:59+00:00)
• netkan.exe should remain the source of the truth. It should be responsible for populating the release_date field (name pending)
  • This would enable us to initially use the system timestamp, but also enhance our transformers to retrieve this information from the relevant APIs where available
• An optional --release-date flag on netkan.exe means we can switch this on when we're ready
• Metadata pull request testing on CKAN-meta to fail if release_date is changed or removed.
• The indexer can be switched from using an MD5 to an equivalence check via the Ckan class. For example, if an existing ckan is found, load it in as a Ckan and compare it to the received inflation. The Ckan class to have a custom __eq__ method that compares the objects, but ignores the release_date
  • This keeps netkan.exe as the source of the truth
  • Allows us to write to disk exactly as it's received from netkan.exe
  • Keeps our existing logic, we just compare inflated == existing instead of the MD5s
  • We can do this in advance to enabling the release_date flag
  • This is trivial to write a test case for and doesn't require any complicated logic changes to the indexer
• We should treat historical data as a separate exercise. We can dig out reasonable approximations from the git history and populate them in all in one pass (disabling webhooks/canceling the tests as needed).

We have some good and flexible tools available now to make this happen, but we don't want to make it any more complicated than it needs to be. Reducing complexity was a huge motivator for re-architecting in the first place and while the big picture is a bit more complicated and contains a lot of moving parts; the individual components are much much simpler.

[techman83]

I've added a formal proposal for the release date field (#2916). Comments/Suggestions welcome!