Handling ETags

[tonytran]

Should I be looking for ETags on a:

repo basis or data basis (each metadata segment has its own like contributors and commits)

and the basic idea is for the harvester to get urls.md, create urls.json
loop through for each repo within urls.json and create a metadata.json which contains all the repos and their metadata and then generate an html file for this data?