[PROPOSAL] Add Download Attribute Transformer

[techman83]

@dbent Based on your feedback and some thoughts I had this morning. Extending on #1703 and addressing extra requirements in #1682

NOTE: @KSP-CKAN/wranglers if this gets merged, the next index run will make a lot of changes.

I've merged DownloadHashTransformer + DownloadSizeTransformer into DownloadAttributeTransformer, along with adding 'download_content_type'. Seemed to make sense to have all those three together. Tests + Spec have also been added/updated.

Following Netkan

{
    "spec_version"   : 1,
    "identifier"     : "DogeCoinFlag",
    "name"           : "Dogecoin Flag",
    "abstract"       : "Such flag. Very currency. Wow.",
    "description"    : "Adorn your craft with your favourite cryptocurrency. To the mün!",
    "$kref"          : "#/ckan/github/pjf/DogeCoinFlag",
    "ksp_version"    : "any",
    "comment"        : "flagmod",
    "license"        : "CC-BY",
    "author"         : "daviddwk",
    "resources"      : {
        "homepage"   : "https://www.reddit.com/r/dogecoin/comments/1tdlgg/i_made_a_more_accurate_dogecoin_and_a_ksp_flag/"
    }
}

Produces the following ckan

{
    "spec_version": 1,
    "comment": "flagmod",
    "identifier": "DogeCoinFlag",
    "name": "Dogecoin Flag",
    "abstract": "Such flag. Very currency. Wow.",
    "description": "Adorn your craft with your favourite cryptocurrency. To the mün!",
    "author": "daviddwk",
    "license": "CC-BY",
    "resources": {
        "homepage": "https://www.reddit.com/r/dogecoin/comments/1tdlgg/i_made_a_more_accurate_dogecoin_and_a_ksp_flag/",
        "repository": "https://github.com/pjf/DogeCoinFlag"
    },
    "version": "v1.02",
    "ksp_version": "any",
    "download": "https://github.com/pjf/DogeCoinFlag/releases/download/v1.02/DogeCoinFlag-1.02.zip",
    "download_size": 53359,
    "download_hash": {
        "sha1": "BFB78381B5565E1AC7E9B2EB9C65B76EF7F6DF71",
        "sha256": "CF70CE1F988F908FB9CF641C1E47CDA2A30D5AD951A4811A5C0C5D30F6FD3BA9"
    },
    "download_content_type": "application/zip",
    "x_generated_by": "netkan"
}

Which passes validation

Validating /tmp/PlaneMode-1.3.1.ckan.. Success!

[dbent]

"array" -> "object"

[dbent]

LGTM

[techman83]

@dbent good spot, I missed updating that.

[techman83]

@KSP-CKAN/wranglers @dbent

I've done a bunch of extra testing on some random netkans and it looks to be doing what is intended.

leon@ultra ~/programs/netkan $ cat output/USI-SRV-0.5.2.1.ckan |grep -E "download|sha"
    "download": "https://github.com/BobPalmer/SrvPack/releases/download/0.5.2.1/SrvPack_0.5.2.1.zip",
    "download_size": 2472704,
    "download_hash": {
        "sha1": "A7F8DF1F0EEA466914FB9E3659495265E0CCF6C3",
        "sha256": "F2C780762B6F0C65665BF6065E9C7503FFE28F42CB02D6BF9F06F155DB44BC64"
    "download_content_type": "application/zip",
leon@ultra ~/programs/netkan $ shasum cache/4EEE8D38-netkan-USI-SRV.zip 
a7f8df1f0eea466914fb9e3659495265e0ccf6c3  cache/4EEE8D38-netkan-USI-SRV.zip
leon@ultra ~/programs/netkan $ shasum -a 256 cache/4EEE8D38-netkan-USI-SRV.zip 
f2c780762b6f0c65665bf6065e9c7503ffe28f42cb02d6bf9f06f155db44bc64  cache/4EEE8D38-netkan-USI-SRV.zip
leon@ultra ~/git/ckan/CKAN $ python bin/ckan-validate.py ~/programs/netkan/output/USI-SRV-0.5.2.1.ckan 
Validating /home/leon/programs/netkan/output/USI-SRV-0.5.2.1.ckan.. Success!

[dbent]

This is good to merge, whenever you're ready.

[techman83]

Neat, I'll let you do the honors if you like @dbent - I'll run the indexer manually and squash all the commits before pushing them.

[dbent]

MERGE PARTY 🍰 🎈 🎉

[techman83]

👯