Add a design for authoritative http downloads using .netrc (#115)

http-archive rule does not have an ability to download from urls which require authorization.
This proposal describes how to make additions to the API to allow various standard / custom protocols using the .netrc file as storage for the credentials / tokens.

README.md

@@ -78,6 +78,7 @@ list.
 
 | Last updated | Title                                                                                                                                                     | Author(s) alias                                                              | Category              |
 | ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | --------------------- |
+|   2019-05-02 | [HTTP downloads with Authorization using .netrc](https://github.com/genrym/proposals/blob/http-netrc-auth/designs/2019-05-01-http-auth.md) | [@genrym](https://github.com/genrym) | External Repositories |
 |   2019-04-22 | [Move //tools/build_defs/pkg to rules_pkg](https://docs.google.com/document/d/1GAbAiW0nVbxGlwsdXhFIcn3owlZJkMJiXBt-ttA9z_k/edit)     | [@aiuto](https://github.com/aiuto) | Bazel |
 |   2019-04-16 | [Common Bazel Constraints](https://docs.google.com/document/d/1bqPF7CjHI03xHZmpy8gOE186uNjeU3dGYU8v2t-qiJ8/edit)     | [@gregestren](https://github.com/gregestren) | Configurability |
 |   2019-04-09 | [Experimental Content-Based Output Paths](https://docs.google.com/document/d/17snvmic26-QdGuwVw55Gl0oOufw9sCVuOAvHqGZJFr4/edit) | [@gregestren](https://github.com/gregestren) | Configurability

designs/2019-05-01-http-auth.md

@@ -0,0 +1,117 @@
+---
+created: 2019-05-01
+last updated: 2019-05-01
+status: To be reviewed
+title: Enable authoritative http downloads using .netrc 
+authors:
+  - genrym
+---
+
+
+# Abstract
+This document describes an extension of the `SkylarkRepositoryContextApi.downladAndExtract` to enable HTTP downloads which
+require authorization, using the [.netrc](https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html) 
+file specification as storage for the credentials.
+
+
+# Background
+Currently, when using the rule _http_archive_ for specifying a download, it is possible to download any anonymously open url.
+However, if the url requires _Authorization_ for the download (be it _Basic Authentication_, _OAuth2_,...) - there is no such
+option. Such an example can be download a source zip from private github repository.
+
+
+# Proposal
+We can use the _.netrc_ file to store credentials for _Authorization_ per host / domain.
+
+Example of .netrc file contents:
+```
+machine github.com password my-auth-token
+machine basic-auth.com login myuser password my-password
+```
+
+For token based authentication, we can use the `password` field to specify the authorization token / API key.
+
+We will add the following properties to _http_archive_ rule and `SkylarkRepositoryContextApi.downloadAndExtract`:
+- netrc_file_path : String
+  - the location of the _.netrc_ file. Default location for the _.netrc_ file will be the user home directory, e.g. 
+  `/Users/<username>/.netrc`
+- netrc_domain_auth_types : SkylarkDict
+  - mapping between a domain / host (which correlates to the _.netrc_ file definition) to _authorization type_, e.g.
+    Basic Authentication, OAuth2, custom prefix. Default will be an empty dictionary.
+  - we use a mapping here to allow different hosts / domains as sources for the same file, which might have different authorization protocols / credentials
+
+Example of _http_archive_ rule definition:
+```
+http_archive(
+    name = 'somefile',
+    url = 'https://basic-auth.com/somefile.zip',
+    sha256 = '87569EEA3732D8A11DCFC8293D08A443D79CE34CD4996822DD3D26F8F46ECEE0',
+    type = 'zip',
+    netrc_file_path = '/Users/user/.netrc',
+    netrc_domain_auth_types = {
+      "basic-auth.com": "basic"
+    },
+)
+```
+The result will be, that the http call will be sent with _Authorization_ header with the value `Basic base64(login:password)`,
+specifically in this case, given the _.netrc_ file example above: `Basic bXl1c2VyOm15LXBhc3N3b3Jk`, according to 
+[rfc2617](https://tools.ietf.org/html/rfc2617#page-5)
+
+## Authorization Type Format
+In order to support various authorization protocols which might also require some custom properties, e.g. a custom prefix 
+protocol, which requires the prefix to be supplied as additional parameter to the authorization protocol definition, we can define the following format for the _authorization type_ value, where the additional properties are optional:
+
+### Alt.1
+
+`<type>[:<prop1>=<val1>,<prop2>=<val2>,...]`
+
+Using the _.netrc_ file example definition above, the defintion of _http_archive_ rule will look like the 
+following for custom prefix protocol. Taking github as example, which uses a non-standard authorization protocol:
+```
+http_archive(
+    name = 'somefile',
+    url = 'https://www.github.com/.../somefile.zip',
+    sha256 = '87569EEA3732D8A11DCFC8293D08A443D79CE34CD4996822DD3D26F8F46ECEE0',
+    type = 'zip',
+    netrc_file_path = '/Users/user/.netrc',
+    netrc_domain_auth_types = {
+      "github.com": "custom_prefix:prefix=token"
+    },
+)
+```
+The result will be, that the http call will be sent with _Authorization_ header with the value `token my-auth-token`.
+
+### Alt.2: 
+```
+{
+   "type": "<type>",
+   "props": {
+      "<prop1>": "<val1>",
+      "<prop2>": "<val2>",
+      ...
+   }
+}
+```
+
+Not sure whether it can be achievable with the SkylarkDict type:
+```
+http_archive(
+    name = 'somefile',
+    url = 'https://www.github.com/.../somefile.zip',
+    sha256 = '87569EEA3732D8A11DCFC8293D08A443D79CE34CD4996822DD3D26F8F46ECEE0',
+    type = 'zip',
+    netrc_file_path = '/Users/user/.netrc',
+    netrc_domain_auth_types = {
+      "github.com": {
+         "type": "custom_prefix",
+         "props": {
+            "prefix": "token"
+         }
+      }
+    },
+)
+```
+
+
+# Backward-compatibility
+The new configuration will be used only if explicitly defined per specific host / domain, thus is backward compatible.