Switch to using a HTTP source #23
Conversation
| run(`tar xvf $tzdata -C $TZDATA_DIR $REGIONS`) | ||
| rm(tzdata) | ||
| @unix_only function extract(archive, directory, files) | ||
| run(`tar xvf $archive --directory=$directory $files`) |
There was a problem hiding this comment.
Shouldn't this be -xvf? Also, maybe remove the v.
There was a problem hiding this comment.
It's pretty normal for tar not to have the hypen:
The first argument to tar should be a function; either one of the letters Acdrtux, or one of the long function names. A function letter need not be prefixed with ``-'', and may be combined with other single-letter options
There was a problem hiding this comment.
As for the verbose option I think it's rather useful here as it shows which individual tzdata files we're extracting:
INFO: Extracting TZ data
x africa
x antarctica
x asia
x australasia
x europe
x northamerica
x southamerica
|
Should |
|
Has this been tested with GNU and BSD tar? I can't find the |
|
We should try tar in addition to 7zip on Windows in case they have tar installed. |
| end | ||
|
|
||
| extract(archive, TZDATA_DIR, REGIONS) | ||
| rm(archive) |
There was a problem hiding this comment.
Seems unnecessarily aggressive to remove the archive.
There was a problem hiding this comment.
We don't have to remove the archive. The file is being downloaded to a temporary file so this is just cleanup. You think it would be useful for debugging to just leave the archive?
There was a problem hiding this comment.
Yeah. Plus a weird combination situation I came up with in my head.
|
@iamed2 quoting happens automatically with I haven't personally tested in with BSD tar but I believe |
|
|
|
I'll note that GNU tar does mention it in the man pages. It just seems to be BSD that doesn't mention it. If you think it would be clearer |
|
|
Previously we were downloading individual uncompressed tzdata files
directly from IANA's FTP server. Unfortunately, downloading from this
FTP server periodically results in:
curl: (56) Recv failure: Connection reset by peer
We managed to work around these connection issues by retrying downloads
but the FTP server was still problematic for machines behind restrictive
firewalls (Note: passive FTP can use a wide range of ports).
In order to address both of these problems the build script was reworked
to attempt to download the compressed tzdata from the HTTP server first.
Note that the `extract` function is based off of BinDeps.jl's
`unpack_cmd`. I was going to use BinDeps directly but we need the
additional functionality of only extracting a subset of files.
|
The Julia package evaluator found that the test cases for TimeZones.jl failed for v0.4 starting 2016-03-17. The failure was due to the "Connection reset by peer" error we get from the FTP source. |
|
Catching up on old emails; +1 to this! |
remove glob mention
Previously we were downloading individual uncompressed tzdata files directly from IANA's FTP server. Unfortunately, downloading from this FTP server periodically results in:
We managed to work around these connection issues by retrying downloads but the FTP server was still problematic for machines behind restricted networks (Note: passive FTP can use a wide range of ports).
This PR switches the build process to download tzdata from an official IANA HTTP server which appears to be much more reliable and won't cause issues with restrictive networks. In order to support this change the we needed to be able to decompress the tar.gz which requires a small additional step in the build process.
Closes #11