Slow Language Model Download Speed

[darcyosullivan]

Im currently downloading the language Models using

python3 -m spacy.en.download all

The download speed is less then 0.00MB/S and there is no issue with my internet connection, any suggestions?

Your Environment

• Operating System: OSX
• Python Version Used: 3.4
• spaCy Version Used:
• Environment Information:

[honnibal]

Reconnect?

[darcyosullivan]

I've tried reconnecting to my internet several times swell as restarting the download

[patxu]

i had this error, mine is flucuating between 0.01 and 0.03 mbs

[jason-feng]

im having this issue using python2.7

[honnibal]

Hmm. It's working fine for me, currently. Is it still not working for you guys?

The model data is hosted on S3. We're looking forward to changing this, as it's expensive and not great overall.

[jason-feng]

It got better for me and is working now, but for an hour or two, it was terribly slow

[ranka47]

I too had this problem while I was working through my campus LAN network.
The moment I switched to open network provided by the service provider on the mobile phone the speed was great.

[xushenkun]

Same problem. Even using VPN to download from S3, It's still very slow.

[eromoe]

Slow in China too, though I konw it is due to GFW...

Downloading parsing model
Downloading...
Downloaded 1.09MB 0.21% 0.02MB/s eta 504m 29s

@honnibal Maybe you can provide the directly download link in docs, so we can download that through browser with proxy , it is the simpliest way since support download over proxy in python need some modification which may be buggy (https, cert or something else, which does happen in nltk).

Additional suggestion, without call spacy.util.set_data_path(),
it is better to tell spacy where the data locate by environment variable.

[ines]

After trying out different solutions, we decided to simply attach the models as archive files to the latest release. The files are still quite large (between 500 and ~700MB), but they can now be downloaded via the browser. The English and German models are already uploaded and I'm currently waiting for the GloVe vectors to finish (which is the largest file).

➡️ https://github.com/explosion/spaCy/releases/tag/v1.6.0

Here's how to install the models manually:

1. Find the default data path. Use spacy.util.get_data_path() to find the directory where spaCy will look for its models, or change the default data path with spacy.util.set_data_path().
2. Unpack the archive and place the contained folder in that directory.
3. Load the model via spacy.load('en') or spacy.load('de').

cc: @darcyosullivan, @patxu, @jason-feng, @ranka47, @xushenkun, @eromoe

[de-code]

I'm just wondering whether there is a reason why you wouldn't want python -m spacy.en.download all to download from GitHub instead rather than requiring the manual download?

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.