Installation error (Windows, Python 3.5)

[jcgoble3]

When attempting to install via pip on a freshly installed Python 3.5 on Windows 10, I get installation errors, where it fails when trying to compile the tokenizer. Obviously this is because it's working off of the source tarball, and I don't have a compiler. After looking at the package's page on PyPI, I only see Windows binaries for Pythons 2.7, 3.3, and 3.4, yet the release notes for v0.4.1 state that Python 3.5 is supported. Any chance of getting some binaries for Python 3.5 on Windows?

(I can provide the full error messages if needed, but as the cause seems obvious, I'm holding them back unless asked since they're rather lengthy.)

[earwig]

There were no Windows builds for Python 3.5 because Appveyor didn't support it when I released 0.4.2. I'll try to add that now.

You should probably give us the error message for good measure, in case it's an unknown issue. If the problem has something to do with vcvarsall.bat, don't bother.

[earwig]

Okay, I'm ready to release 0.4.3 with Python 3.5 builds (since it doesn't make sense going through the effort of re-building/releasing an old version) when we can confirm that your issue isn't a code bug that I should fix first.

[jcgoble3]

Original error output:

Collecting mwparserfromhell==0.4.2 (from -r requirements.txt (line 2))
  Downloading mwparserfromhell-0.4.2.tar.gz (116kB)
    100% |################################| 118kB 470kB/s ta 0:00:01
...
Installing collected packages: pep8, mccabe, pyflakes, flake8, mwparserfromhell, simplemediawiki
  Running setup.py install for mwparserfromhell
    Complete output from command c:\python35\python.exe -c "import setuptools, tokenize;__file__='C:\\Users\\Jonathan\\A
ppData\\Local\\Temp\\pip-build-txtph4hs\\mwparserfromhell\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file
__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\Users\Jonathan\AppData\Local\Temp\pip-bzc5y4ac
-record\install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.5
    creating build\lib.win-amd64-3.5\mwparserfromhell
    copying mwparserfromhell\compat.py -> build\lib.win-amd64-3.5\mwparserfromhell
    copying mwparserfromhell\definitions.py -> build\lib.win-amd64-3.5\mwparserfromhell
    copying mwparserfromhell\smart_list.py -> build\lib.win-amd64-3.5\mwparserfromhell
    copying mwparserfromhell\string_mixin.py -> build\lib.win-amd64-3.5\mwparserfromhell
    copying mwparserfromhell\utils.py -> build\lib.win-amd64-3.5\mwparserfromhell
    copying mwparserfromhell\wikicode.py -> build\lib.win-amd64-3.5\mwparserfromhell
    copying mwparserfromhell\__init__.py -> build\lib.win-amd64-3.5\mwparserfromhell
    creating build\lib.win-amd64-3.5\mwparserfromhell\nodes
    copying mwparserfromhell\nodes\argument.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes
    copying mwparserfromhell\nodes\comment.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes
    copying mwparserfromhell\nodes\external_link.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes
    copying mwparserfromhell\nodes\heading.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes
    copying mwparserfromhell\nodes\html_entity.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes
    copying mwparserfromhell\nodes\tag.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes
    copying mwparserfromhell\nodes\template.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes
    copying mwparserfromhell\nodes\text.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes
    copying mwparserfromhell\nodes\wikilink.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes
    copying mwparserfromhell\nodes\__init__.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes
    creating build\lib.win-amd64-3.5\mwparserfromhell\parser
    copying mwparserfromhell\parser\builder.py -> build\lib.win-amd64-3.5\mwparserfromhell\parser
    copying mwparserfromhell\parser\contexts.py -> build\lib.win-amd64-3.5\mwparserfromhell\parser
    copying mwparserfromhell\parser\tokenizer.py -> build\lib.win-amd64-3.5\mwparserfromhell\parser
    copying mwparserfromhell\parser\tokens.py -> build\lib.win-amd64-3.5\mwparserfromhell\parser
    copying mwparserfromhell\parser\__init__.py -> build\lib.win-amd64-3.5\mwparserfromhell\parser
    creating build\lib.win-amd64-3.5\mwparserfromhell\nodes\extras
    copying mwparserfromhell\nodes\extras\attribute.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes\extras
    copying mwparserfromhell\nodes\extras\parameter.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes\extras
    copying mwparserfromhell\nodes\extras\__init__.py -> build\lib.win-amd64-3.5\mwparserfromhell\nodes\extras
    running build_ext
    building 'mwparserfromhell.parser._tokenizer' extension
    error: [WinError 2] The system cannot find the file specified

    ----------------------------------------
Command "c:\python35\python.exe -c "import setuptools, tokenize;__file__='C:\\Users\\Jonathan\\AppData\\Local\\Temp\\pip-build-txtph4hs\\mwparserfromhell\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'),__file__, 'exec'))" install --record C:\Users\Jonathan\AppData\Local\Temp\pip-bzc5y4ac-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\Jonathan\AppData\Local\Temp\pip-build-txtph4hs\mwparserfromhell

Additional item of note produced when rerunning the command with the -vv (double verbose) flag:

Command "c:\python35\python.exe -c "import setuptools, tokenize;__file__='C:\\Users\\Jonathan\\AppData\\Local\\Temp\\pip-build-_te5uqh5\\mwparserfromhell\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\Users\Jonathan\AppData\Local\Temp\pip-wua90mvg-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\Jonathan\AppData\Local\Temp\
pip-build-_te5uqh5\mwparserfromhell
Exception information:
Traceback (most recent call last):
  File "c:\python35\lib\site-packages\pip\basecommand.py", line 211, in main
    status = self.run(options, args)
  File "c:\python35\lib\site-packages\pip\commands\install.py", line 311, in run
    root=options.root_path,
  File "c:\python35\lib\site-packages\pip\req\req_set.py", line 646, in install
    **kwargs
  File "c:\python35\lib\site-packages\pip\req\req_install.py", line 852, in install
    show_stdout=False,
  File "c:\python35\lib\site-packages\pip\utils\__init__.py", line 733, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command "c:\python35\python.exe -c "import setuptools, tokenize;__file__='C:\\Users\\Jonathan\\AppData\\Local\\Temp\\pip-build-_te5uqh5\\mwparserfromhell\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\Users\Jonathan\AppData\Local\Temp\pip-wua90mvg-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\Jonathan\AppData\Local\Temp\pip-build-_te5uqh5\mwparserfromhell

Thanks for looking into this so quickly. :)

[earwig]

May be related to http://bugs.python.org/issue21821...

[earwig]

@valhallasw, any ideas?

[jayvdb]

I havent looked at Python 3.5 for Windows yet, but https://github.com/ogrisel/python-appveyor-demo is updated to support Python 3.5, and this project may need to copy some of those commits (or dynamically load the files from that project like the pywikibot .appveyor.yml does)
Also there is some ongoing discussion at # 34 there that might be relevant.

[earwig]

I added 3.5 to Appveyor just now in 21fa211... although now I wonder if ogrisel/python-appveyor-demo@09a1c86 is necessary.

Lego just pointed me to http://stevedower.id.au/blog/building-for-python-3-5-part-two/. This looks a lot more frustrating than I thought it would be. Why can't we just drop Windows support?

[jcgoble3]

If you've built a wheel for Windows/Py3.5, email it to me (Gmail address with the same username as here) and I can attempt to install it directly and report back whether it works, or what errors it produces.

[jcgoble3]

Oh, I forgot to say that I need the 64-bit wheel since I have 64-bit Python.

[valhallasw]

@earwig: as far as I understand, that blog is mostly for when you want to build a wheel that will work for other visual studio (> 2015) versions, but that's not too relevant for now.

The error @jcgoble3 has suggests there is no compiler installed (which is consistent with their original report). It's using the .tar.gz, not the .wheel.

[valhallasw]

https://ci.appveyor.com/project/earwig/mwparserfromhell/history doesn't have any recent builds at all?

[earwig]

Sorry for the delay in responding.

@valhallasw, that's because I only have it set up to build on master pushes (i.e., releases). I'm gonna change that to build on master/develop pushes but only upload to PyPI on the former.

[earwig]

Appveyor is a lot, lot faster than three months ago. I like this.

[earwig]

All compiling now, although I had a bunch of failed attempts and I really don't understand why they failed...

[jcgoble3]

Thanks for emailing me the wheel link. I installed from that wheel, and all is good with no errors. Nothing of interest, I think, in the output, but I'm including it here anyway:

C:\Users\Jonathan\Downloads>pip3 install mwparserfromhell-0.4.3.dev0-cp35-none-win_amd64.whl
Processing c:\users\jonathan\downloads\mwparserfromhell-0.4.3.dev0-cp35-none-win_amd64.whl
Installing collected packages: mwparserfromhell
Successfully installed mwparserfromhell-0.4.3.dev0

C:\Users\Jonathan\Downloads>py -3.5
Python 3.5.0 (v3.5.0:374f501f4567, Sep 13 2015, 02:27:37) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import mwparserfromhell
>>> mwparserfromhell.parse('[[target|text]]')
'[[target|text]]'
>>> type(_)
<class 'mwparserfromhell.wikicode.Wikicode'>
>>>

Looks like this can be closed. Thanks for working on this. :)

[earwig]

Hooray!

[jcgoble3]

Looks like Appveyor's final step of uploading the wheels to PyPI failed. Apparently %PYMOD% isn't being recursively expanded in that step. I hate Windows cmd scripts. :P

[earwig]

Yep! Had to do it manually. Is recursive expansion really not allowed?

[earwig]

...It doesn't seem quite right, or else I'm not sure how the other things (like %SETUPPY%) could be working.

[jcgoble3]

I don't know. I gave up trying to understand cmd scripts years ago. Bash is much more intuitive. (I use both Windows and Ubuntu.)

It does seem that %SETUPPY% and %PIP% are being recursively expanded above (in appveyor.yml), so I'm guessing some kind of issue with being in the IF statement.

[earwig]

That would make sense. Unfortunately, I can only guess at this point and I can't really test it without doing another full release. :P

[jcgoble3]

Okay, I found something that may or may not be of relevance: https://technet.microsoft.com/en-us/library/bb490954.aspx Near the bottom, it notes that if you use quotes in defining a variable value, those quotes are interpreted as a literal part of the value. As that command is the only one that doesn't stand completely by itself in appveyor.yml, I suspect that the quotes may be causing the issue, and that removing the quotes in the value at the top may be the answer. YAML doesn't require quotes for an obvious string value anyway.

[earwig]

Hmm, how would that make a difference? YAML seems to strip the quotes when it parses the file.

[jcgoble3]

I don't know. Like I said, it might be relevant, or it might not.

[jcgoble3]

The only concrete suggestion I can make at this point is to post a question on Stack Overflow. There has to be someone there that can solve the problem.

[earwig]

This is the best I can do.