Windows "Usage" incorrect display and difficulty to get a command name correct displayed

[pombreda]

When you create a command with a setuptools console_scripts entry_points (for instance named mycli) the Usage displayed on POSIX (Linux, Mac,...) when running mycli --help will be:
Usage: mycli

In contrast on Windows, I get the following:
Usage: mycli-script.py

This is annoying because the correct invocation on windows is to use mycli, not mycli-script.py.
The script is the same on all OSes, even though behind the scenes setuptools creates:

• on posix, a bin/mycli python script with a correct shebang
• on windows, a Scripts/mycli-script.py and Scripts/mycli.exe. This is called by invoking mycli like on POSIX

The obvious way to force this would be to use the name arg in click.command. ("This defaults to the function name."). But it does not default to the function name and always show the mycli-script.py instead of mycli.

It looks like this comes from Context.info_name which is computed somehow, but I could not find a way to get this info_name forced to something fixed.

[untitaker]

You're looking for prog_name. It's taken from sys.argv[0] or __file__ by default:

    if prog_name is None:
        prog_name = make_str(os.path.basename(
            sys.argv and sys.argv[0] or __file__))

I wonder what sys.argv is in your case.

[untitaker]

From the stdlib docs:

argv[0] is the script name (it is operating system dependent whether this is a full pathname or not)

https://docs.python.org/3/library/sys.html#sys.argv

I start to doubt that this bug is fixable.

[pombredanne]

I guess I could overwrite main here: https://github.com/mitsuhiko/click/blob/fe2f525e720a6f71e8ac4079807b64df13063e8a/click/core.py#L611
But this is a tad hackish. There should be either a way to work around the quirks of setuptools generated scripts or have a parameter to allow passing a prog_name
Note that in this commit, I subclass Command.main to force a prog_name to self.name and this seems to work fine for this case:
aboutcode-org/scancode-toolkit@586a7c7

[selimb]

This was a problem for me too when trying to activate bash completion because the magic environment variable is _<PROG_NAME>_COMPLETE by default. So I tried to work around it, hopefully without doing too much monkey-patching. In my case, the application name is "nx_tools".

So prog_name is actually "nx_tools-script.py", and complete_var is "_NX_TOOLS_SCRIPT.PY_COMPLETE". So I tried executing the following:

_NX_TOOLS_SCRIPT.PY_COMPLETE=source nx_tools > complete.sh

only to get an error:

bash: _NX_TOOLS_SCRIPT.PY_COMPLETE=source: command not found

Bash does not allow periods in variable names.

All in all, not that big a deal, but it could be documented. Would you accept a PR on that?

[gauden]

Adding a "me-too" on this issue. I was pointed to this bug report from my StackOverflow question. I will document this issue in the help text until the bug is resolved...

[untitaker]

Because it was mentioned in the SO thread: You can just call your command with an explicit prog_name argument:

@click.command()
def cli():
    pass

cli(prog_name='foo')

No need for overriding main...

[gauden]

That works, as expected, also when cli() is a click.group and I have updated the SO thread with a reference to your answer. Thanks!

[mitsuhiko]

I'm not sure if we can fix that easily unless someone has an idea how to autodetect it and magically fix it up.

[pombredanne]

@untitaker wrote:

You can just call your command with an explicit prog_name argument

When doing so you might end up having to write some wrapper function to inject progname and use that instead as an entry point... only if you care about windows. but that becomes a solution for all OS which is not so nice.

@mitsuhiko wrote:

I'm not sure if we can fix that easily unless someone has an idea how to autodetect it and magically fix it up.

The problem is only when using setup tools entry points on Windows. entry points are IMHO a great thing. One approach, since the -script.py addition is only a Windows+entry point thing, could be to detect if we are on windows and if the script name ends with -script.py then strip -script.py from the progname. There may be more introspection that can be done with pkg_resources too.

This would be hackish, but simple enough.

[untitaker]

This definetly seems hackish because of the possibility that your actual program name ends with that string.

A proper solution would be part of setuptools.

On 16 September 2015 16:12:06 CEST, Philippe Ombredanne notifications@github.com wrote:

@untitaker wrote:

You can just call your command with an explicit prog_name argument

When doing so you might end up having to write some wrapper function to
inject progname and use that instead as an entry point... only if you
care about windows. but that becomes a solution for all OS which is not
so nice.

@mitsuhiko wrote:

I'm not sure if we can fix that easily unless someone has an idea how
to autodetect it and magically fix it up.

The problem is only when using setup tools entry points on Windows.
entry points are IMHO a great thing. One approach, since the
-script.py addition is only a Windows+entry point thing, could be to
detect if we are on windows and if the script name ends with
-script.py then strip -script.py from the progname. There may be
more introspection that can be done with pkg_resources too.

This would be hackish, but simple enough.

---

Reply to this email directly or view it on GitHub:
#365 (comment)

Sent from my phone. Please excuse my brevity.

[mitsuhiko]

Not happy to just script -script.py. Is there no better way to detect this?

[pombredanne]

@mitsuhiko The simplest thing that comes to mind since this is a windows only whart would be to detect if we are on windows, and we are using entry points and there is an XXX.exe and the command comes out as XXX-script.py, then use XXX instead. Not pretty but working in practice.

[mitsuhiko]

@pombreda how do you detect if you were invoked from an entrypoint?

[pombredanne]

@mitsuhiko
This way:

def is_console_script():
    """
    Return True if the current code was loaded from a setuptools console_script
    entry point. This walks the stack frames up to the root frame and checks if the
    pkg_resources.load_entry_point function is at the root. If yes, we were loaded
    most likely from a console script entry point.
    """
    module = 'pkg_resources'
    ep_func = 'load_entry_point'
    frm = sys._getframe()
    func = None
    while frm is not None:
        func = frm.f_globals.get(ep_func, '')
        frm = frm.f_back
    return func and func.__module__ == module and func.__name__ == ep_func

[pombredanne]

and @dstufft suggested this regex (flawed for now because it does not catch the .py case ) too : https://github.com/pypa/pip/blob/develop/pip/wheel.py#L410-L417

[dstufft]

I think if the regex got updated in pip, then you wouldn't need to deal with it at all in click.

[mitsuhiko]

@dstufft so your suggestion is to fix this in pip/setuptools instead?

[dstufft]

Yes, the purpose of that regex is to make it so that console wrappers have the same sys.argv[0] on both Windows and *nix. It's a bug that it currently isn't doing that.

[mitsuhiko]

In that case I will close this here.

[pombredanne]

excellent! much better! @dstufft Thank!

[pombredanne]

@dstufft @mitsuhiko actually things are a tad more complex:

1. if the entry point launcher script was created by pip from a wheel, there are no issue. If using pip after version 6, pip always creates a wheel first even for sdist. wheel never generates a -script.py, only a .exe launcher. There is no bug in the pip wheel.py and changing the regex at https://github.com/pypa/pip/blob/develop/pip/wheel.py#L410-L417 has no impact, because there is no .py created. Actually the regex could be simplified to only strip .exe . In wheels rather a main.py script is bundled inside the exe launcher and looks like this:

__main__.py
# -*- coding: utf-8 -*-
import re
import sys
from scancode.extract_cli import extractcode
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(extractcode())

This is done here in pip:
https://github.com/pypa/pip/blob/1da0ea13f3f0ab789edd62012d09bf69be045f68/pip/wheel.py#L413

1. if the entry point launcher script was created by pip from a -e requirement or from a setup.py install then this is setuptools easy_install that takes over and it does generate the -script.py. When this happens I do not think there is a way to distinguish things unless we use the function I proposed above unless things are fixed in setuptools too. This how the -script.py looks like:

# EASY-INSTALL-ENTRY-SCRIPT: 'scancode-toolkit==1.3.8','console_scripts','extractcode'
__requires__ = 'scancode-toolkit==1.3.8'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('scancode-toolkit==1.3.8', 'console_scripts', 'extractcode')()
    )

This is done here in setuptools:
https://bitbucket.org/pypa/setuptools/src/4ce518784af886e6977fa2dbe58359d0fe161d0d/setuptools/command/easy_install.py?at=default&fileviewer=file-view-default#easy_install.py-2008

There yet another launcher in @vsajip distlib, but that does not seem to apply here:
https://bitbucket.org/pypa/distlib/src/6ada86eea2700f10edcd81f831139c6ae20edb74/distlib/scripts.py?at=default&fileviewer=file-view-default#scripts.py-41

[dstufft]

Perhaps @jaraco would be open to adding the same regex to setuptools script wrappers.

Sent from my iPhone

On Nov 7, 2015, at 10:52 AM, Philippe Ombredanne notifications@github.com wrote:

@dstufft @mitsuhiko actually things are a tad more complex:

1. if the entry point launcher script was created by pip from a wheel, there are no issue. If using pip after version 6, pip always creates a wheel first even for sdist. wheel never generates a -script.py, only a .exe launcher. There is no bug in the pip wheel.py and changing the regex at https://github.com/pypa/pip/blob/develop/pip/wheel.py#L410-L417 has no impact, because there is no .py created. Actually the regex could be simplified to only strip .exe . In wheels rather a main.py script is bundled inside the exe launcher and looks like this:

main.py

-- coding: utf-8 --

import re
import sys
from scancode.extract_cli import extractcode
if name == 'main':
sys.argv[0] = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv[0])
sys.exit(extractcode())
This is done here in pip:
https://github.com/pypa/pip/blob/1da0ea13f3f0ab789edd62012d09bf69be045f68/pip/wheel.py#L413

if the entry point launcher script was created by pip from a -e requirement or from a setup.py install then this is setuptools easy_install that takes over and it does generate the -script.py. When this happens I do not think there is a way to distinguish things unless we use the function I proposed above unless things are fixed in setuptools too. This how the -script.py looks like:

EASY-INSTALL-ENTRY-SCRIPT: 'scancode-toolkit==1.3.8','console_scripts','extractcode'

requires = 'scancode-toolkit==1.3.8'
import sys
from pkg_resources import load_entry_point

if name == 'main':
sys.exit(
load_entry_point('scancode-toolkit==1.3.8', 'console_scripts', 'extractcode')()
)
This is done here in setuptools:
https://bitbucket.org/pypa/setuptools/src/4ce518784af886e6977fa2dbe58359d0fe161d0d/setuptools/command/easy_install.py?at=default&fileviewer=file-view-default#easy_install.py-2008

There yet another launcher in @vsajip distlib, but that does not seem to apply here:
https://bitbucket.org/pypa/distlib/src/6ada86eea2700f10edcd81f831139c6ae20edb74/distlib/scripts.py?at=default&fileviewer=file-view-default#scripts.py-41

—
Reply to this email directly or view it on GitHub.

[pombredanne]

@dstufft I filed a bug in setuptools: https://bitbucket.org/pypa/setuptools/issues/459/incorrect-sysargv-0-for-generated-console

and one in pip: pypa/pip#3233

[ofek]

This should be fixed soon: pypa/setuptools/pull/736 pypa/pip/pull/3918

[ofek]

Fixed in latest version of setuptools (*_^)/