Supporting Ellipsis as a value

[max-sixty]

We use an Ellipsis as an easy-to-understand, easy-to-import sentinel value to mean "everything else".

But it's not possible to use fully with type checking, since mypy doesn't exclude it from a type when it's been excluded with an if, as it does for None, or Enums per python/typing#240).

I've included below a MCVE of the behavior below, and here's an example of how we use it, given @gvanrossum has already asked whether it's important to use an Ellipsis:

For example, to transpose an array, these are equivalent:

In [1]: import xarray as xr

In [2]: ds = xr.tutorial.scatter_example_dataset()

In [3]: ds
Out[3]:
<xarray.Dataset>
Dimensions:  (w: 4, x: 3, y: 11, z: 4)
Coordinates:
  * x        (x) int64 0 1 2
  * y        (y) float64 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
  * z        (z) int64 0 1 2 3
  * w        (w) <U5 'one' 'two' 'three' 'five'
Data variables:
    A        (x, y, z, w) float64 0.02074 0.04807 -0.1059 ... -0.1809 -0.04862
    B        (x, y, z, w) float64 0.0 0.0 0.0 0.0 ... 1.406 1.414 1.368 1.408

In [4]: ds.transpose('w','x','y','z')
Out[4]:
<xarray.Dataset>
Dimensions:  (w: 4, x: 3, y: 11, z: 4)
Coordinates:
  * x        (x) int64 0 1 2
  * y        (y) float64 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
  * z        (z) int64 0 1 2 3
  * w        (w) <U5 'one' 'two' 'three' 'five'
Data variables:
    A        (w, x, y, z) float64 0.02074 0.02074 0.02074 ... -0.03076 -0.04862
    B        (w, x, y, z) float64 0.0 0.002074 0.004147 ... 1.403 1.405 1.408


In [5]: ds.transpose('w',...) # use an Ellipsis to indicate 'all other dimensions'
Out[5]:
<xarray.Dataset>
Dimensions:  (w: 4, x: 3, y: 11, z: 4)
Coordinates:
  * x        (x) int64 0 1 2
  * y        (y) float64 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
  * z        (z) int64 0 1 2 3
  * w        (w) <U5 'one' 'two' 'three' 'five'
Data variables:
    A        (w, x, y, z) float64 0.02074 0.02074 0.02074 ... -0.03076 -0.04862
    B        (w, x, y, z) float64 0.0 0.002074 0.004147 ... 1.403 1.405 1.408

• Are you reporting a bug, or opening a feature request?
  I think it's a bug but Recognize comparison with Ellipsis #2180 suggests it may be a "Will not fix". Wanted to repost it here given the logic above
• Please insert below the code you are checking with mypy,
  or a mock-up repro if the source is private. We would appreciate
  if you try to simplify your case to a minimal repro.

import builtins
from typing import List, Union


def fun(x: Union[builtins.ellipsis, List], y: List):
    if x is not Ellipsis:
        y = x #  error: Incompatible types in assignment (expression has type "Union[ellipsis, List[Any]]", variable has type "List[Any]")

    return y

# another attempt:
def fun2(x: Union[Ellipsis, List], y: List): # error: Variable "builtins.Ellipsis" is not valid as a type
    if x is not Ellipsis:
        y = x

    return y

• What is the actual behavior/output?

test.py:7: error: Incompatible types in assignment (expression has type "Union[ellipsis, List[Any]]", variable has type "List[Any]")
test.py:12: error: Variable "builtins.Ellipsis" is not valid as a type

• What are the versions of mypy and Python you are using?
  Do you see the same issue after installing mypy from Git master?
  mypy 0.740

Thank you!

[JukkaL]

This would have to fixed/changed in Python first. Ellipsis is not valid as a type -- List[Ellipsis] generates an exception, for example. The name ellipsis is not defined at runtime. Using type(Ellipsis) in a type annotation might be an option, but it's not supported in PEP 484.

If this gets changed in Python, mypy will add support to it, but I'm not sure if this is a common enough use case to add non-standard syntax for this type.

If you want to move this forward, you can propose something at https://github.com/python/typing/issues.

[max-sixty]

Thanks @JukkaL , I'll open an issue in that repo. Feel free to close this

[dimaqq]

Py3.11:

>>> typing.List[str]
typing.List[str]
>>> typing.List[Ellipsis]
typing.List[...]
>>> typing.List[...]
typing.List[...]
>>> a: ... = ...
>>> a
Ellipsis
>>> a: Ellipsis = Ellipsis
>>> 

So... Python seems to allow it, at run time at least.
Is that enough to reopen, or is something else needed?

[yashaka]

@max-sixty Have you opened the issue elsewhere? What was the end of your story?

[max-sixty]

@max-sixty Have you opened the issue elsewhere? What was the end of your story?

Issue linked above!