-
-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement PEP 585 (Generic builtins and __future__.annotations) #7907
Comments
Looks like this should be not hard to support. |
@ilevkivskyi I'd like to take a shot to support this and I may need some hints |
@TH3CHARLie Sure, you can try this. I think the fix should contain two parts. First, whenever encountering the future import a file, you should set some flag (probably in the semantic analyzer), and second, at the point where the error is generated check for this flag, and suppress the error if it is set. |
I think that this needs more discussion. Firstly, it seems pretty clear that However, mypy allows types such as Some types, such sa The current behavior seems inconsistent, but I'm not sure what's the desired semantics (with or without |
Since the possible runtime failures happen soon and are quite obvious, I would stay on the side of having some false negatives if this removes some false positives (from a user point of view). I.e. I am fine with unconditionally allowing |
In my opinion this not a false positive, since we are only following the spec.
I think that this is such as a big change that it deserves a PEP, especially as not supporting this syntax was explicitly mentioned in PEP 563 (if I'm reading it correctly). One of the main points of PEP 484 was that the same annotation syntax would be used by all tools. I think that any new syntax we experiment with should be clearly marked as a non-standard extension, such as through enabling it via A potential way to experiment with this would be to allow |
Sure, there is your opinion, but there is also a user opinion. Seriously what is the point of prohibiting
IMO the main point of PEP 585 is to actually enable Anyway, I am not going to argue about this, I don't think it is something really important. |
I agree that supporting |
Since this thread seems stagnant, I'll post this: from __future__ import annotations
def find(haystack: dict[str, list[int]]) -> int:
... IMO this shows absolute support for parametrising generics in this way, and additionally, this was written in March - so no need to write a new PEP :P |
PEP 585 hasn't been accepted yet, though. Allowing |
FWIW, there is an implementation of PEP 585 for CPython python/cpython#18239 I think for now it would be best to just handle PEP 563, and then soon I will write tests for and land PEP 585 support. |
PEP 585 has been accepted! https://mail.python.org/archives/list/python-dev@python.org/message/HW2NFOEMCVCTAFLBLC3V7MLM6ZNMKP42/ I will start working on the implementation. The summary is:
|
@ethanhs I think generic builtins should also be allowed in quoted annotations (e.g. |
So, It still needs the support for generic builtins for PEP 585, right? |
Yes. Are you volunteering? |
I wasn't. But I can try (with some help / mentoring / tutoring). I not used to mypy code base, and I don't know if this could something feasible as a first contribution. I studied compilers in college and read some of mypy's dev documentation, but still a bit clueless about the changes needed. If it's something worth giving a try, I can asking for some tips and directions about this in Gitter or with someone who volunteer to help me. |
I don't think it's a big change to the type checking logic, so you're in luck -- the big job is probably going to be updating tests. But it's not a trivial change either. The error message we get is |
I have a doubt about the A code like this in those versions will show the not subscriptable error, as well it fails to run:
But using the postponed evaluation of annotations like this, it should run ok:
But mypy currently doesn't show any error, been in 3.7 or 3.8. With the I personally would love it, as just using the future import would allow me to use pep 585 in 3.7 and 3.8, but I wonder if mypy should be treating it like before. About the GenericAlias I think I figure it out, just need more checking and testing. |
I cannot reproduce this. I always see the error. How are you testing it? |
mypy master branch (mypy 0.790+dev.37777b3f52560c6d801e76a2ca58b91f3981f43f) on python 3.9 rc1 (via pyenv). file: n: list[int] = [42]
file from __future__ import annotations
n: list[int] = [42]
I'm asking because to support the GenricAlias it seems that just an additional clause in the if is enough, but testing it brought this doubt: in if (fullname in nongen_builtins
and t.args and
not self.allow_unnormalized and
not self.api.is_future_flag_set("annotations") and
self.options.python_version < (3, 9) #added line
): But if it should give an error also on 3.7/3.8 with future annotations enabled, I should change it. I'll create a fork and PR soon with this. |
This is what was implemented in #7963, mentioned above (this hasn't been released yet, which is presumably why Guido can't repro). We want to accept this in 3.7 and 3.8 with future annotations, for the reasons you mention |
great!
It will do exactly what is needed. It's used in I thought doing it just after the I could add some extra clauses to the if in But the better way I could think is just do this in the main function, after processing the options: if options.python_version >= (3, 9):
from mypy.nodes import nongen_builtins
nongen_builtins.clear() It seems a bit hackish, but it would do the job well, without messing with taking all the places it could be used and changing some if clauses to become (even) more complex. I just don't know if this fits the project style. Maybe this could be in a better place them |
Actually I would prefer it if module-level constants were treated as immutable. We have an API that allows people to invoke mypy's main() repeatedly with different arguments, which include the targeted Python version. (We have one option that for historic reasons has to be global, and we have to do terrible shenanigans to make this work.) So I think the better approach is to do the version check in each place where This would also be the approach to do this in earlier versions as long as |
Since this issue is still open, does that mean that mypy doesn't support PEP 585 yet? |
@Hubro Yes, that's the case:
|
This complains with the OrderedDict, but works with Dict |
@dpinol Documentation states "Deprecated since version 3.9: collections.OrderedDict now supports []. See PEP 585.". You should use |
right thanks! |
I was rechecking the pep585 and I noticed that there are a few things that are missing for full pep585 support, like the deprecations warnings. So I did a list of all tasks that should and was done:
For the support for generic alias, the type I made a table with all the symbols cited in the PEP 585 with the information. Working means that you can use it without generating error warnings and getting the right type. It's based on my PR (#9564) that implements this. Reveal is if the
¹ Using implementation on PR #9564
|
type hint | type_reveal (3.9) | type_reveal (3.8) |
---|---|---|
tuple[int] |
Tuple[builtins.int] |
Tuple[builtins.int] |
list[int] |
builtins.list[builtins.int] |
builtins.list[builtins.int] |
dict[int, str] |
builtins.dict[builtins.int, builtins.str] |
builtins.dict[builtins.int, builtins.str] |
set[int] |
builtins.set[builtins.int] |
builtins.set[builtins.int] |
frozenset[int] |
builtins.frozenset[builtins.int] |
|
type[int] |
Type[builtins.int] |
Type[builtins.int] |
collections.deque[int] |
collections.deque[builtins.int] |
collections.deque[builtins.int] |
collections.defaultdict[int, str] |
collections.defaultdict[builtins.int, builtins.str] |
collections.defaultdict[builtins.int, builtins.str] |
collections.OrderedDict[int, str] |
collections.OrderedDict[builtins.int, builtins.str] |
collections.OrderedDict[builtins.int, builtins.str] |
collections.Counter[int] |
collections.Counter[builtins.int] |
collections.Counter[builtins.int] |
collections.ChainMap[int, str] |
collections.ChainMap[builtins.int, builtins.str] |
collections.ChainMap[builtins.int, builtins.str] |
collections.abc.Awaitable[int] |
typing.Awaitable[builtins.int] |
typing.Awaitable[builtins.int] |
collections.abc.Coroutine[str, int, float] |
typing.Coroutine[builtins.str, builtins.int, builtins.float] |
typing.Coroutine[builtins.str, builtins.int, builtins.float] |
collections.abc.AsyncIterable[int] |
typing.AsyncIterable[builtins.int] |
typing.AsyncIterable[builtins.int] |
collections.abc.AsyncIterator[int] |
typing.AsyncIterator[builtins.int] |
typing.AsyncIterator[builtins.int] |
collections.abc.AsyncGenerator[int, float] |
typing.AsyncGenerator[builtins.int, builtins.float] |
typing.AsyncGenerator[builtins.int, builtins.float] |
collections.abc.Iterable[int] |
typing.Iterable[builtins.int] |
typing.Iterable[builtins.int] |
collections.abc.Iterator[int] |
typing.Iterator[builtins.int] |
typing.Iterator[builtins.int] |
collections.abc.Generator[int, float, str] |
typing.Generator[builtins.int, builtins.float, builtins.str] |
typing.Generator[builtins.int, builtins.float, builtins.str] |
collections.abc.Reversible[int] |
typing.Reversible[builtins.int] |
typing.Reversible[builtins.int] |
collections.abc.Container[int] |
typing.Container[builtins.int] |
typing.Container[builtins.int] |
collections.abc.Collection[int] |
typing.Collection[builtins.int] |
typing.Collection[builtins.int] |
collections.abc.Callable[[int], float] |
def (builtins.int) -> builtins.float |
def (builtins.int) -> builtins.float |
collections.abc.Set[int] |
typing.AbstractSet[builtins.int] |
builtins.set[builtins.int] |
collections.abc.MutableSet[int] |
typing.MutableSet[builtins.int] |
typing.MutableSet[builtins.int] |
collections.abc.Mapping[int, str] |
typing.Mapping[builtins.int, builtins.str] |
typing.Mapping[builtins.int, builtins.str] |
collections.abc.MutableMapping[int, str] |
typing.MutableMapping[builtins.int, builtins.str] |
typing.MutableMapping[builtins.int, builtins.str] |
collections.abc.Sequence[int] |
typing.Sequence[builtins.int] |
typing.Sequence[builtins.int] |
collections.abc.MutableSequence[int] |
typing.MutableSequence[builtins.int] |
typing.MutableSequence[builtins.int] |
collections.abc.ByteString |
typing.ByteString |
typing.ByteString |
collections.abc.MappingView[int, int] |
`` | `` |
collections.abc.KeysView[int] |
typing.KeysView[builtins.int] |
typing.KeysView[builtins.int] |
collections.abc.ItemsView[int, str] |
typing.ItemsView[builtins.int, builtins.str] |
typing.ItemsView[builtins.int, builtins.str] |
collections.abc.ValuesView[str] |
typing.ValuesView[builtins.str] |
typing.ValuesView[builtins.str] |
contextlib.AbstractContextManager[int] |
typing.ContextManager[builtins.int] |
typing.ContextManager[builtins.int] |
contextlib.AbstractAsyncContextManager[int] |
typing.AsyncContextManager[builtins.int] |
typing.AsyncContextManager[builtins.int] |
re.Pattern[str] |
typing.Match[builtins.str] |
typing.Pattern[builtins.str] |
re.Match[str] |
typing.Pattern[builtins.str] |
typing.Match[builtins.str] |
So, as it is shown in the table,the builtins and collections
types are ok, but the others (like collections.abc
, contextlib
and re
) are being revealed as the deprecated generic alias from typing
module. IMHO this should be addressed in a separated PR.
There is one weird thing: when running reveal_type inside the mypy's tests, sometimes it can give different results, like Any
instead of the type. For further information, check this message in my PR (#9564), as this can be just my test that is badly written.
Deprecation warnings
As PEP 585 says
Importing those from typing is deprecated. Due to PEP 563 and the intention to minimize the runtime impact of typing, this deprecation will not generate DeprecationWarnings. Instead, type checkers may warn about such deprecated usage when the target version of the checked program is signalled to be Python 3.9 or newer. It's recommended to allow for those warnings to be silenced on a project-wide basis.
The deprecated functionality will be removed from the typing module in the first Python version released 5 years after the release of Python 3.9.0.
So it should be handled this deprecation warnings for those cases, in a different PR as well. When thinking about this, I have doubt that who will implement this might have that when running on python 3.7 / 3.8 with importing __future__.annotations
should raise those warnings or not.
New issues?
Would be better to open new separated issues to each subtask remaining or just keep using this one?
Thanks, @AllanDaemon, but why do you thumbs-up and heart your own comment? |
I was just playing around with it when I saw it. |
Re: deprecation warnings Re: MappingView Re: reveal_type Re: tuple |
or a mock-up repro if the source is private. We would appreciate if you try to simplify your case to a minimal repro.
"defaultdict" is not subscriptable, use "typing.DefaultDict" instead
from __future__ import annotations
The text was updated successfully, but these errors were encountered: