logging.StreamHandler.stream is sys.stderr by default ; "SupportsWrite['str']" is too narrow

[mr-c]

This prevents code like logging.StreamHandler().stream.close() from being type checked as valid

https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler

This problem was introduced in https://github.com/python/typeshed/pull/5069/files#diff-23ef9e6197f6f95428d7333190fc2357c1c4f110b89c27a7fc71e2fab7d7897aR702

[srittau]

This is a tradeoff. SupportsWrite[str] is the only thing needed by the implementation of StreamHandler and users are free to use any stream that fulfills this protocol. Therefore using any other method from a stream is potentially unsafe (although it's not in practice of setStream() is never used.

I was deliberating whether to make StreamHandler generic over the stream. I refrained from it, because in theory setStream() could change the type of stream. But it's probably the best solution.

[Akuli]

because in theory setStream() could change the type of stream.

I agree that this is a good use case for an invariant TypeVar, since this is the same situation as with lists. For example, list_of_strings.append(None) can change the type of a list, similarly to textiowrapper_handler.setStream(io.StringIO()).

[srittau]

Yes. In theory if someone would do the following, type checkers would warn:

h = StreamHandler()
h.setStream(StringIO())

But I think that's a good thing that users need to be explicit about this:

h: StreamHandler[MyProto] = StreamHandler()
h.setStream(StringIO())

And the following is a bad idea anyway:

h = StreamHandler(open("abc", "w"))
h.setStream(StringIO())
h.stream.getvalue()

[gaborbernat]

@srittau How does typehint the class now with this change?

❯ py
Python 3.10.1 (main, Dec  9 2021, 07:52:48) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from logging import StreamHandler
>>> StreamHandler[str]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'type' object is not subscriptable

h: StreamHandler[MyProto] = StreamHandler()

This only works if the annotation is not evaluated 🤔

[srittau]

Indeed. You have to either use from __future__ import annotations or quote the annotation.

[gaborbernat]

Sometimes you cannot do that, can we revert this in the meantime? 🤔 E.g. now impossible to write:

class MyHanlder(StreamHandler):
      pass

🤔 you need to add a type ignore to the inheritance line otherwise mypy will complain that is missing a generic parameter.

[srittau]

The ultimate solution is to have defaults for generics (python/typing#307), but for now we unfortunately require # type: ignore.

[AlexWaygood]

You could also submit a PR to CPython to add __class_getitem__ to StreamHandler. Previous PRs along those lines have been accepted, e.g. python/cpython#28714

[gaborbernat]

That wouldn't work for the older Python version though only the latest 3.11 or later 🤔

[Akuli]

A bit annoying, but maybe better than making a pull request to CPython or using # type: ignore:

if TYPE_CHECKING:
    _base_class = StreamHandler[MyProto]
else:
    _base_class = StreamHandler

class MyHanlder(_base_class):
    pass

[hauntsaninja]

In case it's useful for future reference, this section of the mypy documentation covers ways to work around this class of issue: https://mypy.readthedocs.io/en/stable/runtime_troubles.html#using-classes-that-are-generic-in-stubs-but-not-at-runtime

[mr-c]

In cases others did not notice it, the stream attribute of logging.StreamHandler is not part of the official API, so in my case we track stream separately, removing the issue.

[AlexWaygood]

I've opened python/cpython#92129 to add __class_getitem__ to logging.StreamHandler in 3.11+.

(Update: merged!)

[vectro]

With this change, how would one type hint handler in

if output_file is None:
    handler = logging.StreamHandler()
else:
    handler = logging.FileHandler(output_file)

Because of the invariant generic, it seems that the only valid type for handler is StreamHandler[Any].