Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Cannot infer type argument 1 of shutil.copyfileobj" error with mypy 0.780 #8962

Closed
deveshks opened this issue Jun 7, 2020 · 3 comments
Closed

Comments

@deveshks
Copy link

deveshks commented Jun 7, 2020

(This was originally filed as python/typeshed#4201, but it was suggested
to file here instead)

The following code used to pass mypy type check with mypy 0.770 but doesn't work for mypy 0.780

import shutil

filename1 = '/foo/bar'
filename2 = '/foo/baz'

srcfp =  open(filename1, 'rb')
destfp = open(filename2, 'wb')

shutil.copyfileobj(srcfp, destfp)

> mypy 0.770

$ mypy --version
mypy 0.770
$  mypy scratch.py
Success: no issues found in 1 source file

> mypy 0.780

$ mypy --version
mypy 0.780

$ scratch.py  
scratch.py:9: error: Cannot infer type argument 1 of "copyfileobj"
@srittau
Copy link
Contributor

srittau commented Jun 8, 2020

Something seems to have changed between mypy 0.770 and 0.780 regarding how overloads are handled. BinaryIO.write() has two overloads: bytearray and bytes in that order (since 2018). When removing the superclass IO[bytes] from BinaryIO, the error message changes to:

error: Argument 1 to "copyfileobj" has incompatible type "BinaryIO"; expected "_Reader[bytearray]"

Here is an example that doesn't rely on typeshed types (assuming that IO is special-cased):

from typing import TypeVar, Protocol, Any, overload, IO

_S_co = TypeVar("_S_co", covariant=True)
_S_contra = TypeVar("_S_contra", contravariant=True)
_T = TypeVar("_T")

class BinaryIO(IO[bytes]):
    @overload
    def write(self, s: bytearray) -> int: ...
    @overload
    def write(self, s: bytes) -> int: ...
    def write(self, s) -> int:
        return 0

class _Reader(Protocol[_S_co]):
    def read(self, n: int) -> _S_co: ...

class _Writer(Protocol[_S_contra]):
    def write(self, s: _S_contra) -> Any: ...

def foo(fsrc: _Reader[_T], fdst: _Writer[_T]) -> None:
    pass

srcfp: BinaryIO
destfp: BinaryIO

foo(srcfp, destfp)

behrmann added a commit to behrmann/mkosi that referenced this issue Jul 6, 2020
Starting with 0.780 (including current 0.782) the type arguments for
shutil.copyfileobj cannot be infered.
behrmann added a commit to behrmann/mkosi that referenced this issue Jul 6, 2020
Starting with 0.780 (including current 0.782) the type arguments for
shutil.copyfileobj cannot be infered.
javajawa added a commit to javajawa/stellaris-empire-exchange that referenced this issue Jul 11, 2020
- Add JS linters to the GitHub pipelines.
- Enable the `--strict` flag for `mypy`.
- Pin `mypy` at version 0.770 to avoid.
  python/mypy#8962
- Add strict types everywhere.
behrmann added a commit to behrmann/mkosi that referenced this issue Jul 17, 2020
Starting with 0.780 (including current 0.782) the type arguments for
shutil.copyfileobj cannot be infered.
dgelessus added a commit to dgelessus/python-rsrcfork that referenced this issue Jul 25, 2020
@McSinyx
Copy link

McSinyx commented Oct 12, 2020

This seems to be fixed by 0.790.

@hauntsaninja
Copy link
Collaborator

Thanks for reporting the fix! Used mypy_primer to bisect it to #9064 , so probably the result of python/typeshed#4203

While I'm here, I also bisected the original regression to #8845 . Since this issue is "fixed" by the typeshed change, I'm going to close — but maybe @msullivan is curious enough to take a closer look.

clrpackages pushed a commit to clearlinux-pkgs/mkosi that referenced this issue Dec 10, 2020
Adrian Freihofer (14):
      Add COMMANDS section to summary
      parse_args: Support removing list entries with ! prefix
      gitignore: __pycache__, PyDev
      Move ArgumentParser creation to function
      Use constants for commands
      parse_args: Improve handling of boolean arguments
      parse_args: verb arg has no attributes
      parse_args: Add compatibility options, set defaults
      parse_args: major refactoring and improvements
      parse_args: Support [foo,bar] syntax
      parse_args: handle kernel_cmd as space separated list
      man page: cover ! in lists
      Add unit tests for argument and config file parsing
      Call pytest from ci/semaphore.sh

Alexander Ried (1):
      mkosi: use threads for .tar.xz creation

Anish Swaminathan (1):
      Add support for Photon OS

Antonio Terceiro (1):
      Add location of OVMF.fd on recent Debian systems

Blair Bonnett (1):
      Fix default source file transfer

Clayton Craft (1):
      Use console=pipe in install_grub() when calling nspawn

Daan De Meyer (149):
      Add editorconfig file
      Print errors in bold + bright red
      Gracefully handle build script failures
      Arch: Do less when building the build script image
      Don't install a boot loader when building the build script image
      Arch: Disable fallback kernel images
      Arch: Avoid installing kernel image twice.
      Arch: fix pacman warnings/errors due to incorrect directory permissions.
      Don't create /boot when creating the build script image
      Arch: Avoid synchronizing repositories in second pacman invocation
      Arch: Fix root login when using systemd-nspawn
      Arch: Remove unnecessary synchronization.
      Arch: Remove pacstrap retired -d flag
      Arch: Move run_pacman and run_pacstrap definitions to their call sites
      Arch: Re-enable mkinitcpio install hook after running pacman
      Arch: Update CI dependencies
      Pass root around instead of workspace
      CentOS: Use mirrorlist.fedoraproject.org for EPEL mirrors
      Stop ignoring umount errors
      Call pacman-key --refresh-keys in Arch CI
      Stop gpg-agent and friends even when execution is interrupted
      Revert "Call pacman-key --refresh-keys in Arch CI"
      Use open_close contextmanager for dir_fd
      Remove noop step in run_finalize_script
      Don't print stacktrace when a binary is not found.
      Simplify CI
      Fix detect_distribution on ubuntu
      Add --qemu-headless option
      Fix --qemu-headless test.
      Add build-script option to --debug
      Install systemd by default on centos systems.
      Fix --qemu-headless so it works for all bootloaders.
      CI: Run apt-get update before installing dependencies
      Unify error handling
      Update default releases
      Require --bootable when using --qemu-headless
      Don't install bootable packages when building the build image
      Delay KeyboardInterrupt() handling when running a subprocess
      Use Ubuntu Focal in all Github Actions workflows
      Remove mkosi.default from repo and add it to .gitignore
      Check if scripts are executable
      Install systemd-udev on bootable CentOS images
      Install systemd by default on Fedora
      Increase default root size to 3G
      Reduce dnf duplication
      Add Mageia to CI
      Add docbook-xsl to CI packages
      Don't depend on python3 being at /usr/bin/python3
      Merge unit-test and mypy workflows
      Drop semaphore CI
      Require --merged-usr support in debootstrap.
      Workaround nspawn PATH bug
      Install correct grub package on openmandriva and mageia
      Fail gracefully when trying to use btrfs on CentOS 8
      Add plain_squashfs to output formats that cannot be booted.
      Fix CentOS EPEL bootloader installation
      Fix BIOS on OpenSUSE
      Add workspace-command debug option
      Install systemd on Clear Linux
      Don't try to install systemd-boot on Clear Linux
      Only mount necessary loop device and partitions when installing bootloaders
      Fail gracefully when trying to install Clear Linux on btrfs
      Gracefully handle hybrid BIOS/UEFI images on Clear Linux
      Gracefully handle BIOS + --minimize/gpt_squashfs combo
      Enable BIOS on all distros
      Add bootable CI builds
      Install zypper along with all other apt packages
      Rename CI steps
      Fix photon default release
      Add photon to CI
      opensuse: Add binutils
      Fedora: Fix --mirror
      CentOS: Fix --mirror
      Fix btrfs EEXIST error when using cached images
      Regenerate unified kernel images when installing/updating kernels in
      Arch: Drop removal of packages
      Arch: Enable pacman colors
      Arch: Add kernel-install pacman hooks
      Arch: Switch to dracut
      Arch: Simplify kernel logic
      Arch: Drop UseSyslog from the pacman config
      Docs: Remove trailing whitespace
      Add --without-unified-kernel-images option
      Add --qemu-headless to option summary
      Collapse print_error() into die()
      Make warning messages yellow and simplify warn()
      Make sure serial-getty override parent directories exist
      Simplify grub installation
      Arch: Drop pacstrap dependency
      Use bootctl install instead of manually copying systemd-boot to $BOOT
      Fix --subvolume --read-only combo
      Arch: Add bootctl update pacman hook
      Fedora: Fall back to fedora.gpg if key id is not in FEDORA_KEYS_MAP.
      Fedora: Capitalize rawhide in repository full name
      Add note to README that `sudo python -m mkosi` should be used to run
      Revert "Add note to README that `sudo python -m mkosi` should be used to run"
      Add setup-mkosi Github Action
      Interpret CLI args as nspawn args when verb == "boot"
      Make run() set check=True by default
      Arch: Make pacman hooks more generic
      Make run_workspace_command take the command as a list
      Ubuntu/Debian: Simplify installation and drop some outdated workarounds
      Drop --verbose flag when running dracut
      Ignore mypy false positives and update to latest version
      OpenSUSE: Simplify
      OpenSUSE: Add systemd package
      Only require --bootable for --qemu-headless when building
      Use --volatile=overlay when booting generated_root() images
      Mageia: Disable pam_securetty
      OpenMandriva: Disable pam_securetty
      CI: Add systemd-nspawn boot test
      Arch: Use Arch mirrorlist generator to get a list of mirrors.
      Only boot with OVMF when building UEFI images
      Arch: Add vmlinuz add/remove hooks when using BIOS
      Fix WithUnifiedKernelImages
      Fall back to software CPU emulation if KVM isn't available
      Mageia: Apply opensuse dracut fix
      Arch: Explicitly install base package first
      Arch: Handle no kernels installed case in kernel-install hook
      Fix grub installation
      Arch: Drop e2fsprogs install logic as its part of base
      Add BIOS support to --qemu-headless
      Ubuntu/Debian: Add universe regardless of UKI or not
      Fedora: Install kernel modules so squashfs is available
      Mageia: Make sure systemd is included in the initramfs
      Fail early when using --without-unified-kernel-images on unsupported
      Fail early when trying to use --bootable with Clear Linux
      Fail early when using --bootable on Photon
      Arch: Quote correctly in mkosi-kernel-remove script
      OpenSUSE: Support passwordless root login
      OpenSUSE: Remove dracut workaround
      Only install dracut config files when necessary
      CI: Stop running setup-mkosi on Ubuntu and Debian
      CI: Add QEMU boot tests
      Cache unified kernel images and secure boot when not doing verity
      Boot nspawn with --volatile=overlay when using dm-verity
      Move action to repository root
      Make sure we apply the btrfs stale devices fix everywhere
      Give prepare scripts access to SRCDIR.
      bump version numbers for v6
      mypy: Remove unused type: ignore comments
      CI: Run apt-get update before install python3-venv
      Disable Openmandriva boot tests
      Check if loop device partition exists in nspawn_params_for_blockdev_access
      setup-mkosi: Update dependencies to latest versions
      bump version numbers for v7
      CI: Drop clear, photon, opensuse, openmandriva and mageia
      CI: Build rpm from source
      bump version numbers for v8

Drew Moseley (2):
      Do not automatically specify rw on kernel command line.
      Fix incorrect script name in README.

Felipe Sateler (2):
      Add custom command to build the manpage
      mkosi.md: convert emdash into regular dash

Ivan Shapovalov (1):
      mkosi: fix `-t submodule` output

James Shubin (4):
      Fix typo in source comments
      Add env to build that specifies which "default" we're using
      Add flag to skip (second) final image build phase
      Optionally include the .git directory with the files it contains

JoelColledge (1):
      Fix literal asterisks in markdown documentation

Joerg Behrmann (32):
      mkosi: provide default xbootldr_partno in argument parsing
      Add the ability to set pre-hashed root passwords
      mkosi: pass architecture to debbootstrap and map to Debian names
      arch: use base package instead of base group
      mypy: add missing type information
      remove unnecessary cast
      fix tests
      mkosi: add explicit check for loopdev being not None to placate mypy
      mkosi: make mkosi mypy --strict compliant
      setup: add mypy section to setup.cfg
      ci: add mypy check
      ci: pin mypy version until python/mypy#8962 is resolved
      mypy: enable warning for unreachable code
      Make test for uefi boot protocol on Photon reachable
      ci: enable mypy on PRs
      ci: add openSUSE
      add whitespace
      Make mkosi a python module and generate script via entrypoint
      Explain the installation in the README
      Fix man page generation
      Install the man page
      Fix entrypoint installation
      Revert: Make mkosi a python module and generate script via entrypoint
      generate a new man page for a release
      Revert "Revert: Make mkosi a python module and generate script via entrypoint"
      Eschew using an entrypoint for an explicit wrapper script
      Statically set the program name in the argument parser
      ci: add installation tests
      add generate-zipapp.sh script to generate a zipapp
      move do-a-release.sh to tools directory
      Don't write .pyc files when bin/mkosi is called with sudo
      bin: use /usr/bin/env to get bash path instead of hardwiring it

John Pitney (1):
      Update ci-arch.yml

Jose Teuttli (1):
      Implemented bootable CentOS 8

Julien Nicoulaud (1):
      add basic support for --repositories option for Arch Linux

Jóhann B. Guðmundsson (3):
      FIX #159, #133
      Add Fedora31
      Update to Fedora31

Jörg Behrmann (1):
      Fix type error on args variable (#368)

Lennart Poettering (18):
      mkosi: do RNG pass-through automatically for qemu
      mkosi: having a "defaults" file is optional, hence conditionalize MKOSI_DEFAULT
      mkosi: add missing double whitespace between functions
      mkosi: force 256byte inodes on ext4
      mkosi: show whether we are in the first or second stage
      mkosi: optionally allow 'minimization' of generated file systems
      mkosi: incremental builds are not supported for squashfs or minimized file systems
      mkosi: introduce --with-network=strict
      mkosi: support old btrfs-tools on semaphore
      ci: use different ppa for python 3.6
      ci: generate more different images
      docs: document new features
      mkosi: optionally create /var/tmp and /var partitions
      mkosi: fixate pbkdf parameters to something low
      mkosi: include repart/debug-generator in images if it exists
      mkosi: fix comment style
      mkosi: rename --with-network=strict → --with-network=never
      mkosi: minor man page fixes

Liam McBirnie (2):
      Fix packages and build packages not being installed on Centos and Mageia
      Add CentOS 8 repositories.

Lucas De Marchi (5):
      Fix pytest
      gitignore: add .pyc
      Fix BootProtocols parsing
      ci: add missing "run" arg to debug
      ci: trigger on pull requests

Ludwig Nussel (4):
      strip quotes from /etc/os-release values
      read os-release's ID_LIKE to find openSUSE
      rename opensuse default repos
      call zypper with mount_api_vfs context

Michal Koutný (5):
      Fix --skip-final-phase builds referencing None
      openSUSE: Download RPMs in advance
      Always fill machine_id in built image
      Install openSUSE packages in one go
      Add OVMF path to openSUSE qemu-ovmf

Michal Sekletár (1):
      Respect TMPDIR variable

Márton Morvai (2):
      Call finalize script as described in documentation
      Added prepare script

Nicolas Di Pietro (3):
      Update editorconfig for space indent.
      Change tab indents to spaces
      Remove redundents indents and align some params.

Ronny Chevalier (1):
      mkosi: fix wrong url to get Fedora's keys

Sequencer (1):
      add full basic repo and epel for better centos container building (#346)

Shengjing Zhu (1):
      Fix mount squashfs on kernel 5.4

Silvio Knizek (31):
      Create ci-ubuntu.yml
      proper name variable
      mkosi needs to run as root
      run ci with explicit python3
      Update ci-ubuntu.yml
      Update ci-ubuntu.yml
      Update ci-ubuntu.yml
      Update ci-ubuntu.yml
      Update ci-ubuntu.yml
      Update ci-ubuntu.yml
      Update ci-ubuntu.yml
      Update ci-ubuntu.yml
      Update ci-ubuntu.yml
      split up github ci definition
      rename variable proper
      make debian based CIs actually build deb based
      use an empty defaults file
      ci: remove subvolume test
      ci-arch: add pacman from source
      ci-arch: add libarchive dependency for pacman
      ci-arch: add the keyring from arch linux as well
      ci-arch: add gpgme dependency for pacman
      ci-arch: include compile fix for pacman
      ci-arch: populate arch keyring before using
      ci-arch: use the newer python and enable debug
      ci-arch: add dependency for pacman
      ci-arch: replace arch-install-script package
      ci-arch: add asciidoc dep for arch-install-scripts
      ci: enable debug for CnetOS, enable ClearOS
      ci-clear: add go bin path to find swupd-extract
      ci-clear: symlink swupd-extract directly into $PATH

Stefan Joosten (2):
      Fix centosplus_url repository selection
      Fix epel_url to use baseurl instead of mirrorlist

Thore Bödecker (6):
      allow optional override of GPT first-lba header
      add helper for determining if CentOS older than 8
      extend mkfs to support adding cmdline args
      fix bootable CentOS 7 efi by using grub2-efi
      run postinstall nspawn with loopdev if bootable
      fix small typo in comment

Vishal Verma (1):
      mkosi: add Fedora 32

Zbigniew Jędrzejewski-Szmek (1):
      Do not allow dd to create the output file

fernandolobato (3):
      Changes to support unified kernel+initrd in Ubuntu
      Add usrmerge for ubuntu previous to 19.04
      ony add this package for bootable images

potens1 (1):
      Factor securetty disabling and apply to debian (#429)

thon (2):
      odroid / armv7l / armhf fix
      add repository print

tpgxyz (1):
      add support for OpenMandriva Linux distribution

zigarrre (1):
      Fix: No empty string argument if no options provided
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants