Wheel files may contain duplicate files in data directory

[wiggin15]

Wheel files contain a directory called <project>-<version>.data with data files, that are moved to the destination installation directory when installing the wheel. These data files may have the same path as files that are already packaged in the "main" directory. i.e., the data files may already be packaged and be in the wheel file twice, to be installed in the same destination.
Selenium is one example of such wheel: SeleniumHQ/selenium#5907
pip and wheel will silently override the files in the main package with the ones from .data and won't fail. setuptools uses os.rename for the data files, which will silently override the files on Linux, but fail on Windows. See issue pypa/setuptools#1362 / PR pypa/setuptools#1371 . The expected behavior is not defined so there are inconsistencies in the implementations.

What should be done about wheel distributions with files that exist both in .data directory and the main directory? Should the installation fail or succeed? Should the data files override the packaged files or the other way around?

[ncoghlan]

Another variant of this problem may arise if a wheel contains filenames that differ only by case, and is installed on a case insensitive filesystem (however, I don't have any specific examples of that arising in the wild - it's just one I've hit with git repos, and realised could also happen for wheel archives).

[pganssle]

It seems to me that the way forward here might be to unify the behavior between setuptools, pip and wheel such that all three of them will use the current permissive behavior, but start raising a warning.

If we don't allow the permissive behavior that will break things that currently exist and are currently working, but it definitely shouldn't be failing silently.

I suppose the other option is to start raising warnings in wheel and pip without enabling this behavior in setuptools, with the goal of breaking them in wheel and pip fairly soon. I kinda think it might be good to allow opting in to this behavior, at the very least. I've been bitten in the past by pointlessly pedantic packaging software that won't let me overwrite basically identical (or equivalent) files like __init__.py.