test_makefile_test_folders fails if test/wheeldata is empty

[befeleme]

Bug report

Bug description:

In Fedora, when packaging Python, we remove the .whl files from Lib/test/wheeldata/. The directory is left empty.
test_makefile has started to skip empty directories as of alpha 6 (#117190). It doesn't include them in the list of used which are then checked for equality with unique_test_dirs. This causes the test to fail in our environment.

The test could be more robust and account for sysconfig.get_config_var('WHEEL_PKG_DIR'). If this is present, then it shouldn't expect that test/wheeldata is among the tested directories. (#105056)

Traceback for completness:

test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders) ... FAIL

======================================================================
FAIL: test_makefile_test_folders (test.test_tools.test_makefile.TestMakefile.test_makefile_test_folders)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.13.0a6/Lib/test/test_tools/test_makefile.py", line 72, in test_makefile_test_folders
    self.assertSetEqual(unique_test_dirs, set(used))
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Items in the first set but not the second:
'test/wheeldata'

----------------------------------------------------------------------
Ran 1 test in 0.033s

FAILED (failures=1)

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

• gh-117711: Only check for 'test/wheeldata' when it's actually used #117712
• gh-117711: Fix test_tools.test_makefile when wheeldata dir is empty #117748
• [3.12] gh-117711: Only check for 'test/wheeldata' when it's actually used (GH-117712) #117749

[befeleme]

The upstream change has been also backported to Python 3.12 and affects Fedora's builds of this version too.

[corona10]

cc @vstinner

[terryjreedy]

One solution is for you to not remove the two files. Why remove them? Given that you remove them, why leave the empty directory to cause a failure? Does anything else break if you remove it also?

[befeleme]

We ship the Python tests in Fedora, so we don't want to include wheels that are not actually used when building and testing our Python version, also removing the files "makes sure" they're really really not used.
Removing the directory is an option, but that would require us to patch Makefile.pre.in (https://github.com/python/cpython/blob/main/Makefile.pre.in#L2473) which lists the folder in TESTSUBDIRS which are then expected in the output.
Given all that, we'd prefer to patch the test rather than the other places.

[vstinner]

I created PR gh-117748 to fix the issue.

[vstinner]

Fixed by change d496387.

I created PR #117748 to fix the issue.

Oh, I didn't notice that @befeleme wrote a PR. I merged her PR, IMO her approach is better.

[befeleme]

Thank you!