test_pty: test_openpty() timed out after 3h 20min on aarch64 RHEL8 Refleaks 3.x:

[vstinner]

Is os.write(fd, data) guaranteed to write all bytes of data? Or can it write only some (first) bytes of data?

Extract of the test:

        debug("Writing chunked output")
        os.write(slave_fd, TEST_STRING_2[:5])
        os.write(slave_fd, TEST_STRING_2[5:])
        s2 = _readline(master_fd) # <==== HERE LINE 181 ===
        self.assertEqual(b'For my pet fish, Eric.\n', normalize_output(s2))

with:

def _readline(fd):
    """Read one line.  May block forever if no newline is read."""
    reader = io.FileIO(fd, mode='rb', closefd=False)
    return reader.readline()

aarch64 RHEL8 Refleaks 3.x:

3:22:48 load avg: 0.00 [467/467/1] test_pty worker non-zero exit code (Exit code 1)
beginning 6 repetitions
123456
/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/pty.py:95: DeprecationWarning: This process (pid=502171) is multi-threaded, use of forkpty() may lead to deadlocks in the child.
  pid, fd = os.forkpty()
hi there
./home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/pty.py:95: DeprecationWarning: This process (pid=502171) is multi-threaded, use of forkpty() may lead to deadlocks in the child.
  pid, fd = os.forkpty()
Timeout (3:20:00)!
Thread 0x0000ffff9f154590 (most recent call first):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/test/test_pty.py", line 68 in _readline
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/test/test_pty.py", line 181 in test_openpty
  (...)

build: https://buildbot.python.org/all/#/builders/551/builds/883

cc @serhiy-storchaka

Linked PRs

• gh-110673: test_pty raises on short write #110677
• [3.12] gh-110673: test_pty raises on short write (GH-110677) #110742
• [3.11] gh-110673: test_pty raises on short write (GH-110677) #110743

[vstinner]

Old issue, 2017: #75341

My previous abandoned attempt in 2017 for "writeall" and "short write": #3037

Short read issue was discussed in #73256 and fixed by commit e6f62f6.

[vstinner]

I only saw that bug once. I added raise Exception(f"short write: os.write(...)"). So if the bug occurs again, next time we will know if a short write occurred, or if it didn't, to make progress in the analysis.

For now, I close the issue.