Temporary files remain after test failures

[ericpassmore]

When tests encounter a failure they should clean out any temporary files.

Encounter this when building leap on one device with /tmp mounted on another device. After the test failed the snapshots and blocks.log remained.

error fatal error: in "restart_chain_tests/test_trim_blocklog_front": boost::filesystem::filesystem_error: boost::filesystem::copy_file: Invalid cross-device link: "/tmp/2555-0aa5-80f5-0baa/blocks/blocks.log", "6393-f05a-59a8-d2be--eos-vm-jit/blocks.log"

Command

ctest -j "8" -LE _tests

Failed Tests

• Start 107: restart_chain_unit_test_eos-vm-oc
• Start 108: restart_chain_unit_test_eos-vm
• Start 109: restart_chain_unit_test_eos-vm-jit
• Start 110: snapshot_unit_test_eos-vm-oc
• Start 111: snapshot_unit_test_eos-vm
• Start 112: snapshot_unit_test_eos-vm-jit

[spoonincode]

boost::filesystem::copy_file: Invalid cross-device link

fwiw this is a defect in boost 1.74, there is even a workaround in CI!

leap/.github/workflows/build.yaml

Lines 134 to 136 in dd59a99

	# jammy's boost 1.74 can stumble on an EXDEV via copy_file_range() it doesn't have a fallback for; re-eval once moving to std::filesystem
	export TMPDIR="$PWD/tmp"
	mkdir -p $TMPDIR

I'd expect after tech debt item #620 this won't be an issue.

That doesn't change the still possible defect that we're not removing files we ought to be.

[heifner]

That doesn't change the still possible defect that we're not removing files we ought to be.

The design for tests are to leave files on errors so they can help with debugging. I think the idea for putting them in /tmp is so they will be cleaned up by the OS on /tmp cleanout schedule (typically reboot I believe).

[ericpassmore]

More data, in this failure case, each run adds new files. The files are empty. The 4K files contain one line of text.

4.0K	/tmp/fc_copy_test_tgt
4.0K	/tmp/fc_copy_test_src
0	/tmp/e846-bf53-17ca-4712
0	/tmp/df35-f32d-ec97-65b0
0	/tmp/6868-6d28-778a-2caf
0	/tmp/684c-a9a1-634b-cc17
0	/tmp/8799-e78a-d9ed-9cb8
0	/tmp/7dab-8911-93d4-2cae
0	/tmp/c9f0-7522-34a6-e4bd
0	/tmp/fb4b-bd0d-e1e6-19bc
0	/tmp/5a1e-663b-20ac-6c95
0	/tmp/3196-5ae9-2ed2-5b2a
0	/tmp/63d7-4742-1bd9-8576
0	/tmp/10c2-7742-391f-abe8
0	/tmp/049b-4aea-67bf-9b88
0	/tmp/7db5-b613-1adb-b503
0	/tmp/fd2c-682e-d405-9ec7
0	/tmp/0129-310c-2d2e-d6bf
0	/tmp/24d9-7b81-9bb3-6b56
0	/tmp/80a4-be8c-f7d4-d495

[ericpassmore]

There is a tmpfiles.d utility that will clean out tmpfs files without a reboot.

[stephenpdeos]

Next step to run text in env that will not surface the above cited boost 1.74 defect. If there are any temporary files still left around, this issue will be updated to take next steps.

Agreed that temp files should be cleaned up for unit tests, even in the case of failure.

[ClaytonCalabrese]

#915 should fix the snapshot unit test that was leaving files in tmp, I think the other time we will see tmp files not get cleaned up is during a segfault, which is already undefined behavior so it is not handled.