Enable result cache by default

[sebastianbergmann]

No description provided.

[keradus]

shall we also change order execution to depends,defects ?
the thing is, without that, cache file would be created, but not used, thus no benefits of creating it

[sebastianbergmann]

Makes sense.

[sebastianbergmann]

@epdenouden I started to work on this in 36ed9fe.

One thing I noticed that if I run PHPUnit's own test suite with --do-not-cache-result then a cache file will still be written. This does not come from PHPUnit's own test suite itself but from one of the test suites used by it. Without --do-not-cache-result this cache file will be overwritten at the end by the cache file for PHPUnit's own test suite.

We probably have to disable the cache for all test suites executed by PHPUnit's own test suite.

[epdenouden]

@sebastianbergmann Yes, unfortunately that is a known issue with (some of?) the PHPT tests. I have never looked that up in detail, however I am aware of it. The saving grace for PHPUnit's own test suite is that it overwrites the other caches at the end.

I think at some point there was a development version that looked at the PHPUNIT_TESTSUITE constant being defined. Complicating factors I found but never fully explored:

• the working directory of some tests results in caches being littered in subdirectories
• caches in the working directory being left lying around when I CTRL-C the run

In newer tests I actively work around the issue by explicitly copying the cache to a temporary file somewhere else and deleting it afterwards. This works really well; example:

phpunit/tests/end-to-end/regression/GitHub/3396/issue-3396-test.phpt

Lines 1 to 18 in 4de2605

	--TEST--
	https://github.com/sebastianbergmann/phpunit/issues/3396
	--FILE--
	<?php
	$tmpResultCache = tempnam(sys_get_temp_dir(), __FILE__);
	file_put_contents($tmpResultCache, file_get_contents(__DIR__ . '/../../../../_files/DataproviderExecutionOrderTest_result_cache.txt'));
	$_SERVER['argv'][1] = '--no-configuration';
	$_SERVER['argv'][2] = '--order-by=defects';
	$_SERVER['argv'][3] = '--debug';
	$_SERVER['argv'][4] = '--cache-result';
	$_SERVER['argv'][5] = '--cache-result-file=' . $tmpResultCache;
	$_SERVER['argv'][6] = \dirname(\dirname(\dirname(__DIR__))) . '/../_files/DataproviderExecutionOrderTest.php';
	require __DIR__ . '/../../../../bootstrap.php';
	PHPUnit\TextUI\Command::main();
	unlink($tmpResultCache);

I am not sure what the best solution is that works in all cases, honestly. Wish I had a better answer for you already :-(