Skip to content
This repository has been archived by the owner on Nov 15, 2024. It is now read-only.

Commit

Permalink
[SS][MINOR] Fix flaky test in DatastreamReaderWriterSuite. temp check…
Browse files Browse the repository at this point in the history
…point dir should be deleted

## What changes were proposed in this pull request?

Stopping query while it is being initialized can throw interrupt exception, in which case temporary checkpoint directories will not be deleted, and the test will fail.

Author: Tathagata Das <tathagata.das1565@gmail.com>

Closes apache#18442 from tdas/DatastreamReaderWriterSuite-fix.

(cherry picked from commit 60043f2)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
  • Loading branch information
tdas authored and MatthewRBruce committed Jul 31, 2018
1 parent 3712d9a commit 1cf241a
Showing 1 changed file with 1 addition and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -641,6 +641,7 @@ class DataStreamReaderWriterSuite extends StreamTest with BeforeAndAfter {
test("temp checkpoint dir should be deleted if a query is stopped without errors") {
import testImplicits._
val query = MemoryStream[Int].toDS.writeStream.format("console").start()
query.processAllAvailable()
val checkpointDir = new Path(
query.asInstanceOf[StreamingQueryWrapper].streamingQuery.checkpointRoot)
val fs = checkpointDir.getFileSystem(spark.sessionState.newHadoopConf())
Expand Down

0 comments on commit 1cf241a

Please sign in to comment.