[SPARK-20131][DStream][Test] Flaky Test: org.apache.spark.streaming.StreamingContextSuite

[uncleGen]

What changes were proposed in this pull request?

do not stop the SparkContext in another separate thread.

How was this patch tested?

Jenkins.

[srowen]

I'm just curious why this solves the problem? if the problem is that the streaming context doesn't shut down or doesn't shut down quickly, then I'd suspect that it's not because shutting down SparkContext is the slow part, but I'm not sure.

[SparkQA]

Test build #75340 has finished for PR 17463 at commit 89d1d35.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #75342 has finished for PR 17463 at commit c68c285.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[uncleGen]

@srowen It's hard to say it's because shutting down SparkContext is the slow part, and we can improve this case by avoiding stopping SparkContext in a separate thread.
cc @zsxwing

[srowen]

Hm, I'm not sure. For example, what if the error is that the context is never stopped? maybe assert that awaitTerminationOrTimeout is true to also check this? In my experience it's the streaming context that can take a while to stop because it has to wait for batches. It's not 'graceful' in this test though, but I've certainly seen it fail to stop quickly.

[srowen]

I think I'd close this one as I don't think it is necessarily fixing the problem, and might mask one.

[zsxwing]

Could you close this one, please? I submitted #17610 to fix the root issue.