[MINOR][DOC] Update deprecation notes on Python/Hadoop/Scala. #18207
Conversation
|
Hi, @marmbrus . |
|
I wasn't clear whether we really truly don't support 2.6 in this 2.2.0 release, given https://issues.apache.org/jira/browse/SPARK-12661 ? |
|
Oh, I see. Then, may I ask this at mailing list to be more clearer? |
|
Test build #77754 has finished for PR 18207 at commit
|
|
/cc @JoshRosen |
|
As of #17355 we no longer test against Python 2.6. That doesn't mean that 2.6 won't work today, but there's nothing stopping 2.6 support from breaking in a future 2.2.x release because we are no longer testing against that release. #17355 replaced our Python 2.6 testing environment with a Python 2.7 release, so we can now begin to use language features and libraries which are only available from 2.7 onwards (such as set and dictionary comprehensions). Therefore, this documentation change looks correct to me. |
|
Thank you for confirming, @JoshRosen . |
|
I believe we still support Python 2.6, given Jenkins runs 2.6... There seems to be no point in removing that support this late in the release cycle. |
|
Jenkins runs with |
|
OK great then we have officially deprecated it, haven't we? |
|
Yep. Right. Then, could you officially resolve SPARK-12661, too? |
| uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version | ||
| ({{site.SCALA_BINARY_VERSION}}.x). | ||
|
|
||
| Note that support for Java 7 was removed as of Spark 2.2.0. | ||
| Note that support for Java 7, Python 2.6 and old Hadoop versions before 2.6.5 were removed as of Spark 2.2.0. |
There was a problem hiding this comment.
It sounds like 2.6 is still more supported than not-support in 2.2.x, but we should call it unsupported for 2.3.x.
Let's say that Python 2.6 support was really removed for 2.3.0, rather than 2.2.0.
It's fine to add a note about Hadoop before 2.6 being unsupported in 2.2.0, that's true.
Then we can merge in master. It's not essential to merge this for 2.2 as the existing text appears to be not-wrong.
There was a problem hiding this comment.
Did you test on Python 2.6 at RC4 testing? And it happens to be supported at 2.2.0?
Actually, Jenkins does not run the tests over two months (since 29th Mar). As @JoshRosen said, there is no guard anymore in order to guarantee still more supported on 2.2.1 or later.
|
Even if Python 2.6 "works by accident" on 2.2.0 that should not impact this
wording: as long as we marked Python 2.6 as deprecated in an earlier
release then it should be fine to claim non support in 2.2.0.
…On Tue, Jun 6, 2017, 8:25 AM Dongjoon Hyun ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In docs/index.md
<#18207 (comment)>:
> uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
({{site.SCALA_BINARY_VERSION}}.x).
-Note that support for Java 7 was removed as of Spark 2.2.0.
+Note that support for Java 7, Python 2.6 and old Hadoop versions before 2.6.5 were removed as of Spark 2.2.0.
Did you test on Python 2.6 at RC4 testing? And it happens to be supported
at 2.2.0?
Actually, Jenkins does not run the tests over two months (since 29th Mar).
As @JoshRosen <https://github.com/joshrosen> said, there is no guard
anymore in order to guarantee still more supported on 2.2.1 or later.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#18207 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AADGPJtuOTgL3_uYB4wzbbBpEBmlb1o2ks5sBW99gaJpZM4NweCt>
.
|
|
Put another way: if we found a bug only impacting Python 2.6 in 2.2.0 would
we fix that bug in 2.2.1? If not then the right thing to do is to say 2.6
isn't supported.
…On Tue, Jun 6, 2017, 8:44 AM Josh Rosen ***@***.***> wrote:
Even if Python 2.6 "works by accident" on 2.2.0 that should not impact
this wording: as long as we marked Python 2.6 as deprecated in an earlier
release then it should be fine to claim non support in 2.2.0.
On Tue, Jun 6, 2017, 8:25 AM Dongjoon Hyun ***@***.***>
wrote:
> ***@***.**** commented on this pull request.
> ------------------------------
>
> In docs/index.md
> <#18207 (comment)>:
>
> > uses Scala {{site.SCALA_BINARY_VERSION}}. You will need to use a compatible Scala version
> ({{site.SCALA_BINARY_VERSION}}.x).
>
> -Note that support for Java 7 was removed as of Spark 2.2.0.
> +Note that support for Java 7, Python 2.6 and old Hadoop versions before 2.6.5 were removed as of Spark 2.2.0.
>
> Did you test on Python 2.6 at RC4 testing? And it happens to be supported
> at 2.2.0?
> Actually, Jenkins does not run the tests over two months (since 29th
> Mar). As @JoshRosen <https://github.com/joshrosen> said, there is no
> guard anymore in order to guarantee still more supported on 2.2.1 or
> later.
>
> —
> You are receiving this because you were mentioned.
>
>
> Reply to this email directly, view it on GitHub
> <#18207 (comment)>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AADGPJtuOTgL3_uYB4wzbbBpEBmlb1o2ks5sBW99gaJpZM4NweCt>
> .
>
|
|
I think the question is whether we can get this change into the 2.2 release. It'd be ideal, but not sure it's worth another RC. Of course, this runs into the discussion we're currently having about when to update docs. |
|
I hope it goes to |
|
Merged to master/2.2 |
## What changes were proposed in this pull request? We had better update the deprecation notes about Python 2.6, Hadoop (before 2.6.5) and Scala 2.10 in [2.2.0-RC4](http://people.apache.org/~pwendell/spark-releases/spark-2.2.0-rc4-docs/) documentation. Since this is a doc only update, I think we can update the doc during publishing. **BEFORE (2.2.0-RC4)**  **AFTER**  ## How was this patch tested? Manual. ``` SKIP_API=1 jekyll build ``` Author: Dongjoon Hyun <dongjoon@apache.org> Closes #18207 from dongjoon-hyun/minor_doc_deprecation. (cherry picked from commit 3218505) Signed-off-by: Sean Owen <sowen@cloudera.com>
|
Thank you all! |
What changes were proposed in this pull request?
We had better update the deprecation notes about Python 2.6, Hadoop (before 2.6.5) and Scala 2.10 in 2.2.0-RC4 documentation. Since this is a doc only update, I think we can update the doc during publishing.
BEFORE (2.2.0-RC4)
AFTER
How was this patch tested?
Manual.