Code

Upstream merge#228

Code

Merged

robert3005 merged 127 commits intomasterpalantir/spark:masterfrom rk/upstreampalantir/spark:rk/upstreamCopy head branch name to clipboardJul 14, 2017

+11,621-9,075

Commits

Commits on Jun 30, 2017

• [SPARK-18710][ML] Add offset in GLM

  Show description for 49d767d
  

  actuaryzhang

  authored and

  yanboliang

  committed
  49d767d

  Copy full SHA for 49d767d

• [SPARK-18294][CORE] Implement commit protocol to support mapred package's committer

  Show description for 3c2fc19
  

  jiangxb1987

  authored and

  cloud-fan

  committed
  3c2fc19

  Copy full SHA for 3c2fc19

• [ML] Fix scala-2.10 build failure of GeneralizedLinearRegressionSuite.

  Show description for 528c928
  yanboliang

  committed
  528c928

  Copy full SHA for 528c928

• [SPARK-21223] Change fileToAppInfo in FsHistoryProvider to fix concurrent issue.

  Show description for 1fe08d6
  

  曾林西

  authored and

  srowen

  committed
  1fe08d6

  Copy full SHA for 1fe08d6

• [SPARK-21129][SQL] Arguments of SQL function call should not be named expressions

  Show description for eed9c4e
  gatorsmile

  committed
  eed9c4e

  Copy full SHA for eed9c4e

• [SPARK-21052][SQL][FOLLOW-UP] Add hash map metrics to join

  Show description for fd13255
  

  viirya

  authored and

  gatorsmile

  committed
  fd13255

  Copy full SHA for fd13255

Commits on Jul 1, 2017

• [SPARK-17528][SQL] data should be copied properly before saving into InternalRow

  Show description for 4eb4187
  cloud-fan

  committed
  4eb4187

  Copy full SHA for 4eb4187

• [SPARK-21127][SQL] Update statistics after data changing commands

  Show description for 61b5df5
  

  wzhfy

  authored and

  cloud-fan

  committed
  61b5df5

  Copy full SHA for 61b5df5

• [SPARK-21273][SQL] Propagate logical plan stats using visitor pattern and mixin

  Show description for b1d719e
  

  rxin

  authored and

  gatorsmile

  committed
  b1d719e

  Copy full SHA for b1d719e

• [SPARK-21275][ML] Update GLM test to use supportedFamilyNames

  Show description for 37ef32e
  

  actuaryzhang

  authored and

  yanboliang

  committed
  37ef32e

  Copy full SHA for 37ef32e

• [SPARK-18518][ML] HasSolver supports override

  Show description for e0b047e
  

  zhengruifeng

  authored and

  yanboliang

  committed
  e0b047e

  Copy full SHA for e0b047e

• [SPARK-21170][CORE] Utils.tryWithSafeFinallyAndFailureCallbacks throws IllegalArgumentException: Self-suppression not permitted

  Show description for 6beca9c
  

  Devaraj K

  authored and

  srowen

  committed
  6beca9c

  Copy full SHA for 6beca9c

Commits on Jul 2, 2017

• [SPARK-21260][SQL][MINOR] Remove the unused OutputFakerExec

  Show description for c605fee
  

  jiangxb1987

  authored and

  srowen

  committed
  c605fee

  Copy full SHA for c605fee

• [SPARK-19852][PYSPARK][ML] Python StringIndexer supports 'keep' to handle invalid data

  Show description for c19680b
  yanboliang

  committed
  c19680b

  Copy full SHA for c19680b

Commits on Jul 3, 2017

• [SPARK-18004][SQL] Make sure the date or timestamp related predicate can be pushed down to Oracle correctly

  Show description for d410719
  

  SharpRay

  authored and

  gatorsmile

  committed
  d410719

  Copy full SHA for d410719

• [SPARK-21250][WEB-UI] Add a url in the table of 'Running Executors' in worker page to visit job page.

  Show description for d913db1
  

  guoxiaolong

  authored and

  cloud-fan

  committed
  d913db1

  Copy full SHA for d913db1

• [SPARK-21137][CORE] Spark reads many small files slowly

  Show description for a9339db
  

  srowen

  authored and

  cloud-fan

  committed
  a9339db

  Copy full SHA for a9339db

• [TEST] Load test table based on case sensitivity

  Show description for eb7a5a6
  

  wzhfy

  authored and

  gatorsmile

  committed
  eb7a5a6

  Copy full SHA for eb7a5a6

• [SPARK-21102][SQL] Refresh command is too aggressive in parsing

  Show description for 17bdc36
  

  aokolnychyi

  authored and

  gatorsmile

  committed
  17bdc36

  Copy full SHA for 17bdc36

• [SPARK-20073][SQL] Prints an explicit warning message in case of NULL-safe equals

  Show description for 363bfe3
  

  maropu

  authored and

  gatorsmile

  committed
  363bfe3

  Copy full SHA for 363bfe3

• [SPARK-21284][SQL] rename SessionCatalog.registerFunction parameter name

  Show description for f953ca5
  

  cloud-fan

  authored and

  gatorsmile

  committed
  f953ca5

  Copy full SHA for f953ca5

• [TEST] Different behaviors of SparkContext Conf when building SparkSession

  Show description for c79c10e
  gatorsmile

  committed
  c79c10e

  Copy full SHA for c79c10e

Commits on Jul 4, 2017

• [SPARK-21283][CORE] FileOutputStream should be created as append mode

  Show description for 6657e00
  

  10110346

  authored and

  cloud-fan

  committed
  6657e00

  Copy full SHA for 6657e00

• [SPARK-21264][PYTHON] Call cross join path in join without 'on' and with 'how'

  Show description for a848d55
  

  HyukjinKwon

  authored and

  ueshin

  committed
  a848d55

  Copy full SHA for a848d55

• [MINOR] Add french stop word "les"

  Show description for 8ca4ebe
  

  ebuildy

  authored and

  srowen

  committed
  8ca4ebe

  Copy full SHA for 8ca4ebe

• [MINOR][SPARK SUBMIT] Print out R file usage in spark-submit

  Show description for 2b1e94b
  

  HyukjinKwon

  authored and

  srowen

  committed
  2b1e94b

  Copy full SHA for 2b1e94b

• [SPARK-19507][SPARK-21296][PYTHON] Avoid per-record type dispatch in schema verification and improve exception message

  Show description for d492cc5
  

  HyukjinKwon

  authored and

  cloud-fan

  committed
  d492cc5

  Copy full SHA for d492cc5

• [SPARK-21256][SQL] Add withSQLConf to Catalyst Test

  Show description for 29b1f6b
  gatorsmile

  committed
  29b1f6b

  Copy full SHA for 29b1f6b

• [SPARK-19726][SQL] Faild to insert null timestamp value to mysql using spark jdbc

  Show description for a3c29fc
  

  YIHAODIAN\wangshuangshuang

  authored and

  gatorsmile

  committed
  a3c29fc

  Copy full SHA for a3c29fc

• [SPARK-20256][SQL] SessionState should be created more lazily

  Show description for 1b50e0e
  

  dongjoon-hyun

  authored and

  gatorsmile

  committed
  1b50e0e

  Copy full SHA for 1b50e0e

• [SPARK-21268][MLLIB] Move center calculations to a distributed map in KMeans

  Show description for 4d6d819
  

  gjgd

  authored and

  srowen

  committed
  4d6d819

  Copy full SHA for 4d6d819

• [SPARK-20889][SPARKR] Grouped documentation for WINDOW column methods

  Show description for cec3921
  

  actuaryzhang

  authored and

  Felix Cheung

  committed
  cec3921

  Copy full SHA for cec3921

• [MINOR][SPARKR] ignore Rplots.pdf test output after running R tests

  Show description for daabf42
  

  wangmiao1981

  authored and

  Felix Cheung

  committed
  daabf42

  Copy full SHA for daabf42

Commits on Jul 5, 2017

• [SPARK-21295][SQL] Use qualified names in error message for missing references

  Show description for de14086
  

  gatorsmile

  authored and

  cloud-fan

  committed
  de14086

  Copy full SHA for de14086

• [SPARK-21300][SQL] ExternalMapToCatalyst should null-check map key prior to converting to internal value.

  Show description for ce10545
  

  ueshin

  authored and

  cloud-fan

  committed
  ce10545

  Copy full SHA for ce10545

• [SPARK-20889][SPARKR][FOLLOWUP] Clean up grouped doc for column methods

  Show description for e9a93f8
  

  actuaryzhang

  authored and

  Felix Cheung

  committed
  e9a93f8

  Copy full SHA for e9a93f8

• [SPARK-21304][SQL] remove unnecessary isNull variable for collection related encoder expressions

  Show description for f2c3b1d
  cloud-fan

  committed
  f2c3b1d

  Copy full SHA for f2c3b1d

• [SPARK-18623][SQL] Add returnNullable to StaticInvoke and modify it to handle properly.

  Show description for a386432
  

  ueshin

  authored and

  cloud-fan

  committed
  a386432

  Copy full SHA for a386432

• [SPARK-21310][ML][PYSPARK] Expose offset in PySpark

  Show description for 4852b7d
  

  actuaryzhang

  authored and

  yanboliang

  committed
  4852b7d

  Copy full SHA for 4852b7d

• [SPARK-16167][SQL] RowEncoder should preserve array/map type nullability.

  Show description for 873f3ad
  

  ueshin

  authored and

  cloud-fan

  committed
  873f3ad

  Copy full SHA for 873f3ad

• [SPARK-20383][SQL] Supporting Create [temporary] Function with the keyword 'OR REPLACE' and 'IF NOT EXISTS'

  Show description for 5787ace
  

  ouyangxiaochen

  authored and

  cloud-fan

  committed
  5787ace

  Copy full SHA for 5787ace

• [SPARK-21286][TEST] Modified StorageTabSuite unit test

  Show description for e3e2b5d
  

  Geek-He

  authored and

  cloud-fan

  committed
  e3e2b5d

  Copy full SHA for e3e2b5d

• [SPARK-20858][DOC][MINOR] Document ListenerBus event queue size

  Show description for 960298e
  

  sadikovi

  authored and

  srowen

  committed
  960298e

  Copy full SHA for 960298e

• [SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFunction Should Support UDAFs

  Show description for 742da08
  

  zjffdu

  authored and

  gatorsmile

  committed
  742da08

  Copy full SHA for 742da08

• [SPARK-21307][SQL] Remove SQLConf parameters from the parser-related classes.

  Show description for c8e7f44
  gatorsmile

  committed
  c8e7f44

  Copy full SHA for c8e7f44

• [SPARK-21278][PYSPARK] Upgrade to Py4J 0.10.6

  Show description for c8d0aba
  

  dongjoon-hyun

  authored and

  holdenk

  committed
  c8d0aba

  Copy full SHA for c8d0aba

Commits on Jul 6, 2017

• [SPARK-21248][SS] The clean up codes in StreamExecution should not be interrupted

  Show description for ab866f1
  zsxwing

  committed
  ab866f1

  Copy full SHA for ab866f1

• [SPARK-21308][SQL] Remove SQLConf parameters from the optimizer

  Show description for 75b168f
  

  gatorsmile

  authored and

  cloud-fan

  committed
  75b168f

  Copy full SHA for 75b168f

• [SPARK-21312][SQL] correct offsetInBytes in UnsafeRow.writeToStream

  Show description for 14a3bb3
  

  Sumedh Wale

  authored and

  cloud-fan

  committed
  14a3bb3

  Copy full SHA for 14a3bb3

• [SS][MINOR] Fix flaky test in DatastreamReaderWriterSuite. temp checkpoint dir should be deleted

  Show description for 60043f2
  tdas

  committed
  60043f2

  Copy full SHA for 60043f2

• [SPARK-21012][SUBMIT] Add glob support for resources adding to Spark

  Show description for 5800144
  

  jerryshao

  authored and

  cloud-fan

  committed
  5800144

  Copy full SHA for 5800144

• [SPARK-20703][SQL] Associate metrics with data writes onto DataFrameWriter operations

  Show description for 6ff05a6
  

  viirya

  authored and

  cloud-fan

  committed
  6ff05a6

  Copy full SHA for 6ff05a6

• [SPARK-21324][TEST] Improve statistics test suites

  Show description for b8e4d56
  

  wzhfy

  authored and

  cloud-fan

  committed
  b8e4d56

  Copy full SHA for b8e4d56

• [SPARK-21273][SQL][FOLLOW-UP] Add missing test cases back and revise code style

  Show description for d540dfb
  

  gengliangwang

  authored and

  cloud-fan

  committed
  d540dfb

  Copy full SHA for d540dfb

• [SPARK-20950][CORE] add a new config to diskWriteBufferSize which is hard coded before

  Show description for 565e7a8
  

  heary-cao

  authored and

  cloud-fan

  committed
  565e7a8

  Copy full SHA for 565e7a8

• [SPARK-21228][SQL] InSet incorrect handling of structs

  Show description for 26ac085
  

  bogdanrdc

  authored and

  cloud-fan

  committed
  26ac085

  Copy full SHA for 26ac085

• [SPARK-21204][SQL] Add support for Scala Set collection types in serialization

  Show description for 48e44b2
  

  viirya

  authored and

  cloud-fan

  committed
  48e44b2

  Copy full SHA for 48e44b2

• [SPARK-21323][SQL] Rename plans.logical.statsEstimation.Range to ValueInterval

  Show description for bf66335
  

  gengliangwang

  authored and

  rxin

  committed
  bf66335

  Copy full SHA for bf66335

Commits on Jul 7, 2017

• [SPARK-21267][SS][DOCS] Update Structured Streaming Documentation

  Show description for 0217dfd
  

  tdas

  authored and

  zsxwing

  committed
  0217dfd

  Copy full SHA for 0217dfd

• [SPARK-20946][SQL] Do not update conf for existing SparkContext in SparkSession.getOrCreate

  Show description for 40c7add
  cloud-fan

  committed
  40c7add

  Copy full SHA for 40c7add

• [SPARK-21329][SS] Make EventTimeWatermarkExec explicitly UnaryExecNode

  Show description for e5bb261
  

  jaceklaskowski

  authored and

  zsxwing

  committed
  e5bb261

  Copy full SHA for e5bb261

• [SPARK-21326][SPARK-21066][ML] Use TextFileFormat in LibSVMFileFormat and allow multiple input paths for determining numFeatures

  Show description for d451b7f
  

  HyukjinKwon

  authored and

  cloud-fan

  committed
  d451b7f

  Copy full SHA for d451b7f

• [SPARK-21327][SQL][PYSPARK] ArrayConstructor should handle an array of typecode 'l' as long rather than int in Python 2.

  Show description for 53c2eb5
  ueshin

  committed
  53c2eb5

  Copy full SHA for 53c2eb5

• [SPARK-21217][SQL] Support ColumnVector.Array.to<type>Array()

  Show description for c09b31e
  

  kiszk

  authored and

  cloud-fan

  committed
  c09b31e

  Copy full SHA for c09b31e

• [SPARK-20703][SQL][FOLLOW-UP] Associate metrics with data writes onto DataFrameWriter operations

  Show description for 5df99bd
  

  viirya

  authored and

  cloud-fan

  committed
  5df99bd

  Copy full SHA for 5df99bd

• [SPARK-21313][SS] ConsoleSink's string representation

  Show description for 7fcbb9b
  

  jaceklaskowski

  authored and

  srowen

  committed
  7fcbb9b

  Copy full SHA for 7fcbb9b

• [SPARK-21285][ML] VectorAssembler reports the column name of unsupported data type

  Show description for 56536e9
  

  facaiy

  authored and

  yanboliang

  committed
  56536e9

  Copy full SHA for 56536e9

• [SPARK-21335][SQL] support un-aliased subquery

  Show description for fef0813
  cloud-fan

  committed
  fef0813

  Copy full SHA for fef0813

• [SPARK-19358][CORE] LiveListenerBus shall log the event name when dropping them due to a fully filled queue

  Show description for fbbe37e
  

  CodingCat

  authored and

  cloud-fan

  committed
  fbbe37e

  Copy full SHA for fbbe37e

• [SPARK-21336] Revise rand comparison in BatchEvalPythonExecSuite

  Show description for a0fe32a
  

  gengliangwang

  authored and

  gatorsmile

  committed
  a0fe32a

  Copy full SHA for a0fe32a

Commits on Jul 8, 2017

• [SPARK-21100][SQL] Add summary method as alternative to describe that gives quartiles similar to Pandas

  Show description for e1a172c
  

  aray

  authored and

  cloud-fan

  committed
  e1a172c

  Copy full SHA for e1a172c

• [SPARK-21281][SQL] Use string types by default if array and map have no argument

  Show description for 7896e7b
  

  maropu

  authored and

  gatorsmile

  committed
  7896e7b

  Copy full SHA for 7896e7b

• [SPARK-20379][CORE] Allow SSL config to reference env variables.

  Show description for 9760c15
  

  Marcelo Vanzin

  authored and

  cloud-fan

  committed
  9760c15

  Copy full SHA for 9760c15

• [SPARK-21069][SS][DOCS] Add rate source to programming guide.

  Show description for d0bfc67
  

  ScrapCodes

  authored and

  zsxwing

  committed
  d0bfc67

  Copy full SHA for d0bfc67

• [SPARK-20307][SPARKR] SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

  Show description for a7b46c6
  

  wangmiao1981

  authored and

  Felix Cheung

  committed
  a7b46c6

  Copy full SHA for a7b46c6

• [SPARK-20456][DOCS] Add examples for functions collection for pyspark

  Show description for f5f02d2
  

  map222

  authored and

  Felix Cheung

  committed
  f5f02d2

  Copy full SHA for f5f02d2

• Mesos doc fixes

  Show description for 01f183e
  

  Joachim Hereth

  authored and

  srowen

  committed
  01f183e

  Copy full SHA for 01f183e

• [SPARK-20609][MLLIB][TEST] manually cleared 'spark.local.dir' before/after a test in ALSCleanerSuite

  Show description for 330bf5c
  

  heary-cao

  authored and

  srowen

  committed
  330bf5c

  Copy full SHA for 330bf5c

• [SPARK-21345][SQL][TEST][TEST-MAVEN] SparkSessionBuilderSuite should clean up stopped sessions.

  Show description for 0b8dd2d
  

  dongjoon-hyun

  authored and

  cloud-fan

  committed
  0b8dd2d

  Copy full SHA for 0b8dd2d

• [SPARK-21083][SQL] Store zero size and row count when analyzing empty table

  Show description for 9fccc36
  

  wzhfy

  authored and

  cloud-fan

  committed
  9fccc36

  Copy full SHA for 9fccc36

• [SPARK-20342][CORE] Update task accumulators before sending task end event.

  Show description for 9131bdb
  

  Marcelo Vanzin

  authored and

  cloud-fan

  committed
  9131bdb

  Copy full SHA for 9131bdb

• [SPARK-21343] Refine the document for spark.reducer.maxReqSizeShuffleToMem.

  Show description for 062c336
  

  jinxing

  authored and

  cloud-fan

  committed
  062c336

  Copy full SHA for 062c336

• [SPARK-21307][REVERT][SQL] Remove SQLConf parameters from the parser-related classes

  Show description for c3712b7
  gatorsmile

  committed
  c3712b7

  Copy full SHA for c3712b7

• [SPARK-21093][R] Terminate R's worker processes in the parent of R's daemon to prevent a leak

  Show description for 08e0d03
  

  HyukjinKwon

  authored and

  Felix Cheung

  committed
  08e0d03

  Copy full SHA for 08e0d03

Commits on Jul 9, 2017

• [SPARK-18016][SQL][FOLLOWUP] merge declareAddedFunctions, initNestedClasses and declareNestedClasses

  Show description for 680b33f
  

  cloud-fan

  authored and

  gatorsmile

  committed
  680b33f

  Copy full SHA for 680b33f

Commits on Jul 10, 2017

• [MINOR][DOC] Improve the docs about how to correctly set configurations

  Show description for 457dc9c
  

  jerryshao

  authored and

  cloud-fan

  committed
  457dc9c

  Copy full SHA for 457dc9c

• [SPARK-21100][SQL][FOLLOWUP] cleanup code and add more comments for Dataset.summary

  Show description for 0e80eca
  

  cloud-fan

  authored and

  gatorsmile

  committed
  0e80eca

  Copy full SHA for 0e80eca

• [SPARK-21219][CORE] Task retry occurs on same executor due to race condition with blacklisting

  Show description for 96d58f2
  

  Eric Vandenberg

  authored and

  cloud-fan

  committed
  96d58f2

  Copy full SHA for 96d58f2

• [MINOR][DOC] Remove obsolete ec2-scripts.md

  Show description for c444d10
  

  dongjoon-hyun

  authored and

  srowen

  committed
  c444d10

  Copy full SHA for c444d10

• [SPARK-20460][SQL] Make it more consistent to handle column name duplication

  Show description for 647963a
  

  maropu

  authored and

  cloud-fan

  committed
  647963a

  Copy full SHA for 647963a

• [SPARK-21342] Fix DownloadCallback to work well with RetryingBlockFetcher.

  Show description for 6a06c4b
  

  jinxing

  authored and

  cloud-fan

  committed
  6a06c4b

  Copy full SHA for 6a06c4b

• [SPARK-21272] SortMergeJoin LeftAnti does not update numOutputRows

  Show description for 18b3b00
  

  juliuszsompolski

  authored and

  gatorsmile

  committed
  18b3b00

  Copy full SHA for 18b3b00

• [SPARK-21266][R][PYTHON] Support schema a DDL-formatted string in dapply/gapply/from_json

  Show description for 2bfd5ac
  

  HyukjinKwon

  authored and

  Felix Cheung

  committed
  2bfd5ac

  Copy full SHA for 2bfd5ac

• [SPARK-13534][PYSPARK] Using Apache Arrow to increase performance of DataFrame.toPandas

  Show description for d03aebb
  

  BryanCutler

  authored and

  holdenk

  committed
  d03aebb

  Copy full SHA for d03aebb

Commits on Jul 11, 2017

• [SPARK-21358][EXAMPLES] Argument of repartitionandsortwithinpartitions at pyspark

  Show description for c3713fd
  

  chie8842

  authored and

  rxin

  committed
  c3713fd

  Copy full SHA for c3713fd

• [SPARK-21043][SQL] Add unionByName in Dataset

  Show description for a2bec6c
  

  maropu

  authored and

  gatorsmile

  committed
  a2bec6c

  Copy full SHA for a2bec6c

• [SPARK-21350][SQL] Fix the error message when the number of arguments is wrong when invoking a UDF

  Show description for 1471ee7
  

  gatorsmile

  authored and

  cloud-fan

  committed
  1471ee7

  Copy full SHA for 1471ee7

• [SPARK-21369][CORE] Don't use Scala Tuple2 in common/network-*

  Show description for 833eab2
  

  zsxwing

  authored and

  cloud-fan

  committed
  833eab2

  Copy full SHA for 833eab2

• [SPARK-21315][SQL] Skip some spill files when generateIterator(startIndex) in ExternalAppendOnlyUnsafeRowArray.

  Show description for 97a1aa2
  

  jinxing

  authored and

  cloud-fan

  committed
  97a1aa2

  Copy full SHA for 97a1aa2

• [SPARK-20456][PYTHON][FOLLOWUP] Fix timezone-dependent doctests in unix_timestamp and from_unixtime

  Show description for d4d9e17
  

  HyukjinKwon

  authored and

  ueshin

  committed
  d4d9e17

  Copy full SHA for d4d9e17

• [SPARK-20331][SQL] Enhanced Hive partition pruning predicate pushdown

  Show description for a4baa8f
  

  Michael Allman

  authored and

  cloud-fan

  committed
  a4baa8f

  Copy full SHA for a4baa8f

• [SPARK-21263][SQL] Do not allow partially parsing double and floats via NumberFormat in CSV

  Show description for 7514db1
  

  HyukjinKwon

  authored and

  srowen

  committed
  7514db1

  Copy full SHA for 7514db1

• [SPARK-21366][SQL][TEST] Add sql test for window functions

  Show description for 66d2168
  

  jiangxb1987

  authored and

  cloud-fan

  committed
  66d2168

  Copy full SHA for 66d2168

• [SPARK-21365][PYTHON] Deduplicate logics parsing DDL type/schema definition

  Show description for ebc124d
  

  HyukjinKwon

  authored and

  cloud-fan

  committed
  ebc124d

  Copy full SHA for ebc124d

• [SPARK-16019][YARN] Use separate RM poll interval when starting client AM.

  Show description for 1cad31f
  

  Marcelo Vanzin

  committed
  1cad31f

  Copy full SHA for 1cad31f

• [SPARK-19285][SQL] Implement UDF0

  Show description for d3e0716
  gatorsmile

  committed
  d3e0716

  Copy full SHA for d3e0716

Commits on Jul 12, 2017

• [SPARK-12139][SQL] REGEX Column Specification

  Show description for 2cbfc97
  

  janewangfb

  authored and

  gatorsmile

  committed
  2cbfc97

  Copy full SHA for 2cbfc97

• [SPARK-21382] The note about Scala 2.10 in building-spark.md is wrong.

  Show description for 24367f2
  

  liu-zhaokun

  authored and

  rxin

  committed
  24367f2

  Copy full SHA for 24367f2

• [SPARK-21146][CORE] Master/Worker should handle and shutdown when any thread gets UncaughtException

  Show description for e16e8c7
  

  Devaraj K

  authored and

  zsxwing

  committed
  e16e8c7

  Copy full SHA for e16e8c7

• [SPARK-21370][SS] Add test for state reliability when one read-only state store aborts after read-write state store commits

  Show description for e0af76a
  

  brkyvz

  authored and

  tdas

  committed
  e0af76a

  Copy full SHA for e0af76a

• [SPARK-20842][SQL] Upgrade to 1.2.2 for Hive Metastore Client 1.2

  Show description for f587d2e
  

  gatorsmile

  authored and

  cloud-fan

  committed
  f587d2e

  Copy full SHA for f587d2e

• [SPARK-21305][ML][MLLIB] Add options to disable multi-threading of native BLAS

  Show description for 5ed134e
  

  Peng Meng

  authored and

  srowen

  committed
  5ed134e

  Copy full SHA for 5ed134e

• [SPARK-21007][SQL] Add SQL function - RIGHT && LEFT

  Show description for aaad34d
  

  10110346

  authored and

  cloud-fan

  committed
  aaad34d

  Copy full SHA for aaad34d

• [SPARK-18619][ML] Make QuantileDiscretizer/Bucketizer/StringIndexer/RFormula inherit from HasHandleInvalid

  Show description for d2d2a5d
  

  zhengruifeng

  authored and

  yanboliang

  committed
  d2d2a5d

  Copy full SHA for d2d2a5d

• [SPARK-17701][SQL] Refactor RowDataSourceScanExec so its sameResult call does not compare strings

  Show description for 780586a
  

  cloud-fan

  authored and

  gatorsmile

  committed
  780586a

  Copy full SHA for 780586a

Commits on Jul 13, 2017

• [SPARK-18646][REPL] Set parent classloader as null for ExecutorClassLoader

  Show description for e08d06b
  

  taroplus

  authored and

  cloud-fan

  committed
  e08d06b

  Copy full SHA for e08d06b

• [SPARK-19810][BUILD][CORE] Remove support for Scala 2.10

  Show description for 425c4ad
  

  srowen

  authored and

  cloud-fan

  committed
  425c4ad

  Copy full SHA for 425c4ad

• [SPARK-21373][CORE] Update Jetty to 9.3.20.v20170531

  Show description for af80e01
  

  kiszk

  authored and

  srowen

  committed
  af80e01

  Copy full SHA for af80e01

• [SPARK-21403][MESOS] fix --packages for mesos

  Show description for d8257b9
  

  skonto

  authored and

  Marcelo Vanzin

  committed
  d8257b9

  Copy full SHA for d8257b9

• [SPARK-15526][MLLIB] Shade JPMML

  Show description for 5c8edfc
  

  srowen

  authored and

  Marcelo Vanzin

  committed
  5c8edfc

  Copy full SHA for 5c8edfc

• Merge branch 'master' into rk/upstream

  

  Robert Kruszewski

  committed
  aa2e951

  Copy full SHA for aa2e951

• resolve conflicts

  

  Robert Kruszewski

  committed
  2ca1ed9

  Copy full SHA for 2ca1ed9

• [SPARK-21376][YARN] Fix yarn client token expire issue when cleaning the staging files in long running scenario

  Show description for cb8d5cc
  

  jerryshao

  authored and

  Marcelo Vanzin

  committed
  cb8d5cc

  Copy full SHA for cb8d5cc

Commits on Jul 14, 2017

• Merge branch 'master' into rk/upstream

  

  Robert Kruszewski

  committed
  1fe3936

  Copy full SHA for 1fe3936

• linting

  

  Robert Kruszewski

  committed
  9c973e4

  Copy full SHA for 9c973e4

• update dependencies

  

  Robert Kruszewski

  committed
  0e0cc0a

  Copy full SHA for 0e0cc0a

• py4j in conda version

  

  Robert Kruszewski

  committed
  d56829a

  Copy full SHA for d56829a