Skip to content

Commit

Permalink
[MINOR][SQL] Fix wrong comments on org.apache.spark.sql.parquet.row.a…
Browse files Browse the repository at this point in the history
…ttributes

## What changes were proposed in this pull request?

This PR fixes the wrong comment on `org.apache.spark.sql.parquet.row.attributes`
which is useful for UDTs like Vector/Matrix. Please see [SPARK-22320](https://issues.apache.org/jira/browse/SPARK-22320) for the usage.

Originally, [SPARK-19411](bf49368#diff-ee26d4c4be21e92e92a02e9f16dbc285L314) left this behind during removing optional column metadatas. In the same PR, the same comment was removed at line 310-311.

## How was this patch tested?

N/A (This is about comments).

Author: Dongjoon Hyun <dongjoon@apache.org>

Closes #20346 from dongjoon-hyun/minor_comment_parquet.
  • Loading branch information
dongjoon-hyun authored and HyukjinKwon committed Jan 22, 2018
1 parent 11daeb8 commit 8142a3b
Showing 1 changed file with 1 addition and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,7 @@ class ParquetFileFormat

ParquetOutputFormat.setWriteSupportClass(job, classOf[ParquetWriteSupport])

// We want to clear this temporary metadata from saving into Parquet file.
// This metadata is only useful for detecting optional columns when pushdowning filters.
// This metadata is useful for keeping UDTs like Vector/Matrix.
ParquetWriteSupport.setSchema(dataSchema, conf)

// Sets flags for `ParquetWriteSupport`, which converts Catalyst schema to Parquet
Expand Down

0 comments on commit 8142a3b

Please sign in to comment.