Skip to content

Commit

Permalink
Raise an error when the row_group_index overflows i16
Browse files Browse the repository at this point in the history
This caused confusing panics down the line because 'ordinal' is
negative.
  • Loading branch information
progval committed Sep 10, 2024
1 parent 704f90b commit 2d17183
Showing 1 changed file with 13 additions and 2 deletions.
15 changes: 13 additions & 2 deletions parquet/src/file/writer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -190,14 +190,25 @@ impl<W: Write + Send> SerializedFileWriter<W> {
/// Creates new row group from this file writer.
/// In case of IO error or Thrift error, returns `Err`.
///
/// There is no limit on a number of row groups in a file; however, row groups have
/// There can be at most 2^15 row groups in a file; and row groups have
/// to be written sequentially. Every time the next row group is requested, the
/// previous row group must be finalised and closed using `RowGroupWriter::close` method.
pub fn next_row_group(&mut self) -> Result<SerializedRowGroupWriter<'_, W>> {
self.assert_previous_writer_closed()?;
let ordinal = self.row_group_index;

self.row_group_index += 1;
let ordinal: i16 = ordinal.try_into().map_err(|_| {
ParquetError::General(format!(
"Parquet does not support more than {} row groups per file (currently: {})",
i16::MAX,
ordinal
))
})?;

self.row_group_index = self
.row_group_index
.checked_add(1)
.expect("SerializedFileWriter::row_group_index overflowed");

let bloom_filter_position = self.properties().bloom_filter_position();
let row_groups = &mut self.row_groups;
Expand Down

0 comments on commit 2d17183

Please sign in to comment.