Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot read E57 file #12

Closed
cry-inc opened this issue Oct 7, 2024 · 2 comments
Closed

Cannot read E57 file #12

cry-inc opened this issue Oct 7, 2024 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@cry-inc
Copy link
Owner

cry-inc commented Oct 7, 2024

This is the C++ code used to create a reproduction case using the libe57format library (latest version 3.2.0):

e57::WriterOptions options;
e57::Writer writer("file.e57", options);
e57::Data3D header;
header.pointCount = 2651;
header.pointFields.colorRedField = true;
header.pointFields.colorGreenField = true;
header.pointFields.colorBlueField = true;
header.pointFields.sphericalAzimuthField = true;
header.pointFields.sphericalElevationField = true;
header.pointFields.sphericalRangeField = true;
header.pointFields.sphericalInvalidStateField = true;
header.pointFields.intensityField = true;
header.pointFields.rowIndexField = true;
header.pointFields.columnIndexField = true;
header.colorLimits.colorRedMaximum = 255;
header.colorLimits.colorGreenMaximum = 255;
header.colorLimits.colorBlueMaximum = 255;
header.intensityLimits.intensityMinimum = 0;
header.intensityLimits.intensityMaximum = 0;
header.pointFields.pointRangeNodeType = e57::NumericalNodeType::Float;
header.pointFields.angleNodeType = e57::NumericalNodeType::Float;
header.pointFields.intensityNodeType = e57::NumericalNodeType::Integer;
header.pointFields.columnIndexMaximum = 5742 - 1;
header.pointFields.rowIndexMaximum = 8534 - 1;
e57::Data3DPointsFloat buffer(header);
for (size_t i = 0; i < header.pointCount; i++) {
	buffer.sphericalAzimuth[i] = 0;
	buffer.sphericalElevation[i] = 0;
	buffer.sphericalRange[i] = 0;
	buffer.sphericalInvalidState[i] = 0;
	buffer.colorRed[i] = 0;
	buffer.colorGreen[i] = 0;
	buffer.colorBlue[i] = 0;
	buffer.intensity[i] = 0;
	buffer.rowIndex[i] = 0;
	buffer.columnIndex[i] = 0;
}
writer.WriteData3DData(header, buffer);
writer.Close();

It fails with the error "Invalid E57 content: Found unknown packet ID when trying to read packet header" when trying to read the very last point in the file.

CloudCompare and other applications can read this file without errors. The libe57format library can be also used to read all points from that file without issues.

@cry-inc cry-inc self-assigned this Oct 7, 2024
@cry-inc
Copy link
Owner Author

cry-inc commented Oct 7, 2024

I tried to reproduce a file with the same content using the Rust library itself. It fails with the same error, but I am not sure if its really the same issue. Here is the Rust test case:

#[test]
fn repro() {
    let file = "repro.e57";
    {
        let mut writer = e57::E57Writer::from_file(file, "file_uuid").unwrap();
        let proto = vec![
            Record {
                name: RecordName::SphericalAzimuth,
                data_type: RecordDataType::Single {
                    min: None,
                    max: None,
                },
            },
            Record {
                name: RecordName::SphericalElevation,
                data_type: RecordDataType::Single {
                    min: None,
                    max: None,
                },
            },
            Record {
                name: RecordName::SphericalRange,
                data_type: RecordDataType::Single {
                    min: None,
                    max: None,
                },
            },
            Record {
                name: RecordName::SphericalInvalidState,
                data_type: RecordDataType::Integer { min: 0, max: 2 },
            },
            Record::COLOR_RED_U8,
            Record::COLOR_GREEN_U8,
            Record::COLOR_BLUE_U8,
            Record {
                name: RecordName::Intensity,
                data_type: RecordDataType::Integer { min: 0, max: 0 },
            },
            Record {
                name: RecordName::ColumnIndex,
                data_type: RecordDataType::Integer {
                    min: 0,
                    max: 5742 - 1,
                },
            },
            Record {
                name: RecordName::RowIndex,
                data_type: RecordDataType::Integer {
                    min: 0,
                    max: 8534 - 1,
                },
            },
        ];
        let mut pc_writer = writer.add_pointcloud("pc_guid", proto).unwrap();
        for _ in 0..2651 {
            let point = vec![
                RecordValue::Single(0.0), // azimuth
                RecordValue::Single(0.0), // elevation
                RecordValue::Single(0.0), // range
                RecordValue::Integer(0),  // invalid state
                RecordValue::Integer(0),  // red
                RecordValue::Integer(0),  // green
                RecordValue::Integer(0),  // blue
                RecordValue::Integer(0),  // intensity
                RecordValue::Integer(0),  // column index
                RecordValue::Integer(0),  // row index
            ];
            pc_writer.add_point(point).unwrap();
        }
        pc_writer.finalize().unwrap();
        writer.finalize().unwrap();
    }
    {
        let mut reader = e57::E57Reader::from_file(file).unwrap();
        let pcs = reader.pointclouds();
        for pc in pcs {
            let iter = reader.pointcloud_raw(&pc).unwrap();
            for (i, point) in iter.enumerate() {
                point.expect(&format!("unable to read next point {i}"));
            }
        }
    }
    std::fs::remove_file(file).unwrap();
}

cry-inc added a commit that referenced this issue Oct 7, 2024
cry-inc added a commit that referenced this issue Oct 7, 2024
cry-inc added a commit that referenced this issue Oct 7, 2024
…nerate for elements with a zero bit size. The old broken code only considered number of bits read from the file while ignoring the bits that were already in the byte stream buffer.
@cry-inc
Copy link
Owner Author

cry-inc commented Oct 7, 2024

Both code snippets above triggered the same bug in the reader.
The bug was fixed in commit bb78a6b.
The fix is included in the new version 0.11.6 on crates.io.

@cry-inc cry-inc closed this as completed Oct 7, 2024
@cry-inc cry-inc added the bug Something isn't working label Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant