Skip to content
This repository has been archived by the owner on Jun 30, 2023. It is now read-only.

Fields with same name and different type can cause issue for avro sink #42

Open
abhishekagarwal87 opened this issue Jan 13, 2020 · 1 comment

Comments

@abhishekagarwal87
Copy link

Assume a following simple protobuf schema

message NestedStruct_1 {
 optional int64 nestedField_1
}

message NestedStruct_2 {
 optional int64 nestedField_2
}

message Struct_1 {
 optional NestedStruct_1 nested = 1;
}

message Struct_2 {
 optional NestedStruct_2 nested = 1;
}

message Test {
   required Struct_1 struct1 = 1;
   required Struct_2 struct2 = 2;
}

Now, protobuf converter can read the data for Test.proto but if you try to write the this data in avro, sink, you will get an exception similar to follows
org.apache.avro.SchemaParseException: Can't redefine: nested

In the original schema, there are two fields with same name but different type. However, when building connect schema in ProtobufData, we lose the type scope and hence it results in conflicts in Avro sinks.

@abhishekagarwal87
Copy link
Author

If we can include the type name in the field, the conflict can be avoided. I can fix this if this solution sounds alright. Other suggestions are welcome as well.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

No branches or pull requests

1 participant