-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MINOR [Go] Add nested struct test for pqarrow #35851
MINOR [Go] Add nested struct test for pqarrow #35851
Conversation
Thanks for opening a pull request! If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project. Then could you also rename the pull request title in the following format?
or
In the case of PARQUET issues on JIRA the title also supports:
See also: |
@zeroshade I found an issue with the test & it should now pass, but I'm really confused about |
I guess the issue can be connected to https://github.com/apache/arrow/blob/main/go/parquet/pqarrow/encode_arrow_test.go#L1040-L1042
|
#172) Extracted from #171 Structs reading is slow because of apache/arrow#35851 Fields copy is because of apache/arrow#35867
I'm closing this for now, however, it would've been nice to be able to use |
…al` (#35872) ### Rationale for this change When comparing `array.Struct` values with `array.ApproxEqual` the validity bitmap of the struct itself should take precedence: > When reading the struct array the parent validity bitmap takes priority. This follows a brief discussion from #35851. ### What changes are included in this PR? `array.arrayApproxEqualStruct` will check the fields data only if the struct elem is valid. ### Are these changes tested? `pqarrow` tests are updated accordingly (no special treatment for structs, just `array.ApproxEqual` ### Are there any user-facing changes? `array.ApproxEqual` behavior changed to match the docs about validity bitmap precedence. * Closes: #35871 Authored-by: candiduslynx <candiduslynx@gmail.com> Signed-off-by: Matt Topol <zotthewizard@gmail.com>
@zeroshade I've found a disturbing behavior with structs (namely, nested ones) in pqarrow.
This blocks cloudquery/filetypes#172
Could you please take a look?