Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store arrow schema when reading catalogs #310

Merged
merged 4 commits into from
Jul 25, 2024

Conversation

camposandro
Copy link
Contributor

When instantiating a catalog, extract the arrow schema from the metadata files and store it.

  • My PR includes a link to the issue that I am addressing

Code Quality

  • I have read the Contribution Guide
  • My code follows the code style of this project
  • My code builds (or compiles) cleanly without any errors or warnings
  • My code contains relevant comments and necessary documentation

@camposandro camposandro self-assigned this Jul 22, 2024
Copy link

codecov bot commented Jul 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.84%. Comparing base (047600e) to head (6977e02).
Report is 37 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #310      +/-   ##
==========================================
+ Coverage   93.78%   93.84%   +0.06%     
==========================================
  Files          58       58              
  Lines        2028     2048      +20     
==========================================
+ Hits         1902     1922      +20     
  Misses        126      126              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Jul 22, 2024

Before [047600e] <v0.3.7> After [6ce811e] Ratio Benchmark (Parameter)
41.7±0.3ms 42.6±0.8ms 1.02 benchmarks.Suite.time_pixel_tree_creation
13.5±0.2ms 13.6±0.2ms 1.01 benchmarks.Suite.time_inner_pixel_alignment
384±2ms 383±4ms 1 benchmarks.Suite.time_outer_pixel_alignment
119±0.7ms 119±0.5ms 1 benchmarks.time_test_alignment_even_sky
20.9±0.5ms 20.8±0.4ms 0.99 benchmarks.MetadataSuite.time_load_partition_info_order6
78.2±2ms 77.7±0.4ms 0.99 benchmarks.MetadataSuite.time_load_partition_join_info
980±3μs 974±20μs 0.99 benchmarks.time_test_cone_filter_multiple_order
79.2±1ms 77.0±0.3ms 0.97 benchmarks.MetadataSuite.time_load_partition_info_order7
91.3±2ms 86.9±2ms 0.95 benchmarks.Suite.time_paths_creation

Click here to view all benchmarks.

@camposandro camposandro marked this pull request as ready for review July 24, 2024 17:59
Copy link
Contributor

@delucchi-cmu delucchi-cmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add this for index catalogs as well? Does it just not matter? They don't inherit from HealpixDataset, just that superclass Dataset.

@camposandro camposandro merged commit 2c625ce into main Jul 25, 2024
12 checks passed
@camposandro camposandro deleted the add-arrow-schema-to-catalog branch July 25, 2024 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants