Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hats reimport pipeline #441

Open
2 of 3 tasks
hombit opened this issue Nov 25, 2024 · 0 comments
Open
2 of 3 tasks

Hats reimport pipeline #441

hombit opened this issue Nov 25, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@hombit
Copy link
Contributor

hombit commented Nov 25, 2024

Feature request

It would be useful to have a HATS reimport pipeline to simplify tasks such as re-partitioning HATS catalogs. This could be implemented as a thin wrapper around ImportArguments, which would handle file paths, the format (parquet), use/add_healpix_29, and default to using the same catalog name.

I had to write something like this to do that, and it took me a while to figure it out.

catalog = read_hats(input_catalog_path)

column_names = catalog.schema.names
column_names.remove('Dir')
column_names.remove('Norder')
column_names.remove('Npix')

args = ImportArguments(
    ra_column=catalog.catalog_info.ra_column,
    dec_column=catalog.catalog_info.dec_column,
    input_path=input_catalog_path / "dataset",
    file_reader=ParquetReader(column_names= column_names),
    output_artifact_name=catalog.catalog_name,
    output_path=output_dir,
    pixel_threshold=100_000,
    use_healpix_29=True,
    add_healpix_29=False,
)

Before submitting
Please check the following:

  • I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
  • I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
  • If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.
@hombit hombit added the enhancement New feature or request label Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: No status
Development

No branches or pull requests

1 participant