Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cosmic masking #258

Merged
merged 11 commits into from
Aug 16, 2016
Merged

Cosmic masking #258

merged 11 commits into from
Aug 16, 2016

Conversation

sbailey
Copy link
Contributor

@sbailey sbailey commented Jul 29, 2016

This PR depends upon changes in desihub/specter#40 to propagate the pixel-level model and goodness-of-fit metrics while doing extractions. Tests will fail until that is merged, but opening this PR to get eyes on it now.

This uses the PSF-weighted pixel-level chi2 to identify and flag questionable flux bins in the extractions. This reduces the rate of unmasked cosmics being mis-identified as H-alpha. Testing on oak-lite (2 spectrographs x 3 exposures) with new code vs. oak (10 spectrographs x 9 exposures) with desispec/0.8.0, this improved the ELG redshift efficiency from 90% to 94%, and LRGs from 97% to 99%. QSO efficiency went down, but the efficiency is so bad already that something else is going on there that still needs to be debugged.

Below shows an example of this successfully masking an incorrect extraction spike from an unmasked cosmic on oak r0 exposure 11 fiber 104 at wavelength=6984 Angstroms.

cosmic-r0-00000011-fiber104-wave6984

Other updates in this PR:

  • enables output of the 2D pixel model via desi_extract_spectra --model option (not enabled in the pipeline by default). Fixes Write CCD chi2 image to further mask cosmics #181.
  • fixes bug in pipeline call to desi_bootcalib (it doesn't have a --qafig option)
  • adds extraction tests, including tests that identified bugs in main_mpi() for extracting partial bundles. These are flagged as expected failures for now, but I'll open a separate issue to fix those after this is merged.

This has been tested on the oak-lite dataset, with the pipeline running end-to-end with no errors (when used with the specter/chi2 branch which will be merged before this).

@julienguy
Copy link
Contributor

  • may add clobber=True in py/desispec/scripts/extract.py line 165
  • code tested successfully (along with specter branch chi2) on one cosmic ray spike (oak1 frame r0-00000006 fiber 413 wave 6519A). with new code, cosmic shows very large CHI2PIX value in frame, and is adequately flagged in MASK.

@sbailey sbailey merged commit 99b95ee into master Aug 16, 2016
@sbailey
Copy link
Contributor Author

sbailey commented Aug 16, 2016

Added clobber option as suggested by @julienguy and updated .travis.yml to use specter/0.6.0 so that tests pass again. Manually merged due to conflicts in changes.rst.

@sbailey sbailey deleted the cosmic_masking branch August 16, 2016 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Write CCD chi2 image to further mask cosmics
3 participants