Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR depends upon changes in desihub/specter#40 to propagate the pixel-level model and goodness-of-fit metrics while doing extractions. Tests will fail until that is merged, but opening this PR to get eyes on it now.
This uses the PSF-weighted pixel-level chi2 to identify and flag questionable flux bins in the extractions. This reduces the rate of unmasked cosmics being mis-identified as H-alpha. Testing on oak-lite (2 spectrographs x 3 exposures) with new code vs. oak (10 spectrographs x 9 exposures) with desispec/0.8.0, this improved the ELG redshift efficiency from 90% to 94%, and LRGs from 97% to 99%. QSO efficiency went down, but the efficiency is so bad already that something else is going on there that still needs to be debugged.
Below shows an example of this successfully masking an incorrect extraction spike from an unmasked cosmic on oak r0 exposure 11 fiber 104 at wavelength=6984 Angstroms.
Other updates in this PR:
desi_extract_spectra --model
option (not enabled in the pipeline by default). Fixes Write CCD chi2 image to further mask cosmics #181.main_mpi()
for extracting partial bundles. These are flagged as expected failures for now, but I'll open a separate issue to fix those after this is merged.This has been tested on the oak-lite dataset, with the pipeline running end-to-end with no errors (when used with the specter/chi2 branch which will be merged before this).