v0.6.0

New v0.6.0 release. One big change is the transition to using a directory as a cache for image data, rather than storing everything in RAM. This should ensure processing is much more scalable e.g on a laptop. Another big change is that processors can run in parallel by default, speeding up back processing. Finally, this release includes enforced coding style, usint black, isort and pylint, to ensure consistency across the project. See the full log for more minor chages.

What's Changed

• Bump to v0.5.0 by @robertdstein in #207
• Remove sextractor redundancy in reference by @virajkaram in #209
• Bump pandas from 1.5.1 to 1.5.2 by @dependabot in #210
• Bump setuptools from 65.5.1 to 65.6.3 by @dependabot in #211
• Download summer images from winter machine by @robertdstein in #212
• Carefully check/enforcing typing in data objects by @robertdstein in #213
• Bump to v0.5.1 by @robertdstein in #214
• Enforce Headers for Image objects by @robertdstein in #216
• Transfer metadata to sourcelist from parent image by @robertdstein in #217
• Unify error reports and propagate processing history to sourcelists by @robertdstein in #218
• Every Processor is a Parallel Processor by @robertdstein in #220
• Update docs with contributing guide by @robertdstein in #225
• Code with Style by @robertdstein in #224
• Lint and document base data structure by @robertdstein in #226
• Write ErrorLog for main.py by @robertdstein in #227
• Rename Gihtub Action jobs by @robertdstein in #228
• Fix main date format, and add try/except by @robertdstein in #229
• Fix CalHunter loading function by @robertdstein in #230
• Linitify errors by @robertdstein in #232
• Lintify BaseProcessor by @robertdstein in #234
• Bump photutils from 1.5.0 to 1.6.0 by @dependabot in #236
• Lintify CSVlog/Fritz by @robertdstein in #237
• Close #197, use cache to massively reduce RAM by @robertdstein in #231

Full Changelog: v0.5.0...v0.6.0

v0.5.1

This is the v0.5.1 release. The major change with the previous version is a bug fix for CalHunter involving returning nested Image batches. This release also contains much more extensive data type checks, to prevent similar mistakes happening in future.

What's Changed

• Bump to v0.5.0 by @robertdstein in #207
• Remove sextractor redundancy in reference by @virajkaram in #209
• Bump pandas from 1.5.1 to 1.5.2 by @dependabot in #210
• Bump setuptools from 65.5.1 to 65.6.3 by @dependabot in #211
• Download summer images from winter machine by @robertdstein in #212
• Carefully check/enforcing typing in data objects by @robertdstein in #213

Full Changelog: v0.5.0...v0.5.1

v0.5.0 Release

The release of v0.5.0. The big (breaking) change here is the introduction of DataBlock objects for processors to work on (see #196 ), replacing the old numpy arrays/headers/dataframe style.

This release also includes documentation for the first time: https://winterdrp.readthedocs.io/en/latest/, and expanded tests.

What's Changed

• Bump to v0.4.3 by @robertdstein in #180
• Better test coverage by @robertdstein in #181
• Create LICENSE by @robertdstein in #183
• Test errors and monitor by @robertdstein in #184
• Pipeline Visualisation by @robertdstein in #187
• Add test for pipeline visualisation by @robertdstein in #188
• Remove unnecessary files by @virajkaram in #186
• remove psfex.xml in summer pipe by @virajkaram in #189
• Bump docker from 6.0.0 to 6.0.1 by @dependabot in #190
• Bump matplotlib from 3.6.1 to 3.6.2 by @dependabot in #192
• Bump setuptools from 65.5.0 to 65.5.1 by @dependabot in #191
• Reorganise error code by @robertdstein in #199
• Update testdata on request by @robertdstein in #200
• Bump numpy from 1.23.4 to 1.23.5 by @dependabot in #201
• Add Docs by @robertdstein in #203
• Document install, contributing, usage by @robertdstein in #204
• Data Objects by @robertdstein in #196

Full Changelog: v0.4.3...v0.5.0

v0.4.3 Release

New v0.4.3 release. Beyond general bugfixes, the main change is a transition to poetry for handling installation/dependency management. There is now a pyproject.toml, and no setup.py or requirements.txt. Automated vpackage updates have now been configured using dependabot. There are several other minor improvements detailed below.

What's Changed

• Bump v0.4.2 release by @robertdstein in #151
• Compress large files for email, run postprocess on one thread only by @robertdstein in #154
• Add timestamp to database by @robertdstein in #155
• Avrofields by @virajkaram in #158
• Save detrended and resampled images by @virajkaram in #162
• Add Skymapper catalog for u-band photcal by @virajkaram in #163
• Bump to sdss 16 by @robertdstein in #165
• Move to Poetry/toml by @robertdstein in #168
• Add dependabot by @robertdstein in #169
• Refactor Vizier Catalog, Check SDSS by @robertdstein in #166
• Add automerge by @robertdstein in #171
• Dehardcode and unify wirc by @robertdstein in #172
• Use toml for metadata by @robertdstein in #174
• Delete old files by @robertdstein in #176
• Close images properly in pyzogy, fix #112 by @robertdstein in #177
• Close images after opening them in psfex by @robertdstein in #179
• Non-deprecated slicing in zogy by @robertdstein in #178
• Bump numpy from 1.22.4 to 1.23.4 by @dependabot in #170
• RawPath to reflect stacking by @robertdstein in #160

New Contributors

• @dependabot made their first contribution in #170

Full Changelog: v0.4.2...v0.4.3

v0.4.2 Release

What's Changed

• Bump to v0.4.1 by @robertdstein in #118
• Aggregate errors in non-verbose summary by @robertdstein in #121
• Create multiple 'PhotometryError's by @robertdstein in #123
• Stop printing the astroquery/ldac tables by @robertdstein in #124
• Dbops by @virajkaram in #125
• Divide the various astrometry errors by @robertdstein in #126
• Force pipelines to have names by @robertdstein in #128
• Group errors by raising line by @robertdstein in #127
• Count affected images in error summary by @robertdstein in #129
• Require PROCFAIL, RAWPATH and BASENAME keys in headers by @robertdstein in #130
• Default pipeline configuration is now 'default' not None by @robertdstein in #131
• Postprocess Step by @robertdstein in #132
• Check file exists before email attaching by @robertdstein in #135
• Make midway processing less email-specific by @robertdstein in #138
• Convert obstype to string, handle corrupted headers by @robertdstein in #139
• Add Try/except and error report logging for monitor errors by @robertdstein in #140
• Also catch attributerror when loading summer images by @robertdstein in #141
• Add log configuration for summer by @robertdstein in #142
• Delete redundant code by @robertdstein in #143
• Multithreading, close #122 by @robertdstein in #145
• Add default of 300s for timeout on shell by @robertdstein in #147
• Reduce log spam by @robertdstein in #149
• Include software version in logs by @robertdstein in #150

Full Changelog: v0.4.1...v0.4.2

v0.4.1 Release

New v0.4.1 release. The big change is the introduction of the Monitor class, which can take a pipeline and watch for new images/reduce those images in realtime. It can be activated most easily with python -m winterdrp --monitor -p pipelinenamegoeshere.

What's Changed

• bump version by @robertdstein in #102
• track processor errors by @virajkaram in #107
• Automatically pull latest test data then select correct tag by @robertdstein in #108
• Dualsextractorcheckimg by @virajkaram in #114
• Summer by @robertdstein in #115
• Set up Monitor by @robertdstein in #71

Full Changelog: v0.4.0...v0.4.1

v0.4.0 release

What's Changed

• v0.3.0 release by @robertdstein in #63
• Add Summer CI by @robertdstein in #64
• Coveralls: track code coverage by @robertdstein in #65
• v0.3.1 release by @robertdstein in #66
• Multiconfigs by @robertdstein in #68
• Fix #69 by @robertdstein in #70
• Fix Candidatestable by @virajkaram in #74
• wirc_imsub on multiple images by @virajkaram in #75
• Cutoutnames by @virajkaram in #76
• Names2objectid by @virajkaram in #77
• Remove old schema files by @robertdstein in #80
• Case sensitive columns by @virajkaram in #82
• Fritzscan by @sukishore12 in #84
• Photometryduplicates by @virajkaram in #86
• Delete cache/egg info by @sukishore12 in #87
• add diffmaglim to dataframe by @virajkaram in #91
• update numpy requirement to specific version, else zogy breaks by @virajkaram in #93
• add isdiffpos field by @virajkaram in #94
• Avrofields by @sukishore12 in #95
• Summer imsub by @virajkaram in #96
• remove image selectors to run on all data by @virajkaram in #97
• Summer imsub by @virajkaram in #98
• update candidate schema by @sukishore12 in #99
• Winterserver by @virajkaram in #100

Full Changelog: v0.3.0...v0.4.0

v0.3.1 Release

v0.3.1 Release. Biggest change from v0.3.0 is the introduction of unit tests for summer, and then the implementation of coveralls to track code coverage.

What's Changed

• v0.3.0 release by @robertdstein in #63
• Add Summer CI by @robertdstein in #64

Full Changelog: v0.3.0...v0.3.1

v0.3.0 Release

v0.3.0 Release contains many changes. The biggest milestone is the completion of CI set up. Each PR will now run unit tests to ensure code remains functional.

What's Changed

• Alphabetise requirements, de-fix versions by @robertdstein in #40
• Exampleenv by @sukishore12 in #43
• Add 'prefix' option for more flexible ssh by @robertdstein in #44
• Sort path list in ImageLoader by @robertdstein in #46
• Add Error Handling by @robertdstein in #47
• Add ErrorStack by @robertdstein in #48
• Add functionality for tracking internal python exceptions by @robertdstein in #50
• Update test to check errorstack by @robertdstein in #51
• Enforce typing on ImageSelector by @robertdstein in #52
• Add Prebuiltcache Processor by @robertdstein in #53
• Add image rejector by @robertdstein in #57
• Add util function for sending email via gmail by @robertdstein in #59
• Candidatestable by @virajkaram in #60
• Updated Summer Pipeline by @robertdstein in #12
• Set SSH_USER env option by @robertdstein in #62
• Wirctest by @robertdstein in #54

Full Changelog: v0.2.0...v0.3.0

v0.2.0 Release

v0.2.0 Release. The major change is the inclusion of image subtraction.

What's Changed

• Optional RAW_IMG_DIR and OUTPUT_IMG_DIR by @robertdstein in #9
• Add unit test for summer pipeline by @robertdstein in #10
• HeaderAnnotate by @robertdstein in #11
• Processor reorganisation by @robertdstein in #13
• Alerts2 by @sukishore12 in #24
• Update to add DataBase importers by @robertdstein in #25
• More tolerant approach to passwords by @robertdstein in #26
• More tolerant approach to passwords (V2) by @robertdstein in #28
• Raise error if imageloader finds nothing by @robertdstein in #30
• Check for kowalski creds only if using it by @virajkaram in #31
• Moretolerantapproachtopasswords by @sukishore12 in #33
• Image Subtraction ("Imsub") by @robertdstein in #39

Full Changelog: v0.1.0...v0.2.0