Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lintify BaseProcessor #234

Merged
merged 12 commits into from
Dec 8, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .github/workflows/continous_integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,18 @@ jobs:
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2

- uses: actions/checkout@v3

- name: Install poetry
run: pipx install poetry

# Set up the python versions
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: "poetry"

# Runs a set of commands using the runners shell
- name: set up ssh
Expand Down
29 changes: 16 additions & 13 deletions .github/workflows/pylint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,19 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- name: Set up Python 3.10
uses: actions/setup-python@v1
with:
python-version: "3.10"
- name: Install dependencies
run: |
pip install --upgrade pip
python -m pip install --upgrade poetry
poetry install
- name: Analysing the code with pylint
run: |
poetry run pylint winterdrp
- uses: actions/checkout@v3
- name: Install poetry
run: pipx install poetry
- name: Set up Python 3.10
uses: actions/setup-python@v4
with:
python-version: "3.10"
cache: "poetry"
- name: Install dependencies
run: |
pip install --upgrade pip
python -m pip install --upgrade poetry
poetry install
- name: Analysing the code with pylint
run: |
poetry run pylint winterdrp
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ repos:
types: [python]
args:
[
"--disable=C",
"--disable=C,fixme",
"-rn", # Only display messages
"-sn", # Don't display the score
]
Expand Down
67 changes: 43 additions & 24 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion winterdrp/data/image_data.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""

Module to specify the input data classes for :class:`winterdrp.processors.base_processor.ImageHandler`
Module to specify the input data classes for
:class:`winterdrp.processors.base_processor.ImageHandler`

"""
import logging
Expand Down
5 changes: 4 additions & 1 deletion winterdrp/errors/error_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import logging
import traceback
from datetime import datetime
from pathlib import Path

from winterdrp.errors.exceptions import BaseProcessorError, NoncriticalProcessingError

Expand All @@ -18,7 +19,9 @@ class ErrorReport:
Class representing a single error raised during processing
"""

def __init__(self, error, processor_name, contents: list[str]):
def __init__(
self, error: Exception, processor_name: str, contents: list[str] | list[Path]
):
self.error = error
self.processor_name = processor_name
self.contents = contents
Expand Down
20 changes: 14 additions & 6 deletions winterdrp/pipelines/summer/blocks.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
"""
Script containing the various :class:`~winterdrp.processors.base_processor.BaseProcessor`
Script containing the various
:class:`~winterdrp.processors.base_processor.BaseProcessor`
lists which are used to build configurations for the
:class:`~winterdrp.pipelines.summer.summer_pipeline.SummerPipeline`.
"""
from winterdrp.downloader.get_test_data import get_test_data_dir
from winterdrp.paths import base_name_key, core_fields
from winterdrp.pipelines.summer.config import (
from winterdrp.pipelines.summer.config import ( # summer_weight_path,
DB_NAME,
PIPELINE_NAME,
SUMMER_PIXEL_SCALE,
Expand All @@ -17,7 +18,6 @@
sextractor_photometry_config,
summer_cal_requirements,
summer_mask_path,
summer_weight_path,
swarp_config_path,
)
from winterdrp.pipelines.summer.config.schema import summer_schema_dir
Expand Down Expand Up @@ -150,7 +150,8 @@
ImageSaver(output_dir_name="detrend", write_mask=True),
Sextractor(
output_sub_dir="sextractor",
weight_image=summer_weight_path,
# TODO: work out why this was ever here...
# weight_image=summer_weight_path,
checkimage_name=None,
checkimage_type=None,
**sextractor_astrometry_config
Expand All @@ -159,7 +160,11 @@
ref_catalog_generator=summer_astrometric_catalog_generator,
scamp_config_path=scamp_path,
),
Swarp(swarp_config_path=swarp_config_path, imgpixsize=2400),
Swarp(
swarp_config_path=swarp_config_path,
# TODO: work out why this was ever here...
# imgpixsize=2400
),
ImageSaver(output_dir_name="processed", write_mask=True),
Sextractor(
output_sub_dir="photprocess",
Expand All @@ -168,7 +173,10 @@
),
PhotCalibrator(ref_catalog_generator=summer_photometric_catalog_generator),
ImageSaver(
output_dir_name="processed", additional_headers=["PROCIMG"], write_mask=True
output_dir_name="processed",
# TODO: work out why this was ever here...
# additional_headers=["PROCIMG"],
write_mask=True,
),
HeaderEditor(edit_keys="procflag", values=1),
DatabaseImageExporter(
Expand Down
Loading