Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration to Object Orientation #39

Merged
merged 71 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
76a929d
Initial commit of object oriented version
dwest77a Jul 26, 2024
cfaeb05
Initial commit of core modules
dwest77a Jul 31, 2024
c750b74
Initial commit of operations, filehandlers and evaluate
dwest77a Jul 31, 2024
c53d437
Initial commit of object oriented pipeline elements
dwest77a Jul 31, 2024
aaf2f9d
Properly set padocc package name
dwest77a Jul 31, 2024
5a451d8
Removed old code, updated license to 2024
dwest77a Sep 4, 2024
4fd811d
Removed old functions, updated license
dwest77a Sep 4, 2024
eb944dd
Formalised imports from modules
dwest77a Sep 4, 2024
624ad26
Added logged operation, docstrings and typehints
dwest77a Sep 6, 2024
7c8c93d
Finalised init design for importing sections
dwest77a Sep 6, 2024
87374e7
Finalised refactoring for object orientation
dwest77a Sep 6, 2024
053190a
First version of operations
dwest77a Sep 6, 2024
c040924
Minor docstring changes
dwest77a Sep 6, 2024
b9f4a3d
Extended filehandler functionality
dwest77a Sep 6, 2024
ffdfb43
Extended operational functionality
dwest77a Sep 6, 2024
04ca12b
Upgraded scan to object orientation
dwest77a Sep 6, 2024
c07bca9
Removed scan config to config.py
dwest77a Sep 6, 2024
09af317
Added configuration for ease of use
dwest77a Sep 6, 2024
7e3f240
Updated with Object Oriented elements
dwest77a Sep 6, 2024
0af0290
Moved content to padocc_cli
dwest77a Sep 6, 2024
31b6b37
Moved allocate to operations
dwest77a Sep 6, 2024
3214678
Removed old scripts, placed into padocc_cli
dwest77a Sep 6, 2024
8582062
Finalised imports design
dwest77a Sep 6, 2024
c807c14
Removed separate config functions
dwest77a Sep 6, 2024
530b235
Added incomplete new scripts for OO conversion
dwest77a Sep 6, 2024
223f3a5
Initial commit of test data and scripts
dwest77a Sep 6, 2024
c079535
Added config options to a central class
dwest77a Sep 6, 2024
256e591
Removed old testargs
dwest77a Sep 6, 2024
0704043
Recommit all padocc scripts, removed Configuration in favour of Group…
dwest77a Oct 30, 2024
a25942e
Upgraded test suite for padocc tests
dwest77a Oct 30, 2024
6931a8c
Updated package managements.py
dwest77a Oct 30, 2024
b7289c0
Added group examples and user guide
dwest77a Oct 30, 2024
1fd4be6
Added lock file
dwest77a Oct 30, 2024
e7a783a
Update README.md with more helpful guidance.
dwest77a Oct 30, 2024
9f8e4db
Updated handling and documentation for mode parameter
dwest77a Oct 30, 2024
9b216ec
Updated CI workflows
dwest77a Oct 30, 2024
db2405c
Updated CI workflows
dwest77a Oct 30, 2024
bf2b590
Added general compute error
dwest77a Nov 4, 2024
b551710
Added substitutions for init file
dwest77a Nov 4, 2024
657381e
Moved tests
dwest77a Nov 4, 2024
18bd208
Added test files to include
dwest77a Nov 4, 2024
0a6f8ab
Added test files to include
dwest77a Nov 4, 2024
003d57a
Fixed usage of substitutions
dwest77a Nov 4, 2024
3b4d598
Added extra substitutions config
dwest77a Nov 4, 2024
af48b71
Various changes to several scripts
dwest77a Nov 11, 2024
33a0618
Added properties mixin for projects
dwest77a Nov 14, 2024
5e9763d
Stashed changes to validator
dwest77a Nov 14, 2024
92fdabc
Initial commit, various scripts
dwest77a Dec 4, 2024
c91d609
Initial commit of CLI
dwest77a Dec 4, 2024
9673689
Merge pull request #40 from cedadev/valid_0412
dwest77a Dec 4, 2024
11ce98c
First validation update commit
dwest77a Dec 10, 2024
03e49c6
Added new properties for all projects
dwest77a Dec 10, 2024
960294e
Added functionality for default contents in JSON filehandler
dwest77a Dec 19, 2024
e8e0433
Added properties mixin, reconfigured default and base configs
dwest77a Dec 19, 2024
17d733b
Reconfigured detail and base configs
dwest77a Dec 19, 2024
e0071e4
Updated Errors from mypy linting
dwest77a Dec 19, 2024
52321b4
Updated detail and base config usage
dwest77a Dec 19, 2024
c0e8f0b
Updated detail and base config usage
dwest77a Dec 19, 2024
cc65a27
Updated use of detail/base configs
dwest77a Dec 19, 2024
45e601b
Finalised validate workflow
dwest77a Dec 19, 2024
7548058
Added test constructions from this branch
dwest77a Dec 19, 2024
f852e6f
Removed dimension finder, replaced with cfa
dwest77a Dec 19, 2024
03eff79
Updated cloud format issue
dwest77a Dec 19, 2024
df3e265
Fixed issues with test data creation and files
dwest77a Dec 20, 2024
86bbb27
Fixed validator, multiple small changes across all modules
dwest77a Dec 20, 2024
ce9316f
Removed auto test directory
dwest77a Dec 20, 2024
e5b2663
Ignore auto test directory
dwest77a Dec 20, 2024
3ce561c
Fixes to allow validation
dwest77a Dec 20, 2024
cc579d1
Added catch for improperly-formatted tuples
dwest77a Dec 20, 2024
6a00135
Merge pull request #43 from cedadev/ValidationOO
dwest77a Dec 20, 2024
9b947b8
Updated Docs workflow for building
dwest77a Dec 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Removed dimension finder, replaced with cfa
  • Loading branch information
dwest77a committed Dec 19, 2024
commit f852e6f89786d3e91292b7fa7743c39c5cf67b58
15 changes: 13 additions & 2 deletions padocc/phases/compute.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
ComputeError
)

from padocc.phases.validate import validate_selection
from padocc.phases.validate import ValidateDatasets
from padocc.core.filehandlers import JSONFileHandler, ZarrStore

CONCAT_MSG = 'See individual files for more details'
Expand Down Expand Up @@ -427,10 +427,18 @@ def _find_concat_dims(self, ds_examples: list, logger=FalseLogger()) -> None:

This Class method is common to all conversion types.
"""

raise NotImplementedError

concat_dims = []
for dim in ds_examples[0].dims:
try:
validate_selection(ds_examples[0][dim], ds_examples[1][dim], dim, 128, 128, logger, bypass=self._bypass)

report = ValidateDatasets(
[ds_examples[0][dim], ds_examples[1][dim]],
['0th','1st']
).validate_data

except ValidationError:
self.logger.debug(f'Non-identical dimension: {dim} - if this dimension should be identical across the files, please inspect.')
concat_dims.append(dim)
Expand All @@ -452,6 +460,9 @@ def _find_identical_dims(self, ds_examples: list, logger=FalseLogger()) -> None:

This Class method is common to all conversion types.
"""

raise NotImplementedError

identical_dims = []
normal_dims = []
for var in ds_examples[0].variables:
Expand Down
10 changes: 5 additions & 5 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ requests = "2.32.3"
aiohttp = "3.10.10"
sphinx = "7.1.2"
sphinx-rtd-theme = "2.0.0"
cfapyx = "2024.10.11"
cfapyx = "2024.11.27"
myst-nb = "^1.1.2"


Expand Down
Loading