Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pre-commit check compliant for github action #793

Merged
merged 58 commits into from
Nov 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
788f711
update pre-commit setting
lee1043 Nov 3, 2021
f038f99
revised files to be pre-commit tests compliant
lee1043 Nov 3, 2021
3fe5901
make setup.cfg to be consistant to that of xcdat
lee1043 Nov 3, 2021
df20443
exclude .nc files from pre-commit
lee1043 Nov 3, 2021
59c1845
add black and use exclude as global instead for individual hooks
lee1043 Nov 3, 2021
c3881f8
add configure for black
lee1043 Nov 4, 2021
40a7de6
revisions from local pre-commit run
lee1043 Nov 4, 2021
915d521
revisions from local pre-commit, in particular from black
lee1043 Nov 4, 2021
7ca80ff
manual fix for py2-style print function
lee1043 Nov 4, 2021
b0c4b61
black and isort revisions
lee1043 Nov 4, 2021
d74d885
clean up
lee1043 Nov 4, 2021
738d93e
revision from isort
lee1043 Nov 4, 2021
d5d88ee
clean up
lee1043 Nov 4, 2021
03f534a
clean up to pass black and isort
lee1043 Nov 4, 2021
79422e5
clean up
lee1043 Nov 4, 2021
c4f59a5
clean up -- flake8
lee1043 Nov 4, 2021
f298b19
clean up --flake8
lee1043 Nov 4, 2021
9ed6992
clean up -- flake8
lee1043 Nov 4, 2021
8e87b88
clean up -- flake8
lee1043 Nov 4, 2021
63cbef1
Add `test_placeholder.py` to workaround error code 5
tomvothecoder Nov 4, 2021
679c91c
Remove Upload Coverage Report job
tomvothecoder Nov 4, 2021
3ed0eb0
Merge pull request #794 from tomvothecoder/787_ljw_pre-commit-hook-co…
lee1043 Nov 4, 2021
ca8851c
clean up and re-org
lee1043 Nov 4, 2021
c7b0a35
remove vcs from monsoon_sperber
lee1043 Nov 5, 2021
72fe5d3
clean up
lee1043 Nov 5, 2021
fae1ff9
remove vcs test for mjo debug plot
lee1043 Nov 5, 2021
1201b78
work in progress for MJO debug plotting
lee1043 Nov 5, 2021
bb34706
pre-commit tests
lee1043 Nov 5, 2021
dabb68f
vcs removed from mjo debug check plot
lee1043 Nov 6, 2021
5856899
add reminder for later fix for those lower priority plot
lee1043 Nov 6, 2021
bb75e7d
pre-commit compliant
lee1043 Nov 6, 2021
2ff09c8
replace vcs by cartopy for mov plots
lee1043 Nov 7, 2021
5a58b38
check plotting for PDO and NPGO (SST based mode)
lee1043 Nov 8, 2021
0e7bb9c
land mask for sst mode plot
lee1043 Nov 8, 2021
e04b0d2
Merge pull request #797 from PCMDI/785_ljw_vcs_branched_from_787
lee1043 Nov 8, 2021
64c117e
clean up
lee1043 Nov 8, 2021
f5d639e
Merge branch 'master' into 787_ljw_pre-commit-hook-compliant
lee1043 Nov 8, 2021
83528c3
Merge branch 'master' into 787_ljw_pre-commit-hook-compliant
lee1043 Nov 8, 2021
afd4878
cdscan part removed because no xml generation needed for files under …
lee1043 Nov 11, 2021
e21521d
Demo 0, 1, 1a to work with obs4mips demo data
lee1043 Nov 11, 2021
cd6d81a
mjo to work with obs4mips demo data
lee1043 Nov 11, 2021
6dfcadc
clean up
lee1043 Nov 11, 2021
353c4b0
clean up to work with obs4mips demo data
lee1043 Nov 11, 2021
cc734c0
Merge pull request #802 from PCMDI/798_ljw_demo1_fix_from_787_ljw_pre…
lee1043 Nov 11, 2021
bdd1f18
Merge branch 'master' into 787_ljw_pre-commit-hook-compliant
lee1043 Nov 11, 2021
7e4a726
plot option turned on as vcs removal completed
lee1043 Nov 11, 2021
51aae77
pre-commit clean up
lee1043 Nov 11, 2021
7f89ae9
typo fix
lee1043 Nov 13, 2021
af6917a
add new feature: split violin plot for 2 groups
lee1043 Nov 13, 2021
f67b456
clean up
lee1043 Nov 13, 2021
1541e23
clean up
lee1043 Nov 13, 2021
27cec5d
clean up, avoid isort for __init__.py in graphics directory
lee1043 Nov 13, 2021
b062227
add seaborn in the pmp env
lee1043 Nov 13, 2021
846b0ab
Merge pull request #803 from PCMDI/801_ljw_violin_plot_from_787_ljw_p…
lee1043 Nov 13, 2021
76cf148
reolve conflict
lee1043 Nov 16, 2021
eb14b4e
reslove conflict
lee1043 Nov 16, 2021
b9dc53c
resolve conflicts
lee1043 Nov 16, 2021
fc30edf
pre-commit clean up: Trim Trailing Whitespace
lee1043 Nov 16, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 62 additions & 68 deletions .github/workflows/build_workflow.yml
Original file line number Diff line number Diff line change
@@ -1,82 +1,76 @@
name: CI/CD Build Workflow

on:
push:
branches: [master]
push:
branches: [master]

pull_request:
branches: [master]
pull_request:
branches: [master]

workflow_dispatch:
workflow_dispatch:

jobs:
check-jobs-to-skip:
runs-on: ubuntu-latest
outputs:
should_skip: ${{ steps.skip_check.outputs.should_skip }}
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@master
with:
cancel_others: true
paths_ignore: '["**/README.rst", "**/docs/**", "**/ISSUE_TEMPLATE/**", "**/pull_request_template.md", "**/.vscode/**"]'
check-jobs-to-skip:
runs-on: ubuntu-latest
outputs:
should_skip: ${{ steps.skip_check.outputs.should_skip }}
steps:
- id: skip_check
uses: fkirc/skip-duplicate-actions@master
with:
cancel_others: true
paths_ignore: '["**/README.md", "**/docs/**", "**/ISSUE_TEMPLATE/**", "**/pull_request_template.md", "**/.vscode/**"]'

pre-commit-hooks:
needs: check-jobs-to-skip
if: ${{ needs.check-jobs-to-skip.outputs.should_skip != 'true'}} || ${{ github.event_name == 'push' }}
runs-on: ubuntu-latest
timeout-minutes: 2
steps:
- name: Checkout Code Repository
uses: actions/checkout@v2
pre-commit-hooks:
needs: check-jobs-to-skip
if: ${{ needs.check-jobs-to-skip.outputs.should_skip != 'true'}} || ${{ github.event_name == 'push' }}
runs-on: ubuntu-latest
timeout-minutes: 2
steps:
- name: Checkout Code Repository
uses: actions/checkout@v2

- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9

# Run all pre-commit hooks on all the files.
# Getting only staged files can be tricky in case a new PR is opened
# since the action is run on a branch in detached head state
- name: Install and Run Pre-commit
uses: pre-commit/action@v2.0.3
# Run all pre-commit hooks on all the files.
# Getting only staged files can be tricky in case a new PR is opened
# since the action is run on a branch in detached head state
- name: Install and Run Pre-commit
uses: pre-commit/action@v2.0.3

build:
needs: check-jobs-to-skip
if: ${{ needs.check-jobs-to-skip.outputs.should_skip != 'true'}} || ${{ github.event_name == 'push' }}
runs-on: ubuntu-latest
defaults:
run:
shell: bash -l {0}
timeout-minutes: 5
steps:
- uses: actions/checkout@v2
build:
needs: check-jobs-to-skip
if: ${{ needs.check-jobs-to-skip.outputs.should_skip != 'true'}} || ${{ github.event_name == 'push' }}
runs-on: ubuntu-latest
defaults:
run:
shell: bash -l {0}
timeout-minutes: 5
steps:
- uses: actions/checkout@v2

- name: Cache Conda
uses: actions/cache@v2
env:
# Increase this value to reset cache if conda/dev.yml has not changed in the workflow
CACHE_NUMBER: 0
with:
path: ~/conda_pkgs_dir
key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-publish
- name: Cache Conda
uses: actions/cache@v2
env:
# Increase this value to reset cache if conda/dev.yml has not changed in the workflow
CACHE_NUMBER: 0
with:
path: ~/conda_pkgs_dir
key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-publish

- name: Set up Conda Environment
uses: conda-incubator/setup-miniconda@v2
with:
activate-environment: "xcdat_dev"
environment-file: conda-env/dev.yml
channel-priority: strict
auto-update-conda: true
# IMPORTANT: This needs to be set for caching to work properly!
use-only-tar-bz2: true
- name: Set up Conda Environment
uses: conda-incubator/setup-miniconda@v2
with:
activate-environment: "pcmdi_metrics_dev"
environment-file: conda-env/dev.yml
channel-priority: strict
auto-update-conda: true
# IMPORTANT: This needs to be set for caching to work properly!
use-only-tar-bz2: true

- name: Run Tests
run: |
pytest
- name: Upload Coverage Report
uses: codecov/codecov-action@v1
with:
file: "tests_coverage_reports/coverage.xml"
fail_ci_if_error: true
- name: Run Tests
run: |
pytest
45 changes: 24 additions & 21 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,27 +1,30 @@
exclude: "docs|node_modules|migrations|.git|.tox"
exclude: "docs|node_modules|migrations|.git|.tox|sample_setups/(external-setups|jsons)|tests/deprecated"
default_stages: [commit]
fail_fast: true

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.4.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.4.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml

# Need to use flake8 GitHub mirror due to CentOS git issue with GitLab
# https://github.com/pre-commit/pre-commit/issues/1206
- repo: https://github.com/pycqa/flake8
rev: 3.9.2
hooks:
- id: flake8
args: ["--config=setup.cfg"]
additional_dependencies: [flake8-isort]
exclude: ^sample_setups/jsons/
- repo: https://github.com/psf/black
rev: 21.9b0
hooks:
- id: black

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.910
hooks:
- id: mypy
args: ["--config=setup.cfg"]
- repo: https://github.com/timothycrosley/isort
rev: 5.9.3
hooks:
- id: isort

# Need to use flake8 GitHub mirror due to CentOS git issue with GitLab
# https://github.com/pre-commit/pre-commit/issues/1206
- repo: https://github.com/pycqa/flake8
rev: 3.9.2
hooks:
- id: flake8
args: ["--config=setup.cfg"]
additional_dependencies: [flake8-isort]
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
BSD 3-Clause License

Copyright (c) 2021, Lawrence Livermore National Security, LLC
Copyright (c) 2021, Lawrence Livermore National Security, LLC
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<img src="share/pcmdi/PMPLogo_1359x1146px_300dpi.png" width="15%" height="15%" align="right" />
<h5 align="right"></h5>
<h5 align="right"></h5>

PCMDI Metrics Package (PMP)
======
Expand All @@ -11,9 +11,9 @@ PCMDI Metrics Package (PMP)
<!--[![CircleCI](https://circleci.com/gh/PCMDI/pcmdi_metrics.svg?style=svg)](https://circleci.com/gh/PCMDI/pcmdi_metrics) -->
<!--[![Coverage Status](https://coveralls.io/repos/github/PCMDI/pcmdi_metrics/badge.svg)](https://coveralls.io/github/PCMDI/pcmdi_metrics) -->

The PCMDI Metrics Package (PMP) is used to provide "quick-look" objective comparisons of Earth System Models (ESMs) with one another and available observations. Results are produced in the context of all model simulations contributed to CMIP6 and earlier CMIP phases. Among other purposes, this enables modeling groups to evaluate changes during the development cycle in the context of the structural error distribution of the multi-model ensemble. Currently, the comparisons emphasize metrics of large- to global-scale annual cycle and both tropcial and extra-tropical modes of variability. Ongoing work in v1.x development branches include established statistics for ENSO, MJO, regional monsoons, and high frequency characteristics of simulated precipitation.
The PCMDI Metrics Package (PMP) is used to provide "quick-look" objective comparisons of Earth System Models (ESMs) with one another and available observations. Results are produced in the context of all model simulations contributed to CMIP6 and earlier CMIP phases. Among other purposes, this enables modeling groups to evaluate changes during the development cycle in the context of the structural error distribution of the multi-model ensemble. Currently, the comparisons emphasize metrics of large- to global-scale annual cycle and both tropcial and extra-tropical modes of variability. Ongoing work in v1.x development branches include established statistics for ENSO, MJO, regional monsoons, and high frequency characteristics of simulated precipitation.

**PCMDI uses the PMP to produce [quick-look simulation summaries across generations of CMIP](https://cmec.llnl.gov/results/physical.html)**
**PCMDI uses the PMP to produce [quick-look simulation summaries across generations of CMIP](https://cmec.llnl.gov/results/physical.html)**

The metrics package consists of four parts: 1) Analysis software, 2) an observationally-based database of global (or near global, land or ocean) [time series and climatologies](https://github.com/PCMDI/PCMDIobs-cmor-tables/tree/master/catalogue), 3) a database of performance metrics computed for CMIP models and 4) [package documentation and interactive demos](http://pcmdi.github.io/pcmdi_metrics/).

Expand Down
Loading