Skip to content

Commit

Permalink
Merge branch 'master' into include-icat-relations-for-non-related-pan…
Browse files Browse the repository at this point in the history 
…osc-fields
  • Loading branch information
@VKTB
VKTB committed Feb 1, 2022
2 parents e1f199d + b275fdb commit 1faba30
Show file tree
Hide file tree
Showing 15 changed files with 1,323 additions and 311 deletions.
270 changes: 145 additions & 125 deletions .github/workflows/ci-build.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,146 +11,153 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ['3.6', '3.7', '3.8', '3.9']
python-version: ["3.6", "3.7", "3.8", "3.9"]
name: Python ${{ matrix.python-version }} Build & Tests
steps:
- name: Add apt repo
run: sudo add-apt-repository universe

# Setup Java & Python
- name: Setup Java
uses: actions/setup-java@v1
with:
java-version: 8
java-package: jdk
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
architecture: x64

# ICAT Ansible clone and install dependencies
- name: Checkout icat-ansible
uses: actions/checkout@v2
with:
repository: icatproject-contrib/icat-ansible
ref: master
path: icat-ansible
- name: Install Ansible
run: pip install -r icat-ansible/requirements.txt

# Prep for running the playbook
- name: Create hosts file
run: echo -e "[icatdb-minimal-hosts]\nlocalhost ansible_connection=local" > icat-ansible/hosts
- name: Prepare vault pass
run: echo -e "icattravispw" > icat-ansible/vault_pass.txt
- name: Move vault to directory it'll get detected by Ansible
run: mv icat-ansible/vault.yml icat-ansible/group_vars/all
- name: Replace default payara user with Actions user
run: |
sed -i -e "s/^payara_user: \"glassfish\"/payara_user: \"runner\"/" icat-ansible/group_vars/all/vars.yml
# Force hostname to localhost - bug fix for previous ICAT Ansible issues on Actions
- name: Change hostname to localhost
run: sudo hostname -b localhost

# Create local instance of ICAT
- name: Run ICAT Ansible Playbook
run: |
ansible-playbook icat-ansible/icatdb-minimal-hosts.yml -i icat-ansible/hosts --vault-password-file icat-ansible/vault_pass.txt -vv
- name: Checkout DataGateway API
uses: actions/checkout@v2

# Prep for using the API for tests
- name: Create log file
run: touch logs.log
- name: Configure log file location
run: echo "`jq -r --arg REPO_DIR "$GITHUB_WORKSPACE/logs.log" \
'.log_location=$REPO_DIR' datagateway_api/config.json.example`" > datagateway_api/config.json.example
- name: Create config.json
run: cp datagateway_api/config.json.example datagateway_api/config.json
- name: Create search_api_mapping.json
run: cp datagateway_api/search_api_mapping.json.example datagateway_api/search_api_mapping.json

# Install Nox, Poetry and API's dependencies
- name: Install Nox
run: pip install nox==2020.8.22
- name: Install Poetry
run: pip install poetry==1.1.9
- name: Install dependencies
run: poetry install

- name: Add dummy data to icatdb
run: |
poetry run python -m util.icat_db_generator
# Run Nox tests session, saves and uploads a coverage report to codecov
- name: Run Nox tests session
run: nox -p ${{ matrix.python-version }} -s tests -- --cov=datagateway_api --cov-report=xml
- name: Upload code coverage report
if: matrix.python-version == '3.6'
uses: codecov/codecov-action@v1
- name: Add apt repo
run: sudo add-apt-repository universe

# Setup Java & Python
- name: Setup Java
uses: actions/setup-java@v1
with:
java-version: 8
java-package: jdk
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
architecture: x64

# ICAT Ansible clone and install dependencies
- name: Checkout icat-ansible
uses: actions/checkout@v2
with:
repository: icatproject-contrib/icat-ansible
ref: master
path: icat-ansible
- name: Install Ansible
run: pip install -r icat-ansible/requirements.txt

# Prep for running the playbook
- name: Create hosts file
run: echo -e "[icatdb-minimal-hosts]\nlocalhost ansible_connection=local" > icat-ansible/hosts
- name: Prepare vault pass
run: echo -e "icattravispw" > icat-ansible/vault_pass.txt
- name: Move vault to directory it'll get detected by Ansible
run: mv icat-ansible/vault.yml icat-ansible/group_vars/all
- name: Replace default payara user with Actions user
run: |
sed -i -e "s/^payara_user: \"glassfish\"/payara_user: \"runner\"/" icat-ansible/group_vars/all/vars.yml
# Force hostname to localhost - bug fix for previous ICAT Ansible issues on Actions
- name: Change hostname to localhost
run: sudo hostname -b localhost

# Create local instance of ICAT
- name: Run ICAT Ansible Playbook
run: |
ansible-playbook icat-ansible/icatdb-minimal-hosts.yml -i icat-ansible/hosts --vault-password-file icat-ansible/vault_pass.txt -vv
- name: Checkout DataGateway API
uses: actions/checkout@v2

# Prep for using the API for tests
- name: Create log file
run: touch logs.log
- name: Configure log file location
run: echo "`jq -r --arg REPO_DIR "$GITHUB_WORKSPACE/logs.log" \
'.log_location=$REPO_DIR' datagateway_api/config.json.example`" > datagateway_api/config.json.example
- name: Create config.json
run: cp datagateway_api/config.json.example datagateway_api/config.json
- name: Create search_api_mapping.json
run: cp datagateway_api/search_api_mapping.json.example datagateway_api/search_api_mapping.json

# Install Nox, Poetry and API's dependencies
- name: Install Nox
run: pip install nox==2020.8.22
- name: Install Poetry
run: pip install poetry==1.1.9

# Installing an older version of setuptools for reasons explained at: https://github.com/icatproject/python-icat/issues/99
- name: Uninstall setuptools
run: poetry run pip uninstall -y setuptools
- name: Install older setuptools
run: poetry run pip install 'setuptools<58.0.0'

- name: Install dependencies
run: poetry install

- name: Add dummy data to icatdb
run: |
poetry run python -m util.icat_db_generator
# Run Nox tests session, saves and uploads a coverage report to codecov
- name: Run Nox tests session
run: nox -p ${{ matrix.python-version }} -s tests -- --cov=datagateway_api --cov-report=xml
- name: Upload code coverage report
if: matrix.python-version == '3.6'
uses: codecov/codecov-action@v1

linting:
runs-on: ubuntu-20.04
name: Linting
steps:
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: '3.9.7'
architecture: x64
- name: Checkout DataGateway API
uses: actions/checkout@v2

- name: Install Nox
run: pip install nox==2020.8.22
- name: Install Poetry
run: pip install poetry==1.1.9

- name: Run Nox lint session
run: nox -s lint
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: "3.9.7"
architecture: x64
- name: Checkout DataGateway API
uses: actions/checkout@v2

- name: Install Nox
run: pip install nox==2020.8.22
- name: Install Poetry
run: pip install poetry==1.1.9

- name: Run Nox lint session
run: nox -s lint

formatting:
runs-on: ubuntu-20.04
name: Code Formatting
steps:
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: '3.9.7'
architecture: x64
- name: Checkout DataGateway API
uses: actions/checkout@v2

- name: Install Nox
run: pip install nox==2020.8.22
- name: Install Poetry
run: pip install poetry==1.1.9

- name: Run Nox black session
run: nox -s black
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: "3.9.7"
architecture: x64
- name: Checkout DataGateway API
uses: actions/checkout@v2

- name: Install Nox
run: pip install nox==2020.8.22
- name: Install Poetry
run: pip install poetry==1.1.9

- name: Run Nox black session
run: nox -s black

safety:
runs-on: ubuntu-20.04
name: Dependency Safety
steps:
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: '3.9.7'
architecture: x64
- name: Checkout DataGateway API
uses: actions/checkout@v2

- name: Install Nox
run: pip install nox==2020.8.22
- name: Install Poetry
run: pip install poetry==1.1.9

- name: Run Nox safety session
run: nox -s safety
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: "3.9.7"
architecture: x64
- name: Checkout DataGateway API
uses: actions/checkout@v2

- name: Install Nox
run: pip install nox==2020.8.22
- name: Install Poetry
run: pip install poetry==1.1.9

- name: Run Nox safety session
run: nox -s safety

generator-script-testing:
runs-on: ubuntu-20.04
Expand All @@ -169,7 +176,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: '3.9.7'
python-version: "3.9.7"
architecture: x64

# ICAT Ansible clone and install dependencies
Expand Down Expand Up @@ -211,6 +218,13 @@ jobs:
run: cp datagateway_api/search_api_mapping.json.example datagateway_api/search_api_mapping.json
- name: Install Poetry
run: pip install poetry==1.1.9

# Installing an older version of setuptools for reasons explained at: https://github.com/icatproject/python-icat/issues/99
- name: Uninstall setuptools
run: poetry run pip uninstall -y setuptools
- name: Install older setuptools
run: poetry run pip install 'setuptools<58.0.0'

- name: Install dependencies
run: poetry install

Expand Down Expand Up @@ -238,7 +252,6 @@ jobs:
- name: Diff SQL dumps
run: diff -s ~/generator_script_dump_1.sql ~/generator_script_dump_2.sql


# Drop and re-create icatdb to remove generated data
- name: Drop icatdb
run: mysql -picatdbuserpw -uicatdbuser -e 'DROP DATABASE icatdb;'
Expand All @@ -257,6 +270,13 @@ jobs:
run: cp datagateway_api/config.json.example datagateway_api/config.json
- name: Create search_api_mapping.json
run: cp datagateway_api/search_api_mapping.json.example datagateway_api/search_api_mapping.json

# Installing an older version of setuptools for reasons explained at: https://github.com/icatproject/python-icat/issues/99
- name: Uninstall setuptools
run: poetry run pip uninstall -y setuptools
- name: Install older setuptools
run: poetry run pip install 'setuptools<58.0.0'

- name: Install dependencies
run: poetry install

Expand Down
40 changes: 40 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,46 @@

<!--next-version-placeholder-->

## v3.5.1 (2022-01-31)
### Fix
* Fix nested relations bug #261 ([`67fcbfe`](https://github.com/ral-facilities/datagateway-api/commit/67fcbfe2a35ca2b7e007a1a6d78105b2e46b0b5f))
* Reference self instead of fixed instance #301 ([`40f5662`](https://github.com/ral-facilities/datagateway-api/commit/40f566279543871c5b7b1eab2c8b74050d8b3525))

## v3.5.0 (2022-01-31)
### Feature
* Implement basic version of `SearchAPIIncludeFilter` #261 ([`f2f53c9`](https://github.com/ral-facilities/datagateway-api/commit/f2f53c92229d052ae697787eb80a35dcd2ea3b45))

### Fix
* Fix list type field checking in Python 3.6 #265 ([`691a59e`](https://github.com/ral-facilities/datagateway-api/commit/691a59ea3f850475572c3a877fb739e5216c6fe7))

### Documentation
* Add new comments and fix existing #265 ([`3f1b1cf`](https://github.com/ral-facilities/datagateway-api/commit/3f1b1cffdd1e57ab4eb1227b13e0906424adefd0))

## v3.4.0 (2022-01-31)
### Feature
* Implement `regexp` operator #297 ([`bf3fe0e`](https://github.com/ral-facilities/datagateway-api/commit/bf3fe0ef2ac582d55dbd881edf6a81a93625ce91))
* Implement `neq` operator #297 ([`9094bbb`](https://github.com/ral-facilities/datagateway-api/commit/9094bbb894ead20a53fadfd0e24b264af29548b9))
* Implement `nin` operator #297 ([`00dbba5`](https://github.com/ral-facilities/datagateway-api/commit/00dbba525d5cd86cb5577f3b1621a7042cdd2fa0))
* Implement `inq` operator #297 ([`fc1cf19`](https://github.com/ral-facilities/datagateway-api/commit/fc1cf194454a4da60652b1f68df278c4624ddc11))
* Implement `between` operator #297 ([`4736888`](https://github.com/ral-facilities/datagateway-api/commit/4736888bf76cda0dbc00f997443ed565f0f5e760))

## v3.3.0 (2022-01-31)
### Feature
* Add function to get PaNOSC to ICAT mapping for where filter #260 ([`34b1d81`](https://github.com/ral-facilities/datagateway-api/commit/34b1d819482aa3efdb4f8da321125d3e40d76617))
* Convert PaNOSC to ICAT for where filter fields #260 ([`ff9595d`](https://github.com/ral-facilities/datagateway-api/commit/ff9595d2f571211db79dea02f702d4148b8879f3))

### Fix
* Fix example mapping file ([`3802cc9`](https://github.com/ral-facilities/datagateway-api/commit/3802cc9ee71a355d0ad87529f65112e8c3f8b881))
* Update `__str__()` for WHERE filter to cope with applying filter #260 ([`8d259d7`](https://github.com/ral-facilities/datagateway-api/commit/8d259d75a28414f26cc569293720ef4e306e6844))

### Documentation
* Add docstring to static function #260 ([`618f6b9`](https://github.com/ral-facilities/datagateway-api/commit/618f6b9fead88f61a346b90cb2b85a90877b0410))

## v3.2.0 (2022-01-31)
### Feature
* Add class to represent nested conditions #259 ([`583cbf2`](https://github.com/ral-facilities/datagateway-api/commit/583cbf29744b72c020429b61ae7442b19acef231))
* Add first pass of query param implementation #259 ([`ee668e3`](https://github.com/ral-facilities/datagateway-api/commit/ee668e38cd43354851163616a93924ad84e14b90))

## v3.1.1 (2021-12-15)
### Fix
* Correct reference to class name #264 ([`fc4c180`](https://github.com/ral-facilities/datagateway-api/commit/fc4c18085ab496d838e8d1e9e3f8c77f07826e9d))
Expand Down
11 changes: 8 additions & 3 deletions datagateway_api/src/common/filters.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,16 @@ def __init__(self, field, value, operation):
self.value = value
self.operation = operation

if self.operation == "in":
if self.operation in ["in", "nin", "inq", "between"]:
if not isinstance(self.value, list):
raise BadRequestError(
"When using the 'in' operation for a WHERE filter, the values must"
" be in a list format e.g. [1, 2, 3]",
f"When using the {self.operation} operation for a WHERE filter, the"
f" values must be in a list format e.g. [1, 2]",
)
if self.operation == "between" and len(self.value) != 2:
raise BadRequestError(
"When using the 'between' operation for a WHERE filter, the list"
"must contain two values e.g. [1, 2]",
)


Expand Down
Loading

0 comments on commit 1faba30

Please sign in to comment.