Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update to sfapi v3.0.0 #282

Merged
merged 14 commits into from
Aug 6, 2024
46 changes: 18 additions & 28 deletions .github/workflows/cicd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,9 @@ on:
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 10
timeout-minutes: 20

services:

elasticsearch_8_svc:
image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
env:
Expand Down Expand Up @@ -63,22 +62,31 @@ jobs:
OPENSEARCH_JAVA_OPTS: -Xms512m -Xmx512m
ports:
- 9202:9202

strategy:
matrix:
python-version: [ "3.8", "3.9", "3.10", "3.11", "3.12"]
backend: [ "elasticsearch7", "elasticsearch8", "opensearch"]

name: Python ${{ matrix.python-version }} testing
name: Python ${{ matrix.python-version }} testing with ${{ matrix.backend }}

steps:
- name: Check out repository code
uses: actions/checkout@v4

# Setup Python (faster than using Python container)
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Cache pipenv packages
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/Pipfile.lock') }}
restore-keys: |
${{ runner.os }}-pip-

- name: Lint code
if: ${{ matrix.python-version == 3.11 }}
run: |
Expand All @@ -101,35 +109,17 @@ jobs:
run: |
pip install ./stac_fastapi/opensearch[dev,server]

- name: Run test suite against Elasticsearch 7.x
run: |
pipenv run pytest -svvv
env:
ENVIRONMENT: testing
ES_PORT: 9400
ES_HOST: 172.17.0.1
ES_USE_SSL: false
ES_VERIFY_CERTS: false
BACKEND: elasticsearch

- name: Run test suite against Elasticsearch 8.x
- name: Install pytest-timeout
run: |
pipenv run pytest -svvv
env:
ENVIRONMENT: testing
ES_PORT: 9200
ES_HOST: 172.17.0.1
ES_USE_SSL: false
ES_VERIFY_CERTS: false
BACKEND: elasticsearch
pip install pytest-timeout

- name: Run test suite against OpenSearch 2.11.1
- name: Run test suite
run: |
pipenv run pytest -svvv
pipenv run pytest -svvv --timeout=300
env:
ENVIRONMENT: testing
ES_PORT: 9202
ES_PORT: ${{ matrix.backend == 'elasticsearch7' && '9400' || matrix.backend == 'elasticsearch8' && '9200' || '9202' }}
ES_HOST: 172.17.0.1
ES_USE_SSL: false
ES_VERIFY_CERTS: false
BACKEND: opensearch
BACKEND: ${{ matrix.backend == 'elasticsearch7' && 'elasticsearch' || matrix.backend == 'elasticsearch8' && 'elasticsearch' || 'opensearch' }}
10 changes: 4 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,11 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [Unreleased]
## [v3.0.0] - 2024-08-14

### Changed
- Aggregation bug fixes [#281](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/281)
- Updated stac-fastapi libraries to v3.0.0 [#282](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/282)

## [v3.0.0a3] - 2024-07-17

Expand Down Expand Up @@ -238,11 +239,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- Added db_to_stac serializer to item_collection method in core.py.


[Unreleased]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v3.0.0a3...main>
[v3.0.0a3]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v3.0.0a2...v3.0.0a3>
[v3.0.0a2]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v3.0.0a1...v3.0.0a2>
[v3.0.0a1]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v3.0.0a0...v3.0.0a1>
[v3.0.0a0]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v2.4.1...v3.0.0a0>
[Unreleased]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v3.0.0...main>
[v3.0.0]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v2.4.1...v3.0.0>
[v2.4.1]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v2.4.0...v2.4.1>
[v2.4.0]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v2.3.0...v2.4.0>
[v2.3.0]: <https://github.com/stac-utils/stac-fastapi-elasticsearch/tree/v2.2.0...v2.3.0>
Expand Down
6 changes: 3 additions & 3 deletions stac_fastapi/core/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
"attrs>=23.2.0",
"pydantic[dotenv]",
"stac_pydantic>=3",
"stac-fastapi.types==3.0.0a4",
"stac-fastapi.api==3.0.0a4",
"stac-fastapi.extensions==3.0.0a4",
"stac-fastapi.types==3.0.0",
"stac-fastapi.api==3.0.0",
"stac-fastapi.extensions==3.0.0",
"orjson",
"overrides",
"geojson-pydantic",
Expand Down
15 changes: 4 additions & 11 deletions stac_fastapi/core/stac_fastapi/core/core.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""Core client."""
import logging
import re
from datetime import datetime as datetime_type
from datetime import timezone
from enum import Enum
Expand Down Expand Up @@ -279,8 +278,8 @@ async def item_collection(
collection_id: str,
bbox: Optional[BBox] = None,
datetime: Optional[DateTimeType] = None,
limit: int = 10,
token: str = None,
limit: Optional[int] = 10,
token: Optional[str] = None,
**kwargs,
) -> stac_types.ItemCollection:
"""Read items from a specific collection in the database.
Expand All @@ -302,6 +301,8 @@ async def item_collection(
Exception: If any error occurs while reading the items from the database.
"""
request: Request = kwargs["request"]
token = request.query_params.get("token")

base_url = str(request.base_url)

collection = await self.get_collection(
Expand Down Expand Up @@ -490,14 +491,6 @@ async def get_search(
"query": orjson.loads(query) if query else query,
}

# this is borrowed from stac-fastapi-pgstac
# Kludgy fix because using factory does not allow alias for filter-lan
query_params = str(request.query_params)
if filter_lang is None:
match = re.search(r"filter-lang=([a-z0-9-]+)", query_params, re.IGNORECASE)
if match:
filter_lang = match.group(1)

if datetime:
base_args["datetime"] = self._format_datetime_range(datetime)

Expand Down
13 changes: 0 additions & 13 deletions stac_fastapi/core/stac_fastapi/core/extensions/aggregation.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""Request model for the Aggregation extension."""

import re
from datetime import datetime
from datetime import datetime as datetime_type
from typing import Dict, List, Literal, Optional, Union
Expand Down Expand Up @@ -351,18 +350,6 @@ async def aggregate(
if datetime:
base_args["datetime"] = self._format_datetime_range(datetime)

# this is borrowed from stac-fastapi-pgstac
# Kludgy fix because using factory does not allow alias for filter-lang
# If the value is the default, check if the request is different.
query_params = str(request.query_params)
if filter_lang is None:
match = re.search(
r"filter-lang=([a-z0-9-]+)", query_params, re.IGNORECASE
)
if match:
filter_lang = match.group(1)
else:
filter_lang = "cql2-text"
if filter:
base_args["filter"] = self.get_filter(filter, filter_lang)
aggregate_request = EsAggregationExtensionPostRequest(**base_args)
Expand Down
2 changes: 1 addition & 1 deletion stac_fastapi/core/stac_fastapi/core/version.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""library version."""
__version__ = "3.0.0a3"
__version__ = "3.0.0"
2 changes: 1 addition & 1 deletion stac_fastapi/elasticsearch/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
desc = f.read()

install_requires = [
"stac-fastapi.core==3.0.0a3",
"stac-fastapi.core==3.0.0",
"elasticsearch[async]==8.11.0",
"elasticsearch-dsl==8.11.0",
"uvicorn",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""library version."""
__version__ = "3.0.0a3"
__version__ = "3.0.0"
2 changes: 1 addition & 1 deletion stac_fastapi/opensearch/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
desc = f.read()

install_requires = [
"stac-fastapi.core==3.0.0a3",
"stac-fastapi.core==3.0.0",
"opensearch-py==2.4.2",
"opensearch-py[async]==2.4.2",
"uvicorn",
Expand Down
2 changes: 1 addition & 1 deletion stac_fastapi/opensearch/stac_fastapi/opensearch/version.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""library version."""
__version__ = "3.0.0a3"
__version__ = "3.0.0"
Loading