PGStac #126
Merged
PGStac #126
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kylebarron
reviewed
Apr 22, 2021
| @property | ||
| def reader_connection_string(self): | ||
| """Create reader psql connection string.""" | ||
| return f"postgresql://{self.postgres_user}:{self.postgres_pass}@{self.postgres_host_reader}:{self.postgres_port}/{self.postgres_dbname}" |
There was a problem hiding this comment.
I see pydantic has a PostgresDsn type. Maybe it would be useful for validation purposes to have reader_connection_string be a field with init=False that's set within __post_init__? Then you'd have fuller validation?
There was a problem hiding this comment.
We can look at that later (across both pgstac and sqlalchemy - which this is set up identically to right now). I know I ran into some issue initially when I did use the PostgresDsn type, I think it had some issues when you didn't directly supply some non-required fields IIRC.
kylebarron
reviewed
May 4, 2021
Co-authored-by: Jeff Albrecht <jeff@arturo.ai>
Co-authored-by: Jeff Albrecht <jeff@arturo.ai>
Co-authored-by: Jeff Albrecht <jeff@arturo.ai>
Co-authored-by: Jeff Albrecht <jeff@arturo.ai>
remove unused load_test_data fixtures Co-authored-by: Jeff Albrecht <jeff@arturo.ai>
…nctions. add type hints.
|
lossyrob
reviewed
Jun 16, 2021
docker-compose.yml
Outdated
| @@ -51,10 +83,10 @@ services: | |||
| volumes: | |||
| - ./:/app | |||
| command: > | |||
| bash -c "sleep 10 && alembic upgrade head && python /app/scripts/ingest_joplin.py" | |||
| bash -c "sleep 10 && cd stac_fastapi/sqlalchemy && alembic upgrade head && python scripts/ingest_joplin.py" | |||
There was a problem hiding this comment.
Migration is broken with this change, fixed by:
Suggested change
| bash -c "sleep 10 && cd stac_fastapi/sqlalchemy && alembic upgrade head && python scripts/ingest_joplin.py" | |
| bash -c "sleep 10 && cd stac_fastapi/sqlalchemy && alembic upgrade head && cd ../.. && python scripts/ingest_joplin.py" |
|
@bitner I was able to work around the venv issue by not loading the entire source into the container, which I think is better practice anyway. diff --git a/Dockerfile b/Dockerfile
index ee0412f..21b7c64 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -12,7 +12,7 @@ ARG install_dev_dependencies=true
WORKDIR /app
# Install stac_fastapi.types
-COPY . /app
+COPY ./stac_fastapi /app/stac_fastapi
ENV PATH=$PATH:/install/bin
diff --git a/docker-compose.yml b/docker-compose.yml
index c5124ac..bd6a98e 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -25,7 +25,7 @@ services:
ports:
- "8081:8081"
volumes:
- - ./:/app
+ - ./stac_fastapi:/app/stac_fastapi
depends_on:
- database
command:
@@ -55,7 +55,7 @@ services:
ports:
- "8082:8082"
volumes:
- - ./:/app
+ - ./stac_fastapi:/app/stac_fastapi
depends_on:
- database
command:
@@ -81,7 +81,8 @@ services:
- POSTGRES_HOST=database
- POSTGRES_PORT=5432
volumes:
- - ./:/app
+ - ./stac_fastapi:/app/stac_fastapi
+ - ./scripts:/app/scripts
command: >
bash -c "sleep 10 && cd stac_fastapi/sqlalchemy && alembic upgrade head && python scripts/ingest_joplin.py"
depends_on:Now I'm getting new errors on (there's a bunch of those, all related to too many clients). Confused on how that's not working for me and working for others - there doesn't seem to be any environment variables that I may be loading in and it's all containerized...any ideas on this one? |
… stac_fastapi directories
|
Hey @lossyrob , That looks like you are running out of PG connections, wondering if things just were finishing quicker on my machine so I wasn't hitting that?. I just pushed a change for docker-compose that
Famous words: "works on my system", but maybe if you still are running into issues we hop on a call to try to debug? |
geospatial-jeff
approved these changes
Jun 17, 2021
|
Looking good, but The script points to a non-existent data_dir, should be: def ingest_joplin_data(data_dir=Path.cwd() / "stac_fastapi" / "sqlalchemy" / "tests" / "data" / "joplin"):Seems like this broke during the subpackage reorg, but I'd recommend fixing here before merge. Edit: I see this is mentioned in #150 |
lossyrob
requested changes
Jun 17, 2021
There was a problem hiding this comment.
Is there a development data ingest step for pgstac? If not, we should implement one similar to ingest_joplin.py (or just reuse that). That way it'll be easier to use in the dev environment in general, and manually inspect the JSON responses for this review. Thanks!
| """ | ||
| request = kwargs["request"] | ||
| base_url = str(request.base_url) | ||
| landing_page = LandingPage( |
There was a problem hiding this comment.
Missing the "collections" link
stac-fastapi/stac_fastapi/types/stac_fastapi/types/core.py
Lines 148 to 152 in f6a9007
Perhaps we can define a def get_landing_page(self) method that both implementations could use, to ensure consistency?
There was a problem hiding this comment.
@lossyrob get_landing_page is still dependent on the backend (to get the list of collections), so the direct get_landing_page() still needs to be defined for both backends. I do think that we should look to move the Link generation code into a common place across backends, but I think it's best to do that after this has been merged so we can track things a bit more granularly that are touching both backends. For this PR, I've tried to avoid doing too many things that would change/affect the sqlalchemy backend.
I just pushed a commit that:
- moves the test data out of sqlalchemy
- modifies ingest_joplin.py to take an app_host parameter
- adds ingest_joplin.py runner to docker-compose for pgstac backend
- modifies /collections to return empty array rather than not found error (and changes test that was expecting not found)
- adds missing collections/ link to landing page
There was a problem hiding this comment.
Gotcha - thanks for the quick commits, I'll take another look at this and report back!
There was a problem hiding this comment.
Hitting http://localhost:8082/collections/joplin/items with the development data, looks like the null issue has popped up again?
Response JSON
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-94.6334839,
37.0595608
],
[
-94.6334839,
37.0332547
],
[
-94.6005249,
37.0332547
],
[
-94.6005249,
37.0595608
],
[
-94.6334839,
37.0595608
]
]
]
},
"properties": {
"gsd": 0.5971642834779395,
"title": null,
"width": 2500,
"height": 2500,
"created": null,
"mission": null,
"updated": null,
"datetime": "2000-02-02T00:00:00Z",
"platform": null,
"proj:epsg": 3857,
"providers": null,
"description": null,
"instruments": null,
"orientation": "nadir",
"end_datetime": null,
"constellation": null,
"start_datetime": null
},
"id": "fe916452-ba6f-4631-9154-c249924a122d",
"bbox": [
-94.6334839,
37.0332547,
-94.6005249,
37.0595608
],
"stac_version": "1.0.0-beta.2",
"assets": {
"COG": {
"gsd": null,
"href": "https://arturo-stac-api-test-data.s3.amazonaws.com/joplin/images/may24C355000e4102500n.tif",
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
"bands": null,
"roles": null,
"title": "NOAA STORM COG",
"created": null,
"mission": null,
"updated": null,
"platform": null,
"multihash": null,
"providers": null,
"description": null,
"instruments": null,
"end_datetime": null,
"constellation": null,
"polarizations": null,
"start_datetime": null
}
},
"links": [
{
"href": "http://localhost:8082/collections/joplin",
"rel": "collection",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items/fe916452-ba6f-4631-9154-c249924a122d",
"rel": "self",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/collections/joplin/items/fe916452-ba6f-4631-9154-c249924a122d/tiles",
"rel": "alternate",
"type": "application/json",
"title": "tiles"
}
],
"collection": "joplin"
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-94.6911621,
37.1033841
],
[
-94.6911621,
37.0770932
],
[
-94.6582031,
37.0770932
],
[
-94.6582031,
37.1033841
],
[
-94.6911621,
37.1033841
]
]
]
},
"properties": {
"gsd": 0.5971642834779395,
"title": null,
"width": 2500,
"height": 2500,
"created": null,
"mission": null,
"updated": null,
"datetime": "2000-02-02T00:00:00Z",
"platform": null,
"proj:epsg": 3857,
"providers": null,
"description": null,
"instruments": null,
"orientation": "nadir",
"end_datetime": null,
"constellation": null,
"start_datetime": null
},
"id": "f7f164c9-cfdf-436d-a3f0-69864c38ba2a",
"bbox": [
-94.6911621,
37.0770932,
-94.6582031,
37.1033841
],
"stac_version": "1.0.0-beta.2",
"assets": {
"COG": {
"gsd": null,
"href": "https://arturo-stac-api-test-data.s3.amazonaws.com/joplin/images/may24C350000e4107500n.tif",
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
"bands": null,
"roles": null,
"title": "NOAA STORM COG",
"created": null,
"mission": null,
"updated": null,
"platform": null,
"multihash": null,
"providers": null,
"description": null,
"instruments": null,
"end_datetime": null,
"constellation": null,
"polarizations": null,
"start_datetime": null
}
},
"links": [
{
"href": "http://localhost:8082/collections/joplin",
"rel": "collection",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items/f7f164c9-cfdf-436d-a3f0-69864c38ba2a",
"rel": "self",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/collections/joplin/items/f7f164c9-cfdf-436d-a3f0-69864c38ba2a/tiles",
"rel": "alternate",
"type": "application/json",
"title": "tiles"
}
],
"collection": "joplin"
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-94.4631958,
37.0836668
],
[
-94.4631958,
37.057369
],
[
-94.4329834,
37.057369
],
[
-94.4329834,
37.0836668
],
[
-94.4631958,
37.0836668
]
]
]
},
"properties": {
"gsd": 0.5971642834779395,
"title": null,
"width": 2500,
"height": 2500,
"created": null,
"mission": null,
"updated": null,
"datetime": "2000-02-02T00:00:00Z",
"platform": null,
"proj:epsg": 3857,
"providers": null,
"description": null,
"instruments": null,
"orientation": "nadir",
"end_datetime": null,
"constellation": null,
"start_datetime": null
},
"id": "f734401c-2df0-4694-a353-cdd3ea760cdc",
"bbox": [
-94.4631958,
37.057369,
-94.4329834,
37.0836668
],
"stac_version": "1.0.0-beta.2",
"assets": {
"COG": {
"gsd": null,
"href": "https://arturo-stac-api-test-data.s3.amazonaws.com/joplin/images/may24C370000e4105000n.tif",
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
"bands": null,
"roles": null,
"title": "NOAA STORM COG",
"created": null,
"mission": null,
"updated": null,
"platform": null,
"multihash": null,
"providers": null,
"description": null,
"instruments": null,
"end_datetime": null,
"constellation": null,
"polarizations": null,
"start_datetime": null
}
},
"links": [
{
"href": "http://localhost:8082/collections/joplin",
"rel": "collection",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items/f734401c-2df0-4694-a353-cdd3ea760cdc",
"rel": "self",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/collections/joplin/items/f734401c-2df0-4694-a353-cdd3ea760cdc/tiles",
"rel": "alternate",
"type": "application/json",
"title": "tiles"
}
],
"collection": "joplin"
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-94.43573,
37.0836668
],
[
-94.43573,
37.0595608
],
[
-94.402771,
37.0595608
],
[
-94.402771,
37.0836668
],
[
-94.43573,
37.0836668
]
]
]
},
"properties": {
"gsd": 0.5971642834779395,
"title": null,
"width": 2500,
"height": 2500,
"created": null,
"mission": null,
"updated": null,
"datetime": "2000-02-02T00:00:00Z",
"platform": null,
"proj:epsg": 3857,
"providers": null,
"description": null,
"instruments": null,
"orientation": "nadir",
"end_datetime": null,
"constellation": null,
"start_datetime": null
},
"id": "f490b7af-0019-45e2-854b-3854d07fd063",
"bbox": [
-94.43573,
37.0595608,
-94.402771,
37.0836668
],
"stac_version": "1.0.0-beta.2",
"assets": {
"COG": {
"gsd": null,
"href": "https://arturo-stac-api-test-data.s3.amazonaws.com/joplin/images/may24C372500e4105000n.tif",
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
"bands": null,
"roles": null,
"title": "NOAA STORM COG",
"created": null,
"mission": null,
"updated": null,
"platform": null,
"multihash": null,
"providers": null,
"description": null,
"instruments": null,
"end_datetime": null,
"constellation": null,
"polarizations": null,
"start_datetime": null
}
},
"links": [
{
"href": "http://localhost:8082/collections/joplin",
"rel": "collection",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items/f490b7af-0019-45e2-854b-3854d07fd063",
"rel": "self",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/collections/joplin/items/f490b7af-0019-45e2-854b-3854d07fd063/tiles",
"rel": "alternate",
"type": "application/json",
"title": "tiles"
}
],
"collection": "joplin"
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-94.6884155,
37.0595608
],
[
-94.6884155,
37.0332547
],
[
-94.6554565,
37.0332547
],
[
-94.6554565,
37.0595608
],
[
-94.6884155,
37.0595608
]
]
]
},
"properties": {
"gsd": 0.5971642834779395,
"title": null,
"width": 2500,
"height": 2500,
"created": null,
"mission": null,
"updated": null,
"datetime": "2000-02-02T00:00:00Z",
"platform": null,
"proj:epsg": 3857,
"providers": null,
"description": null,
"instruments": null,
"orientation": "nadir",
"end_datetime": null,
"constellation": null,
"start_datetime": null
},
"id": "f2cca2a3-288b-4518-8a3e-a4492bb60b08",
"bbox": [
-94.6884155,
37.0332547,
-94.6554565,
37.0595608
],
"stac_version": "1.0.0-beta.2",
"assets": {
"COG": {
"gsd": null,
"href": "https://arturo-stac-api-test-data.s3.amazonaws.com/joplin/images/may24C350000e4102500n.tif",
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
"bands": null,
"roles": null,
"title": "NOAA STORM COG",
"created": null,
"mission": null,
"updated": null,
"platform": null,
"multihash": null,
"providers": null,
"description": null,
"instruments": null,
"end_datetime": null,
"constellation": null,
"polarizations": null,
"start_datetime": null
}
},
"links": [
{
"href": "http://localhost:8082/collections/joplin",
"rel": "collection",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items/f2cca2a3-288b-4518-8a3e-a4492bb60b08",
"rel": "self",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/collections/joplin/items/f2cca2a3-288b-4518-8a3e-a4492bb60b08/tiles",
"rel": "alternate",
"type": "application/json",
"title": "tiles"
}
],
"collection": "joplin"
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-94.6609497,
37.0595608
],
[
-94.6609497,
37.0332547
],
[
-94.6279907,
37.0332547
],
[
-94.6279907,
37.0595608
],
[
-94.6609497,
37.0595608
]
]
]
},
"properties": {
"gsd": 0.5971642834779395,
"title": null,
"width": 2500,
"height": 2500,
"created": null,
"mission": null,
"updated": null,
"datetime": "2000-02-02T00:00:00Z",
"platform": null,
"proj:epsg": 3857,
"providers": null,
"description": null,
"instruments": null,
"orientation": "nadir",
"end_datetime": null,
"constellation": null,
"start_datetime": null
},
"id": "ea0fddf4-56f9-4a16-8a0b-f6b0b123b7cf",
"bbox": [
-94.6609497,
37.0332547,
-94.6279907,
37.0595608
],
"stac_version": "1.0.0-beta.2",
"assets": {
"COG": {
"gsd": null,
"href": "https://arturo-stac-api-test-data.s3.amazonaws.com/joplin/images/may24C352500e4102500n.tif",
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
"bands": null,
"roles": null,
"title": "NOAA STORM COG",
"created": null,
"mission": null,
"updated": null,
"platform": null,
"multihash": null,
"providers": null,
"description": null,
"instruments": null,
"end_datetime": null,
"constellation": null,
"polarizations": null,
"start_datetime": null
}
},
"links": [
{
"href": "http://localhost:8082/collections/joplin",
"rel": "collection",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items/ea0fddf4-56f9-4a16-8a0b-f6b0b123b7cf",
"rel": "self",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/collections/joplin/items/ea0fddf4-56f9-4a16-8a0b-f6b0b123b7cf/tiles",
"rel": "alternate",
"type": "application/json",
"title": "tiles"
}
],
"collection": "joplin"
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-94.6060181,
37.0595608
],
[
-94.6060181,
37.0332547
],
[
-94.5730591,
37.0332547
],
[
-94.5730591,
37.0595608
],
[
-94.6060181,
37.0595608
]
]
]
},
"properties": {
"gsd": 0.5971642834779395,
"title": null,
"width": 2500,
"height": 2500,
"created": null,
"mission": null,
"updated": null,
"datetime": "2000-02-02T00:00:00Z",
"platform": null,
"proj:epsg": 3857,
"providers": null,
"description": null,
"instruments": null,
"orientation": "nadir",
"end_datetime": null,
"constellation": null,
"start_datetime": null
},
"id": "e0a02e4e-aa0c-412e-8f63-6f5344f829df",
"bbox": [
-94.6060181,
37.0332547,
-94.5730591,
37.0595608
],
"stac_version": "1.0.0-beta.2",
"assets": {
"COG": {
"gsd": null,
"href": "https://arturo-stac-api-test-data.s3.amazonaws.com/joplin/images/may24C357500e4102500n.tif",
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
"bands": null,
"roles": null,
"title": "NOAA STORM COG",
"created": null,
"mission": null,
"updated": null,
"platform": null,
"multihash": null,
"providers": null,
"description": null,
"instruments": null,
"end_datetime": null,
"constellation": null,
"polarizations": null,
"start_datetime": null
}
},
"links": [
{
"href": "http://localhost:8082/collections/joplin",
"rel": "collection",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items/e0a02e4e-aa0c-412e-8f63-6f5344f829df",
"rel": "self",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/collections/joplin/items/e0a02e4e-aa0c-412e-8f63-6f5344f829df/tiles",
"rel": "alternate",
"type": "application/json",
"title": "tiles"
}
],
"collection": "joplin"
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-94.4659424,
37.1077651
],
[
-94.4659424,
37.0814756
],
[
-94.4329834,
37.0814756
],
[
-94.4329834,
37.1077651
],
[
-94.4659424,
37.1077651
]
]
]
},
"properties": {
"gsd": 0.5971642834779395,
"title": null,
"width": 2500,
"height": 2500,
"created": null,
"mission": null,
"updated": null,
"datetime": "2000-02-02T00:00:00Z",
"platform": null,
"proj:epsg": 3857,
"providers": null,
"description": null,
"instruments": null,
"orientation": "nadir",
"end_datetime": null,
"constellation": null,
"start_datetime": null
},
"id": "da6ef938-c58f-4bab-9d4e-89f6ae667da2",
"bbox": [
-94.4659424,
37.0814756,
-94.4329834,
37.1077651
],
"stac_version": "1.0.0-beta.2",
"assets": {
"COG": {
"gsd": null,
"href": "https://arturo-stac-api-test-data.s3.amazonaws.com/joplin/images/may24C370000e4107500n.tif",
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
"bands": null,
"roles": null,
"title": "NOAA STORM COG",
"created": null,
"mission": null,
"updated": null,
"platform": null,
"multihash": null,
"providers": null,
"description": null,
"instruments": null,
"end_datetime": null,
"constellation": null,
"polarizations": null,
"start_datetime": null
}
},
"links": [
{
"href": "http://localhost:8082/collections/joplin",
"rel": "collection",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items/da6ef938-c58f-4bab-9d4e-89f6ae667da2",
"rel": "self",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/collections/joplin/items/da6ef938-c58f-4bab-9d4e-89f6ae667da2/tiles",
"rel": "alternate",
"type": "application/json",
"title": "tiles"
}
],
"collection": "joplin"
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-94.5758057,
37.0836668
],
[
-94.5758057,
37.057369
],
[
-94.5455933,
37.057369
],
[
-94.5455933,
37.0836668
],
[
-94.5758057,
37.0836668
]
]
]
},
"properties": {
"gsd": 0.5971642834779395,
"title": null,
"width": 2500,
"height": 2500,
"created": null,
"mission": null,
"updated": null,
"datetime": "2000-02-02T00:00:00Z",
"platform": null,
"proj:epsg": 3857,
"providers": null,
"description": null,
"instruments": null,
"orientation": "nadir",
"end_datetime": null,
"constellation": null,
"start_datetime": null
},
"id": "d8461d8c-3d2b-4e4e-a931-7ae61ca06dbf",
"bbox": [
-94.5758057,
37.057369,
-94.5455933,
37.0836668
],
"stac_version": "1.0.0-beta.2",
"assets": {
"COG": {
"gsd": null,
"href": "https://arturo-stac-api-test-data.s3.amazonaws.com/joplin/images/may24C360000e4105000n.tif",
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
"bands": null,
"roles": null,
"title": "NOAA STORM COG",
"created": null,
"mission": null,
"updated": null,
"platform": null,
"multihash": null,
"providers": null,
"description": null,
"instruments": null,
"end_datetime": null,
"constellation": null,
"polarizations": null,
"start_datetime": null
}
},
"links": [
{
"href": "http://localhost:8082/collections/joplin",
"rel": "collection",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items/d8461d8c-3d2b-4e4e-a931-7ae61ca06dbf",
"rel": "self",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/collections/joplin/items/d8461d8c-3d2b-4e4e-a931-7ae61ca06dbf/tiles",
"rel": "alternate",
"type": "application/json",
"title": "tiles"
}
],
"collection": "joplin"
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-94.6609497,
37.1033841
],
[
-94.6609497,
37.0770932
],
[
-94.6279907,
37.0770932
],
[
-94.6279907,
37.1033841
],
[
-94.6609497,
37.1033841
]
]
]
},
"properties": {
"gsd": 0.5971642834779395,
"title": null,
"width": 2500,
"height": 2500,
"created": null,
"mission": null,
"updated": null,
"datetime": "2000-02-02T00:00:00Z",
"platform": null,
"proj:epsg": 3857,
"providers": null,
"description": null,
"instruments": null,
"orientation": "nadir",
"end_datetime": null,
"constellation": null,
"start_datetime": null
},
"id": "d4eccfa2-7d77-4624-9e2a-3f59102285bb",
"bbox": [
-94.6609497,
37.0770932,
-94.6279907,
37.1033841
],
"stac_version": "1.0.0-beta.2",
"assets": {
"COG": {
"gsd": null,
"href": "https://arturo-stac-api-test-data.s3.amazonaws.com/joplin/images/may24C352500e4107500n.tif",
"type": "image/tiff; application=geotiff; profile=cloud-optimized",
"bands": null,
"roles": null,
"title": "NOAA STORM COG",
"created": null,
"mission": null,
"updated": null,
"platform": null,
"multihash": null,
"providers": null,
"description": null,
"instruments": null,
"end_datetime": null,
"constellation": null,
"polarizations": null,
"start_datetime": null
}
},
"links": [
{
"href": "http://localhost:8082/collections/joplin",
"rel": "collection",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items/d4eccfa2-7d77-4624-9e2a-3f59102285bb",
"rel": "self",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/collections/joplin/items/d4eccfa2-7d77-4624-9e2a-3f59102285bb/tiles",
"rel": "alternate",
"type": "application/json",
"title": "tiles"
}
],
"collection": "joplin"
}
],
"stac_version": "1.0.0-beta.2",
"links": [
{
"href": "http://localhost:8082/collections/joplin/items",
"rel": "items",
"type": "application/geo+json"
},
{
"href": "http://localhost:8082/",
"rel": "parent",
"type": "application/json"
},
{
"href": "http://localhost:8082/",
"rel": "root",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin",
"rel": "self",
"type": "application/json"
},
{
"href": "http://localhost:8082/collections/joplin/items?token=next:d191a6fd-7881-4421-805c-e246371e5cc4",
"rel": "next",
"type": "application/json",
"method": "GET",
"merge": false
}
],
"timeStamp": "2021-06-17T14:58:21.337549+00:00"
} There was a problem hiding this comment.
(Unresolving as there's a new issue mentioned, should made a new thread)
There was a problem hiding this comment.
@kylebarron you going to just keep thumbsupping contradicting comments? ;-)
There was a problem hiding this comment.
Calling me out 🙈 Well first I read @geospatial-jeff 's comment, and that made sense, but then I read your comment, and that made sense too 😂
...unthumbsupping comments to hide my ignorance 😄
There was a problem hiding this comment.
Honestly, I claim ignorance on a lot of the intricacies with async & python, so I really have no clue which would be more correct. My laziness definitely tends towards the solution with the least amount of work/repetition.
There was a problem hiding this comment.
If your route handler is async def you need to wrap any synchronous code (ex. sqlalchemy pre v1.4) with a thread pool executor to prevent synchronous calls from blocking the event loop. If this is not the case a potentially long database call might prevent the fastapi application from handling new requests because the event loop is blocked. Using a synchronous route handler wraps the entire function call in a thread pool executor which is why you can safely execute synchronous code in a non-blocking fashion.
Also because the backend clients are decoupled from the API layer and may not necessarily be called within the context of FastAPI, we generally don't want async def functions that are running blocking code to begin with.
We could switch everything to async and just make people aware that they need to be executing synchronous code in threads but I'm not a fan of being overly opinionated about how code is executed as this will change based on the context.
There was a problem hiding this comment.
My preference is still creating an AsyncBaseClient and BaseClient so we can explicitly support both concurrency models.
…ker, make no collections return empty array
lossyrob
approved these changes
Jun 17, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds pgstac backend to stac_fastapi.
Supersedes #119 with automatic detection of async endpoints.
Adds 1:1 tests for all tests in sqlalchemy branch with the exception of the Context and BulkLoad plugin tests which are not supported on PGStac.
Updates the Makefile, Docker, and CI configurations to add support for running/testing pgstac.