Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade titiler and titiler-pgstac #148

Merged
merged 18 commits into from
Apr 24, 2023
Merged

Upgrade titiler and titiler-pgstac #148

merged 18 commits into from
Apr 24, 2023

Conversation

ividito
Copy link
Collaborator

@ividito ividito commented Jan 25, 2023

Fixes: #147

Tested with a local DB instance, front-end seemed to work fine with a handful of collections. This upgrade might not require as many front-end changes as we expected.

One thing this does break/change are asset_expressions (ref1, ref2).

Version changes:

@ividito ividito force-pushed the fix/dependency-upgrade branch from 95fda14 to 7bbaf4b Compare January 30, 2023 15:19
@ividito ividito marked this pull request as ready for review January 30, 2023 15:31
@ividito
Copy link
Collaborator Author

ividito commented Jan 30, 2023

Ready for review now that #149 has been merged.

@vincentsarago
Copy link
Contributor

by using titiler-pgstac==0.2.0 you can remove the custom factory in https://github.com/NASA-IMPACT/veda-backend/blob/develop/raster_api/runtime/src/factory.py#L61-L258

which is now directly in titiler-pgstac https://github.com/stac-utils/titiler-pgstac/blob/0.2.0/titiler/pgstac/factory.py#L721-L879 (you need to pass add_mosaic_list=True in MosaicTilerFactory(add_mosaic_list=True) instead of https://github.com/NASA-IMPACT/veda-backend/blob/develop/raster_api/runtime/src/app.py#L50

@anayeaye anayeaye mentioned this pull request Mar 14, 2023
5 tasks
@ividito ividito force-pushed the fix/dependency-upgrade branch from 7bbaf4b to d6db149 Compare March 28, 2023 15:15
@ividito ividito mentioned this pull request Mar 28, 2023
3 tasks
.github/workflows/pr.yml Outdated Show resolved Hide resolved
@ividito ividito requested a review from anayeaye April 19, 2023 20:02
@anayeaye
Copy link
Collaborator

anayeaye commented Apr 21, 2023

I rebuilt and ran with docker locally. Looks like /cog/viewer is replaced by /cog/map which offers more configuration options but may be a tad less intuitive for less technical users. I.e these are the differences I noticed and want to confirm are expected:

  • url /cog/viewer --> /cog/map
  • users must use url parameter in browser because there isn't an automatic input (I think most everyone does this anyway)
  • underlying map context is no longer present
  • configuration widget/side bar is gone (although the side bar offered less options than the new /cog/map endpoint provides)

former-cog-viewer

cog-map

If this is the expected behavior I'll approve the changes, we probably need to do a light refactor on our data provider documentation to make sure users know how to the new endpoint works and to update any references to the old viewer.

@j08lue
Copy link

j08lue commented Apr 24, 2023

Hmm, from the new viewer template, it seems like the UI elements for setting some of the URL parameters should still be there: https://github.com/NASA-IMPACT/veda-backend/pull/148/files#diff-42a81a69a9d962db79e4c01b3f757dbdb666fa17d46967b4e030ab8e2a3b1070R1. So the absence from your deployment might not be intended, @anayeaye.

@vincentsarago, what do you think about the expected changes with our update here regarding /cog/viewer -> /cog/map?

From a project perspective: we have not been relying a lot on /cog/viewer or promoted the use of it for our data contributors and curators, but it has come in handy sometimes to discover and debug data issues. So if we can keep a functioning viewer that at least has a background map (so users can confirm correct geo-reference fo their data), that would be great. If not, no problem to remove it.

@vincentsarago
Copy link
Contributor

/cog/viewer should still be there

@cog.router.get("/viewer", response_class=HTMLResponse)
def cog_demo(request: Request):
"""COG Viewer."""
return templates.TemplateResponse(
name="viewer.html",
context={
"request": request,
"tilejson_endpoint": cog.url_for(request, "tilejson"),
"info_endpoint": cog.url_for(request, "info"),
"statistics_endpoint": cog.url_for(request, "statistics"),
},
media_type="text/html",
)

there is also a new /cog/map that is provided directly by the tiler factory

@anayeaye
Copy link
Collaborator

I rebuilt and ran the changes locally and now see the viewer which is working as expected. Not sure what I had running on Friday, sorry about that!

@ividito ividito merged commit 36c34ee into develop Apr 24, 2023
@j08lue j08lue deleted the fix/dependency-upgrade branch April 25, 2023 18:32
slesaad added a commit to US-GHG-Center/ghgc-backend that referenced this pull request Jun 1, 2023
* Point README to veda-docs (NASA-IMPACT#171)

* Upgrade titiler and titiler-pgstac

* Fix br/gzip header test

* Remove factory (it's included in new titiler-pgstac version)
NASA-IMPACT#148 (comment)

* Remove pin on cramjam

* Load test data in container (actions not updated)

* fix actions tests

* adjust container strategy

* actions connections fix

* remove -it flag (it broke actions)

* re-enable lint, propagate test changes to other actions

* docker-compose -> docker compose

* Fix template import

* include private subnet config

* fix subnet type

* format changes

* pub accessible default true

* remove publicly accessible variable

* Add sql connection and execution abstraction

* Add sql logic to fix projection extension types

* add script to delete null stac_extensions

* Lint and move sql command scripts to support_scripts dir

* Use pythonic naming conventions

* pgstac readme change

* pin to urllib3<2

* pin urllib3 in test

* format

* revert

* pin httpx

* revert

* docker compose pgstac version

* pre-deploy action fix

* RDS Proxy initial implementation

* proxy secret + urlllib fix

* use pgstac 0.7.4

---------

Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
slesaad added a commit to US-GHG-Center/ghgc-backend that referenced this pull request Jun 14, 2023
… if needed (#8)

* Point README to veda-docs (NASA-IMPACT#171)

* Upgrade titiler and titiler-pgstac

* Fix br/gzip header test

* Remove factory (it's included in new titiler-pgstac version)
NASA-IMPACT#148 (comment)

* Remove pin on cramjam

* Load test data in container (actions not updated)

* fix actions tests

* adjust container strategy

* actions connections fix

* remove -it flag (it broke actions)

* re-enable lint, propagate test changes to other actions

* docker-compose -> docker compose

* Fix template import

* include private subnet config

* fix subnet type

* format changes

* pub accessible default true

* remove publicly accessible variable

* Add sql connection and execution abstraction

* Add sql logic to fix projection extension types

* add script to delete null stac_extensions

* Lint and move sql command scripts to support_scripts dir

* Use pythonic naming conventions

* pgstac readme change

* docker compose pgstac version

* pre-deploy action fix

* RDS Proxy initial implementation

* proxy secret + urlllib fix

* use pgstac 0.7.9

* Update set environment

* Add support for permissions boundary

* use titiler custom JSONResponse to handle NaN values

---------

Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: Caden Helbling <caden.helbling@gmail.com>
Co-authored-by: vincentsarago <vincent.sarago@gmail.com>
Co-authored-by: Alexandra Kirk <alexandra@developmentseed.org>
slesaad added a commit to US-GHG-Center/ghgc-backend that referenced this pull request Jun 14, 2023
… if needed (#9)

* Point README to veda-docs (NASA-IMPACT#171)

* Upgrade titiler and titiler-pgstac

* Fix br/gzip header test

* Remove factory (it's included in new titiler-pgstac version)
NASA-IMPACT#148 (comment)

* Remove pin on cramjam

* Load test data in container (actions not updated)

* fix actions tests

* adjust container strategy

* actions connections fix

* remove -it flag (it broke actions)

* re-enable lint, propagate test changes to other actions

* docker-compose -> docker compose

* Fix template import

* include private subnet config

* fix subnet type

* format changes

* pub accessible default true

* remove publicly accessible variable

* Add sql connection and execution abstraction

* Add sql logic to fix projection extension types

* add script to delete null stac_extensions

* Lint and move sql command scripts to support_scripts dir

* Use pythonic naming conventions

* pgstac readme change

* docker compose pgstac version

* pre-deploy action fix

* RDS Proxy initial implementation

* proxy secret + urlllib fix

* use pgstac 0.7.9

* Update set environment

* Add support for permissions boundary

* use titiler custom JSONResponse to handle NaN values

---------

Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: Caden Helbling <caden.helbling@gmail.com>
Co-authored-by: vincentsarago <vincent.sarago@gmail.com>
Co-authored-by: Alexandra Kirk <alexandra@developmentseed.org>
slesaad pushed a commit to US-GHG-Center/ghgc-backend that referenced this pull request Nov 10, 2023
* Fix mangum db connection pool

* lint

* Point README to veda-docs (NASA-IMPACT#171)

* Upgrade titiler and titiler-pgstac

* Fix br/gzip header test

* Remove factory (it's included in new titiler-pgstac version)
NASA-IMPACT#148 (comment)

* Remove pin on cramjam

* Load test data in container (actions not updated)

* fix actions tests

* adjust container strategy

* actions connections fix

* remove -it flag (it broke actions)

* re-enable lint, propagate test changes to other actions

* docker-compose -> docker compose

* Fix template import

* include private subnet config

* fix subnet type

* format changes

* pub accessible default true

* remove publicly accessible variable

* Add sql connection and execution abstraction

* Add sql logic to fix projection extension types

* add script to delete null stac_extensions

* Lint and move sql command scripts to support_scripts dir

* Use pythonic naming conventions

* pgstac readme change

* pin to urllib3<2

* pin urllib3 in test

* format

* revert

* pin httpx

* revert

* docker compose pgstac version

* pre-deploy action fix

* RDS Proxy initial implementation

* proxy secret + urlllib fix

* use pgstac 0.7.4

* use titiler custom JSONResponse to handle NaN values

* change db search path

* hacky fix

* try exporting os env vars

* export os env vars

* lint

* comment

* move to APISettings check

* black changes

* hide route

* Pin pydantic for now

* put raster assume role config in lambda construct

* pass lambda env variable bool as string

* NAT SG rules

* order of operation

* remove implicity SGs

* black formatted

* fix invalid string in rule description

* updates to deployment docs in readme

* update pgstac version in docker compose

* use max/end datetime in periodic collection datetime summary

* correct table name typo

* use NAT Gateway instead of EC2 Instance as NAT

* aws_ec2 mistake

* use aws_ec2.SubnetType.PRIVATE_WITH_EGRESS

* add back in NAT count

* add rds subnet group manual removal step to instructions for destroying a stack

* Enable tilematrixsets

* feat(database): RDS Configurations (NASA-IMPACT#218)

* feat(database): RDS Configurations

* chore: string configurations

* feat(database): Configurable RDS Configurations

* consolidate configuration docs in readme

* update table header

---------

Co-authored-by: anayeaye <alexandra@developmentseed.org>

* raster path prefix

* stac path prefix

* format

* config changes

* format

* Revert "format"

This reverts commit 3a26c1c.

* Revert "config changes"

This reverts commit 1425de6.

* feat: add configuration and github actions for python-semantic-release

* feat: lint conventional pr

* ci: cdk deploy should not require approval in github actions workflow

* docs: add contributing.md

* chore(ci): fix typo

* chore: explain versioning init choice

* chore: expalin automated version in setup.py

* chore: typo

Co-authored-by: Jamison French <50224594+jjfrench@users.noreply.github.com>

* docs: typos and improvements in contributing docs

* docs: typos in contributing

* ci: handle semantic version releases with tags only

* chore: add exclude commit patterns from release changelog

* cloudfront in veda backend

* format

* fix: if provided, apply permissions boundary to all roles created within a stack

* mypy, pydocstyle fixes

pydocstyle fixes

format

isort changes

optional config

format

config changes

format

config changes

construct change

* include apigateway parameter mapping

* remove unused import

* update readme

* remove unused config

* docs: s/veda-data-pipelines/veda-data-airflow/

I can't update the diagram b/c it's draw.io.

* include parameter mapping in raster api

* ci: branch specific pre-deploy check on pull request action

* ci: pre-deploy test against dev stack by default

* fix github var name in action

* typo

* include conditional param mapping

* change param mapping kwargs

* fix cloudfront subdomain

* incude missing argument

* formatting

* feat(raster)!: update titiler-pgstac from 0.2.3 to 0.8.0

* add dependencies

* add tests deps

* fix tests

* 🤦

* update handler

* pin pydantic to 1.* for cdk deployment

* fix deps

* fix(raster) use root path for prefixing and add to gateway base url in handler

* fix(stac) use root path for prefixing urls

* root path in fastapi app

* updates to for using a proxy

* lint

* only add additional servers to fast api if root path in config

* root path setting is optional

* cleanup

* cleanup

* do not configure alias for shared subdomain

* pr review cleanup

* fastapi additional servers not needed with apigw base path=app.root_path

* add custom colormap for EPA dataset

* Update raster_api/runtime/src/cmap_data/README.md

* rename cmap

* fix(apis)!: overwrite host in api integration if custom host provided

* fix(apis)!: overwrite host in api integration if custom host provided (p2)

* fix(routes)!: ingest api root path is /api/publish

* change ingestor api cf behavior

* format

* fix(domain)! only create custom api subdomains when configured

* improve config descriptions

* typo

* improve domain config docs

* account for custom staged url of veda-stac-ingestor

* do not run lint conventional pr

* merge duplicate corrected

* lint

* merge mismatched rds config naming

* merge inconsistencies

* merge inconsistencies

---------

Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: vincentsarago <vincent.sarago@gmail.com>
Co-authored-by: ranchodeluxe <greg.corradini@gmail.com>
Co-authored-by: sudobangbang <gregcorradini@gmail.com>
Co-authored-by: Jamison French <50224594+jjfrench@users.noreply.github.com>
Co-authored-by: Pete Gadomski <pete.gadomski@gmail.com>
slesaad added a commit to US-GHG-Center/ghgc-backend that referenced this pull request Nov 14, 2023
* Sync updates from upstream, add permissions boundary to created roles if needed (#8)

* Point README to veda-docs (NASA-IMPACT#171)

* Upgrade titiler and titiler-pgstac

* Fix br/gzip header test

* Remove factory (it's included in new titiler-pgstac version)
NASA-IMPACT#148 (comment)

* Remove pin on cramjam

* Load test data in container (actions not updated)

* fix actions tests

* adjust container strategy

* actions connections fix

* remove -it flag (it broke actions)

* re-enable lint, propagate test changes to other actions

* docker-compose -> docker compose

* Fix template import

* include private subnet config

* fix subnet type

* format changes

* pub accessible default true

* remove publicly accessible variable

* Add sql connection and execution abstraction

* Add sql logic to fix projection extension types

* add script to delete null stac_extensions

* Lint and move sql command scripts to support_scripts dir

* Use pythonic naming conventions

* pgstac readme change

* docker compose pgstac version

* pre-deploy action fix

* RDS Proxy initial implementation

* proxy secret + urlllib fix

* use pgstac 0.7.9

* Update set environment

* Add support for permissions boundary

* use titiler custom JSONResponse to handle NaN values

---------

Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: Caden Helbling <caden.helbling@gmail.com>
Co-authored-by: vincentsarago <vincent.sarago@gmail.com>
Co-authored-by: Alexandra Kirk <alexandra@developmentseed.org>

* Update bootstrapper lambda timeout (#11)

* Fix order of db search path (#13)

* Fix order of db search path

* Remove environment from predeploy

* Make RDS instance type configurable (#18)

* Fix missing rds config update (#22)

* Update rds type example config value (#25)

* Fix missing rds config update

* Update rds type config example value

* update collection summaries methods to include max end datetime

* fix docker compose in pr action

* Add DB storage encryption configurability (#34)

* Only add storage_encrypted db config if true (#35)

* use NAT Gateway

* Add TMS route for tile matrices (#40)

* Optionally overwrite host param in stac api (#43)

* Update mappingvalue to custom

* Overwrite host header for both raster and stac apis (#46)

* Overwrite host header for both raster and stac apis

* Add docstrings

* Fix stac config env prefix

* Host param override for api gateway (#48)

* Fetch upstream changes (#51)

* Fix mangum db connection pool

* lint

* Point README to veda-docs (NASA-IMPACT#171)

* Upgrade titiler and titiler-pgstac

* Fix br/gzip header test

* Remove factory (it's included in new titiler-pgstac version)
NASA-IMPACT#148 (comment)

* Remove pin on cramjam

* Load test data in container (actions not updated)

* fix actions tests

* adjust container strategy

* actions connections fix

* remove -it flag (it broke actions)

* re-enable lint, propagate test changes to other actions

* docker-compose -> docker compose

* Fix template import

* include private subnet config

* fix subnet type

* format changes

* pub accessible default true

* remove publicly accessible variable

* Add sql connection and execution abstraction

* Add sql logic to fix projection extension types

* add script to delete null stac_extensions

* Lint and move sql command scripts to support_scripts dir

* Use pythonic naming conventions

* pgstac readme change

* pin to urllib3<2

* pin urllib3 in test

* format

* revert

* pin httpx

* revert

* docker compose pgstac version

* pre-deploy action fix

* RDS Proxy initial implementation

* proxy secret + urlllib fix

* use pgstac 0.7.4

* use titiler custom JSONResponse to handle NaN values

* change db search path

* hacky fix

* try exporting os env vars

* export os env vars

* lint

* comment

* move to APISettings check

* black changes

* hide route

* Pin pydantic for now

* put raster assume role config in lambda construct

* pass lambda env variable bool as string

* NAT SG rules

* order of operation

* remove implicity SGs

* black formatted

* fix invalid string in rule description

* updates to deployment docs in readme

* update pgstac version in docker compose

* use max/end datetime in periodic collection datetime summary

* correct table name typo

* use NAT Gateway instead of EC2 Instance as NAT

* aws_ec2 mistake

* use aws_ec2.SubnetType.PRIVATE_WITH_EGRESS

* add back in NAT count

* add rds subnet group manual removal step to instructions for destroying a stack

* Enable tilematrixsets

* feat(database): RDS Configurations (NASA-IMPACT#218)

* feat(database): RDS Configurations

* chore: string configurations

* feat(database): Configurable RDS Configurations

* consolidate configuration docs in readme

* update table header

---------

Co-authored-by: anayeaye <alexandra@developmentseed.org>

* raster path prefix

* stac path prefix

* format

* config changes

* format

* Revert "format"

This reverts commit 3a26c1c.

* Revert "config changes"

This reverts commit 1425de6.

* feat: add configuration and github actions for python-semantic-release

* feat: lint conventional pr

* ci: cdk deploy should not require approval in github actions workflow

* docs: add contributing.md

* chore(ci): fix typo

* chore: explain versioning init choice

* chore: expalin automated version in setup.py

* chore: typo

Co-authored-by: Jamison French <50224594+jjfrench@users.noreply.github.com>

* docs: typos and improvements in contributing docs

* docs: typos in contributing

* ci: handle semantic version releases with tags only

* chore: add exclude commit patterns from release changelog

* cloudfront in veda backend

* format

* fix: if provided, apply permissions boundary to all roles created within a stack

* mypy, pydocstyle fixes

pydocstyle fixes

format

isort changes

optional config

format

config changes

format

config changes

construct change

* include apigateway parameter mapping

* remove unused import

* update readme

* remove unused config

* docs: s/veda-data-pipelines/veda-data-airflow/

I can't update the diagram b/c it's draw.io.

* include parameter mapping in raster api

* ci: branch specific pre-deploy check on pull request action

* ci: pre-deploy test against dev stack by default

* fix github var name in action

* typo

* include conditional param mapping

* change param mapping kwargs

* fix cloudfront subdomain

* incude missing argument

* formatting

* feat(raster)!: update titiler-pgstac from 0.2.3 to 0.8.0

* add dependencies

* add tests deps

* fix tests

* 🤦

* update handler

* pin pydantic to 1.* for cdk deployment

* fix deps

* fix(raster) use root path for prefixing and add to gateway base url in handler

* fix(stac) use root path for prefixing urls

* root path in fastapi app

* updates to for using a proxy

* lint

* only add additional servers to fast api if root path in config

* root path setting is optional

* cleanup

* cleanup

* do not configure alias for shared subdomain

* pr review cleanup

* fastapi additional servers not needed with apigw base path=app.root_path

* add custom colormap for EPA dataset

* Update raster_api/runtime/src/cmap_data/README.md

* rename cmap

* fix(apis)!: overwrite host in api integration if custom host provided

* fix(apis)!: overwrite host in api integration if custom host provided (p2)

* fix(routes)!: ingest api root path is /api/publish

* change ingestor api cf behavior

* format

* fix(domain)! only create custom api subdomains when configured

* improve config descriptions

* typo

* improve domain config docs

* account for custom staged url of veda-stac-ingestor

* do not run lint conventional pr

* merge duplicate corrected

* lint

* merge mismatched rds config naming

* merge inconsistencies

* merge inconsistencies

---------

Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: vincentsarago <vincent.sarago@gmail.com>
Co-authored-by: ranchodeluxe <greg.corradini@gmail.com>
Co-authored-by: sudobangbang <gregcorradini@gmail.com>
Co-authored-by: Jamison French <50224594+jjfrench@users.noreply.github.com>
Co-authored-by: Pete Gadomski <pete.gadomski@gmail.com>

* Remove unnecessary remanants of merge

* Remove "host" config var

---------

Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: Caden Helbling <caden.helbling@gmail.com>
Co-authored-by: vincentsarago <vincent.sarago@gmail.com>
Co-authored-by: Alexandra Kirk <alexandra@developmentseed.org>
Co-authored-by: Abdelhak Marouane <33136280+amarouane-ABDELHAK@users.noreply.github.com>
Co-authored-by: ranchodeluxe <greg.corradini@gmail.com>
Co-authored-by: sudobangbang <gregcorradini@gmail.com>
Co-authored-by: Jamison French <50224594+jjfrench@users.noreply.github.com>
Co-authored-by: Pete Gadomski <pete.gadomski@gmail.com>
slesaad added a commit to US-GHG-Center/ghgc-backend that referenced this pull request Nov 15, 2023
* Fix STAC catalog urls, add support for alternative projections in titiler (#44)

* Header host override for stac and raster apis (#47)

* Fetch upstream changes (#52)

* Sync updates from upstream, add permissions boundary to created roles if needed (#8)

* Point README to veda-docs (NASA-IMPACT#171)

* Upgrade titiler and titiler-pgstac

* Fix br/gzip header test

* Remove factory (it's included in new titiler-pgstac version)
NASA-IMPACT#148 (comment)

* Remove pin on cramjam

* Load test data in container (actions not updated)

* fix actions tests

* adjust container strategy

* actions connections fix

* remove -it flag (it broke actions)

* re-enable lint, propagate test changes to other actions

* docker-compose -> docker compose

* Fix template import

* include private subnet config

* fix subnet type

* format changes

* pub accessible default true

* remove publicly accessible variable

* Add sql connection and execution abstraction

* Add sql logic to fix projection extension types

* add script to delete null stac_extensions

* Lint and move sql command scripts to support_scripts dir

* Use pythonic naming conventions

* pgstac readme change

* docker compose pgstac version

* pre-deploy action fix

* RDS Proxy initial implementation

* proxy secret + urlllib fix

* use pgstac 0.7.9

* Update set environment

* Add support for permissions boundary

* use titiler custom JSONResponse to handle NaN values

---------

Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: Caden Helbling <caden.helbling@gmail.com>
Co-authored-by: vincentsarago <vincent.sarago@gmail.com>
Co-authored-by: Alexandra Kirk <alexandra@developmentseed.org>

* Update bootstrapper lambda timeout (#11)

* Fix order of db search path (#13)

* Fix order of db search path

* Remove environment from predeploy

* Make RDS instance type configurable (#18)

* Fix missing rds config update (#22)

* Update rds type example config value (#25)

* Fix missing rds config update

* Update rds type config example value

* update collection summaries methods to include max end datetime

* fix docker compose in pr action

* Add DB storage encryption configurability (#34)

* Only add storage_encrypted db config if true (#35)

* use NAT Gateway

* Add TMS route for tile matrices (#40)

* Optionally overwrite host param in stac api (#43)

* Update mappingvalue to custom

* Overwrite host header for both raster and stac apis (#46)

* Overwrite host header for both raster and stac apis

* Add docstrings

* Fix stac config env prefix

* Host param override for api gateway (#48)

* Fetch upstream changes (#51)

* Fix mangum db connection pool

* lint

* Point README to veda-docs (NASA-IMPACT#171)

* Upgrade titiler and titiler-pgstac

* Fix br/gzip header test

* Remove factory (it's included in new titiler-pgstac version)
NASA-IMPACT#148 (comment)

* Remove pin on cramjam

* Load test data in container (actions not updated)

* fix actions tests

* adjust container strategy

* actions connections fix

* remove -it flag (it broke actions)

* re-enable lint, propagate test changes to other actions

* docker-compose -> docker compose

* Fix template import

* include private subnet config

* fix subnet type

* format changes

* pub accessible default true

* remove publicly accessible variable

* Add sql connection and execution abstraction

* Add sql logic to fix projection extension types

* add script to delete null stac_extensions

* Lint and move sql command scripts to support_scripts dir

* Use pythonic naming conventions

* pgstac readme change

* pin to urllib3<2

* pin urllib3 in test

* format

* revert

* pin httpx

* revert

* docker compose pgstac version

* pre-deploy action fix

* RDS Proxy initial implementation

* proxy secret + urlllib fix

* use pgstac 0.7.4

* use titiler custom JSONResponse to handle NaN values

* change db search path

* hacky fix

* try exporting os env vars

* export os env vars

* lint

* comment

* move to APISettings check

* black changes

* hide route

* Pin pydantic for now

* put raster assume role config in lambda construct

* pass lambda env variable bool as string

* NAT SG rules

* order of operation

* remove implicity SGs

* black formatted

* fix invalid string in rule description

* updates to deployment docs in readme

* update pgstac version in docker compose

* use max/end datetime in periodic collection datetime summary

* correct table name typo

* use NAT Gateway instead of EC2 Instance as NAT

* aws_ec2 mistake

* use aws_ec2.SubnetType.PRIVATE_WITH_EGRESS

* add back in NAT count

* add rds subnet group manual removal step to instructions for destroying a stack

* Enable tilematrixsets

* feat(database): RDS Configurations (NASA-IMPACT#218)

* feat(database): RDS Configurations

* chore: string configurations

* feat(database): Configurable RDS Configurations

* consolidate configuration docs in readme

* update table header

---------

Co-authored-by: anayeaye <alexandra@developmentseed.org>

* raster path prefix

* stac path prefix

* format

* config changes

* format

* Revert "format"

This reverts commit 3a26c1c.

* Revert "config changes"

This reverts commit 1425de6.

* feat: add configuration and github actions for python-semantic-release

* feat: lint conventional pr

* ci: cdk deploy should not require approval in github actions workflow

* docs: add contributing.md

* chore(ci): fix typo

* chore: explain versioning init choice

* chore: expalin automated version in setup.py

* chore: typo

Co-authored-by: Jamison French <50224594+jjfrench@users.noreply.github.com>

* docs: typos and improvements in contributing docs

* docs: typos in contributing

* ci: handle semantic version releases with tags only

* chore: add exclude commit patterns from release changelog

* cloudfront in veda backend

* format

* fix: if provided, apply permissions boundary to all roles created within a stack

* mypy, pydocstyle fixes

pydocstyle fixes

format

isort changes

optional config

format

config changes

format

config changes

construct change

* include apigateway parameter mapping

* remove unused import

* update readme

* remove unused config

* docs: s/veda-data-pipelines/veda-data-airflow/

I can't update the diagram b/c it's draw.io.

* include parameter mapping in raster api

* ci: branch specific pre-deploy check on pull request action

* ci: pre-deploy test against dev stack by default

* fix github var name in action

* typo

* include conditional param mapping

* change param mapping kwargs

* fix cloudfront subdomain

* incude missing argument

* formatting

* feat(raster)!: update titiler-pgstac from 0.2.3 to 0.8.0

* add dependencies

* add tests deps

* fix tests

* 🤦

* update handler

* pin pydantic to 1.* for cdk deployment

* fix deps

* fix(raster) use root path for prefixing and add to gateway base url in handler

* fix(stac) use root path for prefixing urls

* root path in fastapi app

* updates to for using a proxy

* lint

* only add additional servers to fast api if root path in config

* root path setting is optional

* cleanup

* cleanup

* do not configure alias for shared subdomain

* pr review cleanup

* fastapi additional servers not needed with apigw base path=app.root_path

* add custom colormap for EPA dataset

* Update raster_api/runtime/src/cmap_data/README.md

* rename cmap

* fix(apis)!: overwrite host in api integration if custom host provided

* fix(apis)!: overwrite host in api integration if custom host provided (p2)

* fix(routes)!: ingest api root path is /api/publish

* change ingestor api cf behavior

* format

* fix(domain)! only create custom api subdomains when configured

* improve config descriptions

* typo

* improve domain config docs

* account for custom staged url of veda-stac-ingestor

* do not run lint conventional pr

* merge duplicate corrected

* lint

* merge mismatched rds config naming

* merge inconsistencies

* merge inconsistencies

---------

Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: vincentsarago <vincent.sarago@gmail.com>
Co-authored-by: ranchodeluxe <greg.corradini@gmail.com>
Co-authored-by: sudobangbang <gregcorradini@gmail.com>
Co-authored-by: Jamison French <50224594+jjfrench@users.noreply.github.com>
Co-authored-by: Pete Gadomski <pete.gadomski@gmail.com>

* Remove unnecessary remanants of merge

* Remove "host" config var

---------

Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: Caden Helbling <caden.helbling@gmail.com>
Co-authored-by: vincentsarago <vincent.sarago@gmail.com>
Co-authored-by: Alexandra Kirk <alexandra@developmentseed.org>
Co-authored-by: Abdelhak Marouane <33136280+amarouane-ABDELHAK@users.noreply.github.com>
Co-authored-by: ranchodeluxe <greg.corradini@gmail.com>
Co-authored-by: sudobangbang <gregcorradini@gmail.com>
Co-authored-by: Jamison French <50224594+jjfrench@users.noreply.github.com>
Co-authored-by: Pete Gadomski <pete.gadomski@gmail.com>

* Fix rds encryption var in example .env file (#55)

---------

Co-authored-by: Julia Signell <jsignell@gmail.com>
Co-authored-by: ividito <isayah@developmentseed.org>
Co-authored-by: smohiudd <saadiq@developmentseed.org>
Co-authored-by: Saadiq Mohiuddin <34844565+smohiudd@users.noreply.github.com>
Co-authored-by: Nathan Zimmerman <npzimmerman@gmail.com>
Co-authored-by: Caden Helbling <caden.helbling@gmail.com>
Co-authored-by: vincentsarago <vincent.sarago@gmail.com>
Co-authored-by: Alexandra Kirk <alexandra@developmentseed.org>
Co-authored-by: Abdelhak Marouane <33136280+amarouane-ABDELHAK@users.noreply.github.com>
Co-authored-by: ranchodeluxe <greg.corradini@gmail.com>
Co-authored-by: sudobangbang <gregcorradini@gmail.com>
Co-authored-by: Jamison French <50224594+jjfrench@users.noreply.github.com>
Co-authored-by: Pete Gadomski <pete.gadomski@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error requesting statistics using a FeatureCollection
4 participants