Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(db)!: update collection summaries on load items #344

Merged
merged 15 commits into from
Mar 28, 2024

Conversation

anayeaye
Copy link
Collaborator

@anayeaye anayeaye commented Mar 26, 2024

What

  • Improve veda loader method that updates the summaries and extents of all items in a collection with a new collection_temporal_extent_max function that includes the maximum item end_datetime which is significantly different from an item's nominal datetime in collections of items that represent an entire year or range or dates.
  • Removes ingest api large number test that is no longer needed because we no longer convert json values for the DDB schema.
  • Pins fastapi at for ingest api tests (tech debt). Note that python-multipart is pinned at the new and secure version so it is still OK for us to use older versions of fastapi and starlette for now.
  • Sets moto version to fix tests.

Note
This requires incrementing VEDA_DB_SCHEMA_VERSION to cause the dashboard schema update to run so I am making it a breaking change.

How tested

I removed the latest date from the geoglam collection's temporal extent and datetime summaries then ingested a single item to /ingestions which triggered both the summaries and extent functions to be executed. Verified in dev.openveda.cloud/api/stac/collections/geoglam.

@anayeaye anayeaye changed the title fix: update collection summaries on load items fix(db)!: update collection summaries on load items Mar 26, 2024
@anayeaye
Copy link
Collaborator Author

Ingest api tests are now failing with the same moto version issue that @stephenkilbourn has been fighting I think. I will dig into that next, too.

@@ -1,11 +1,13 @@
import base64
import json
from datetime import timedelta
from math import isclose

# from math import isclose
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this import line just be deleted?

@anayeaye anayeaye merged commit 6e00fc2 into develop Mar 28, 2024
4 checks passed
@anayeaye anayeaye deleted the fix/update-collection-summaries-on-load-items branch March 28, 2024 17:56
botanical added a commit that referenced this pull request May 6, 2024
### Breaking
- #356

#### Breaking changes notes
Breaking: `VEDA_COGNITO_DOMAIN` configuration now required along with
one time administrator step to update existing user pool client allowed
callback urls with the ingest-api's URL

### Added
- #342
- #330
- #323

### Changed/Updated
- #355
- #340

### Fixed

1. - #367
2. - #365
3. - #361
4. - #360
5. - #358
8. - #345
9. - #344
12. - #339
13. - #338
14. - #337
15. - #335
16. - #334
17. - #331
20. - #329
21. - #327
22. - #326
23. - #325
24. - #324
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants