Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust datetime filter for STAC search #515

Open
ividito opened this issue Apr 20, 2023 · 2 comments
Open

Adjust datetime filter for STAC search #515

ividito opened this issue Apr 20, 2023 · 2 comments

Comments

@ividito
Copy link

ividito commented Apr 20, 2023

When filtering a STAC /search by datetime, the method currently being used will not always return desired results (veda-backend/#170). Instead of using a filter to refine /search requests, we should use the datetime parameter, which will find overlapping data instead of data where the start_datetime falls within the search range.

For example, the following request:

curl --request POST \
  --url https://staging-stac.delta-backend.com/search \
  --header 'content-type: application/json' \
  --data '{"filter-lang":"cql2-json","filter":{"op":"and","args":[{"op":">=","args":[{"property":"datetime"},"2085-02-14T00:00:00.000Z"]},{"op":"<=","args":[{"property":"datetime"},"2085-02-14T23:59:59.999Z"]},{"op":"eq","args":[{"property":"collection"},"CMIP245-winter-median-ta"]}]},"limit":500}'

Should be converted to:

curl --request POST \
  --url https://staging-stac.delta-backend.com/search \
  --header 'content-type: application/json' \
  --data '{"filter-lang":"cql2-json","datetime": "2085-02-14T00:00:00Z/2085-02-14T23:59:59Z","filter":{"op":"and","args":[{"op":"eq","args":[{"property":"collection"},"CMIP245-winter-median-ta"]}]},"limit":500}'

This change should also avoid the need to merge veda-config/#240

@anayeaye
Copy link

@ividito it just now occurred to me to check if this collection used a nominal datetime or a time range (start/end). It looks like a range was used for the item datetime properties instead of a single nominal datetime. As in:

https://staging-stac.delta-backend.com/collections/CMIP245-winter-median-ta/items/CMIP6ensembleMedian_diff_winter_tas_ssp245_20850214.cog
Item id CMIP6ensembleMedian_diff_winter_tas_ssp245_20850214.cog
start_datetime 2085-01-01T00:00:00+00:00
end_datetime 2085-03-31T12:00:00+00:00
datetime not defined

So in this case pgstac is behaving as expected for the filter search. If a nominal datetime of 2085-02-14T00:00:00.000Z had been used for this item, the cql2-json filter would catch it. For the current items the filter needs to use the start and end datetime properties. Restructuring the request for this individual collection to use the base stac-api datetime parameter is a good solution.

@j08lue
Copy link
Contributor

j08lue commented Aug 19, 2024

Looks like the old method is still being used

export function getFilterPayload(date: Date, collection: string) {
return {
op: 'and',
args: [
{
op: '>=',
args: [{ property: 'datetime' }, userTzDate2utcString(startOfDay(date))]
},
{
op: '<=',
args: [{ property: 'datetime' }, userTzDate2utcString(endOfDay(date))]
},
{
op: 'eq',
args: [{ property: 'collection' }, collection]
}
]
};
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants