Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temporal and other searching #24

Merged
merged 13 commits into from
Feb 25, 2024

Conversation

kdesjard
Copy link
Contributor

For your consideration, a pull request to enable temporal searching based on:

https://docs.ogc.org/is/17-069r4/17-069r4.html#_parameter_datetime

As well arbitrary user-defined fields (via configuration) mapped to columns in the postgis DB.

https://docs.ogc.org/is/17-069r4/17-069r4.html#_parameters_for_filtering_on_feature_properties

…ed into a vec of tuple strings. Extract out the offset, limit, datetime and bbox params as standard items.
…e sql generation to use SQLx's QueryBuilder and handle the temporal field if necessary, and the other columns
…ld. add a temporal type to classify the OGC datetime ranges. Parse the datetime field into its parts with the temporal function
@kdesjard kdesjard force-pushed the temporal_and_other_searching branch from fa7edc5 to 4bd62c8 Compare December 12, 2023 10:33
@pka
Copy link
Contributor

pka commented Dec 15, 2023

That's a great addition, thanks!

Do you have a dataset to test with?

@kdesjard
Copy link
Contributor Author

How do I test against a Pg DB?

@pka
Copy link
Contributor

pka commented Feb 9, 2024

I've added the following test:

    #[tokio::test]
    #[ignore]
    async fn pg_datetime_filter() {
        let filter = FilterParams {
            limit: None,
            offset: None,
            bbox: None,
            datetime: Some("2024-01-01T00:00:00Z".to_string()),
            filters: HashMap::new(),
        };
        let ds = PgDatasource::new_pool("postgresql://t_rex:t_rex@127.0.0.1:5439/t_rex_tests")
            .await
            .unwrap();
        let source = PgCollectionSource {
            ds,
            sql: "SELECT *, '2024-01-01 00:00:00Z'::timestamptz - (fid-1) * INTERVAL '1 day' AS ts FROM ne.ne_10m_rivers_lake_centerlines".to_string(),
            geometry_column: "wkb_geometry".to_string(),
            pk_column: Some("fid".to_string()),
            temporal_column: Some("ts".to_string()),
            other_columns: HashMap::new(),
        };
        let items = source.items(&filter).await.unwrap();
        assert_eq!(items.features.len(), 1);
    }

which can be run with cargo test -- --ignored

The test database must be started with docker run -p 127.0.0.1:5439:5432 -d --name trextestdb --rm sourcepole/trextestdb

If you give me push permissions to your PR, I can add my test directly (see https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork).

@kdesjard
Copy link
Contributor Author

Allow edits is enabled.

@pka
Copy link
Contributor

pka commented Feb 25, 2024

Get an error:

To github.com:kdesjard/bbox.git
 ! [remote rejected] kdesjard-temporal_and_other_searching -> kdesjard-temporal_and_other_searching (permission denied)

Nevermind. Will merge and add my test.

@pka pka merged commit e309bc2 into bbox-services:main Feb 25, 2024
7 of 8 checks passed
@@ -143,17 +157,21 @@ pub struct PgCollectionSource {
geometry_column: String,
/// Primary key column, None if multi column key.
pk_column: Option<String>,
temporal_column: Option<String>,
other_columns: HashMap<String, u8>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does the u8 value mean? Shouldn't it be an array of strings for allowed values?

Copy link
Contributor Author

@kdesjard kdesjard Feb 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pushed another 2 commits to use HashSet instead of HashMap

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants