Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] Proposal for fixing datetime query examples (#110) #111

Merged
merged 2 commits into from
Jun 4, 2024

Conversation

MCRE-BE
Copy link
Contributor

@MCRE-BE MCRE-BE commented Jun 3, 2024

Wanted to try address #110 :)
If it helps to save you some time.

I would also have changed the Wiki, but I can't easily fork it. I would have changed it to:

The utility allows users to run some simple SQL-like queries on the Parquet data.

Query Syntax

The syntax for queries can be found by clicking on the Filter Query (?) label:

The syntax is very similar to SQL except for how Dates are handled. Example query formats can be found below:

Type of Data Example(s)
NULL Check WHERE field_name IS NULL
WHERE field_name IS NOT NULL
Datetime WHERE field_name >= #2000/12/31#
Numeric WHERE field_name <= 123.4
WHERE field_name <> 10
String WHERE field_name LIKE '%value%'
WHERE field_name = 'equals value'
WHERE field_name <> 'not equals'
Using Multiple Conditions WHERE (field_1 > #2000/12/31# AND field_1 < #2001/12/31#) OR field_2 <> 100 OR field_3 = 'string value'
List/Map/Struct WHERE my_list LIKE '%elem1%'
WHERE my_map = '(a,1)'
WHERE my_map LIKE '%key_or_value%'
WHERE my_struct LIKE '%field_or_value%'

Note: List, Map, and Struct fields are automatically cast to String type for querying.
Note: The following datetime structures are accepted yyyy/MM/dd and the North American MM/dd/yyyy. (according to stackoverflow

Escaping field names

You will need to escape field names with square brackets if they contain spaces or punctuation:

WHERE [field with spaces and punctuation!] <> 'not equals'

Running the query

The query can be entered in the Query Box located at the top of the UI:

To execute you may either hit Enter or click the Execute button. The grid below will be updated with your results. This can be verified by looking at the bottom-left side of the status bar which will show how many records have been filtered by the query:

Before Query After Query

The Clear button will only remove the filter from the results in the grid below and will not clear the query text that you have entered. You may hit the Esc key while editing the query to quickly clear any existing query filters.

Before Clear After Clear

It should be noted that queries will not run against the entire Parquet file but rather only the records that have been loaded into memory. For more information please see Query Scope.

Query Scope

Currently, queries that are run only apply to records that have been loaded into the application (First 1000 records by default). To run your queries against more records you must increase the Record Count so that more data from the Apache Parquet file is loaded into the application.

Loading more data into the application requires more RAM so this might become troublesome for really large files. See Tips For Large Files for some hints on how to deal with that.

@mukunku mukunku changed the title [FIX] Proposal for fixing datetime query examples [FIX] Proposal for fixing datetime query examples (#110) Jun 4, 2024
@mukunku mukunku merged commit b94c733 into mukunku:v3.0.0-release Jun 4, 2024
1 check failed
@mukunku
Copy link
Owner

mukunku commented Jun 4, 2024

Thanks for the help. I'll make sure this gets into the next release. I also updated the wiki now.

@MCRE-BE
Copy link
Contributor Author

MCRE-BE commented Jun 4, 2024

No problem. I didn't know how I could test the changes though 🤷‍♂️

@MCRE-BE MCRE-BE deleted the datetime_query_fix branch August 5, 2024 06:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants