Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add regex extract functionality to comparisons #1203

Merged
merged 37 commits into from
May 10, 2023
Merged

Add regex extract functionality to comparisons #1203

merged 37 commits into from
May 10, 2023

Commits on Apr 24, 2023

  1. Added option to include a regex match to exact match level base

    @zslade
    zslade committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    a0c8c68 View commit details
    Browse the repository at this point in the history

  2. added function to produce sql string with accepted duckdb regex extra… 

    …ct syntax
    @zslade
    zslade committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    3e27cff View commit details
    Browse the repository at this point in the history

  3. added function to produce sql string with accepted spark regex extrac… 

    …t syntax
    @zslade
    zslade committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    5e2410f View commit details
    Browse the repository at this point in the history

  4. updated with error message to user if regex extract function doesn't … 

    …exist in sql backend
    @zslade
    zslade committed Apr 24, 2023
    Configuration menu
    Copy the full SHA
    387c414 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2023

  1. 0 not required for duckdb backend

    @zslade
    zslade committed Apr 25, 2023
    Configuration menu
    Copy the full SHA
    c819cf5 View commit details
    Browse the repository at this point in the history

  2. Updated regex function so that can be used with string comparitors an… 

    …d reverse match levels
    @zslade
    zslade committed Apr 25, 2023
    Configuration menu
    Copy the full SHA
    86a871c View commit details
    Browse the repository at this point in the history

  3. updated string comparitor levels and reverse match level to accept re… 

    …gex extract option
    @zslade
    zslade committed Apr 25, 2023
    Configuration menu
    Copy the full SHA
    03c6214 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2023

  1. Added option to perform a regex match

    @zslade
    zslade committed Apr 26, 2023
    Configuration menu
    Copy the full SHA
    8582f11 View commit details
    Browse the repository at this point in the history

  2. required regex parameter to be str

    @zslade
    zslade committed Apr 26, 2023
    Configuration menu
    Copy the full SHA
    bcc98aa View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2023

  1. Regex extract sql added to Athena backend

    @zslade
    zslade committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    acf9e95 View commit details
    Browse the repository at this point in the history

  2. Updated error message

    @zslade
    zslade committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    ff2d7a6 View commit details
    Browse the repository at this point in the history

  3. added error message for spark when user enters python escape characte… 

    …r in regex pattern
    @zslade
    zslade committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    2738ab3 View commit details
    Browse the repository at this point in the history

  4. Updated doc strings with examples for using regex_extract option

    @zslade
    zslade committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    c7e985d View commit details
    Browse the repository at this point in the history

Commits on May 2, 2023

  1. removed erroneous bracket

    @zslade
    zslade committed May 2, 2023
    Configuration menu
    Copy the full SHA
    e57ab69 View commit details
    Browse the repository at this point in the history

  2. updated scripts to latest versions to work with Athena linker

    @zslade
    zslade committed May 2, 2023
    Configuration menu
    Copy the full SHA
    2f7730d View commit details
    Browse the repository at this point in the history

Commits on May 4, 2023

  1. Added tests for regex levels for duckdb and spark linker

    @zslade
    zslade committed May 4, 2023
    Configuration menu
    Copy the full SHA
    205c6b8 View commit details
    Browse the repository at this point in the history

  2. Added test for invalid regex

    @zslade
    zslade committed May 4, 2023
    Configuration menu
    Copy the full SHA
    930f221 View commit details
    Browse the repository at this point in the history

  3. Linted and formatted script test script

    @zslade
    zslade committed May 4, 2023
    Configuration menu
    Copy the full SHA
    3105f51 View commit details
    Browse the repository at this point in the history

Commits on May 5, 2023

  1. linted and formatted

    @zslade
    zslade committed May 5, 2023
    Configuration menu
    Copy the full SHA
    08c622e View commit details
    Browse the repository at this point in the history

Commits on May 7, 2023

  1. Added option valid_string_regex to treat col as null if no match on… 

    … regex pattern
    @zslade
    zslade committed May 7, 2023
    Configuration menu
    Copy the full SHA
    2348bc1 View commit details
    Browse the repository at this point in the history

  2. Updated comparisons with valid_string_regex option

    @zslade
    zslade committed May 7, 2023
    Configuration menu
    Copy the full SHA
    93f1489 View commit details
    Browse the repository at this point in the history

  3. removed comments

    @zslade
    zslade committed May 7, 2023
    Configuration menu
    Copy the full SHA
    18d6041 View commit details
    Browse the repository at this point in the history

  4. linted

    @zslade
    zslade committed May 7, 2023
    Configuration menu
    Copy the full SHA
    c8eb64b View commit details
    Browse the repository at this point in the history

  5. linted

    @zslade
    zslade committed May 7, 2023
    Configuration menu
    Copy the full SHA
    e584e2b View commit details
    Browse the repository at this point in the history

  6. add valid_string_regex option to date comparison

    @zslade
    zslade committed May 7, 2023
    Configuration menu
    Copy the full SHA
    3d0faca View commit details
    Browse the repository at this point in the history

  7. Added a check for regex_extract and valid_string_regegx parameters

    @zslade
    zslade committed May 7, 2023
    Configuration menu
    Copy the full SHA
    a14137d View commit details
    Browse the repository at this point in the history

  8. linted scripts

    @zslade
    zslade committed May 7, 2023
    Configuration menu
    Copy the full SHA
    ad293b5 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2023

  1. Merge branch 'master' into regex_extract

    @RossKen
    RossKen committed May 9, 2023
    Configuration menu
    Copy the full SHA
    622acd1 View commit details
    Browse the repository at this point in the history

  2. Update docs and add null level for dates

    @RossKen
    RossKen committed May 9, 2023
    Configuration menu
    Copy the full SHA
    8c1471a View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into regex_extract

    @RossKen
    RossKen committed May 9, 2023
    Configuration menu
    Copy the full SHA
    efc2f27 View commit details
    Browse the repository at this point in the history

  4. revert athena linker rebase error

    @RossKen
    RossKen committed May 9, 2023
    Configuration menu
    Copy the full SHA
    787b72c View commit details
    Browse the repository at this point in the history

  5. fix up template library and null level

    @RossKen
    RossKen committed May 9, 2023
    Configuration menu
    Copy the full SHA
    9cc5ddd View commit details
    Browse the repository at this point in the history

Commits on May 10, 2023

  1. add tests for date format parsing

    @RossKen
    RossKen committed May 10, 2023
    Configuration menu
    Copy the full SHA
    5afe4e6 View commit details
    Browse the repository at this point in the history

  2. linting

    @RossKen
    RossKen committed May 10, 2023
    Configuration menu
    Copy the full SHA
    dd6803b View commit details
    Browse the repository at this point in the history

  3. lint with black

    @RossKen
    RossKen committed May 10, 2023
    Configuration menu
    Copy the full SHA
    9f664b3 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'regex_extract' of github.com:moj-analytical-services/sp… 

    …link into regex_extract
    @RossKen
    RossKen committed May 10, 2023
    Configuration menu
    Copy the full SHA
    e44e321 View commit details
    Browse the repository at this point in the history

  5. reinstate cll testing

    @RossKen
    RossKen committed May 10, 2023
    Configuration menu
    Copy the full SHA
    9501ede View commit details
    Browse the repository at this point in the history