Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move table and regex option to dpl #414

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

elliVM
Copy link
Contributor

@elliVM elliVM commented Nov 14, 2024

Feature

Support for teragrep exec bloom create/update to get spark configuration options dpl.pth_06.bloom.pattern and dpl.pth_06.bloom.table.name from dpl.

Example dpl after change

| teragrep exec bloom create table 'table_name' regex '\w{8}' estimates 'R_estimate(tokens)' input 'tokens'

Changes

  • use pth-03 version 9.3.0
  • move table and regex options from fetching from config to TeragrepTransformation
  • remove helper methods tableName(), journalDBname() and pattern() from FilterTypes object
  • refactor responsibility of saving filter types to database to FilterTypes object from helper method
  • add JournalDBName class to fetch journaldb name from config
  • update and add missing equals/hashCode methods

@elliVM elliVM requested a review from 51-code November 14, 2024 13:34
@elliVM elliVM self-assigned this Nov 14, 2024
@elliVM elliVM requested a review from 51-code November 19, 2024 08:47
Copy link

@51-code 51-code left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@51-code 51-code requested a review from eemhu November 20, 2024 08:24
Copy link
Contributor

@eemhu eemhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, LGTM. I think the ContextValue seems like a good idea.

…m bloom filter table, secondary constructor for Pattern.compile
@elliVM elliVM requested a review from eemhu November 20, 2024 10:27
@elliVM elliVM requested a review from kortemik November 20, 2024 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants