Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Control for equailty fIlter for Mapping rule & table #1863

Closed
benedeki opened this issue Jul 26, 2021 · 1 comment · Fixed by #1929
Closed

Control for equailty fIlter for Mapping rule & table #1863

benedeki opened this issue Jul 26, 2021 · 1 comment · Fixed by #1929
Assignees
@dk1844
Labels
feature New feature Menas Menas component affected priority: medium Important but not urgent UX Something concerning UX

Comments

@benedeki
Copy link
Collaborator

benedeki commented Jul 26, 2021 •
edited
Loading

Background

Mapping tables are possible to filter. Either on the level of the table itself or within the Mapping rule using it. It's not possible to set up the filter in Menas though yet.

Feature

Create a control within Mapping table setup and Mapping Conformance rule edit that will allow to add a filter using the equality operator and a (constant) value, with choosing the value type.
@benedeki benedeki added feature New feature Menas Menas component affected UX Something concerning UX priority: undecided Undecided priority to be assigned after discussion priority: medium Important but not urgent and removed priority: undecided Undecided priority to be assigned after discussion labels Jul 26, 2021
@dk1844 dk1844 self-assigned this Aug 10, 2021
dk1844 added a commit that referenced this issue Aug 25, 2021
@dk1844
#1863 mt filter is shown in a field and in a TreeTable. Labels are cu…
f4a2b7c 
…stomized (OR, AND, ==, !=, NOT, isNull) recursively. Very crude.
dk1844 added a commit that referenced this issue Aug 25, 2021
@dk1844
#1863 mt filter is shown in a field (debug )and in a sap.m.Tree. Labe…
2fb152e 
…ls and iconds are customized (OR, AND, ==, !=, NOT, isNull) recursively.
dk1844 added a commit that referenced this issue Aug 26, 2021
@dk1844
#1863 treeTable binding problem
59b7161
dk1844 added a commit that referenced this issue Aug 26, 2021
@dk1844
#1863 treeTable binding problem fixed
3b29458
dk1844 added a commit that referenced this issue Aug 26, 2021
@dk1844
#1863 existing filter update works
d666f97
dk1844 added a commit that referenced this issue Aug 27, 2021
@dk1844
#1863 very crudely implemented addAND and removeSelected filter on Fi…
4f3d2b0 
…lter TreeTable
dk1844 added a commit that referenced this issue Aug 27, 2021
@dk1844
#1863 very crudely implemented MenuButton with filter/treeTable opera…
0e3d1a9 
…tions. Need to fix the default values to be empty/notAllowed on UI?
dk1844 added a commit that referenced this issue Sep 13, 2021
@dk1844
#1863 editFilterTree has human-readable label, filterData utils exist…
1f5f218 
… for in-depth processing
dk1844 added a commit that referenced this issue Sep 15, 2021
@dk1844
#1863 editFilterTree handles empty filter (filling->saving)
a829474
dk1844 added a commit that referenced this issue Sep 15, 2021
@dk1844
#1863 editFilterTree handles empty filter (emptying->saving) + newly …
cc52a3b 
…added child gets expanded by default,

some fields renaming.
dk1844 added a commit that referenced this issue Sep 15, 2021
@dk1844
#1863 editFilterTree handles empty filter (emptying->saving) + newly …
b131873 
…added child gets expanded by default,

some fields renaming.
dk1844 added a commit that referenced this issue Sep 16, 2021
@dk1844
#1863 mt filter tree (viewing) is mapped on model declaratively;
17035fe 
 default expanded level for filter edit = 1
dk1844 added a commit that referenced this issue Sep 16, 2021
@dk1844
#1863 mt filter tree edit: column type hinting from a predefined list…
3e8747b 
… (but open to new values)
dk1844 added a commit that referenced this issue Sep 16, 2021
@dk1844
#1863 mt filter tree edit: showing/hiding input is based on type (exp…
9ab6e37 
…ression-binding impl in declarative xml)
dk1844 added a commit that referenced this issue Sep 21, 2021
@dk1844
#1863 mt filter tree edit: bugfix: blankFilter behavior fix - after r…
926f76c 
…emoval, the internal state could be [null]
dk1844 added a commit that referenced this issue Sep 21, 2021
@dk1844
#1863 mt filter tree edit: empty cName/value/valueType + placeholders
6556500
dk1844 added a commit that referenced this issue Sep 21, 2021
@dk1844
#1863 mt filter tree edit: validation of the tree - todo needs retest…
26c0dbf 
…ing and validation cleanup/reuse

 - validation uses extended model fields: entity>{filter|columnName|value|valueType}_valueState(Text)
dk1844 added a commit that referenced this issue Sep 21, 2021
@dk1844
#1863 mt filter tree edit: filter reset reusable
0321f81
dk1844 added a commit that referenced this issue Sep 21, 2021
@dk1844
#1863 mt filter tree edit: cleanup
25f821e
dk1844 added a commit that referenced this issue Sep 22, 2021
@dk1844
#1863 mt filter tree edit: validateFilterData extracted
83f186b
dk1844 added a commit that referenced this issue Sep 23, 2021
@dk1844
#1863 cr filter view: added the view, uses the same routine as mf fil…
e4ee1be 
…ter (generalized into FilterTreeUtils.js)

mt filter tree view: expanded to 1 level
dk1844 added a commit that referenced this issue Sep 23, 2021
@dk1844
#1863 cr filter view: edit submit works. Validation is missing.
e96383c 
 - CR filter is done using a fragment (non-shared with MT)
 - MappingTableDialog.removeNiceNamesFromFilterData moved to FilterTreeUtils.removeNiceNamesFromFilterData
dk1844 added a commit that referenced this issue Sep 29, 2021
@dk1844
#1863 cr filter edit, mt filter edit: add icons corrected
c797ac3
dk1844 added a commit that referenced this issue Oct 1, 2021
@dk1844
#1863 mapping cr & mt fitler successfully reuses the same fragment (b…
2a3a254 
…oth using the same named model)

 - todo reuse validation, reuse manipulation methods
dk1844 added a commit that referenced this issue Oct 5, 2021
@dk1844
#1863 FilterEdit.js allows reusing filterEdit TreeTable logic between…
7f0f1c9 
… mCR and MT editings
dk1844 added a commit that referenced this issue Oct 5, 2021
@dk1844
#1863 mCT editing validation enabled (commons from FilterEdit.js)
2bab4db
dk1844 added a commit that referenced this issue Oct 5, 2021
@dk1844
#1863 mCT datatype hinting hinting enabled (commons from DataTypeUtil…
0e539df 
…s.js)
dk1844 added a commit that referenced this issue Oct 5, 2021
@dk1844
#1863 mCR/MT edit dialog default width=950px, some cleanup
a278969
dk1844 added a commit that referenced this issue Oct 5, 2021
@dk1844
#1863 mCR/MT tree view with no data - corrected UX
1865651
dk1844 added a commit that referenced this issue Oct 5, 2021
@dk1844
#1863 mCR/MT tree view - debug filterJson removed
604cbe4
dk1844 added a commit that referenced this issue Oct 6, 2021
@dk1844
#1863 cleanup
b8e5d89
dk1844 added a commit that referenced this issue Oct 6, 2021
@dk1844
#1863 bugfixes: directly creating MT with filter (fix on accepting th…
3cfc458 
…e field), UI fix for MT filter model initialization
@dk1844 dk1844 mentioned this issue Oct 6, 2021
Merged
@dk1844 dk1844 linked a pull request Oct 6, 2021 that will close this issue
Feature/1863 mapping table filtering #1929
Merged
dk1844 added a commit that referenced this issue Oct 6, 2021
@dk1844
#1863 npm audit fix
c10fbc5
dk1844 added a commit that referenced this issue Oct 7, 2021
@dk1844
#1863 bugfix: adding new mCR (when no edit MCR dialog has been opened…
cda0810 
… yet) did not work - fixed
dk1844 added a commit that referenced this issue Oct 13, 2021
@dk1844
#1863 selecting mapping column from MT schema works (for all schema l…
bb6bd1c 
…evels) for edit. TODO = Schema type support
dk1844 added a commit that referenced this issue Oct 13, 2021
@dk1844
#1863 (only for MT, mCR is now broken)
964f2eb 
 - filter type (when applied) is derived from selectedSchema on submit
 - schema selection changed to a single object instead of /schemaName & /schemaVersion - because these have been set individually and were inconsistent at between-times (applies to all schema-selection-based dialogs)
dk1844 added a commit that referenced this issue Oct 15, 2021
@dk1844
#1863 mCR - schema-based columns suggested for filter, value types fi…
b62b7ef 
…lled in silently during submit, too.
dk1844 added a commit that referenced this issue Oct 15, 2021
@dk1844
#1863 bugfix: empty MT - schema may be empty
09a4834
dk1844 added a commit that referenced this issue Oct 18, 2021
@dk1844
#1863 bugfix: removing a filter left a null node - cleanup was needed…
adb2b4d 
… (otherwise view would fail)

logging cleanup
dk1844 added a commit that referenced this issue Oct 18, 2021
@dk1844
#1863 unreachable code cleanup
96124c1
dk1844 added a commit that referenced this issue Oct 18, 2021
@dk1844
#1863 select list item now shows valueType as additionalText, cleanup
377db34
dk1844 added a commit that referenced this issue Oct 19, 2021
@dk1844
#1863 nonEmptyAndNonNullFilled - map->filter bug fixed.
ba5fc1a
dk1844 added a commit that referenced this issue Oct 21, 2021
@dk1844
#1863 typo for null filter
d1cbec8
dk1844 added a commit that referenced this issue Oct 22, 2021
@dk1844 @benedeki
Feature/1863 mapping table filtering (#1929)
24a7fba 
* #1863 mapping cr & mt fitler successfully reuses the same fragment (both using the same named model)
 - todo reuse validation, reuse manipulation methods

* #1863 FilterEdit.js allows reusing filterEdit TreeTable logic between mCR and MT editings

* #1863 mCT editing validation enabled (commons from FilterEdit.js)

* #1863 mCT datatype hinting hinting enabled (commons from DataTypeUtils.js)

* #1863 mCR/MT edit dialog default width=950px, some cleanup
* #1863 bugfixes: directly creating MT with filter (fix on accepting the field), UI fix for MT filter model initialization

* #1863 npm audit fix

* #1863 bugfix: adding new mCR (when no edit MCR dialog has been opened yet) did not work - fixed

* #1863 selecting mapping column from MT schema works (for all schema levels) for edit. TODO = Schema type support

 #1863 mCR - schema-based columns suggested for filter, value types filled in silently during submit, too.

* #1863 bugfix: empty MT - schema may be empty

* #1863 bugfix: removing a filter left a null node - cleanup was needed (otherwise view would fail)
logging cleanup

* #1863 select list item now shows valueType as additionalText, cleanup

* #1863 nonEmptyAndNonNullFilled - map->filter bug fixed.

* #1863 typo for null filter

Co-authored-by: David Benedeki <14905969+benedeki@users.noreply.github.com>
@dk1844
Copy link
Contributor

dk1844 commented Oct 22, 2021

Release notes
Mapping table UI and MappingConformance UI now show mapping filters set and offer controls to edit these filters, too.

dk1844 added a commit that referenced this issue Nov 4, 2021
@dk1844 @benedeki
@Zejnilovic @AdrianOlosutean @yruslan
Merge/merging release 2.25 into develop ver 3 (#1963)
10213a2 
* Update for next development version 2.24.0-SNAPSHOT

* Suppress download noise in license check

* Suppress compiler warning of obsolete Java (#1892)

* 1868 statistics with missing counts and datasets missing proprties (#1873)

* 1868 statistics with missing counts and datasets missing proprties

* 1843 Summary page for properties (#1880)

* 1843 Home page with properties,  side panel with missing counts and summary page for properties with tab containing datasets missing that particular property

* Feature/1603 mapping table filtering general (#1879)

* #1603 serde tests for CR and MT DataFrameFilters
(mongo-bson-based serde tests for CR and MT DataFrameFilters, mongo-bson-based serde tests extended for CR with a blank mappingTableFilter)

* #1909 Increase the limit of columns shown in menas column selection

* 1903 Add validation for complex default values in mapping tables on import

* Project config and management updates (#1908)

Project config and management updates
* poc issue template
* CODEOWNERS update
* developers update
* Badges to README.md

* 1881 HyperConformance enceladus_info_version from payload  (#1896)

1881 HyperConformance enceladus_info_version from payload

* #1887 defaultTimestampTimeZone can be source type specific (#1899)

#1887 defaultTimestampTimeZone can be source type specific
* `DefaultsByFormat` extends the `Defaults` trait, being able to read defaults from configuration files
* `DefaultsByFormat` offers further granularity by first checking the format specific setting only then taking the global one
* Basic `GlobalDefaults` are not configuration dependent anymore
* Standardization now user `DefaultsByFormat` for its defaults, where rawFormat is used for format parameter
* Switched to configuration path to be `enceladus.defaultTimestampTimeZone.default` and `enceladus.defaultTimestampTimeZone.[rawFormat]` respectively
* `defaultTimestampTimeZone` is still supported/read as an obsolete fallback
Co-authored-by: Daniel K <dk1844@gmail.com>

* #1887 defaultTimestampTimeZone can be source type specific (#1916)

#1887 defaultTimestampTimeZone can be source type specific
* rename of the configuration prefix from `enceladus.` to `standardization.`

* #172 Save original timezone information in metadata file (#1900)

* Upgrade of Atum to 3.6.0
* Writing the default time zones for timestamps and dates into _INFO file

* #1894 `HadoopFsPersistenceFactory` - adding Spline S3 write support (#1912)

* #1894 Spline S3 support via custom persistence factory `HadoopFsPersistenceFactory`.
Co-authored-by: David Benedeki <14905969+benedeki@users.noreply.github.com>

* Update versions for release v2.24.0

* Update for next development version 2.25.0-SNAPSHOT

* #1926 Add executor extra java opts to helper scripts

* #1931 Add switch for running kinit in helper scripts

* #1882 Update Cobrix dependency to v.2.3.0

* #1882 Remove explicit "collapse_root" since it is the default since Cobrix 2.3.0

* #1882 Update Cobrix to 2.4.1 and update Cobol test suite for ASCII files.

* #1882 Bump up Cobrix version to 2.4.2.

* #1927 Spline _LINEAGE and Atum _INFO files permission alignment (#1934)

* #1927 - testing setup: set both spline _LINEAGE and atum _INFO to hdfs file permissions 733 -> the result on EMR HDFS was 711 (due to 022 umask there) -> evidence of working

* #1927 - cleanup of test settings of 733 fs permissions

* #1927 Atum final version 3.7.0 used instead of the snapshot (same code)

* #1927 comment change

* #1927 - default 644 FS permissions for both _INFO and _LINEAGE files.

* 1937 limit output file size (#1941)

* 1937 limit output file size

* 1937 limit output file size

* 1937 renamings + constants

* 1937 more conditions

* 1937 rename params

* 1937 feedback + script params

* 1937 more feedback

* 1937 final feedback

* #1951: Windows Helper scripts - add missing features
* `ADDITIONAL_JVM_EXECUTOR_CONF`
* Kerberos configuration
* Trust store configuration
* kinit execution option
* `--min-processing-block-size` & `--max-processing-block-size`
* logo improvement

* * --min-processing-block-size -> --min-processing-partition-size
* --max-processing-block-size -> --max-processing-partition-size

* #1869: SparkJobs working with LoadBalanced Menas (#1935)

* `menas.rest.retryCount` - configuration, how many times an url should be retried if failing with retry-able error implemented
* `menas.rest.availability.setup` - configuration, how the url list should be handled
* _Standardization_, _Conformance_ and _HyperConformance_ changed to provide retry count and availability setup to Dao, read from configuration
* `ConfigReader` enhanced and unified to read configurations more easily and universally
* Mockito upgraded to 1.16.42

Co-authored-by: Daniel K <dk1844@gmail.com>

* Feature/1863 mapping table filtering (#1929)

* #1863 mapping cr & mt fitler successfully reuses the same fragment (both using the same named model)
 - todo reuse validation, reuse manipulation methods

* #1863 FilterEdit.js allows reusing filterEdit TreeTable logic between mCR and MT editings

* #1863 mCT editing validation enabled (commons from FilterEdit.js)

* #1863 mCT datatype hinting hinting enabled (commons from DataTypeUtils.js)

* #1863 mCR/MT edit dialog default width=950px, some cleanup
* #1863 bugfixes: directly creating MT with filter (fix on accepting the field), UI fix for MT filter model initialization

* #1863 npm audit fix

* #1863 bugfix: adding new mCR (when no edit MCR dialog has been opened yet) did not work - fixed

* #1863 selecting mapping column from MT schema works (for all schema levels) for edit. TODO = Schema type support

 #1863 mCR - schema-based columns suggested for filter, value types filled in silently during submit, too.

* #1863 bugfix: empty MT - schema may be empty

* #1863 bugfix: removing a filter left a null node - cleanup was needed (otherwise view would fail)
logging cleanup

* #1863 select list item now shows valueType as additionalText, cleanup

* #1863 nonEmptyAndNonNullFilled - map->filter bug fixed.

* #1863 typo for null filter

Co-authored-by: David Benedeki <14905969+benedeki@users.noreply.github.com>

* Update versions for release v2.25.0

* [merge] build fix

* [merge] npm audit fix

* [merge] npm audit fix

* [merge] buildfix (menas->rest_api packaging fix)

* [merge] review updates

Co-authored-by: David Benedeki <benedeki@volny.cz>
Co-authored-by: Saša Zejnilović <zejnils@gmail.com>
Co-authored-by: David Benedeki <14905969+benedeki@users.noreply.github.com>
Co-authored-by: Adrian Olosutean <adi.olosutean@gmail.com>
Co-authored-by: Ruslan Iushchenko <yruslan@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dk1844 dk1844

Labels
feature New feature Menas Menas component affected priority: medium Important but not urgent UX Something concerning UX
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Feature/1863 mapping table filtering AbsaOSS/enceladus
2 participants
@dk1844 @benedeki