Fix #13603: Configurable Sample Data Rows for Profiler #13807

ulixius9 · 2023-11-01T08:38:00Z

Describe your changes:

Fixes

I worked on ... because ...

Type of change:

Checklist:

I have read the CONTRIBUTING document.
My PR title is Fixes <issue-number>: <short explanation>
I have commented on my code, particularly in hard-to-understand areas.
For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

github-actions · 2023-11-01T08:44:51Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

github-actions · 2023-11-01T11:47:33Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

github-actions · 2023-11-05T02:18:39Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

github-actions · 2023-11-06T07:46:26Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

github-actions · 2023-11-06T08:07:37Z

Jest test Coverage

UI tests summary

Lines	Statements	Branches	Functions
	49.04% (23516/47953)	30.8% (8810/28603)	30.48% (2680/8792)

github-actions · 2023-11-06T09:33:16Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

github-actions · 2023-11-06T10:37:29Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

github-actions · 2023-11-06T10:44:01Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

github-actions · 2023-11-07T16:28:03Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

github-actions · 2023-11-08T05:40:43Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

github-actions · 2023-11-08T05:51:52Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/DatabaseRepository.java

pmbrull · 2023-11-08T06:45:10Z

...ec/src/main/resources/json/schema/entity/services/connections/database/athenaConnection.json

+    },
+    "sampleDataStorageConfig": {
+      "title": "S3 Config for Sample Data",
+      "$ref": "../connectionBasicType.json#/definitions/sampleDataStorageConfig"


should the services have only the sampleDataStorageConfig or the overall ProfilerConfig?

Ummm I think we only discussed to add the storage config, do we want to add the profiler config here?

ah ok then I might be remembering it wrong. If that's what we agreed then we can move on cc @TeddyCr

ingestion/src/metadata/ingestion/ometa/mixins/es_mixin.py

ingestion/src/metadata/profiler/api/models.py

ingestion/src/metadata/profiler/interface/profiler_interface.py

github-actions · 2023-11-08T13:01:16Z

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

sonarqubecloud · 2023-11-08T13:23:05Z

[open-metadata-ui] Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
1 Code Smell

0.0% Coverage
0.0% Duplication

sonarqubecloud · 2023-11-08T13:38:31Z

[open-metadata-ingestion] Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

75.3% Coverage
0.0% Duplication

* Fix #13603: Configurable Sample Data Rows * Fix #13603: Configurable Sample Data Rows for Profiler * fix table config * support configurable overwriting of sample data * add support for schema and database profiler configuration * chore(ui): put sampleDataStorageConfig under advanced config * fix tests * py format * chore(ui): add sampleDataCount in table profiler config * fix tests * pylint & tests * feat(ui): add profiler settings tab in database and database schema page * chore(ui): show different inputs for profile sample type * schema changes to make default storange config null * add unit test * schema changes to fix api * update profiler setting schema * move profiler settings to manage button * sync locals * fix(ui): unit tests * fix tests * py format * fix lint * minor improvements * chore(ui): update profiler settings schema * resolve review comments * pytest --------- Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>

github-actions bot added Ingestion backend safe to test Add this label to run secure Github workflows on PRs labels Nov 1, 2023

ulixius9 force-pushed the issue-13603-1 branch from 36f8566 to 4b296d1 Compare November 1, 2023 11:40

github-actions bot added the UI UI specific issues label Nov 6, 2023

ulixius9 and others added 16 commits November 8, 2023 11:05

Fix #13603: Configurable Sample Data Rows

d1488e5

Fix #13603: Configurable Sample Data Rows for Profiler

0e565bd

fix table config

b58bad1

support configurable overwriting of sample data

17d333c

add support for schema and database profiler configuration

fbb7c12

chore(ui): put sampleDataStorageConfig under advanced config

fd50670

fix tests

91f34ed

py format

3c619c7

chore(ui): add sampleDataCount in table profiler config

720bba6

fix tests

5ce4e0c

pylint & tests

9e90f4b

feat(ui): add profiler settings tab in database and database schema page

9988618

chore(ui): show different inputs for profile sample type

a02f037

schema changes to make default storange config null

1042083

add unit test

2b35d67

schema changes to fix api

a41b9d4

Sachin-chaurasiya and others added 3 commits November 8, 2023 11:06

sync locals

606c789

fix(ui): unit tests

4e3e2eb

fix tests

b14eadc

ulixius9 force-pushed the issue-13603-1 branch from 5da60bb to b14eadc Compare November 8, 2023 05:36

py format

3838458

fix lint

d0ee708

pmbrull reviewed Nov 8, 2023

View reviewed changes

Sachin-chaurasiya and others added 4 commits November 8, 2023 12:55

minor improvements

21ef958

chore(ui): update profiler settings schema

f58ddcc

Merge branch 'main' into issue-13603-1

f07129c

resolve review comments

cc195f6

pytest

948fa91

ulixius9 requested a review from pmbrull November 8, 2023 13:24

ulixius9 marked this pull request as ready for review November 9, 2023 05:48

ulixius9 requested review from a team, harshach, karanh37 and chirag-madlani as code owners November 9, 2023 05:48

pmbrull approved these changes Nov 9, 2023

View reviewed changes

chirag-madlani approved these changes Nov 9, 2023

View reviewed changes

Sachin-chaurasiya merged commit a8145a8 into main Nov 9, 2023

Sachin-chaurasiya deleted the issue-13603-1 branch November 9, 2023 13:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix #13603: Configurable Sample Data Rows for Profiler #13807

Fix #13603: Configurable Sample Data Rows for Profiler #13807

ulixius9 commented Nov 1, 2023

github-actions bot commented Nov 1, 2023

github-actions bot commented Nov 1, 2023

github-actions bot commented Nov 5, 2023

github-actions bot commented Nov 6, 2023

github-actions bot commented Nov 6, 2023 •

edited

Loading

github-actions bot commented Nov 6, 2023

github-actions bot commented Nov 6, 2023

github-actions bot commented Nov 6, 2023

github-actions bot commented Nov 7, 2023

github-actions bot commented Nov 8, 2023

github-actions bot commented Nov 8, 2023

pmbrull Nov 8, 2023

ulixius9 Nov 8, 2023

pmbrull Nov 8, 2023

github-actions bot commented Nov 8, 2023

sonarqubecloud bot commented Nov 8, 2023

sonarqubecloud bot commented Nov 8, 2023

Fix #13603: Configurable Sample Data Rows for Profiler #13807

Fix #13603: Configurable Sample Data Rows for Profiler #13807

Conversation

ulixius9 commented Nov 1, 2023

Describe your changes:

Type of change:

Checklist:

github-actions bot commented Nov 1, 2023

github-actions bot commented Nov 1, 2023

github-actions bot commented Nov 5, 2023

github-actions bot commented Nov 6, 2023

github-actions bot commented Nov 6, 2023 • edited Loading

Jest test Coverage

UI tests summary

github-actions bot commented Nov 6, 2023

github-actions bot commented Nov 6, 2023

github-actions bot commented Nov 6, 2023

github-actions bot commented Nov 7, 2023

github-actions bot commented Nov 8, 2023

github-actions bot commented Nov 8, 2023

pmbrull Nov 8, 2023

Choose a reason for hiding this comment

ulixius9 Nov 8, 2023

Choose a reason for hiding this comment

pmbrull Nov 8, 2023

Choose a reason for hiding this comment

github-actions bot commented Nov 8, 2023

sonarqubecloud bot commented Nov 8, 2023

sonarqubecloud bot commented Nov 8, 2023

github-actions bot commented Nov 6, 2023 •

edited

Loading