Fix #13790: Cost analysis Data Insights Optimisations

[OnkarVO7]

Describe your changes:

Fixes #13790

Type of change:

• Bug fix
• Improvement
• New feature
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes <issue-number>: <short explanation>
• I have commented on my code, particularly in hard-to-understand areas.
• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[TeddyCr]

@OnkarVO7 thanks for the PR. Should we add a way to clear that entity_cache once it is not needed (i.e. we finished processing the cost analysis part of things). Otherwise we'll keep these data for the whole lifecycle of the DI workflow. wdyt?

[OnkarVO7]

@OnkarVO7 thanks for the PR. Should we add a way to clear that entity_cache once it is not needed (i.e. we finished processing the cost analysis part of things). Otherwise we'll keep these data for the whole lifecycle of the DI workflow. wdyt?

Yes that makes sense let me clean up the cache once it is not needed

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
1 New issue

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud