Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate airbyte repo with SonarQube #7881

Closed
6 tasks done
Tracked by #8400
keu opened this issue Nov 11, 2021 · 2 comments · Fixed by #8362 or #8668
Closed
6 tasks done
Tracked by #8400

Integrate airbyte repo with SonarQube #7881

keu opened this issue Nov 11, 2021 · 2 comments · Fixed by #8362 or #8668
Assignees
Labels
area/connectors Connector related issues type/enhancement New feature or request

Comments

@keu
Copy link
Contributor

keu commented Nov 11, 2021

Tell us about the problem you're trying to solve

Many issues we have nowadays can be avoided even with static analysis.

We're going to evaluate the code quality using SonarQube. Using that tool we would be able to see all connectors' info in one place and have a good overview of the project code quality.

Describe the solution you’d like

We're going to generate the SonarQube analytics, base on our tests, pylint and other inner code analytics tools.

To do that we need to implement the following steps:

  • Fix github CI to make each job run proper and stable (for now it could fail with unknown reasons, so we could not rely on it).
  • Paste together all possible code validation reports (such as pylint, unit tests, flake etc.) and generate the SonarQube assessments based on it. So we don't need to run the code evaluation from the SonarQube itself, just pass the reports there.

Subtasks

Describe the alternative you’ve considered or used

No alternative with given flow

@keu keu added the type/enhancement New feature or request label Nov 11, 2021
@sherifnada sherifnada added the area/connectors Connector related issues label Nov 15, 2021
@VasylLazebnyk VasylLazebnyk changed the title Integrate airbyte repo with SonarQube or CodeClimate Integrate airbyte repo with SonarQube Dec 2, 2021
@VasylLazebnyk VasylLazebnyk added this to the Connectors Dec 10 2021 milestone Dec 2, 2021
@antixar
Copy link
Contributor

antixar commented Dec 7, 2021

Follow up issue https://github.com/airbytehq/airbyte-cloud/issues/659.
This task has to close posible security variabilities in the future.
About the future SonarQube projects. The Airbyte repo is aggregation of several stacks, languages and services. And one project will be not enough. I propose to classify projects by language and functional target:

  1. Airbyte Python Connectors
  2. Airbyte Java Connectors
  3. Airbyte WebApp
  4. Airbyte Server
  5. Airbyte Workers
    etc
    Every project will have own properties and test scenarios.

@antixar antixar linked a pull request Dec 7, 2021 that will close this issue
@antixar antixar assigned antixar and htrueman and unassigned htrueman Dec 8, 2021
@htrueman htrueman linked a pull request Dec 9, 2021 that will close this issue
@htrueman
Copy link
Contributor

As were discussed, we should implement the following flow:

  • Search the modules which were changed in PR. E.g. assume that airbyte-cdk/python/airbyte_cdk/logger.py and airbyte-integrations/connectors/source-salesforce/source_salesforce/api.py were changed. That would mean the airbyte-cdk and airbyte-integrations/connectors/source-salesforce should be found.
  • Build reports for the modules found in prev step in separate github action.
  • Pass these module to the SonarQube project for the following module.
  • Show SonarQube analytics results on the github PR page as a comment (similar to SAT results shown now).

@oustynova oustynova moved this to Ready for implementation in GL Roadmap Dec 15, 2021
@oustynova oustynova moved this from Ready for implementation to Implementation in progress in GL Roadmap Dec 23, 2021
@sherifnada sherifnada moved this from Implementation in progress to Ready for implementation in GL Roadmap Dec 23, 2021
@oustynova oustynova moved this from Ready for implementation to Implementation in progress in GL Roadmap Dec 28, 2021
@bazarnov bazarnov moved this from Implementation in progress to Done in GL Roadmap Dec 31, 2021
@sherifnada sherifnada moved this from Done to Ready for implementation in GL Roadmap Jan 12, 2022
@lazebnyi lazebnyi moved this from Ready for implementation to Done in GL Roadmap Jan 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues type/enhancement New feature or request
Projects
No open projects
Archived in project
Development

Successfully merging a pull request may close this issue.

6 participants