Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SonarQube to CI #8362

Merged
merged 194 commits into from
Jan 19, 2022
Merged

Add SonarQube to CI #8362

merged 194 commits into from
Jan 19, 2022

Conversation

h-chlor
Copy link
Contributor

@h-chlor h-chlor commented Dec 1, 2021

Using the SonarQube as tool for aggregation of all issues, bugs, vulnerabilities, coverage reports. GitHub CI should run all tests and necessary scripts automatically and add summary reports as a new comment of source PR.

An UML draft of this solution is here.

Now we are trying to cover Python and Java connectors only. But in the future this logic can be expanded for other languages and project components/modules.

The following files were updated for testing of issues and changes detections. Please ignore these because they will be reverted before merging of this PR.

  • airbyte-integrations/connectors/destination-mongodb/*
  • airbyte-integrations/connectors/source-airtable/*
  • airbyte-integrations/connectors/source-github/*

You can see examples of generated reports into comments of this PR. This report is generated by our script and we can edit and format its template anytime if needed


This change is Reviewable

Created .properties file for sonarqube
@CLAassistant
Copy link

CLAassistant commented Dec 1, 2021

CLA assistant check
All committers have signed the CLA.

@h-chlor h-chlor self-assigned this Dec 1, 2021
@h-chlor h-chlor temporarily deployed to more-secrets December 1, 2021 07:17 Inactive
@h-chlor h-chlor requested a review from antixar December 2, 2021 04:50
@antixar antixar mentioned this pull request Dec 7, 2021
40 tasks
@antixar antixar temporarily deployed to more-secrets December 7, 2021 10:48 Inactive
@antixar antixar temporarily deployed to more-secrets December 7, 2021 11:59 Inactive
@antixar antixar temporarily deployed to more-secrets December 7, 2021 12:06 Inactive
@antixar antixar temporarily deployed to more-secrets December 7, 2021 12:07 Inactive
@antixar antixar temporarily deployed to more-secrets December 7, 2021 12:11 Inactive
@antixar antixar changed the title Create sonar-project.properties Add SonarQube to CI Dec 7, 2021
@antixar antixar temporarily deployed to more-secrets December 7, 2021 12:13 Inactive
@antixar antixar temporarily deployed to more-secrets December 7, 2021 13:12 Inactive
@antixar antixar temporarily deployed to more-secrets December 7, 2021 13:13 Inactive
@antixar antixar temporarily deployed to more-secrets December 7, 2021 13:15 Inactive
@antixar antixar temporarily deployed to more-secrets December 7, 2021 13:54 Inactive
@antixar antixar temporarily deployed to more-secrets December 7, 2021 14:00 Inactive
@antixar antixar temporarily deployed to more-secrets December 7, 2021 14:07 Inactive
@antixar antixar temporarily deployed to more-secrets January 10, 2022 11:20 Inactive
@antixar antixar requested review from sherifnada and removed request for antixar January 10, 2022 11:32
Copy link
Contributor

@sherifnada sherifnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@h-chlor could you add an .md file to the docs under contributing to airbyte section explaining at a high level how sonarQube is integrated? e.g: where is it hosted, how is it run/triggered, etc... basically if you were onboarding someone into how to use sonarqube, what info do they need to know to become effective at working with it?

@antixar
Copy link
Contributor

antixar commented Jan 13, 2022

@h-chlor could you add an .md file to the docs under contributing to airbyte section explaining at a high level how sonarQube is integrated? e.g: where is it hosted, how is it run/triggered, etc... basically if you were onboarding someone into how to use sonarqube, what info do they need to know to become effective at working with it?

Hi @sherifnada , @h-chlor had initialised this PR only and I finished it. DevOps team prepared a similar doc
About docs did you validate my UML diagram ? I didn't write some official documentation because I added several parts of this solution myself and I'm not sure to their are relevant and useful. I'm gonna add docs after approval of Airbyte team only.

@sherifnada
Copy link
Contributor

@antixar indeed, the UML diagram is great! I think it's worth screenshotting + leaving a link to the whimsical in an .md doc in the repo itself, just to have a clear entrypoint for people. Also adding any links to relevant files/entrypoints etc..

I wasn't able to look at the code in details, but I think having something like this is a great insurance policy to help people make modifications to the code in the future

@github-actions github-actions bot added the area/documentation Improvements or additions to documentation label Jan 14, 2022
@antixar antixar requested a review from sherifnada January 14, 2022 18:15
@antixar antixar temporarily deployed to more-secrets January 14, 2022 18:15 Inactive
Copy link
Contributor

@sherifnada sherifnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs look great! Excited to have it in the repo. Just a few comments then feel free to merge.

docs/contributing-to-airbyte/sonar-qube-workflow.md Outdated Show resolved Hide resolved
docs/contributing-to-airbyte/sonar-qube-workflow.md Outdated Show resolved Hide resolved
docs/contributing-to-airbyte/sonar-qube-workflow.md Outdated Show resolved Hide resolved
docs/contributing-to-airbyte/sonar-qube-workflow.md Outdated Show resolved Hide resolved
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
@antixar antixar temporarily deployed to more-secrets January 19, 2022 07:44 Inactive
antixar and others added 3 commits January 19, 2022 09:47
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
@antixar antixar temporarily deployed to more-secrets January 19, 2022 07:49 Inactive
@github-actions github-actions bot removed the area/connectors Connector related issues label Jan 19, 2022
@antixar antixar temporarily deployed to more-secrets January 19, 2022 16:22 Inactive
@antixar antixar merged commit 1832c69 into master Jan 19, 2022
@antixar antixar deleted the h-chlor-patch-1 branch January 19, 2022 16:25
@antixar
Copy link
Contributor

antixar commented Jan 19, 2022

Follow-up issues:
Java coverage
CDK + SAT detection

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate airbyte repo with SonarQube
5 participants