The "che-dashboard-next" testing roadmap

[Ohrimenko1988]

0 stage

(responsibility of the "dashboard-next" team)

• 1) Coverage by component tests should be more than 50% (Current coverage is ~ 48%) [new UD] Provide unit tests for testing roadmap stage 0 #18414

1-st stage

(responsibility of the "Che QE" team)

• 1) Rework existing che devfiles tests for using new dashboard. Keep changes in separate branch for been ready to switching dashboard. (Cover Get Started Tab with e2e tests in dashboard-next #16911)

• 2) Rework the "Happy Path" test for using new dashboard. Keep changes in separate branch for been ready to switching dashboard. (Cover Get Started Tab with e2e tests in dashboard-next #18428)

• 3) Review other tests from the "che/tests/e2e" folder and update them if they depend to the new dashboard. Keep changes in separate branch for been ready to switching dashboard. (When the dashboard is applied by default and will be accessible by the "<che_url>/dashboard" URL, tests should work as expected. Pageobjects already reworked.)

• 4) Create a smoke test for validation workspace creation from the dashboard (basically may be used 1 devfile test ). Done Cover Get Started Tab with e2e tests in dashboard-next #18428

• 5) Implement the "quality gate" using a smoke test for the "dashboard next" repo (run test for each PR and for changes inside PR ) Implement checking each PR by smoke E2E test for the "che-dashboard-next" repo #18454

2-nd stage

(responsibility of the "Che QE" and "dashboard-next" teams)

• 1) Add to "quality gate" running of the existing component tests and coverage measuring.

• 2) Create the "e2e" test for covering the most critical functionality (1 userstory)

• 3) Apply created test for the "quality gate" instead of "smoke" test

• 4) Create integration tests for covering other important functionality

• 5) Add created integrational tests to the "quality gate"

Related issue: #18348

[Ohrimenko1988]

Current coverage by component tests is something close to 20%

@l0rd @rhopp @dmytro-ndp @olexii4 @akurinnoy @sleshchenko WDYT ?

[psrna]

Thanks for preparing the plan, Ihor. It looks solid. However I suggest to highlight ownership of the respective stages to make it clear who is responsible for what, e.g. stage 0 is owned by dashboard.next engineering team., stage 1 Che QE team etc.

For promoting dashboard.next to be the default, I consider completion of stage 0 & 1 as a requirement. Otherwise we will be creating technical debt.

[Ohrimenko1988]

Thanks for preparing the plan, Ihor. It looks solid. However I suggest to highlight ownership of the respective stages to make it clear who is responsible for what, e.g. stage 0 is owned by dashboard.next engineering team., stage 1 Che QE team etc.

For promoting dashboard.next to be the default, I consider completion of stage 0 & 1 as a requirement. Otherwise we will be creating technical debt.

Done.

[Ohrimenko1988]

Current coverage by unit tests is ~ 30%

[gazarenkov]

Hi @psrna

First of all I highly appreciate the initiative of making test unit coverage analysis recommendations, I strongly believe it is a really good goal toward improving product quality.
I am curious whether we have the same requirement (50% of code should be covered with test units as a strong, phase 0 prerequisite) for all the components or it is a privilege of the Dashboard.next only? :)

[psrna]

Hi @gazarenkov

thank you for bringing up this topic.

I believe that we should treat all components the same. Dashboard.next should not be unique. And you probably correctly identified that we have a lot of technical debt in other components as well.

But for dashboard.next I think we should do it right rather then burning technical debt later. We might argue about the coverage number. However the number itself is not important. What is important, is that we are DONE if we do have sufficient test coverage that gives us confidence in the quality. Component level tests are the foundations and the biggest part of the testing pyramid. Compared to the Integration tests, they have the benefits of being cheap in terms of execution time and stability and tremendously help preventing bugs.

Would you agree that we don't have the required confidence with the current level of component tests?

[gazarenkov]

@psrna

Would you agree that we don't have the required confidence with the current level of component tests?

I would agree that we do not pay enough attention to the unit test coverage analysis. It is even difficult to know what is the current coverage level in general... Do we have the actual numbers for all the components somewhere?

[psrna]

@gazarenkov,

I agree with your statement and I don't think we have a consolidated coverage report for all the components. I'll bring it up as a goal/topic to discuss in the next internal call.

[sleshchenko]

The current test coverage is a bit less than 50%

But as we agreed on Che Internal Call, the number is not the goal.
Code coverage in existing components is going to be improved along the way, new components should be provided along with proper test coverage.
So, 0 stage is complete

[nickboldt]

Slip to 7.26 since not completed for 7.25

[sleshchenko]

The stage 1 is done in 7.26. Removing milestone since there is no concrete timeframe for stage 2 at this point.

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.