TS-35531 - TIA Gradle Plugin: tiaTests doesn't execute cucumber tests #515
Conversation
…ests_doesn't_execute_cucumber_tests
WalkthroughThe changes introduce a comprehensive Gradle build configuration for a system testing framework that integrates Cucumber and Teamscale. Key additions include build scripts, a calculator class for basic arithmetic tests, and step definitions for behavior-driven development. Together, these enhancements streamline testing processes, facilitate coverage reporting, and ensure robust validation of application functionality. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Cucumber
participant Calculator
participant TeamscaleMockServer
User->>Cucumber: Start Test
Cucumber->>Calculator: Initialize
Cucumber->>Calculator: Perform Addition
Calculator-->>Cucumber: Return Result
Cucumber->>TeamscaleMockServer: Upload Report
TeamscaleMockServer-->>Cucumber: Acknowledge Report
Cucumber-->>User: Test Complete
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Outside diff range, codebase verification and nitpick comments (6)
system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java (5)
15-18: Consider using a more conventional initial value.Initializing
totalto -999 might be for debugging, but using a more conventional value like 0 can improve readability and maintainability.- total = -999; + total = 0;
20-23: Add an interface comment.The method lacks an interface comment, which is important for understanding its purpose.
/** * Initializes the calculator object. */ public void initializeCalculator() throws Throwable { calculator = new Calculator(); }Tools
GitHub Check: teamscale-findings
[warning] 21-21: system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java#L21
Interface comment missing
https://cqse.teamscale.io/findings/details/teamscale-jacoco-agent?t=ts%2F35531_tia_gradle_plugin_tiatests_doesn%27t_execute_cucumber_tests%3AHEAD&id=0C06381A4CA296E2F0047CB283CBD993
25-28: Add an interface comment.The method lacks an interface comment, which is important for understanding its purpose.
/** * Tests the addition functionality of the calculator. * @param num1 The first number to add. * @param num2 The second number to add. */ public void testAdd(int num1, int num2) throws Throwable { total = calculator.add(num1, num2); }Tools
GitHub Check: teamscale-findings
[warning] 26-26: system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java#L26
Interface comment missing
https://cqse.teamscale.io/findings/details/teamscale-jacoco-agent?t=ts%2F35531_tia_gradle_plugin_tiatests_doesn%27t_execute_cucumber_tests%3AHEAD&id=79FF30066C6BD2EFB3AA9F306F51FA10
30-33: Add an interface comment.The method lacks an interface comment, which is important for understanding its purpose.
/** * Validates the result of the addition. * @param result The expected result of the addition. */ public void validateResult(int result) throws Throwable { Assert.assertThat(total, Matchers.equalTo(result)); }Tools
GitHub Check: teamscale-findings
[warning] 31-31: system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java#L31
Interface comment missing
https://cqse.teamscale.io/findings/details/teamscale-jacoco-agent?t=ts%2F35531_tia_gradle_plugin_tiatests_doesn%27t_execute_cucumber_tests%3AHEAD&id=8845ADDCDE0F74C51C9586A5903068B1
10-10: Add an interface comment.The class lacks an interface comment, which is important for understanding its purpose.
/** * Step definitions for Cucumber tests. */ public class StepDefinitions { // Class implementation }Tools
GitHub Check: teamscale-findings
[warning] 10-10: system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java#L10
Interface comment missing
https://cqse.teamscale.io/findings/details/teamscale-jacoco-agent?t=ts%2F35531_tia_gradle_plugin_tiatests_doesn%27t_execute_cucumber_tests%3AHEAD&id=C3766E4D0E566A5C8D40AC8ABC251669system-tests/gradle-cucumber/src/test/java/com/teamscale/tia/CucumberGradleTiaSystemTest.java (1)
16-16: Add an interface comment.The class lacks an interface comment, which is important for understanding its purpose.
/** * System tests for Cucumber and Teamscale integration. */ public class CucumberGradleTiaSystemTest { // Class implementation }
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files ignored due to path filters (1)
system-tests/gradle-cucumber/gradle-project/gradle/wrapper/gradle-wrapper.jaris excluded by!**/*.jar
Files selected for processing (11)
- system-tests/gradle-cucumber/build.gradle.kts (1 hunks)
- system-tests/gradle-cucumber/gradle-project/build.gradle.kts (1 hunks)
- system-tests/gradle-cucumber/gradle-project/gradle/wrapper/gradle-wrapper.properties (1 hunks)
- system-tests/gradle-cucumber/gradle-project/gradlew (1 hunks)
- system-tests/gradle-cucumber/gradle-project/gradlew.bat (1 hunks)
- system-tests/gradle-cucumber/gradle-project/settings.gradle.kts (1 hunks)
- system-tests/gradle-cucumber/gradle-project/src/main/java/hellocucumber/Calculator.java (1 hunks)
- system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/RunCucumberTest.java (1 hunks)
- system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java (1 hunks)
- system-tests/gradle-cucumber/gradle-project/src/test/resources/hellocucumber/calculator.feature (1 hunks)
- system-tests/gradle-cucumber/src/test/java/com/teamscale/tia/CucumberGradleTiaSystemTest.java (1 hunks)
Files skipped from review due to trivial changes (5)
- system-tests/gradle-cucumber/gradle-project/gradle/wrapper/gradle-wrapper.properties
- system-tests/gradle-cucumber/gradle-project/gradlew
- system-tests/gradle-cucumber/gradle-project/gradlew.bat
- system-tests/gradle-cucumber/gradle-project/settings.gradle.kts
- system-tests/gradle-cucumber/gradle-project/src/main/java/hellocucumber/Calculator.java
Additional context used
GitHub Check: teamscale-findings
system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java
[warning] 21-21: system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java#L21
Interface comment missing
https://cqse.teamscale.io/findings/details/teamscale-jacoco-agent?t=ts%2F35531_tia_gradle_plugin_tiatests_doesn%27t_execute_cucumber_tests%3AHEAD&id=0C06381A4CA296E2F0047CB283CBD993
[warning] 26-26: system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java#L26
Interface comment missing
https://cqse.teamscale.io/findings/details/teamscale-jacoco-agent?t=ts%2F35531_tia_gradle_plugin_tiatests_doesn%27t_execute_cucumber_tests%3AHEAD&id=79FF30066C6BD2EFB3AA9F306F51FA10
[warning] 31-31: system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java#L31
Interface comment missing
https://cqse.teamscale.io/findings/details/teamscale-jacoco-agent?t=ts%2F35531_tia_gradle_plugin_tiatests_doesn%27t_execute_cucumber_tests%3AHEAD&id=8845ADDCDE0F74C51C9586A5903068B1
[warning] 10-10: system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java#L10
Interface comment missing
https://cqse.teamscale.io/findings/details/teamscale-jacoco-agent?t=ts%2F35531_tia_gradle_plugin_tiatests_doesn%27t_execute_cucumber_tests%3AHEAD&id=C3766E4D0E566A5C8D40AC8ABC251669system-tests/gradle-cucumber/src/test/java/com/teamscale/tia/CucumberGradleTiaSystemTest.java
[warning] 40-45: system-tests/gradle-cucumber/src/test/java/com/teamscale/tia/CucumberGradleTiaSystemTest.java#L40-L45
[New] Commented Out Code
https://cqse.teamscale.io/findings/details/teamscale-jacoco-agent?t=ts%2F35531_tia_gradle_plugin_tiatests_doesn%27t_execute_cucumber_tests%3AHEAD&id=681ECABBABFA850FEFF47F2401441F46
Gitleaks
system-tests/gradle-cucumber/gradle-project/build.gradle.kts
21-21: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
Additional comments not posted (49)
system-tests/gradle-cucumber/build.gradle.kts (11)
1-1: LGTM!The plugins block is correctly defined.
2-2: LGTM!The
com.teamscale.system-test-conventionplugin is applied correctly.
3-3: LGTM!The
com.teamscale.coverageplugin is applied correctly.
4-4: LGTM!The plugins block is correctly closed.
5-5: LGTM!The empty line improves readability.
6-6: LGTM!The task configuration for
testis correctly defined.
7-7: LGTM!The environment variable
AGENT_VERSIONis set correctly.
8-8: LGTM!The environment variable
TEAMSCALE_PORTis set correctly.
9-9: LGTM!The comment is clear and informative.
10-10: LGTM!The task dependency is correctly defined.
11-11: LGTM!The task configuration is correctly closed.
system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/RunCucumberTest.java (15)
1-1: LGTM!The package declaration is correct.
2-2: LGTM!The empty line improves readability.
3-3: LGTM!The import statement for
ConfigurationParameteris correct.
4-4: LGTM!The import statement for
IncludeEnginesis correct.
5-5: LGTM!The import statement for
SelectClasspathResourceis correct.
6-6: LGTM!The import statement for
Suiteis correct.
7-7: LGTM!The empty line improves readability.
8-8: LGTM!The static import statement for
PLUGIN_PROPERTY_NAMEis correct.
9-9: LGTM!The empty line improves readability.
10-10: LGTM!The
Suiteannotation is correctly applied.
11-11: LGTM!The
IncludeEnginesannotation is correctly applied.
12-12: LGTM!The
SelectClasspathResourceannotation is correctly applied.
13-13: LGTM!The
ConfigurationParameterannotation is correctly applied.
14-14: LGTM!The class declaration is correct.
15-15: LGTM!The class is correctly closed.
system-tests/gradle-cucumber/gradle-project/src/test/resources/hellocucumber/calculator.feature (14)
1-1: LGTM!The feature declaration is correct.
2-2: LGTM!The user story is correctly defined.
3-3: LGTM!The user story continuation is correctly defined.
4-4: LGTM!The user story conclusion is correctly defined.
5-5: LGTM!The empty line improves readability.
6-6: LGTM!The scenario outline declaration is correct.
7-7: LGTM!The given step is correctly defined.
8-8: LGTM!The when step is correctly defined.
9-9: LGTM!The then step is correctly defined.
10-10: LGTM!The empty line improves readability.
11-11: LGTM!The scenario outline declaration is correct.
12-12: LGTM!The given step is correctly defined.
13-13: LGTM!The when step is correctly defined.
14-14: LGTM!The then step is correctly defined.
system-tests/gradle-cucumber/gradle-project/src/test/java/hellocucumber/StepDefinitions.java (1)
1-8: LGTM! Import statements are appropriate.The import statements include necessary libraries for Cucumber, JUnit, and Hamcrest.
system-tests/gradle-cucumber/gradle-project/build.gradle.kts (5)
3-6: LGTM! Plugins section is appropriate.The plugins section includes necessary plugins for Java and Teamscale.
8-9: LGTM! Group and version properties are appropriate.The
groupandversionproperties are set appropriately.
11-14: LGTM! Repositories section is appropriate.The
repositoriessection includes Maven local and central repositories.
36-44: LGTM! Dependencies section is appropriate.The
dependenciessection includes necessary dependencies for JUnit, Cucumber, and Hamcrest.
46-56: LGTM! Tasks section is appropriate.The
taskssection includes configurations fortestandtiaTeststasks.system-tests/gradle-cucumber/src/test/java/com/teamscale/tia/CucumberGradleTiaSystemTest.java (3)
20-25: LGTM! Method implementation is appropriate.The
startFakeTeamscaleServermethod starts a mock Teamscale server with impacted tests and accepts report uploads.
27-30: LGTM! Method implementation is appropriate.The
stopFakeTeamscaleServermethod stops the mock Teamscale server.
1-10: LGTM! Import statements are appropriate.The import statements include necessary libraries for JUnit, AssertJ, and Teamscale.
| teamscale { | ||
|
|
||
| server { | ||
| url = "http://localhost:${System.getenv("TEAMSCALE_PORT")}/" | ||
| userName = "admin" | ||
| userAccessToken = "mrllRGxXJrsyL0UID3gGcJnZuQT4EyWr" | ||
| project = "cucumber-gradle" | ||
| } | ||
|
|
||
| report { | ||
| testwiseCoverage { | ||
| partition.set("Cucumber Tests") | ||
| } | ||
| } | ||
|
|
||
| commit { | ||
| revision = "abc1234" | ||
| } | ||
| } |
There was a problem hiding this comment.
Remove the hard-coded access token.
The server configuration includes a hard-coded access token, which is a security risk. Use environment variables for sensitive information.
- userAccessToken = "mrllRGxXJrsyL0UID3gGcJnZuQT4EyWr"
+ userAccessToken = System.getenv("TEAMSCALE_ACCESS_TOKEN")Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| teamscale { | |
| server { | |
| url = "http://localhost:${System.getenv("TEAMSCALE_PORT")}/" | |
| userName = "admin" | |
| userAccessToken = "mrllRGxXJrsyL0UID3gGcJnZuQT4EyWr" | |
| project = "cucumber-gradle" | |
| } | |
| report { | |
| testwiseCoverage { | |
| partition.set("Cucumber Tests") | |
| } | |
| } | |
| commit { | |
| revision = "abc1234" | |
| } | |
| } | |
| teamscale { | |
| server { | |
| url = "http://localhost:${System.getenv("TEAMSCALE_PORT")}/" | |
| userName = "admin" | |
| userAccessToken = System.getenv("TEAMSCALE_ACCESS_TOKEN") | |
| project = "cucumber-gradle" | |
| } | |
| report { | |
| testwiseCoverage { | |
| partition.set("Cucumber Tests") | |
| } | |
| } | |
| commit { | |
| revision = "abc1234" | |
| } | |
| } |
Tools
Gitleaks
21-21: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
| @Test | ||
| public void testImpactedCucumberTestsAreRun() throws Exception { | ||
| SystemTestUtils.runGradle("gradle-project", "clean", "tiaTests", "--impacted", "teamscaleReportUpload", "--no-daemon"); | ||
|
|
||
| assertThat(teamscaleMockServer.uploadedReports).hasSize(1); | ||
| assertThat(teamscaleMockServer.uploadedReports).allMatch(report -> report.getPartition().equals("Cucumber Tests")); | ||
|
|
||
| TestwiseCoverageReport testReport = teamscaleMockServer.parseUploadedTestwiseCoverageReport(0); | ||
| // We can't just assert for testReport.test == 1 here because the Gradle plugin uploads all test cases. | ||
| // The ones that were not executed have a mostly empty body, though. E.g. | ||
| // { | ||
| // "uniformPath" : "hellocucumber/RunCucumberTest/hellocucumber/calculator.feature/Add two numbers 10 & 15 #1", | ||
| // "paths" : [ ] | ||
| // } | ||
| assertThat(testReport.tests.stream().filter(test -> test.result != null).count()).isEqualTo(1); | ||
| } |
There was a problem hiding this comment.
Remove the commented-out code.
The method includes commented-out code, which should be removed for better readability.
- // We can't just assert for testReport.test == 1 here because the Gradle plugin uploads all test cases.
- // The ones that were not executed have a mostly empty body, though. E.g.
- // {
- // "uniformPath" : "hellocucumber/RunCucumberTest/hellocucumber/calculator.feature/Add two numbers 10 & 15 #1",
- // "paths" : [ ]
- // }Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| @Test | |
| public void testImpactedCucumberTestsAreRun() throws Exception { | |
| SystemTestUtils.runGradle("gradle-project", "clean", "tiaTests", "--impacted", "teamscaleReportUpload", "--no-daemon"); | |
| assertThat(teamscaleMockServer.uploadedReports).hasSize(1); | |
| assertThat(teamscaleMockServer.uploadedReports).allMatch(report -> report.getPartition().equals("Cucumber Tests")); | |
| TestwiseCoverageReport testReport = teamscaleMockServer.parseUploadedTestwiseCoverageReport(0); | |
| // We can't just assert for testReport.test == 1 here because the Gradle plugin uploads all test cases. | |
| // The ones that were not executed have a mostly empty body, though. E.g. | |
| // { | |
| // "uniformPath" : "hellocucumber/RunCucumberTest/hellocucumber/calculator.feature/Add two numbers 10 & 15 #1", | |
| // "paths" : [ ] | |
| // } | |
| assertThat(testReport.tests.stream().filter(test -> test.result != null).count()).isEqualTo(1); | |
| } | |
| @Test | |
| public void testImpactedCucumberTestsAreRun() throws Exception { | |
| SystemTestUtils.runGradle("gradle-project", "clean", "tiaTests", "--impacted", "teamscaleReportUpload", "--no-daemon"); | |
| assertThat(teamscaleMockServer.uploadedReports).hasSize(1); | |
| assertThat(teamscaleMockServer.uploadedReports).allMatch(report -> report.getPartition().equals("Cucumber Tests")); | |
| TestwiseCoverageReport testReport = teamscaleMockServer.parseUploadedTestwiseCoverageReport(0); | |
| assertThat(testReport.tests.stream().filter(test -> test.result != null).count()).isEqualTo(1); | |
| } |
Tools
GitHub Check: teamscale-findings
[warning] 40-45: system-tests/gradle-cucumber/src/test/java/com/teamscale/tia/CucumberGradleTiaSystemTest.java#L40-L45
[New] Commented Out Code
https://cqse.teamscale.io/findings/details/teamscale-jacoco-agent?t=ts%2F35531_tia_gradle_plugin_tiatests_doesn%27t_execute_cucumber_tests%3AHEAD&id=681ECABBABFA850FEFF47F2401441F46
DreierF
deleted the
ts/35531_tia_gradle_plugin_tiatests_doesn't_execute_cucumber_tests
branch
Addresses issue TS-35531
Please respect the vote of the Teamscale bot or flag irrelevant findings as tolerated or false positives. If you feel that the Teamscale config needs adjustment, please state so in a comment and discuss this with your reviewer.
Summary by CodeRabbit
New Features
Calculatorclass for basic arithmetic operations.Bug Fixes
Documentation
Chores