-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Repeat mode for integration tests #1911
Conversation
src/integrationTest/kotlin/com/sourcegraph/cody/edit/DocumentCodeTest.kt
Outdated
Show resolved
Hide resolved
src/integrationTest/kotlin/com/sourcegraph/cody/edit/DocumentCodeTest.kt
Outdated
Show resolved
Hide resolved
@@ -508,6 +508,7 @@ tasks { | |||
description = "Runs the integration tests." | |||
sharedIntegrationTestConfig(buildCodyDir, "replay") | |||
dependsOn("processIntegrationTestResources") | |||
project.properties["repeatTests"]?.let { systemProperty("repeatTests", it) } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you need it to be a system property you don't need to do that, but instead you can run gradle with -DrepeatTests
instead of -PrepeatTests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to Stackoverflow, the -D
argument won't necessarily pass through to the tests.
import org.junit.runners.BlockJUnit4ClassRunner | ||
import org.junit.runners.model.FrameworkMethod | ||
|
||
class CustomJunitClassRunner(klass: Class<*>?) : BlockJUnit4ClassRunner(klass) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe also something with Repeatable
or Repeat
in name? Custom
doesn't bring too much context
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This class doesn't really repeat anything or make anything repeatable. I don't mind renaming it, but I can't think of any good name.
Could you somehow attach example off test report or at least screenshot of it? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it easily doable to replace
DocumentCodeTest. testGetsFoldingRanges
DocumentCodeTest. testGetsFoldingRanges
DocumentCodeTest. testGetsFoldingRanges
...
with
DocumentCodeTest. testGetsFoldingRanges (0)
DocumentCodeTest. testGetsFoldingRanges (1)
DocumentCodeTest. testGetsFoldingRanges (2)
...
❓
Not without writing a lot more code. I will try to do that in a follow-up PR. |
To use it, add the option
-PrepeatTests
to the command line. Example:Test plan