Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding gradle task for running integ tests in remote cluster #266

Merged
merged 4 commits into from
Jan 25, 2022
+33 −1
Merged

Adding gradle task for running integ tests in remote cluster #266

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d8fca6c
Adding gradle task for running integ tests in remote cluster
martin-gaievski Jan 21, 2022
019586c
Excluding BWC tests
martin-gaievski Jan 24, 2022
107335d
Adding params and intructions for integ tests on remote secure cluster
martin-gaievski Jan 25, 2022
c2cdb94
Adding num of cluster nodes as param for task, set default in gradle …
martin-gaievski Jan 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions DEVELOPER_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,12 @@ In order to run the integration tests with a 3 node cluster, run this command:
./gradlew :integTest -PnumNodes=3
```

Integration tests can be run with remote cluster. For that run the following command and replace host/port/cluster name values with ones for the target cluster:

```
./gradlew :integTestRemote -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="integTest-0" -Dhttps=false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we include an example with secure cluster (username/password) as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, that makes sense. Let me update the guide

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added instructions for secure cluster

```

### Debugging

Sometimes it is useful to attach a debugger to either the OpenSearch cluster or the integration test runner to see what's going on. For running unit tests, hit **Debug** from the IDE's gutter to debug the tests. For the OpenSearch cluster, first, make sure that the debugger is listening on port `5005`. Then, to debug the cluster code, run:
Expand Down
13 changes: 13 additions & 0 deletions build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

import java.util.concurrent.Callable
import org.opensearch.gradle.testclusters.StandaloneRestIntegTestTask
import org.opensearch.gradle.test.RestIntegTestTask

buildscript {
ext {
Expand Down Expand Up @@ -217,6 +218,18 @@ testClusters.integTest {
systemProperty("java.library.path", "$rootDir/jni/release")
}

task integTestRemote(type: RestIntegTestTask) {
testClassesDirs = sourceSets.test.output.classesDirs
classpath = sourceSets.test.runtimeClasspath

// Run tests with remote cluster only if rest case is defined
if (System.getProperty("tests.rest.cluster") != null) {
filter {
includeTestsMatching "org.opensearch.knn.*IT"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, you should add this in the filter to fix the BWC issue which Jack mentioned in review comments
excludeTestsMatching "org.opensearch.knn.bwc.*IT"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference - https://github.com/opensearch-project/k-NN/blob/main/build.gradle#L176

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Naveen, it's a good point as we need to have multiple external clusters for BWC. For now I've added bwc tests to exclusion matching rule

}
}
}

// bwcFilePath contains the gradlew assemble binary files of k-NN plugins
String baseName = "knnBwcCluster"
String bwcFilePath = "src/test/resources/org/opensearch/knn/bwc/"
Expand Down