-
Notifications
You must be signed in to change notification settings - Fork 144
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add github action for secure integ tests #836
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,96 @@ | ||
name: Security test workflow for k-NN | ||
on: | ||
navneet1v marked this conversation as resolved.
Show resolved
Hide resolved
|
||
push: | ||
branches: | ||
- "*" | ||
- "feature/**" | ||
pull_request: | ||
branches: | ||
- "*" | ||
- "feature/**" | ||
|
||
jobs: | ||
Build-ad: | ||
strategy: | ||
matrix: | ||
java: [ 11,17 ] | ||
os: [ubuntu-latest] | ||
navneet1v marked this conversation as resolved.
Show resolved
Hide resolved
|
||
fail-fast: false | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why this is false? I think we should make it true. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ack |
||
|
||
name: Security test workflow for k-NN Plugin | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. update the name as suggested above. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ack |
||
runs-on: ${{ matrix.os }} | ||
|
||
steps: | ||
- name: Checkout k-NN | ||
uses: actions/checkout@v1 | ||
|
||
- name: Setup Java ${{ matrix.java }} | ||
uses: actions/setup-java@v1 | ||
with: | ||
java-version: ${{ matrix.java }} | ||
|
||
- name: Install dependencies on ubuntu | ||
if: startsWith(matrix.os,'ubuntu') | ||
run: | | ||
sudo apt-get install libopenblas-dev gfortran -y | ||
|
||
- name: Assemble k-NN | ||
run: | | ||
./gradlew assemble | ||
# example of variables: | ||
# plugin = opensearch-knn-2.7.0.0-SNAPSHOT.zip | ||
# version = 2.7.0 | ||
# plugin_version = 2.7.0.0 | ||
# qualifier = `SNAPSHOT` | ||
- name: Pull and Run Docker | ||
run: | | ||
plugin=`basename $(ls build/distributions/*.zip)` | ||
version=`echo $plugin|awk -F- '{print $3}'| cut -d. -f 1-3` | ||
plugin_version=`echo $plugin|awk -F- '{print $3}'| cut -d. -f 1-4` | ||
qualifier=`echo $plugin|awk -F- '{print $4}'| cut -d. -f 1-1` | ||
if [ $qualifier != `SNAPSHOT` ]; | ||
then | ||
docker_version=$version-$qualifier | ||
else | ||
docker_version=$version | ||
fi | ||
echo plugin version plugin_version qualifier docker_version | ||
echo "($plugin) ($version) ($plugin_version) ($qualifier) ($docker_version)" | ||
|
||
cd .. | ||
if docker pull opensearchstaging/opensearch:$docker_version | ||
then | ||
echo "FROM opensearchstaging/opensearch:$docker_version" >> Dockerfile | ||
# knn plugin cannot be deleted until there are plugin that has dependency on it | ||
echo "RUN if [ -d /usr/share/opensearch/plugins/opensearch-neural-search ]; then /usr/share/opensearch/bin/opensearch-plugin remove opensearch-neural-search; fi" >> Dockerfile | ||
echo "RUN if [ -d /usr/share/opensearch/plugins/opensearch-performance-analyzer ]; then /usr/share/opensearch/bin/opensearch-plugin remove opensearch-performance-analyzer; fi" >> Dockerfile | ||
# saving pre-built artifacts of native libraries as we can't build it with gradle assemle | ||
echo "RUN if [ -d /usr/share/opensearch/plugins/opensearch-knn ]; then cp -r /usr/share/opensearch/plugins/opensearch-knn/lib /usr/share/opensearch/knn-libs; fi" >> Dockerfile | ||
echo "RUN if [ -d /usr/share/opensearch/plugins/opensearch-knn ]; then /usr/share/opensearch/bin/opensearch-plugin remove opensearch-knn; fi" >> Dockerfile | ||
echo "ADD k-NN/build/distributions/$plugin /tmp/" >> Dockerfile | ||
echo "RUN /usr/share/opensearch/bin/opensearch-plugin install --batch file:/tmp/$plugin" >> Dockerfile | ||
# moving pre-built artifacts of native libraries back to plugin folder | ||
echo "RUN if [ -d /usr/share/opensearch/knn-libs ]; then mv /usr/share/opensearch/knn-libs /usr/share/opensearch/plugins/opensearch-knn/lib; fi" >> Dockerfile | ||
docker build -t opensearch-knn:test . | ||
echo "imagePresent=true" >> $GITHUB_ENV | ||
else | ||
echo "imagePresent=false" >> $GITHUB_ENV | ||
fi | ||
|
||
- name: Run Docker Image | ||
if: env.imagePresent == 'true' | ||
run: | | ||
cd .. | ||
docker run -p 9200:9200 -d -p 9600:9600 -e "discovery.type=single-node" opensearch-knn:test | ||
sleep 90 | ||
- name: Run k-NN Integ Test | ||
if: env.imagePresent == 'true' | ||
run: | | ||
security=`curl -XGET https://localhost:9200/_cat/plugins?v -u admin:admin --insecure |grep opensearch-security|wc -l` | ||
if [ $security -gt 0 ] | ||
then | ||
echo "Security plugin is available" | ||
./gradlew integTest -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="docker-cluster" -Dhttps=true -Duser=admin -Dpassword=admin | ||
else | ||
echo "Security plugin is NOT available, skipping integration tests" | ||
fi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename this to, test k-NN on Secure Cluster
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ack