Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command-line only plugin install action #2266

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
949a635
Rebase
stephen-crawford Nov 10, 2022
e92d0cd
Undo bad file changes
stephen-crawford Nov 10, 2022
30c8adb
reset accidental changes
stephen-crawford Nov 10, 2022
705f9d9
Swap the version number
stephen-crawford Nov 10, 2022
282a500
final changes
stephen-crawford Nov 11, 2022
8e6fdc6
Fix typos
stephen-crawford Nov 11, 2022
858068c
Remove linux name
stephen-crawford Nov 11, 2022
c477372
fix rename
stephen-crawford Nov 11, 2022
710a8a2
Make sure passes without jdk 11
stephen-crawford Nov 14, 2022
aa59db2
Fully working once JDK 11 is fixed
stephen-crawford Nov 14, 2022
15c4005
Reset to working version and add fail fast
stephen-crawford Nov 15, 2022
3258eb9
Remove docker
stephen-crawford Nov 16, 2022
b531417
Test Without Docker
stephen-crawford Nov 16, 2022
a6b8113
Test Without Docker
stephen-crawford Nov 16, 2022
3f8e05a
Hard coded OSs since casing breaks download
stephen-crawford Nov 16, 2022
24e967d
Hard coded OSs since casing breaks download
stephen-crawford Nov 16, 2022
85d7d35
Swap file paths in download to their actual path
stephen-crawford Nov 16, 2022
a8b9fbd
Fix bracket position
stephen-crawford Nov 16, 2022
0faf6e9
try to fix the file suffixes
stephen-crawford Nov 16, 2022
3d0a1fa
Remove -f from del
stephen-crawford Nov 16, 2022
0f14071
LS directories
stephen-crawford Nov 16, 2022
cb6bf84
Fix name paths
stephen-crawford Nov 16, 2022
9bf4fb2
Remove SNAPSHOT mention from Linux
stephen-crawford Nov 16, 2022
2ec2a27
Fix missing bracket
stephen-crawford Nov 16, 2022
8044b7f
Path changes
stephen-crawford Nov 16, 2022
9d62afe
Remove doubled directory name
stephen-crawford Nov 16, 2022
e8ef729
Remove doubled directory name
stephen-crawford Nov 16, 2022
cf8a512
Check plugin install path names
stephen-crawford Nov 16, 2022
1b23eb7
Change plugin file name
stephen-crawford Nov 16, 2022
562d45b
Change plugin file name
stephen-crawford Nov 16, 2022
5c34160
Add echos
stephen-crawford Nov 17, 2022
e9971bd
Edit file names for mv and del
stephen-crawford Nov 17, 2022
f0d8cbd
Add extra 0 to OS sec version
stephen-crawford Nov 17, 2022
fa6cc22
removed echos
stephen-crawford Nov 17, 2022
7e29565
Fix wildcard in windows extraction and change pipe in linux run
stephen-crawford Nov 17, 2022
6b45e15
Add echoes every step
stephen-crawford Nov 17, 2022
bc6960c
Add echoes every step
stephen-crawford Nov 17, 2022
0c38049
Cd back and then installl
stephen-crawford Nov 17, 2022
3e97659
Use min builds instead of distributions
stephen-crawford Nov 17, 2022
99cb135
Add SNAPSHOT suffix
stephen-crawford Nov 17, 2022
fa406a2
add .sh
stephen-crawford Nov 17, 2022
b2cf121
Test linux, fixed path name
stephen-crawford Nov 17, 2022
bb7f5f3
Fix pipe format
stephen-crawford Nov 17, 2022
c1a3977
Add echo for steps
stephen-crawford Nov 17, 2022
6a7dd18
Add echo for steps
stephen-crawford Nov 17, 2022
2b09732
use arrows instead of bars
stephen-crawford Nov 17, 2022
f1aabcb
Add echo to front of install
stephen-crawford Nov 17, 2022
83e81dc
Try process again
stephen-crawford Nov 17, 2022
3d2c12e
Try to make sure file path is correct for plugin file install
stephen-crawford Nov 17, 2022
7ff3310
Use pwd in linux install
stephen-crawford Nov 17, 2022
a17cc0a
Use pwd in linux install
stephen-crawford Nov 17, 2022
fe030ba
remove echo for pipe
stephen-crawford Nov 17, 2022
c8146a6
Try with quotes
stephen-crawford Nov 17, 2022
2d35d95
Try printf '%s\n' y n n y y n
stephen-crawford Nov 17, 2022
ff5ff60
Try no pipinh
stephen-crawford Nov 17, 2022
b3d3f57
Bin/bash -c
stephen-crawford Nov 17, 2022
c330eb6
Bin/bash -c
stephen-crawford Nov 17, 2022
51eac85
Add ls for os dir paths
stephen-crawford Nov 17, 2022
e8b18ee
Add ls for os dir paths
stephen-crawford Nov 17, 2022
49a27b2
Fix permissions
stephen-crawford Nov 17, 2022
0639752
Work?
stephen-crawford Nov 17, 2022
62d62bd
Ls OS bin:
stephen-crawford Nov 17, 2022
03fb2de
Remove suffix
stephen-crawford Nov 17, 2022
771c205
Test Linux and JDK 17
stephen-crawford Nov 17, 2022
639f217
specify bin/bash for run
stephen-crawford Nov 17, 2022
009bcf7
Craig's suggestion
stephen-crawford Nov 17, 2022
318e6c6
Test on windows and linux fully without docker
stephen-crawford Nov 17, 2022
cc6e627
Re enable workflows
stephen-crawford Nov 17, 2022
134cc46
Merge branch 'main' into UniversalPluginInstall
stephen-crawford Nov 17, 2022
10da06a
readd spaces around workflow_dispatch trigger
stephen-crawford Nov 21, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 12 additions & 29 deletions .github/actions/start-opensearch-with-one-plugin/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@ inputs:
description: 'The file name for the configuration script for the plugin such as install_demo_configurations -- may not be needed for every plugin'
required: false

docker-host-plugin-zip:
description: 'The name of the zip file for the plugin hosted on docker-host i.e. security-plugin.zip '
required: true

runs:
using: "composite"
steps:
Expand All @@ -34,19 +30,21 @@ runs:
with:
url: https://artifacts.opensearch.org/snapshots/core/opensearch/${{ inputs.opensearch-version }}-SNAPSHOT/opensearch-min-${{ inputs.opensearch-version }}-SNAPSHOT-windows-x64-latest.zip

# Download OpenSearch
- uses: peternied/download-file@v1
if: ${{ runner.os == 'Linux' }}
with:
url: https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/${{ inputs.opensearch-version }}/latest/linux/x64/tar/builds/opensearch/dist/opensearch-min-${{ inputs.opensearch-version }}-linux-x64.tar.gz
url: https://artifacts.opensearch.org/snapshots/core/opensearch/${{ inputs.opensearch-version }}-SNAPSHOT/opensearch-min-${{ inputs.opensearch-version }}-SNAPSHOT-linux-x64-latest.tar.gz
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why change the source? https://ci.opensearch.org/... [1] is the freshest place to get them from according to the OpenSearch-Build team [2]

[1] https://github.com/opensearch-project/opensearch-build/blob/main/README.md#latest-distribution-url
[2] opensearch-project/opensearch-build#2754

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I swapped the source for two reasons: 1) We do not have the same type of catalogue for Windows as we do Linux so what was available to Linux was not for Windows which made the processes for the two more divergent then I liked 2) Related to that, the most recent distributions seemed to have security already installed which would require that I make the script uninstall and reinstall the plugin to actually test what they are meant to test. Likewise, they probably would have had other plugins installed and I am not sure how much I could account for the differences in what those install and removal processes looked like. With the minimums snapshots, there is basically a barebones framework of OpenSearch to work with which I think makes it easier to install into but also gives you a better idea of whether your particular plugin is working right.

Good question though--I actually thought a bit about this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the most recent distributions seemed to have security already installed

That doesn't sound right, the opensearch-min-* is suppose to be only the core version of OpenSearch with no plugins, could you see about making a bug on OpenSearch-Build if that is the case.


# Extract downloaded zip
- name: Extract downloaded zip for Linux
- name: Extract downloaded zip
stephen-crawford marked this conversation as resolved.
Show resolved Hide resolved
if: ${{ runner.os == 'Linux' }}
run: |
tar -xzf opensearch-*.tar.gz
rm -f opensearch-*.tar.gz
shell: bash


- name: Extract downloaded zip for Windows
if: ${{ runner.os == 'Windows' }}
run: |
Expand All @@ -63,37 +61,24 @@ runs:
- name: Install Plugin into OpenSearch for Linux
if: ${{ runner.os == 'Linux'}}
run: |
cat > os-ep.sh <<EOF
yes | opensearch-plugin install file:///docker-host/${{ inputs.docker-host-plugin-zip }}.zip
chmod +x plugins/${{ inputs.plugin-name }}/tools/${{ inputs.plugin-start-script }}.sh
yes | plugins/${{ inputs.plugin-name }}/tools/${{ inputs.plugin-start-script }}.sh
chown 1001:1001 -R /opensearch
su -c "/opensearch/bin/opensearch" -s /bin/bash opensearch
EOF
docker build -t opensearch-test -f- . <<EOF
FROM ubuntu:latest
COPY --chown=1001:1001 os-ep.sh /docker-host/
COPY --chown=1001:1001 ${{ inputs.plugin-name }}.zip /docker-host/${{ inputs.docker-host-plugin-zip }}.zip
COPY --chown=1001:1001 opensearch* /opensearch/
RUN chmod +x /docker-host/os-ep.sh
RUN useradd -u 1001 -s /sbin/nologin opensearch
ENV PATH="/opensearch/bin:${PATH}"
WORKDIR /opensearch/
ENTRYPOINT /docker-host/os-ep.sh
EOF

chmod +x ./opensearch-${{ inputs.opensearch-version }}-SNAPSHOT/bin/opensearch-plugin
/bin/bash -c "yes | ./opensearch-${{ inputs.opensearch-version }}-SNAPSHOT/bin/opensearch-plugin install file:$(pwd)/${{ inputs.plugin-name }}.zip"
chmod +x ./opensearch-${{ inputs.opensearch-version }}-SNAPSHOT/plugins/${{ inputs.plugin-name }}/tools/${{ inputs.plugin-start-script }}.sh
/bin/bash -c "yes | ./opensearch-${{ inputs.opensearch-version }}-SNAPSHOT/plugins/${{ inputs.plugin-name }}/tools/${{ inputs.plugin-start-script }}.sh"
stephen-crawford marked this conversation as resolved.
Show resolved Hide resolved
shell: bash

- name: Install Plugin into OpenSearch for Windows
if: ${{ runner.os == 'Windows'}}
run: |
'y' | .\opensearch-${{ inputs.opensearch-version }}-SNAPSHOT\bin\opensearch-plugin.bat install file:$pwd\${{ inputs.plugin-name }}.zip
'y' | .\opensearch-${{ inputs.opensearch-version }}-SNAPSHOT\bin\opensearch-plugin.bat install file:$(pwd)\${{ inputs.plugin-name }}.zip
'y', 'y', 'N' | .\opensearch-${{ inputs.opensearch-version }}-SNAPSHOT\plugins\${{ inputs.plugin-name }}\tools\${{ inputs.plugin-start-script }}.bat
shell: pwsh

# Run OpenSearch
- name: Run OpenSearch with plugin on Linux
if: ${{ runner.os == 'Linux'}}
run: docker run --name ops -d -p 9200:9200 -p 9600:9600 -i opensearch-test:latest
run: /bin/bash -c "./opensearch-${{ inputs.opensearch-version }}-SNAPSHOT/bin/opensearch &"
shell: bash

- name: Run OpenSearch with plugin on Windows
Expand All @@ -110,9 +95,7 @@ runs:
# Verify that the server is operational
- name: Check OpenSearch Running on Linux
if: ${{ runner.os != 'Windows'}}
run: |
docker logs ops
curl https://localhost:9200/_cat/plugins -u 'admin:admin' -k -v
run: curl https://localhost:9200/_cat/plugins -u 'admin:admin' -k -v
shell: bash

- name: Check OpenSearch Running on Windows
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/plugin_install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ jobs:
opensearch-version: 3.0.0
plugin-name: opensearch-security
plugin-start-script: install_demo_configuration
docker-host-plugin-zip: security-plugin.zip

- name: Run sanity tests
uses: gradle/gradle-build-action@v2
Expand Down