Skip to content

Commit

Permalink
Add publishToNuget lib and fix workspace overwrite issue for signing (#…
Browse files Browse the repository at this point in the history
…117)

Signed-off-by: Sayali Gaikawad <gaiksaya@amazon.com>
  • Loading branch information
gaiksaya authored Feb 3, 2023
1 parent 6f79c91 commit 8422ac8
Show file tree
Hide file tree
Showing 27 changed files with 311 additions and 22 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ lib = library(identifier: 'jenkins@<tag>', retriever: modernSCM([
| [standardReleasePipelineWithGenericTrigger.groovy](./vars/standardReleasePipelineWithGenericTrigger.groovy) | A standard release pipeline for OpenSearch projects including generic triggers. A tag or a draft release can be used as a trigger using this library. The defaults are all set to trigger via a draft release. If the release is successful, the release can be published by using right params.. Check how to use the [default](./tests/jenkins/jobs/StandardReleasePipelineWithGenericTriggers_Jenkinsfile) in your workflow and how to [overide](./tests/jenkins/jobs/StandardReleasePipelineWithGenericTriggersTag_Jenkinsfile) values. |
| [publishToNpm.groovy](./vars/publishToNpm.groovy) | A library to publish artifacts to [NPM registry](https://www.npmjs.com/) under @opensearch-project namespace. You can use [PublishToNpmLibTester](./tests/jenkins/lib-testers/PublishToNpmLibTester.groovy) to add tests in your repository. See how to use the lib in your [jenkinsFile](./tests/jenkins/jobs/PublishToNpm_Jenkinsfile). |
| [publishToPyPi.groovy](./vars/publishToPyPi.groovy) | A library to publish artifacts to [PyPi registry](https://pypi.org/) with [OpenSearch](https://pypi.org/user/OpenSearch/) as the maintainer. This library takes care of signing the artifacts before publishing. You can use [PublishToPyPiLibTester](./tests/jenkins/lib-testers/PublishToPyPiLibTester.groovy) to add tests in your repository. See how to use the lib in your [jenkinsFile](./tests/jenkins/jobs/PublishToPyPi_Jenkinsfile). |
| [publishToRubyGems.groovy](./vars/publishToRubyGems.groovy) | A library to publish gems to [rubygems.org](https://rubygems.org/) with [opensearchproject](https://rubygems.org/profiles/opensearchproject) as the owner. Please note that this library expects the gems provided to be signed. You can use [PublishToRubyGemsLibTester](./tests/jenkins/lib-testers/PublishToRubyGemsLibTester.groovy) to add tests in your repository. See how to use the lib in your [jenkinsFile](./tests/jenkins/jobs/PublishToRubyGems_JenkinsFile). |
| [publishToRubyGems.groovy](./vars/publishToRubyGems.groovy) | A library to publish gems to [rubygems.org](https://rubygems.org/) with [opensearchproject](https://rubygems.org/profiles/opensearchproject) as the owner. Please note that this library expects the gems to be pre-signed. You can use [PublishToRubyGemsLibTester](./tests/jenkins/lib-testers/PublishToRubyGemsLibTester.groovy) to add tests in your repository. See how to use the lib in your [jenkinsFile](./tests/jenkins/jobs/PublishToRubyGems_JenkinsFile). |
| [publishToMaven.groovy](./vars/publishToMaven.groovy) | A library to sign and deploy opensearch maven artifacts to sonatype staging repository, it also has an optional parameter `autoPublish` to auto-release artifacts from staging repo to prod without manual intervention. You can use [PublishToMavenLibTester](./tests/jenkins/lib-testers/PublishToMavenLibTester.groovy) to add tests in your repository. See how to use the lib in your [jenkinsFile](./tests/jenkins/jobs/PublishToMaven_JenkinsFile). |

| [publishToNuget.groovy](./vars/publishToNuget.groovy) | A library to build, sign and publish dotnet artifacts to [Nuget Gallery](https://www.nuget.org/). Please check if the [default docker](https://github.com/opensearch-project/opensearch-build/blob/main/docker/ci/dockerfiles/current/release.centos.clients.x64.arm64.dockerfile) file contains the required dotnet sdk. You can use [PublishToNugetLibTester](./tests/jenkins/lib-testers/PublishToNugetLibTester.groovy) to add tests in your repository. See how to use the lib in your [jenkinsFile](./tests/jenkins/jobs/PublishToNuget_Jenkinsfile).

## Contributing

Expand Down
1 change: 1 addition & 0 deletions tests/jenkins/TestPromoteLinuxArtifacts.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ class TestPromoteLinuxArtifacts extends BuildPipelineTest {
binding.setVariable('DISTRIBUTION_PLATFORM', 'linux')
binding.setVariable('DISTRIBUTION_ARCHITECTURE', 'x64')
binding.setVariable('WORKSPACE', 'tests/jenkins')
helper.addFileExistsMock('tests/jenkins/sign.sh', true)
binding.setVariable('GITHUB_BOT_TOKEN_NAME', 'github_bot_token_name')
def configs = ["role": "dummy_role",
"external_id": "dummy_ID",
Expand Down
1 change: 1 addition & 0 deletions tests/jenkins/TestPromoteWindowsArtifacts.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ class TestPromoteWindowsArtifacts extends BuildPipelineTest {
binding.setVariable('DISTRIBUTION_ARCHITECTURE', 'x64')
binding.setVariable('DISTRIBUTION_NAME', 'zip')
binding.setVariable('WORKSPACE', 'tests/jenkins')
helper.addFileExistsMock('tests/jenkins/sign.sh', true)
binding.setVariable('GITHUB_BOT_TOKEN_NAME', 'github_bot_token_name')
def configs = ["role": "dummy_role",
"external_id": "dummy_ID",
Expand Down
1 change: 1 addition & 0 deletions tests/jenkins/TestPromoteYumRepos.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ class TestPromoteYumRepos extends BuildPipelineTest {
"unsigned_bucket": "dummy_unsigned_bucket",
"signed_bucket": "dummy_signed_bucket"]
binding.setVariable('configs', configs)
helper.addFileExistsMock('/tmp/workspace/sign.sh', true)
helper.registerAllowedMethod("readJSON", [Map.class], {c -> configs})
helper.registerAllowedMethod("git", [Map])
helper.registerAllowedMethod("withCredentials", [Map, Closure], { args, closure ->
Expand Down
72 changes: 72 additions & 0 deletions tests/jenkins/TestPublishToNuget.groovy
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/

package jenkins.tests

import jenkins.tests.BuildPipelineTest
import static com.lesfurets.jenkins.unit.MethodCall.callArgsToString
import static org.hamcrest.CoreMatchers.hasItem
import static org.hamcrest.CoreMatchers.hasItems
import static org.hamcrest.MatcherAssert.assertThat
import org.junit.Before
import org.junit.Test

class TestPublishToNuget extends BuildPipelineTest {
@Override
@Before
void setUp() {

this.registerLibTester(new PublishToNugetLibTester(
'https://github.com/opensearch-project/opensearch-net',
'1.2.0',
'net-api-key',
'test-solution-file.sln'))
super.setUp()
}

@Test
public void test() {
super.testPipeline("tests/jenkins/jobs/PublishToNuget_Jenkinsfile")
}

@Test
void 'verify build command'(){
runScript('tests/jenkins/jobs/PublishToNuget_Jenkinsfile')

def buildCommands = getShellCommands('build')
assertThat(buildCommands, hasItem('\n dotnet build /tmp/workspace/test-solution-file.sln --configuration Release\n find src -name OpenSearch*.dll>/tmp/workspace/dlls.txt\n '))
}

@Test
void 'verify_pack_command'(){
runScript('tests/jenkins/jobs/PublishToNuget_Jenkinsfile')
def packCommand = getShellCommands('pack')
assertThat(packCommand, hasItem('\n dotnet pack /tmp/workspace/test-solution-file.sln --configuration Release --no-build\n for package in `find src -name OpenSearch*.nupkg`\n do\n dotnet nuget push $package --api-key API_KEY --source https://api.nuget.org/v3/index.json\n done\n '))
}

@Test
void 'verify_signer_call'(){
runScript('tests/jenkins/jobs/PublishToNuget_Jenkinsfile')
def signcommand = getShellCommands('sign.sh')
assertThat(signcommand, hasItems('\n #!/bin/bash\n set +x\n export ROLE=SIGNER_WINDOWS_ROLE\n export EXTERNAL_ID=SIGNER_WINDOWS_EXTERNAL_ID\n export UNSIGNED_BUCKET=SIGNER_WINDOWS_UNSIGNED_BUCKET\n export SIGNED_BUCKET=SIGNER_WINDOWS_SIGNED_BUCKET\n export PROFILE_IDENTIFIER=SIGNER_WINDOWS_PROFILE_IDENTIFIER\n export PLATFORM_IDENTIFIER=SIGNER_WINDOWS_PLATFORM_IDENTIFIER\n\n /tmp/workspace/opensearch-build/sign.sh one.dll --platform windows --overwrite \n ',
'\n #!/bin/bash\n set +x\n export ROLE=SIGNER_WINDOWS_ROLE\n export EXTERNAL_ID=SIGNER_WINDOWS_EXTERNAL_ID\n export UNSIGNED_BUCKET=SIGNER_WINDOWS_UNSIGNED_BUCKET\n export SIGNED_BUCKET=SIGNER_WINDOWS_SIGNED_BUCKET\n export PROFILE_IDENTIFIER=SIGNER_WINDOWS_PROFILE_IDENTIFIER\n export PLATFORM_IDENTIFIER=SIGNER_WINDOWS_PLATFORM_IDENTIFIER\n\n /tmp/workspace/opensearch-build/sign.sh two.dll --platform windows --overwrite \n ',
'\n #!/bin/bash\n set +x\n export ROLE=SIGNER_WINDOWS_ROLE\n export EXTERNAL_ID=SIGNER_WINDOWS_EXTERNAL_ID\n export UNSIGNED_BUCKET=SIGNER_WINDOWS_UNSIGNED_BUCKET\n export SIGNED_BUCKET=SIGNER_WINDOWS_SIGNED_BUCKET\n export PROFILE_IDENTIFIER=SIGNER_WINDOWS_PROFILE_IDENTIFIER\n export PLATFORM_IDENTIFIER=SIGNER_WINDOWS_PLATFORM_IDENTIFIER\n\n /tmp/workspace/opensearch-build/sign.sh three.dll --platform windows --overwrite \n '))
}

def getShellCommands(searchString) {
def shCommands = helper.callStack.findAll { call ->
call.methodName == 'sh'
}.collect { call ->
callArgsToString(call)
}.findAll { command ->
command.contains(searchString)
}
return shCommands
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@
createSignatureFiles.signArtifacts({sigtype=.sig, artifactPath=tests/jenkins/tests/jenkins/file/found.zip})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(tests/jenkins/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@
createSignatureFiles.signArtifacts({sigtype=.sig, artifactPath=tests/jenkins/tests/jenkins/file/found.zip})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(tests/jenkins/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@
createSignatureFiles.signArtifacts({sigtype=.sig, artifactPath=tests/jenkins/tests/jenkins/file/found.zip})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(tests/jenkins/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@
createSignatureFiles.signArtifacts({sigtype=.sig, artifactPath=tests/jenkins/tests/jenkins/file/found.zip})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(tests/jenkins/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down
2 changes: 0 additions & 2 deletions tests/jenkins/jobs/PromoteArtifacts_actions_Jenkinsfile.txt
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@
createSignatureFiles.signArtifacts({sigtype=.sig, artifactPath=tests/jenkins/artifacts/tar/vars-build/1.3.0/33/linux/x64/tar/builds/opensearch/core-plugins})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(tests/jenkins/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down Expand Up @@ -68,7 +67,6 @@
createSignatureFiles.signArtifacts({sigtype=.sig, artifactPath=tests/jenkins/tests/jenkins/file/found.zip})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(tests/jenkins/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@
createSignatureFiles.signArtifacts({sigtype=.sig, artifactPath=tests/jenkins/tests/jenkins/file/found.zip})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(tests/jenkins/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@
createSignatureFiles.signArtifacts({sigtype=.sig, artifactPath=tests/jenkins/tests/jenkins/file/found.zip})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(tests/jenkins/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@
createSignatureFiles.signArtifacts({sigtype=.sig, artifactPath=tests/jenkins/tests/jenkins/file/found.zip})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(tests/jenkins/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down
1 change: 0 additions & 1 deletion tests/jenkins/jobs/PromoteYumRepos_Jenkinsfile.txt
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@
promoteYumRepos.signArtifacts({artifactPath=/tmp/workspace/artifacts/releases/bundle/opensearch/1.x/yum/repodata/repomd.pom, sigtype=.sig, platform=linux})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(/tmp/workspace/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,6 @@ fi
publishToMaven.signArtifacts({artifactPath=/path/to/signing/manifest.yml, type=maven, platform=linux, sigtype=.asc})
signArtifacts.echo(PGP or Windows Signature Signing)
signArtifacts.fileExists(workspace/sign.sh)
signArtifacts.git({url=https://github.com/opensearch-project/opensearch-build.git, branch=main})
signArtifacts.sh(curl -sSL https://artifacts.opensearch.org/publickeys/opensearch.pgp | gpg --import -)
signArtifacts.usernamePassword({credentialsId=github_bot_token_name, usernameVariable=GITHUB_USER, passwordVariable=GITHUB_TOKEN})
signArtifacts.string({credentialsId=jenkins-signer-client-role, variable=SIGNER_CLIENT_ROLE})
Expand Down
Loading

0 comments on commit 8422ac8

Please sign in to comment.