Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding documentation for maven publishing automation #646

Merged
merged 46 commits into from
Sep 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
2e8ae1b
Adding workflow to automate maven publish
jainr Aug 24, 2022
7b0ea3e
Merge pull request #1 from jainr/rijai/mavenworkflow
jainr Aug 24, 2022
e0c8134
Updating maven workflow
jainr Aug 24, 2022
9f3dac3
Updating maven workflow
jainr Aug 24, 2022
28c1cb1
Updating maven workflow file
jainr Aug 24, 2022
475046e
Adding gpg passphrase
jainr Aug 24, 2022
3c45ffd
Fixing signing failure
jainr Aug 24, 2022
f059143
Fixing signing failure
jainr Aug 25, 2022
868b6c1
Reverting back to not passing passphare value for Java setup action
jainr Aug 25, 2022
a175bd7
Trying a new plugin
jainr Aug 26, 2022
7ed1dd7
Adding sbt ci-release
jainr Aug 26, 2022
0734ce2
Updating version for sbt ci-release
jainr Aug 26, 2022
99305ef
Bumping version for testing
jainr Aug 26, 2022
f0f86a8
Addig sbt compile explicitly
jainr Aug 26, 2022
fb1de0c
Addig sbt compile explicitly
jainr Aug 26, 2022
d5cc3b8
Adding triger for tags
jainr Aug 26, 2022
515be6f
fixing ioctl issue
jainr Aug 26, 2022
420dd43
Adding decode key script to resolve signing issue
jainr Aug 27, 2022
9ed7b73
Adding decode key script to resolve signing issue
jainr Aug 27, 2022
9da579a
Adding GPG_OPTIONS to decode script
jainr Aug 29, 2022
0e5537d
Adding tty options
jainr Aug 29, 2022
5347c64
Bringing TTY back
jainr Aug 29, 2022
c782600
Upgrading pgp version
jainr Aug 30, 2022
8111247
Debugging GPG Signing part in maven publish workflow
jainr Aug 30, 2022
b58b8d7
Manual trigger workflow
jainr Aug 30, 2022
7a3dd3d
Manual trigger workflow
jainr Aug 30, 2022
67b291c
Manual trigger workflow
jainr Aug 30, 2022
702857b
Adding gpg import
jainr Aug 31, 2022
af3d793
Falling back on ci-release pipeline
jainr Aug 31, 2022
7330b8c
Falling back on ci-release pipeline
jainr Aug 31, 2022
a4a4984
Changing the hsot for sonatype credential
jainr Aug 31, 2022
f3f0dca
Cleaning up the workflow file to do final test
jainr Aug 31, 2022
03ff93b
Cleaning up and adding some more comments
jainr Aug 31, 2022
9ffa0a9
Adding more comments
jainr Aug 31, 2022
7f2afd5
Fixing merge conflicts
jainr Aug 31, 2022
ff73684
Merge branch 'main' of https://github.com/linkedin/feathr into rijai/…
jainr Aug 31, 2022
7c6cdbc
Merge branch 'rijai/mavenworkflow' into main
jainr Aug 31, 2022
64ce340
Merge pull request #2 from jainr/main
jainr Aug 31, 2022
0d212ff
Merge branch 'rijai/mavenworkflow' of github.com:jainr/feathr into ri…
jainr Aug 31, 2022
9b23571
Resolving merge issue
jainr Sep 6, 2022
b952507
Merge branch 'main' of github.com:jainr/feathr into main
jainr Sep 6, 2022
8ddf87d
Merging from main
jainr Sep 6, 2022
9c85e67
Adding documentation for maven workflow
jainr Sep 6, 2022
62a882d
Fixing wrong env credentials
jainr Sep 6, 2022
7faebee
Merge branch 'main' of github.com:jainr/feathr into rijai/mavenworkflow
jainr Sep 6, 2022
1a48648
Adding doc for Maven Automation steps and gotchas
jainr Sep 6, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/publish-to-maven.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ jobs:
java-version: "8"
distribution: "temurin"
server-id: ossrh
server-username: OSSRH_USERNAME
server-password: OSSRH_PASSWORD
server-username: SONATYPE_USERNAME
server-password: SONATYPE_PASSWORD
gpg-private-key: ${{ secrets.MAVEN_GPG_PRIVATE_KEY }}
gpg-passphrase: MAVEN_GPG_PASSPHRASE
gpg-passphrase: PGP_PASSPHRASE

# CI release command defaults to publishSigned
# Sonatype release command defaults to sonaTypeBundleRelease
Expand Down
66 changes: 59 additions & 7 deletions docs/dev_guide/publish_to_maven.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,15 +99,67 @@ parent: Developer Guides
- If the published jar fails to run in Spark with error `java.lang.UnsupportedClassVersionError: com/linkedin/feathr/common/exception/FeathrInputDataException has been compiled by a more recent version of the Java Runtime (class file version 62.0), this version of the Java Runtime only recognizes class file versions up to 52.0`, make sure you complied with the right Java version with -java-home parameter in sbt console.

## CI Automatic Publishing

(TBD)

There is a Github Action that automates the above process, you can find it [here](../../.github/workflows/publish-to-maven.yml). This action is triggered anytime a new tag is created, which is usually for release purposes. To manually trigger the pipeline for testing purposes tag can be created using following commands

```bash

git tag -a <version> -m "Test tag"
git push --tags

```

Following are some of the things to keep in mind while attempting to do something similar, since signing issues are hard to debug.

1. There are four secrets that needs to be set for the Github workflow action to work
```bash
PGP_PASSPHRASE: This is the passphrase that you provided during GPG key pair creation.
PGP_SECRET: The Private Key from GPG key pair created above.
SONATYPE_PASSWORD: Password for oss sonatype repository.
SONATYPE_USERNAME: Username for oss sonatype repository.
```

1. As noted in previous steps, you need to use gpg to create a public-private key pair on your dev machine. The public key is uploaded to a Key server for verification purpose. The private gpg key is used to sign the package being uploaded to maven. We export this private key to be used for signing on Github agent using the following command

```bash

gpg --export-secret-keys --armor YOUR_PRIVATE_KEY_ID > privatekey.asc
```
Copy everything from the privatekey.asc file and put it as Github secret with name PGP_SECRET

To get the private key id you can run the following command and use id under section sec (stands for secret)

```bash
$ gpg --list-secret-keys
/Users/myuser/.gnupg/pubring.kbx
-------------------------------

sec abc123 2022-08-24 [SC] [expires: 2024-08-23]
3203203SD.......
uid [ultimate] YOUR NAME <YOUR_EMAIL>
ssb abc123 2022-08-24 [E] [expires: 2024-08-23]
```
1. Make sure you are using the right credential host in [sonatype.sbt](../../sonatype.sbt)
- For accounts created before Feb 2021 use __oss.sonatype.org__
- For accounts created after Feb 2021 use __s01.oss.sonatype.org__


1. Make sure you are using latest release of sbt-pgp package, or atleast the one close to the dev box on which gpg keypair is generated. You can change the version in [build.sbt](../../build.sbt)
```bash
addSbtPlugin("com.github.sbt" % "sbt-pgp" % "2.1.2")
```

1. We are using sbt-ci-release plugin, that makes the publishing process easier. Read more about it [here](https://github.com/sbt/sbt-ci-release). You can add this in [build.sbt](../../build.sbt)
```bash
addSbtPlugin("com.github.sbt" % "sbt-ci-release" % "1.5.10")
```
### References

https://github.com/xerial/sbt-sonatype
- https://github.com/xerial/sbt-sonatype

- https://www.linuxbabe.com/security/a-practical-guide-to-gpg-part-1-generate-your-keypair

https://www.linuxbabe.com/security/a-practical-guide-to-gpg-part-1-generate-your-keypair
- https://central.sonatype.org/publish/publish-guide/#deployment

https://central.sonatype.org/publish/publish-guide/#deployment
- https://www.scala-sbt.org/1.x/docs/Using-Sonatype.html

https://www.scala-sbt.org/1.x/docs/Using-Sonatype.html
- https://github.com/sbt/sbt-ci-release