ReleaseGuide

Release Guide

This is meant to be a living document for performing releases of Blazegraph.

Project releases are done as branches in GIT. The branch name has the general form BIGDATA_RELEASE_X_Y_Z which corresponds to version X.Y.Z. This way the project name makes it into the tag and hence into the directory when it is checked out from GIT. Use a branch for the release since there is an expectation that we will have to support the branch.

Preconditions

• Setup for Maven Central

• notify developers that we are closing the branch down for a release and wait for the all clear.

• merge all tickets into master

• push snapshot to SF master

• create release candidate branch. BLAZEGRAPH_X_Y_Z_RC

• Validate CI in branch

• Update maven versions

  ./scripts/updateVersion.sh X_Y_Z

• edit scripts/version.properties to set the new version

• Commit pom updates

  git commit -a -m "Release Version Updates for X_Y_Z"

• verify performance against standard benchmarks : SPARQL Benchmarks

  • LUBM
  • BSBM
  • Govtrack (see bigdata-perf/CI/govtrack)

• Build Artifacts (For verification) ./scripts/buildArtifacts.sh

• verify artifact deployments: For each artifact, start it with artifact-specific method and verify the version and git hash from the start-up log.

  • Verify executable jar (java -jar bigdata.jar and java -jar blazegraph.jar)
  • Verify WAR deployment (tomcat). (bigdata.war and blazegraph.war)
  • Verify Debian Deployer (Requires Ubuntu)
  • Verify RPM Deployer (Requires CentOS, RHEL, AMI)
  • Verify Tarball (unzip to local directory and run bin/blazegraph.sh start)

• verify CI builds.

• Review release notes in JIRA. Close or move any uncompleted tickets.

  • write (and commit) release notes in bigdata/src/releases (make sure the TAG URL is reflected in the release notes!). Copy and paste text version from JIRA and make any updates.

• Update the tag in Github with the release notes and binary artifacts

• Configured your git repository to be able to push to the remote sourceforge repository

$ git remote -v
origin	git@github.com:SYSTAP/bigdata.git (fetch)
origin	git@github.com:SYSTAP/bigdata.git (push)

git remote add sourceforge ssh://beebs@git.code.sf.net/p/bigdata/git
git remote add blazegraph git@github.com:blazegraph/database.git

$ git remote -v
origin	git@github.com:SYSTAP/bigdata.git (fetch)
origin	git@github.com:SYSTAP/bigdata.git (push)
sourceforge	ssh://beebs@git.code.sf.net/p/bigdata/git (fetch)
sourceforge	ssh://beebs@git.code.sf.net/p/bigdata/git (push)
blazegraph	git@github.com:blazegraph/database.git (fetch)
blazegraph	git@github.com:blazegraph/database.git (push)

#If you need to delete and repush tags see.
git tag -d 12345 #delete local
git push origin :refs/tags/12345 #delete remote

Release

• Use the release.sh script with the new version number in the form X.Y.Z. This script builds, creates a new git branch and tag, updates the POM versions, deploys to maven central, pushes the javadocs, and pushes a new branch for the reverse merge. MAKE SURE THAT YOUR KEY SIGNING CREDENTIALS ARE ENABLED. ./scripts/release.sh 2.1.0

• Post Release

  • Deploy artifacts with the release version
  • Verify the artifacts are on Maven Central
  • Verify the Javadocs are online with the correct version.
  • Copy the contents of bigdata/artifacts/ into the Github tag as release artifacts.

• (LEGACY: Now done in release.sh) Create a TAG for the release in GIT. ** The TAG is the immutable snapshot of the code as of the release. Maintenance will be done on the BRANCH from which the release was issued. The TAG should use the tags path and have the general pattern .../bigdata/tags/BIGDATA_RELEASE_X_Y_Z. git push sourceforge BIGDATA_RELEASE_X_Y_Z git tag -a BIGDATA_RELEASE_X_Y_Z -m "Bigdata release X_Y_Z" git push origin --tags git push sourceforge --tags

• (Legacy - no longer using Sourceforge) Create a new directory on sourceforge for the release [2]. Upload the release notes into the directory on sourceforge and rename as "README" (they will be displayed at the bottom of the folder listing on sourceforge once you do this and refresh the page view).

• Upload the artifacts generated by release.sh in the artifacts directory.

  • Download these artifacts and verify that the downloaded files are the same (md5/md5sum). There have been a lot of issues recently with sourceforge file uploads failing silently so you can not rely on the file being posted correctly the first time around.
  • "ant war" - generates bigdata.war (Standalone WAR deployment)
  • "ant executable jar" -- generates bigdata-X_Y_Z-bundled.jar; rename to bigdata-bundled.jar (executable jar deployment)
  • "ant deploy-artifact" - generates REL.bigdata.version.tgz (HA deployment)
  • (LEGACY) "ant javadoc" - generates javadoc. Upload to web site using rsync (see 'bigdata-project-release.txt' in dropbox for details), and verify the javadoc online.
  • (LEGACY) Rsync: rsync -qrzPl --perms --chmod=ugo+X,ugo+r -e ssh ant-build/docs www.blazegraph.com:/var/www/html/www.blazegraph.com/files/
  • (LEGACY) "ant ant-install-artifact" - generates a source code distribution. See 'bigdata-project-release.txt' in dropbox for important details on how and where to install this.
  • (LEGACY) Publish bigdata JAR to the systap maven repository: NOTE: If the fastutil version has changed you will need to upload this to the maven repository. See comments in the POM.xml file.
  • (LEGACY) Change the version in pom.xml (bump version number, remove "-SNAPSHOT", do NOT commit).
  • (LEGACY) "mvn clean deploy" - deploys the new bigdata jar to the systap maven repository (alternatively try 'ant maven-deploy").
  • (LEGACY) Append "-SNAPSHOT" to the version pom.xml and commit to resume snapshot releases in CI.

• Review the download page and edit as necessary. TODO: Document process to snapshot user guide / wiki.

• CAUTION: We maintain multiple releases. However, sourceforge will direct people to the most recently published release for which a download artifact exists. When publishing a release of a maintenance branch, you MUST (a) wait for the release to be distributed to enough mirrors that it will be assigned a download URL at which point it becomes the default download; and (b) go back and "touch" the most current release. Verify that the most current release is still the default download!!!

After actions

• make sure that 'snapshot=true' and "#javadoc=" in build.properties and that pom.xml has -SNAPSHOT for the new version (and commit).
• reverse merge to github (the release notes, build.properties and pom.xml are modified during the release)
• Verify reverse merge, e.g., using Araxis Merge.
  • SF master, github master should be equal.
  • SF release tag, github release tag should be equal.
• (Legacy) notify developers that the branch is open again.
• (Legacy) blog it [3] and elsewhere.
• (Legacy) post the release notes to the sourceforge project news (logged in as an admin, choose "News" from the tabs and then "Post New")
• (Legacy) Update the website download link
• (Legacy) Make sure the new files are set as the default download on Sourceforge
• (Legacy) send out an email announcement (we maintain a mailing list).
• send out a tinkerpop mailing list announcement
• Update Wikipedia
• update semanticweb.org (version and release date).
• review W3 Wiki page for bigdata.
• (Legacy) update the Roadmap

Maven

We publish maven artifacts for bigdata releases (starting with 1.2.3), for bigdata snapshots (from CI builds), and for bigdata dependencies that are not widely available in maven repositories. The maven repository for bigdata is:

Maven Central Guide

Maven Central Maven Deployment

PGP Key Setup Here is a guide to moving existing PGP keys.

Encrypted Passwords

Setup your maven installation to have encrypted passwords Maven Password Encryption.