[RELEASE] Release version 1.1.1

[github-actions]

Release OpenSearch and OpenSearch Dashboards 1.1.1

(As of 2021/11/17 We only make changes to OpenSearch-Dashboards to 1.1.1, therefore, OpenSearch stays 1.1.0.)

(We will release both OpenSearch and OpenSearch-Dashboards 1.1.1 as both need changes to reports plugin BE/FE)

We are pausing 1.1.1 for now due to CVE-2021-44228.

Related to this:
https://discuss.opendistrocommunity.dev/t/log4j-patch-for-cve-2021-44228/7950

OpenSearch is actively working on a patch, will need to change our Ansible playbook to it once it is ready.

Please follow the following checklist to make a release.

This release is contingent on completion of #567

Preparation

• Assign this issue to a release owner.
• Declare a pencils down date for new features to be merged. 2021/12/07 is pencils down
• Update this issue so all RELEASE- placeholders have actual dates.
• Declare a pencils down date for new features to be merged.
  - [ ] Finalize scope and feature set and update the Public Roadmap.
• Create a version label in each component repo.
• Create a release issue in every component repo that links back to this issue, update Components section with these links.
• Add 1.1.1 issue template and input manifests #1275
  - [ ] Ensure that all release issues created above are assigned to an owner in the component team.

CI/CD - 2021/12/06 - 2021/12/08

• Create Jenkins workflows that run daily snapshot builds for OpenSearch and OpenSearch Dashboards.
  • Enable builds of 1.1.1 in Jenkins now #1277
• Increment each component version to 1.1.1 and ensure working CI in component repositories.
• Make pull requests to add each component to manifests/1.1.1/opensearch-1.1.1.yml and manifests/1.1.1/opensearch-dashboards-1.1.1.yml with the corresponding checks.

Release testing - 2021/12/09 - 2021/12/10

• Code Complete (2021/12/09): Teams test their component within the distribution, ensuring integration, backwards compatibility, and perf tests pass.
• Sanity Testing (2021/12/10): Sanity testing and fixing of critical issues found by teams.

Release - 2021/12/13 - 2021/12/14

• Declare a release candidate build, and publish all test results.
• Verify all issued labeled v1.1.1 in all projects have been resolved.
• Complete documentation for this release.
• Author blog post for this release.
• Gather, review and publish release notes.
• 2021/12/14 - Publish this release on opensearch.org.
• 2021/12/14 - Publish blog post - release is launched!

Post Release

• Create release tags for each component.
• Update this template with any new or missed steps.
• Conduct a retrospect, and publish its results. [Retrospect] Release 1.1.1 #984

Patching Bugs

• [Bug]Separate OpenSearch Min Version and Bundle Version in Input Manifest #1266
• [Enhancement]Multi-Region Failover for S3 and enable Versions for duplicates #1267
• Update dashboards-reports build.sh so it builds with dashboards #1278
• [Bug]Jenkins Opensearch is building snapshot that will fail on a patch release #1279
• Fix: Relax compatible versions checks for snapshot dependencies. #1280
• Always create a maven directory, but do not download. #1281
• Fix: check for maven directory in build. #1283
• [Bug]Patch bundle tar has min version number #1286
  • Rename min bundle folder name in patch releases. #1287

Components

Component	On track	Notes
opensearch-project/OpenSearch#1670	🟢	IN_PROGRESS
opensearch-project/alerting#251	🟢	IN_PROGRESS
opensearch-project/anomaly-detection#337	🟢	IN_PROGRESS
opensearch-project/asynchronous-search#75	🟢	IN_PROGRESS
opensearch-project/common-utils#102	🟢	IN_PROGRESS
opensearch-project/cross-cluster-replication#264	🟢	IN_PROGRESS
opensearch-project/index-management#217	🟢	IN_PROGRESS
opensearch-project/job-scheduler#107	🟢	IN_PROGRESS
opensearch-project/k-NN#239	🟢	IN_PROGRESS
opensearch-project/performance-analyzer#95	🟢	IN_PROGRESS
opensearch-project/security#1509	🟢	IN_PROGRESS
opensearch-project/sql#315	🟢	IN_PROGRESS
opensearch-project/OpenSearch-Dashboards#1008	🟢	IN_PROGRESS
opensearch-project/observability#303	🟢	IN_PROGRESS
opensearch-project/alerting-dashboards-plugin#157	🟢	IN_PROGRESS
opensearch-project/index-management-dashboards-plugin#136	🟢	IN_PROGRESS
opensearch-project/anomaly-detection-dashboards-plugin#161	🟢	IN_PROGRESS
opensearch-project/dashboards-notebooks#109	🟢	IN_PROGRESS
opensearch-project/reporting#246	🟢	IN_PROGRESS
opensearch-project/dashboards-visualizations#45	🟢	IN_PROGRESS
opensearch-project/security-dashboards-plugin#874	🟢	IN_PROGRESS

Legend

Symbol	Meaning
🟢	On track with overall release
🟡	Missed last milestone
🔴	Missed multiple milestones

Issue tracking

State	Total	Bug	Enhancement
Open	2	0	0
Closed	2	2	0

Release testing

To aid in release testing we are providing all teams with access to docker images of OpenSearch to aid in the rapid setup/tear down for testing, for additional discussion about test documentation see #933

All links and commands for testing will be available once the artifacts are ready.

---

• Use Docker-Compose to setup a cluster

  • Download this gist as docker-compose.yml on your machine for 1.1.1
  • Get latest image versions docker-compose pull
  • Start the cluster docker-compose up

• OpenSearch docker

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch:1.1.1 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:1.1.1
    • Connect command curl https://localhost:9200
  • Start with security
    • Docker command docker pull opensearchstaging/opensearch:1.1.1 && docker run -it -p 9200:9200 -e "discovery.type=single-node" opensearchstaging/opensearch:1.1.1
    • Connect command curl --insecure https://admin:admin@localhost:9200

• OpenSearch-Dashboards docker

  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:1.1.1 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:1.1.1
    • URL https://localhost:5601
  • Start without security
    • Docker command docker pull opensearchstaging/opensearch-dashboards:1.1.1 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:1.1.1
    • URL https://localhost:5601

• Use TARs to deploy OpenSearch Manually

  • OpenSearch - Build distribution 8

    • arm64 [manifest] [tar]
    • x64 [manifest] [tar]

  • OpenSearch Dashboards - Build distribution 7

    • arm64 [manifest] [tar]
    • x64 [manifest] [tar]

From Docker Release

[peternied]

This release's date is contingent on #567 completion, this is due to manual processes and the intention is to mitigate risk of accidentally including a 1.2.0+ changes in the 1.1.1 components.

[dblock]

We should assemble a 1.1.1 manifest with all the components though earlier than later and include it in automated builds.

[peternied]

@anirudha For this point release can you help me work back the dates that we are targeting? Roughing working off of @bbarani planning notes we had the following:

• 1 day for updating build versions and creating branches
• 1 day for getting a viable distribution of OpenSearch and OpenSearch Dashboards
• 3 day for executing all testing
• 1 day for all release notes/notifications

This isn't factoring in the time itself to make the actual fixes for the point release. Do we need additional time account before starting?

Starting the day after the 1.2 release, Development starts on Nov 17th and +6 days is Release on the Nov 24th. This is a super tight window with Thanksgiving impacting team availability if there is any day/day slip

[dblock]

I renamed the issue back or the bot keeps creating new ones.

[peternied]

I renamed the issue back or the bot keeps creating new ones.

Stop being so helpful automation!

[peternied]

Offline: @anirudha confirmed that are targeting the release on Nov 24th

[peternied]

[Triage] Moved this to @bbarani to confirm the release dates and manage the work back schedule. Since this is a point release there is flexibility, but the more flexing that is done the more risk that is introduced to the schedule.

[bbarani]

1.1.1 is currently schedule for Nov 23 2021. Lets work backward from the release date to close out the preparation work.

[gaiksaya]

[Triage] @bbarani will be updating the final release date.

[bbarani]

We are targeting Dec 2 for 1.1.1 release.

[peterzhuamazon]

Hi @anirudha

Would you mind confirming that we will only change dashboards-reports FrontEnd plugin?

If so, we will only release OpenSearch-Dashboards Bundle 1.1.1 with TAR + Docker Images (x64 + arm64).
The OpenSearch bundle will not be changed, and will remain at version 1.1.0.

Thanks,
Peter

[peterzhuamazon]

$ curl https://localhost:9200/ -u admin:admin --insecure
{
  "name" : "3c22fb21b0a4.ant.amazon.com",
  "cluster_name" : "opensearch",
  "cluster_uuid" : "Q3w1csxZRhWaNZNz1Tv1_w",
  "version" : {
    "distribution" : "opensearch",
    "number" : "1.1.0",
    "build_type" : "tar",
    "build_hash" : "15e9f137622d878b79103df8f82d78d782b686a1",
    "build_date" : "2021-10-04T21:29:03.079792Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}


$ curl https://localhost:9200/_cat/plugins -u admin:admin --insecure
3c22fb21b0a4.ant.amazon.com opensearch-alerting                  1.1.0.0
3c22fb21b0a4.ant.amazon.com opensearch-anomaly-detection         1.1.0.0
3c22fb21b0a4.ant.amazon.com opensearch-asynchronous-search       1.1.0.0
3c22fb21b0a4.ant.amazon.com opensearch-cross-cluster-replication 1.1.0.0
3c22fb21b0a4.ant.amazon.com opensearch-index-management          1.1.0.0
3c22fb21b0a4.ant.amazon.com opensearch-job-scheduler             1.1.0.0
3c22fb21b0a4.ant.amazon.com opensearch-knn                       1.1.0.0
3c22fb21b0a4.ant.amazon.com opensearch-notebooks                 1.1.0.0
3c22fb21b0a4.ant.amazon.com opensearch-performance-analyzer      1.1.0.0
3c22fb21b0a4.ant.amazon.com opensearch-reports-scheduler         1.1.1.0
3c22fb21b0a4.ant.amazon.com opensearch-security                  1.1.0.0
3c22fb21b0a4.ant.amazon.com opensearch-sql                       1.1.0.0

[peterzhuamazon]

$ curl -s localhost:5601/api/status | jq
{
  "name": "3c22fb21b0a4.ant.amazon.com",
  "uuid": "9a486556-2e4c-4e8f-9666-1d542c4d1981",
  "version": {
    "number": "1.1.0",
    "build_hash": "eebb950d0902eea3e384d7ff6485363475109448",
    "build_number": 36526,
    "build_snapshot": false
  },
  "status": {
    "overall": {
      "since": "2021-12-09T23:46:16.475Z",
      "state": "green",
      "title": "Green",
      "nickname": "Looking good",
      "icon": "success",
      "uiColor": "secondary"
    },

$ ./bin/opensearch-dashboards-plugin list
alertingDashboards@1.1.0.0
anomalyDetectionDashboards@1.1.0.0
ganttChartDashboards@1.1.0.0
indexManagementDashboards@1.1.0.0
notebooksDashboards@1.1.0.0
queryWorkbenchDashboards@1.1.0.0
reportsDashboards@1.1.1.0
securityDashboards@1.1.0.0
traceAnalyticsDashboards@1.1.0.0

[dblock]

This release is postponed given the log4j RCE, we'll re-evaluate whether we do it at all. See #1290.

[dblock]

Let's close all 1.1.1 issues since there will be no such thing? @peterzhuamazon

[peterzhuamazon]

Let's close all 1.1.1 issues since there will be no such thing? @peterzhuamazon

Shall we just close all plugin repo issue 1 by 1 then close this one?

[dblock]

Let's close all 1.1.1 issues since there will be no such thing? @peterzhuamazon

Shall we just close all plugin repo issue 1 by 1 then close this one?

Yes please, but we need a paragraph of why we're not releasing 1.1.1 from @bbarani.

[bbarani]

We will not be releasing 1.1.1 version of OpenSearch as we decided to prioritize the release of 1.2.x version of OpenSearch to fix critical log4j vulnerabilities. Please upgrade to the latest version of OpenSearch to take advantage of the new features along with multiple bug fixes and patches.