Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RELEASE] Release version 2.17.1 #5046

Closed
31 of 62 tasks
github-actions bot opened this issue Sep 21, 2024 · 31 comments
Closed
31 of 62 tasks

[RELEASE] Release version 2.17.1 #5046

github-actions bot opened this issue Sep 21, 2024 · 31 comments
Assignees

Comments

@github-actions
Copy link
Contributor

github-actions bot commented Sep 21, 2024

Release OpenSearch and OpenSearch Dashboards 2.17.1

I noticed that a manifest was automatically created in manifests/2.17.1. Please follow the following checklist to make a release.

How to use this issue

This Release Issue

This issue captures the state of the OpenSearch release, its assignee (Release Manager) is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. For more information check the the Release Process OpenSearch Guide.

Please refer to the following link for the release version dates: Release Schedule and Maintenance Policy.

Entrance Criteria

Criteria Status Description  Comments
Each component release issue has an assigned owner 🟢
Documentation draft PRs are up and in tech review for all component changes 🟢
Sanity testing is done for all components 🟢
Code coverage has not decreased (all new code has tests) 🟢
Release notes are ready and available for all components 🟢
Roadmap is up-to-date (information is available to create release highlights) 🟢
Release ticket is cut, and there's a forum post announcing the start of the window 🟢
Any necessary security reviews are complete 🟢

OpenSearch 2.17.1 exit criteria status:

Criteria Status Description  Comments
Performance tests are run, results are posted to the release ticket and there no unexpected regressions 🟢
No unpatched vulnerabilities of medium or higher severity that have been publicly known for more than 60 days 🟢
Documentation has been fully reviewed and signed off by the documentation community. 🟢
All integration tests are passing 🟢

OpenSearch-Dashboards 2.17.1 exit criteria status:

Criteria Status Description  Comments
Documentation has been fully reviewed and signed off by the documentation community 🟢
No unpatched vulnerabilities of medium or higher severity that have been publicly known for more than 60 days 🟢
All integration tests are passing 🟢

Preparation

Campaigns

Version Increment - _Ends September 24th 2024

Code Complete - _Ends September 24th 2024

Release Candidate Creation and Testing - _Ends September 25th 2024

Performance testing validation - _Ends September 25th 2024

  • Post the benchmark-tests
  • Longevity tests do not show any issues.

Pre Release - _Ends September 30th 2024

Release - _Ends October 1st 2024

Release Checklist.


Release Checklist

Pre-Release activities

  • Promote Repos.
      • OS
      • OSD
  • Promote Artifacts.
      • Windows
      • Linux Debian
      • Linux RPM
      • Linux TAR
  • Consolidated Release Notes.

Release activities

  • Docker Promotion.
  • Release Validation part 1.
      • OpenSearch and OpenSearch Dashboard Validation.
      • Validate the native plugin installation.
  • Merge consolidated release notes PR.
  • Website and Documentation Changes.
      • Merge staging website PR.
      • Promote the website changes to prod.
      • Add website alert.
  • Release Validation part 2.
      • Validate the artifact download URL's and signatures.
  • Release Validation part 3.
      • Trigger the validation build (Search for Completed validation for <> in the logs).
  • Maven Promotion.
  • Publish blog posts.
  • Advertise on Social Media.
  • Post on public slack and Github Release issue.


Post Release

Components

Replace with links to all component tracking issues.

Component On track Release Notes
{COMPONENT_ISSUE_LINK} {INDICATOR} {STATUS}
Legend

Symbol Meaning
🟢 On track with overall release
🟡 Missed last milestone
🔴 Missed multiple milestones

@peterzhuamazon
Copy link
Member

peterzhuamazon commented Sep 23, 2024

We have recently found an issue related to OSD, due to the Chrome browser update:

The issue seems coming from chrome itself, with an issue opened in here:

A temp workaround in OSD core has been sent as well:

We want to analyze the situation and see if it grants a potential patch release such as 2.17.1.

  • How many versions of Chrome browser carry this bug?
  • Is there a quick workflow that users can apply, so that we dont need to do a immediate new patch release?1
  • What is the impact of this issue?

cc: @Pallavi-AWS @dblock @anirudha @ashwin-pc @AMoo-Miki @getsaurabh02

Thanks.

@getsaurabh02
Copy link
Member

Thanks @peterzhuamazon for adding details. @ashwin-pc Can we also add if there is any workaround from the Chrome side, which users can do and we can share in form of documentation?

I experienced the same issue while using regular browsing, but it worked in private browsing. I cleared my cache and cookies, which resolved the problem.

@ashwin-pc
Copy link
Member

Thanks for opening the issue @peterzhuamazon @getsaurabh02. To answer your questions:

How many versions of Chrome browser carry this bug?

its started in version 129 and still exists in the latest 130 version

Is there a quick workflow that users can apply, so that we dont need to do a immediate new patch release?1

Users can use an alternate browser Both Edge and Safari dont seem to have this bug. That being said, many of the other browsers are built on top of Chromium, so its very likely that they will also have this bug when they update to use the latest version of Chrome.

What is the impact of this issue?

Discover is unusable since the results table is covered by a red mask

Can we also add if there is any opensearch-project/OpenSearch-Dashboards#8250 (comment) from the Chrome side, which users can do and we can share in form of documentation?

The work arounds suggested here expect the user to change the CSS on their page using the developer tools panel. Using an alternate browser is an easier workaround though and the suggestions in the issue were most for the code fix we needed to make.

@getsaurabh02
Copy link
Member

Thanks @ashwin-pc. One last thing since user here mentioned the below workaround irrespective of the CSS update. Is it not working?

I experienced the same issue while using regular browsing, but it worked in private browsing. I cleared my cache and cookies, which resolved the problem.

Adding maintainers for any feedback/concerns @AMoo-Miki @dbbaughe @msfroh

@ashwin-pc
Copy link
Member

The user there was calling out that the issue for them was not always reproducible, but their fix/workaround does always work. The issue for me and @AMoo-Miki however was always reproducible.

@getsaurabh02
Copy link
Member

getsaurabh02 commented Sep 23, 2024

Thanks, @ashwin-pc.

I will allow today for any final feedback or concerns from the maintainers. If none are raised, we will start the code freeze tomorrow and proceed to release as soon as the final RC is ready. The release window is typically one week, so we expect to have the final RC ready for release by Tuesday, October 1st.

Please ensure that no other commits (aside from those discussed on this thread) are included in this patch release. Any other changes on the 2.17 branch, apart from those tagged in the patch issue, will be reverted.

Tagging @Divyaasm and @peterzhuamazon for support on this release. We will begin the announcement on the public slack.

@getsaurabh02
Copy link
Member

@ashwin-pc. Few more questions, while we prep for the patch release in parallel:

The user there was calling out that the issue for them was not always reproducible, but their fix/workaround does always work. The issue for me and @AMoo-Miki however was always reproducible.

Can we please confirm this, since it will determine the urgency/need of the fix and patch release.

its started in version 129 and still exists in the latest 130 version

Is there any eta on the issue being tracked from the Chrome side? Is it okay to ask user to use other versions of Chrome of other browser, since this issue anyways affects all versions of OSD.

@Divyaasm Divyaasm removed the untriaged Issues that have not yet been triaged label Sep 23, 2024
@Divyaasm Divyaasm self-assigned this Sep 23, 2024
@ashwin-pc
Copy link
Member

@ashwin-pc. Few more questions, while we prep for the patch release in parallel:

The user there was calling out that the issue for them was not always reproducible, but their fix/workaround does always work. The issue for me and @AMoo-Miki however was always reproducible.

Can we please confirm this, since it will determine the urgency/need of the fix and patch release.

Yes we have confirmed the issue is always reproducable and we have even submitted the exact reproduction steps to the chrome team which they were also able to reproduce it on their end

its started in version 129 and still exists in the latest 130 version

Is there any eta on the issue being tracked from the Chrome side? Is it okay to ask user to use other versions of Chrome of other browser, since this issue anyways affects all versions of OSD.

They havent provided me any ETA yet on the issue. They have acknowledged the issue though. Asking users to use another browser is a valid workaround but given that al major browsers besides Firefox and Safari are built on top of Chrome, this issue will grow in size even if it is patched in Chrome. I dont think we should rely on the browser fix alone to mitigate this issue.

@AMoo-Miki
Copy link
Contributor

Is there any eta on the issue being tracked from the Chrome side? Is it okay to ask user to use other versions of Chrome of other browser, since this issue anyways affects all versions of OSD.

@getsaurabh02, according to Chrome's issue tracking system, the bug we reported is marked as a P1 with an S2 level, and is slated for the 131 release. If the fix makes it into their 131 release, it is expected to be available on November 12, 2024. However, if the Chrome team misses the 131 release window, their next opportunity to address the issue will be with the 132 release on January 7, 2025. In the best-case scenario, we are anticipating a minimum of another 50 days before Chrome users will see a resolution to this bug. The next best scenario would involve a wait of 106 days.

@Divyaasm Divyaasm pinned this issue Sep 24, 2024
@eirsep
Copy link
Member

eirsep commented Sep 24, 2024

opensearch-project/security-analytics#1319
opensearch-project/security-analytics#1258
Users have reported bugs in threat intelligence feature which are critical to correct functioning and causing stuck tasks if they are not fixed

Cc: @getsaurabh02

@Divyaasm
Copy link
Collaborator

Divyaasm commented Sep 24, 2024

Please review the list of commits added to 2.17 branch after minor release

And kindly include only the commits that are related to CVE, critical bug fixes or backward incompatible fixes (for non-experimental feature) or integ-test-fixes which were already tagged in this issue.

Revert all the commits which are not applicable.

Attaching 2.17 build manifest for commit reference
OS manifest
OSD manifest

List of commits:
OS:
opensearch-project/OpenSearch#15935

k-nn:
opensearch-project/k-NN#2132
opensearch-project/k-NN#2121
opensearch-project/k-NN#2123

ml-commons:
opensearch-project/ml-commons#2966
opensearch-project/ml-commons#2957
opensearch-project/ml-commons#2953
opensearch-project/ml-commons#2947
opensearch-project/ml-commons#2940
opensearch-project/ml-commons#2936

OSD:
https://github.com/opensearch-project/OpenSearch-Dashboards/commits/2.17/ - Commits after Sep 13th

opensearch-dashboards-functional-test:
opensearch-project/opensearch-dashboards-functional-test@1d64b04
opensearch-project/opensearch-dashboards-functional-test@8f8ec2a

anomaly-detection-dashboards-plugin
opensearch-project/anomaly-detection-dashboards-plugin@e803545 - Reverted

dashboards-notifications:
https://github.com/opensearch-project/dashboards-notifications/commits/2.17/
Commits from Sep 17th

alerting-dashboards-plugin:
https://github.com/opensearch-project/alerting-dashboards-plugin/commits/2.17/
Commits from Sep 17th

@getsaurabh02
Copy link
Member

getsaurabh02 commented Sep 24, 2024

@eirsep is threat intelligence a new feature and under feature flag? Why these cannot wait until until 2.18 given this was not called out yet? Can you elaborate the impact? Patch releases are only for critical fixes (not important), also exercised on prominent code path.

https://github.com/opensearch-project/security-analytics/issues/1319
https://github.com/opensearch-project/security-analytics/issues/1258

@Zhangxunmt
Copy link
Contributor

The above commits in ml-commons are all bug fixes and tutorial document improvement. They are all needed in 2.17.1.

@vamshin
Copy link
Member

vamshin commented Sep 24, 2024

The above commits in k-NN are all critical performance improvements. These improvements are mainly in the areas of disk optimized vector search, which is a new feature being launched in 2.17.0.
These fixes have gone through the perf benchmarking and testing. They are all needed in 2.17.1.

@getsaurabh02
Copy link
Member

getsaurabh02 commented Sep 24, 2024

@Zhangxunmt Thanks for dropping in the comment. Can you please help anwer few of the questions on the fixes to help evaluate the urgency/criticality :

  • Is this related to new feature introduced in 2.17?
  • Is this feature under a feature flag or explicit setting?
  • How does this bug surface? Is it the most exercised flow?
  • What is the impact of this bug?
  • Why was this not called out yet?
  • Why this cannot wait until 2.18?

I will like to iterate that this patch release should only contain critical bugs or cve fixes to unblock open-source users. For ex this change doesn't look like a critical fix.

@Zhangxunmt
Copy link
Contributor

Zhangxunmt commented Sep 24, 2024

  • Is this related to new feature introduced in 2.17?

Yes they are all related to the new feature in 2.17.

  • Is this feature under a feature flag or explicit setting?

Yes, 1 more feature flag to include in this PR. opensearch-project/ml-commons#2982

  • How does this bug surface? Is it the most exercised flow?

These are the bugs after the bug bash in 2.17. The bug bash happened only after 2.17 RC was available. After the bugs fixed, there weren't enough time left for 2.17.

  • What is the impact of this bug?

These bugs will impact the usefulness of the new feature in 2.17.

  • Why was this not called out yet?

They were called out after multiple times before the release. But they were not included in the final RC.

  • Why this cannot wait until 2.18?

This comes along with the 2.17 features so it's better to have them in 2.17.

@shatejas
Copy link

opensearch-project/k-NN#2132
opensearch-project/k-NN#2140

  • Is this related to new feature introduced in 2.17?

Both are performance related
2132 - Customers will not get concurrent segment search if the setting is auto introduced in 2.17 on core
2140 - Customers will see higher latencies due to a refactor done to reduce memory footprint and support disk based ann both of which are introduced in 2.17

Is this feature under a feature flag or explicit setting?
2132 - yes its a concurrent segment search setting
2140 - no

How does this bug surface? Is it the most exercised flow?
2132 - Slower query latencies for customers with auto setting
2140 - yes performance degradation on force merge for all customers

What is the impact of this bug?

Performance impact

Why was this not called out yet?

These were merged pretty close to release and the performance impact was identified as a part of review nightly benchmark runs metrics. The review happens weekly

Why this cannot wait until 2.18?

User will have a bad experience with higher latencies

@getsaurabh02
Copy link
Member

@Zhangxunmt @shatejas Anything introduced as a new feature in 2.17 or under a Feature Flag do not qualify as a critical fix for patch release. Critical Fixes are only for changes where an open-source user will need to revert back due to failure or poor experience post upgrade.

Can we please eliminate the ones which are new features or under feature flag, and list those still qualifying as critical.

@ryanbogan
Copy link
Member

ryanbogan commented Sep 25, 2024

opensearch-project/k-NN#2123

  • Is this related to new feature introduced in 2.17?
    Yes, this is the only way for users to know when the cache is full for a new feature (quantization) in 2.17.
  • Is this feature under a feature flag or explicit setting?
    No
  • How does this bug surface? Is it the most exercised flow?
    This bug occurs whenever a user bulk ingests data (standard use) and the cache runs out of space.
  • What is the impact of this bug?
    The user will not be able to tell if the cache is full, which could cause performance issues for the cluster.
  • Why was this not called out yet?
    This was merged close to the release deadline and was supposed to be included pre-release.
  • Why this cannot wait until 2.18?
    User can run into performance issues with no way to see the root cause and fix. This will give them a bad experience with the feature that we released.

@jmazanec15
Copy link
Member

opensearch-project/k-NN#2121

  • Is this related to new feature introduced in 2.17?
    Yes
  • Is this feature under a feature flag or explicit setting?
    No
  • How does this bug surface? Is it the most exercised flow?
    User has gap in mental model of the feature
  • What is the impact of this bug?
    illogical user experience. fairly minor but also low risk.
  • Why was this not called out yet?
    Not sure
  • Why this cannot wait until 2.18?
    It probably can, although its very low risk as its just a 1 character change

@amsiglan
Copy link

amsiglan commented Sep 25, 2024

[Notifications] Fix for broken deny list and update to check for all resolved IPs in the deny list to be included in this patch release - opensearch-project/notifications#964

@Divyaasm
Copy link
Collaborator

Release Candidate Details for 2.17.1

OpenSearch - Build 10331
OpenSearch Dashboards - Build 7969

  • Use the following Docker-Compose to setup a cluster

    docker-compose.yml

      
    version: '3'
    services:
      opensearch-node1:
        image: opensearchstaging/opensearch:2.17.1.10331
        container_name: opensearch-node1
        environment:
          - cluster.name=opensearch-cluster
          - node.name=opensearch-node1
          - discovery.seed_hosts=opensearch-node1,opensearch-node2
          - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
          - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
          - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
        ulimits:
          memlock:
            soft: -1
            hard: -1
          nofile:
            soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
            hard: 65536
        volumes:
          - opensearch-data1:/usr/share/opensearch/data
        ports:
          - 9200:9200
          - 9600:9600 # required for Performance Analyzer
        networks:
          - opensearch-net
      opensearch-node2:
        image: opensearchstaging/opensearch:2.17.1.10331
        container_name: opensearch-node2
        environment:
          - cluster.name=opensearch-cluster
          - node.name=opensearch-node2
          - discovery.seed_hosts=opensearch-node1,opensearch-node2
          - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
          - bootstrap.memory_lock=true
          - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
        ulimits:
          memlock:
            soft: -1
            hard: -1
          nofile:
            soft: 65536
            hard: 65536
        volumes:
          - opensearch-data2:/usr/share/opensearch/data
        networks:
          - opensearch-net
      opensearch-dashboards:
        image: opensearchstaging/opensearch-dashboards:2.17.1.7969
        container_name: opensearch-dashboards
        ports:
          - 5601:5601
        expose:
          - "5601"
        environment:
          OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
        networks:
          - opensearch-net
    volumes:
      opensearch-data1:
      opensearch-data2:
    

    networks:
    opensearch-net:

    • Download the above docker-compose.yml on your machine.
    • Get latest image versions docker-compose pull.
    • Start the cluster docker-compose up.
  • OpenSearch docker 2.17.1.10331

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch:2.17.1.10331 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.17.1.10331
      • Connect command curl http://localhost:9200/
    • Start with security
      • Docker command
      docker pull opensearchstaging/opensearch:2.17.1.10331 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!" opensearchstaging/opensearch:2.17.1.10331
      
      • Connect command curl --insecure 'https://admin:myStrongPassword123!@localhost:9200/'
  • OpenSearch-Dashboards docker 2.17.1.7969

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.1.7969 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.17.1.7969
      • URL http://localhost:5601/
    • Start with security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.1.7969 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.17.1.7969
      • URL http://localhost:5601/
  • Use TARs to deploy OpenSearch Manually

Check how to install opensearch and dashboards on different platforms

@opensearch-ci-bot
Copy link
Collaborator

opensearch-ci-bot commented Sep 26, 2024

Core Components CommitID(after 2024-09-27) & Release Notes info

Repo Branch CommitID Commit Date Release Notes Exists
OpenSearch [1893d20797e30110e5877170e44d42275ce5951e] False
OpenSearch-Dashboards [62cc0320399aef63aa09689aaaf000adafbedeef] False
alerting [bb01d2724dbfa4d2851d840203b84c6c4745be85] False
alertingDashboards [d9fe4d06369717fb7e3a3ca2bb20f4751c8b975f] False
anomaly-detection [59ec24d9c3f6033c3f9cdee035e366cecf694224] False
anomalyDetectionDashboards [296b44d811d7b0adbe4ef8218d6f84a2757ac484] False
assistantDashboards [bed2c93e0f9b29f616fc0eb44cecc8eb1f8c5dda] False
asynchronous-search [a31fb4268dd0bec733aef4e1764a8487deb2e053] False
common-utils [80e0945a1e7a7c1ee2725a9dcff1c5d4ac8d46f0] False
cross-cluster-replication [d72750e1e7c3bafc16af217cfa1d52fa83f50f22] False
custom-codecs [49130900d9006e8f00cb4f55d93b0ba9ca3e1a60] False
customImportMapDashboards [8997a363fb6f25892e7f8873eecd1ea5bb53c42f] False
flow-framework [e907cd00eb610e1aa7685ff9887f7a18c0d09904] False
functionalTestDashboards [2.17] False
ganttChartDashboards [c40915b7534466832afad19042d422e347c05c40] False
geospatial [51510fa0a424140562fa795f3430f1af0286aa86] False
index-management [635a535951b683fd9f36d99b927a7cc5d9b30470] False
indexManagementDashboards [3ef3b556eb8b23557aa430e23a1bdeec8aca2aa2] False
job-scheduler [91825ce10c8075dd5bbad5de0c309968d610658e] False
k-NN [5dc38430d24431e9545b01f65d461ebc96d89f6c] False
ml-commons [9e4844f112fdc05c719475aa52b093bed6ae2497] False
mlCommonsDashboards [8a01341cadbeffd1d51e6156189eea1a096ed3e8] False
neural-search [9e529b8b92a967242a50bd3a3c4e41dbf766e4f5] False
notifications [df45059728f4ee0ee62bc1167df08ad4113cd7db] False
notifications-core [df45059728f4ee0ee62bc1167df08ad4113cd7db] False
notificationsDashboards [dedf3851752da6da58ef4357c4a12188315b74e7] False
observabilityDashboards [62cc940a727134cca3f2fa35ca232fef1be9c5e4] False
opensearch-observability [552e04e9064806b6a8b77a24faf2f97e1f8fd328] False
opensearch-reports [5af0de7a195d08ddf9410db0d2bb3b9d2cf4657d] False
opensearch-system-templates [3ffa8567a980d6a9f1f2c37d6ccf4d47804dd29b] False
performance-analyzer [759d2ef6604625e1f41dc0eb6d27dbbbfa8ccda8] False
query-insights [7183618210865b69af20e817ec899085febeb7ba] False
queryWorkbenchDashboards [b92d5cec8be1ecc75bb0285d530910a2f87a0b82] False
reportsDashboards [41815cb16ee41cad43b68497e1373342c473ae39] False
searchRelevanceDashboards [29225dbb6b7024596ec13880796eb4516a157096] False
security [d47df0642f4a8db7617adfdcd2cac539dd3dbae5] False
security-analytics [d7e9295503f1106047ba4cf126c5998a7eda2cb2] False
securityAnalyticsDashboards [7d22d22d8714bea0576649289acfc6b0cd9bc121] False
securityDashboards [6bc2e342da90a938956289f6bc36c6313a9a07b7] False
skills [e526edea4cb168e475684242e0b261c1d57abce4] False
sql [fe219794aecd57cd9c8aa4711d676302a14893a8] False

@Divyaasm
Copy link
Collaborator

Latest Release Candidate Details for 2.17.1

OpenSearch - Build 10332
OpenSearch Dashboards - Build 7969

  • Use the following Docker-Compose to setup a cluster

    docker-compose.yml

      
    version: '3'
    services:
      opensearch-node1:
        image: opensearchstaging/opensearch:2.17.1.10332
        container_name: opensearch-node1
        environment:
          - cluster.name=opensearch-cluster
          - node.name=opensearch-node1
          - discovery.seed_hosts=opensearch-node1,opensearch-node2
          - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
          - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
          - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
        ulimits:
          memlock:
            soft: -1
            hard: -1
          nofile:
            soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
            hard: 65536
        volumes:
          - opensearch-data1:/usr/share/opensearch/data
        ports:
          - 9200:9200
          - 9600:9600 # required for Performance Analyzer
        networks:
          - opensearch-net
      opensearch-node2:
        image: opensearchstaging/opensearch:2.17.1.10332
        container_name: opensearch-node2
        environment:
          - cluster.name=opensearch-cluster
          - node.name=opensearch-node2
          - discovery.seed_hosts=opensearch-node1,opensearch-node2
          - cluster.initial_cluster_manager_nodes=opensearch-node1,opensearch-node2
          - bootstrap.memory_lock=true
          - OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!
        ulimits:
          memlock:
            soft: -1
            hard: -1
          nofile:
            soft: 65536
            hard: 65536
        volumes:
          - opensearch-data2:/usr/share/opensearch/data
        networks:
          - opensearch-net
      opensearch-dashboards:
        image: opensearchstaging/opensearch-dashboards:2.17.1.7969
        container_name: opensearch-dashboards
        ports:
          - 5601:5601
        expose:
          - "5601"
        environment:
          OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]'
        networks:
          - opensearch-net
    volumes:
      opensearch-data1:
      opensearch-data2:
    

    networks:
    opensearch-net:

    • Download the above docker-compose.yml on your machine.
    • Get latest image versions docker-compose pull.
    • Start the cluster docker-compose up.
  • OpenSearch docker 2.17.1.10332

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch:2.17.1.10332 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" opensearchstaging/opensearch:2.17.1.10332
      • Connect command curl http://localhost:9200/
    • Start with security
      • Docker command
      docker pull opensearchstaging/opensearch:2.17.1.10332 && docker run -it -p 9200:9200 -e "discovery.type=single-node" -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=myStrongPassword123!" opensearchstaging/opensearch:2.17.1.10332
      
      • Connect command curl --insecure 'https://admin:myStrongPassword123!@localhost:9200/'
  • OpenSearch-Dashboards docker 2.17.1.7969

    • Start without security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.1.7969 && docker run -it --network="host" -e "DISABLE_SECURITY_DASHBOARDS_PLUGIN=true" opensearchstaging/opensearch-dashboards:2.17.1.7969
      • URL http://localhost:5601/
    • Start with security
      • Docker command docker pull opensearchstaging/opensearch-dashboards:2.17.1.7969 && docker run -it --network="host" opensearchstaging/opensearch-dashboards:2.17.1.7969
      • URL http://localhost:5601/
  • Use TARs to deploy OpenSearch Manually

Check how to install opensearch and dashboards on different platforms

@Divyaasm
Copy link
Collaborator

Divyaasm commented Sep 27, 2024

Integration Test Results

Thank you

@Vikasht34
Copy link

opensearch-project/k-NN/2152
Is this related to new feature introduced in 2.17?
Yes
Is this feature under a feature flag or explicit setting?
No
How does this bug surface? Is it the most exercised flow?
Yes , if user does not provide higher rescoring factor then it will be defaulted to lower rescoring factor which gives low re-c all which is not a good user experince.
What is the impact of this bug?
The user will not get good re-call on semantics search
Why was this not called out yet?
This was merged close to the release deadline and was supposed to be included pre-release.
Why this cannot wait until 2.18?
User can run into performance issues. This will give them a bad experience with the feature that we released.

@Divyaasm
Copy link
Collaborator

Integration Test Results

Thank you

@Divyaasm
Copy link
Collaborator

Divyaasm commented Oct 1, 2024

Native Plugin Validation

-> Installing repository-s3
-> Downloading repository-s3 from opensearch
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission config#plus read
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.NetPermission setDefaultAuthenticator
* java.net.SocketPermission * connect,resolve
* java.util.PropertyPermission aws.configFile read,write
* java.util.PropertyPermission aws.sharedCredentialsFile read,write
* java.util.PropertyPermission opensearch.allow_insecure_settings read,write
* java.util.PropertyPermission opensearch.path.conf read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed repository-s3 with folder name repository-s3```

@Divyaasm
Copy link
Collaborator

Divyaasm commented Oct 1, 2024

Signature validation

gpg --verify opensearch-2.17.1-linux-x64.tar.gz.sig opensearch-2.17.1-linux-x64.tar.gz
gpg: Signature made Tue Oct  1 13:39:56 2024 PDT
gpg:                using RSA key C2EE2AF6542C03B4
gpg: Good signature from "OpenSearch project <opensearch@amazon.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: C5B7 4989 65EF D1C2 924B  A9D5 39D3 1987 9310 D3FC
     Subkey fingerprint: 2187 3199 B103 0FCD 49DA  83F8 C2EE 2AF6 542C 03B4

@Divyaasm
Copy link
Collaborator

Divyaasm commented Oct 2, 2024

OpenSearch 2.17.1 has been released to the public

@Divyaasm Divyaasm closed this as completed Oct 2, 2024
@github-project-automation github-project-automation bot moved this from 🏗 In progress to ✅ Done in Engineering Effectiveness Board Oct 2, 2024
@dbwiddis
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: ✅ Done
Development

No branches or pull requests