Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor visibility migration code and add support for OpenSearch visibility migration #6284

Merged
merged 11 commits into from
Sep 26, 2024

Conversation

neil-xie
Copy link
Member

@neil-xie neil-xie commented Sep 12, 2024

What changed?

  • Add option of os-visibility to support ES -> OS migration
  • Refactor visibility triple manager to support the new case
  • Add dual indexer which can write to both ES and OS datastores
  • Add new flipr AdvancedVisibilityMigrationWritingMode which can supports different write mode during migration
Screenshot 2024-09-12 at 3 57 19 PM

Why?
OpenSearch migration

How did you test it?
Unit test and manual test locally

Potential risks

Release notes

Documentation Changes

Copy link

codecov bot commented Sep 12, 2024

Codecov Report

Attention: Patch coverage is 80.89172% with 30 lines in your changes missing coverage. Please review.

Project coverage is 73.23%. Comparing base (957f6ef) to head (a059a7d).
Report is 20 commits behind head on master.

Files with missing lines Patch % Lines
common/persistence/client/factory.go 81.15% 8 Missing and 5 partials ⚠️
service/worker/indexer/esProcessor.go 75.67% 7 Missing and 2 partials ⚠️
service/worker/indexer/migration_indexer.go 76.47% 2 Missing and 2 partials ⚠️
service/worker/indexer/indexer.go 75.00% 3 Missing ⚠️
common/persistence/visibility_triple_manager.go 95.45% 0 Missing and 1 partial ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
common/config/elasticsearch.go 42.42% <ø> (ø)
common/persistence/client/bean.go 0.00% <ø> (ø)
common/resource/resourceImpl.go 2.31% <ø> (ø)
common/persistence/visibility_triple_manager.go 98.46% <95.45%> (-1.54%) ⬇️
service/worker/indexer/indexer.go 9.61% <75.00%> (+9.61%) ⬆️
service/worker/indexer/migration_indexer.go 76.47% <76.47%> (ø)
service/worker/indexer/esProcessor.go 74.31% <75.67%> (+0.11%) ⬆️
common/persistence/client/factory.go 67.71% <81.15%> (+5.97%) ⬆️

... and 15 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 957f6ef...a059a7d. Read the comment docs.

Copy link
Member

@shijiesheng shijiesheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also title would be misleading. It's actually introduce OpenSearch visibility rather than a pure refactoring

cmd/server/cadence/server.go Outdated Show resolved Hide resolved
common/persistence/client/factory.go Outdated Show resolved Hide resolved
common/persistence/client/factory.go Outdated Show resolved Hide resolved
common/persistence/client/factory.go Show resolved Hide resolved
common/persistence/client/factory.go Outdated Show resolved Hide resolved
service/worker/indexer/indexer.go Show resolved Hide resolved
service/worker/indexer/indexer.go Outdated Show resolved Hide resolved
@neil-xie neil-xie changed the title Refactor visibility migration code Refactor visibility migration code and add support for OpenSearch visibility migration Sep 23, 2024
@neil-xie neil-xie merged commit 7b6846d into cadence-workflow:master Sep 26, 2024
20 checks passed
@neil-xie neil-xie deleted the CDNC-10433-1 branch September 26, 2024 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants