Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor MetadataService to Use ContentTransformer #1069

Merged
merged 3 commits into from
Dec 10, 2024

Conversation

minwoox
Copy link
Member

@minwoox minwoox commented Dec 4, 2024

Motivation:
Previously, the Central Dogma server in MetadataService would repeatedly attempt to commit when

  • ChangeConflictException is raised.
  • The revision for fetching the content differs from the repository head revision.
    With the introduction of ContentTransformer in Add Transforming Content Push Command #1064, the server can now handle transformations in a single attempt, avoiding unnecessary retries.

Modifications:

  • Updated MetadataService to utilize ContentTransformer for commit operations.

Result:

  • Improved efficiency in MetadataService by eliminating repeated commit attempts.

To-do:

  • Encapsulate tokenRepo and metadataRepo in MetadataService into its class respectively.

Motivation:
Previously, the Central Dogma server in `MetadataService` would repeatedly attempt to commit until it succeeded. With the introduction of `ContentTransformer` in line#1064, the server can now handle transformations in a single attempt, avoiding unnecessary retries.

Modifications:
- Updated `MetadataService` to utilize `ContentTransformer` for commit operations.

Result:
- Improved efficiency in `MetadataService` by eliminating repeated commit attempts.
@minwoox minwoox added this to the 0.72.0 milestone Dec 4, 2024
Copy link
Contributor

@jrhee17 jrhee17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked that:

  1. there are no significant differences between previous/migrated logic
  2. there is no additional IO/blocking calls within ContentTransformer

I'm unsure about the scope since only some methods in MetadataService seem to be migrated while others haven't been migrated, but the changes look good to me

final Tokens tokens = tokens(node);
final Token token = tokens.get(appId); // Raise an exception if not found.
if (token.deactivation() == null) {
throw new IllegalArgumentException("The token is already activated: " + appId);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this type of logic may be less performant in practice since a lock is always held (whereas previously this was done with an optimistic-lock).

Having said this, I agree with this approach since this API is not used often and I prefer code clarity over minor optimizations.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's something I overlooked. 😅
If an exception is raised, the ZooKeeper lock won't be held at all.

As you mentioned, it won't be an issue in most cases. If it becomes a problem, I can update the implementation to read the content first.

@minwoox
Copy link
Member Author

minwoox commented Dec 6, 2024

there are no significant differences between previous/migrated logic
there is no additional IO/blocking calls within ContentTransformer

That is correct. 👍

I'm unsure about the scope since only some methods in MetadataService seem to be migrated while others haven't been migrated, but the changes look good to me

The rest of them are migrated in #1061
This PR migrates the methods that use changeSupplier.

@minwoox
Copy link
Member Author

minwoox commented Dec 10, 2024

Let me merge this to keep working on #1061.
If there's something I need to fix, I will fix it in #1061.

@minwoox minwoox merged commit 810d02f into line:main Dec 10, 2024
9 of 10 checks passed
@minwoox minwoox deleted the refactor_metadataService branch December 10, 2024 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants