Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-11418. leader executor framework #7406

Draft
wants to merge 7 commits into
base: LeaderExecutor_Feature_2
Choose a base branch
from

Conversation

sumitagrawl
Copy link
Contributor

What changes were proposed in this pull request?

  1. Minimal injection to existing flow for Leader execution framework
  • Table expose rawTable to update byte[] directly
  • SystemAction for new audit log
  • OzoneManager with OmGateway init, start, stop
  • OzoneManagerStateMachine to redirect its execution in compatible way
  • OzoneManagerProtocolServerSideTranslatorPB to redirect specific flow to leader execution
  1. Leader execution for new flow (OBS flow)
  • granular locking
  • gateway for entry of execution
  • request flow with preprocess, authorize, lock, unlock, and process for OBS code simplification
  • bucket quota resource for lockless quota validation
  • leaderRequestExecutor for request execution and ratis update
  • new Request factory for the flow
  • OBS flow: create and commit

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-11418

How was this patch tested?

  • TODO

public OMResponse submitInternal(OMRequest omRequest, String clientId, long callId) throws ServiceException {
requestInProgress.incrementAndGet();
RequestContext requestContext = new RequestContext();
requestContext.setRequest(omRequest);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems redundant

(r, th) -> handleAfterExecution(requestContext, th));
OmLockOpr.OmLockInfo lockInfo = null;
try {
OMRequestBase requestBase = OmRequestFactory.createClientRequest(omRequest, om);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reusing Request name seemed wrong in the original code itself. Request == a bunch of attributes (nouns) and a verb for what needs to be done. These ClientRequests embed the entire logic for executing the code. They in essense capture the lamba to be performed for each request.

Suggested change
OMRequestBase requestBase = OmRequestFactory.createClientRequest(omRequest, om);
OMRequestBase requestBase = OmRequestFactory.createRequestExecutor(omRequest, om);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or createRequestProcessors

requestContext.setUuidClientId(clientId);
requestContext.setCallId(callId);
requestContext.setRequestBase(requestBase);
requestContext.getRequestBase().authorize(om);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checkACLs should be fine, it will make it very clear what is expected to take place.

requestContext.setRequestBase(requestBase);
requestContext.getRequestBase().authorize(om);
lockInfo = requestContext.getRequestBase().lock(omLockOpr);
validate(requestContext.getRequest());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's ok to break up validate into independent methods which validate only one thing

  1. ValidateParams
  2. ValidatePrepareState
  3. ...

There will be a long list of validations that occur in submitInternal and that is fine.

if (om.isRatisEnabled()) {
om.getOmRatisServer().getOmStateMachine().getPostExecutor().registerIndexNotifier(this::indexNotifier);
} else {
uniqueIndex.set(PostExecutor.initLeaderIndex(om));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleaner to initialize this first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants