Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][broker] Optimise msgOutCounter and bytesOutCounter (#16214) #16286

Merged
merged 1 commit into from
Jul 1, 2022

Conversation

nahguam
Copy link
Contributor

@nahguam nahguam commented Jun 29, 2022

Motivation

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

Modification

Provided a scoped implementation of the above methods for the single
counter required for each getter.

Fixes #16214

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change added tests and can be verified as follows:

mvn clean install -Pcore-modules,-main -DskipTests
mvn test -pl pulsar-broker -Dtest="AbstractTopicTest,AbstractSubscriptionTest,ConsumerTest"

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: ( no)
  • Anything that affects deployment: (no)

Documentation

Check the box below or label this PR directly.

Need to update docs?

  • doc-required
    (Your PR needs to update docs and you will update later)

  • doc-not-needed
    (Please explain why)
    Implementation optimisation only

  • doc
    (Your PR contains doc changes)

  • doc-complete
    (Docs have been already added)

…16214)

### Motivation

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

### Modification

Provided a scoped implementation of the above methods for the single
counter required for each getter.
@nahguam nahguam force-pushed the topic-out-counters branch from 0a1afed to e237170 Compare June 29, 2022 11:42
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Jun 29, 2022
@codelipenghui codelipenghui added this to the 2.11.0 milestone Jun 29, 2022
@codelipenghui codelipenghui added type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages area/metrics release/2.10.2 release/2.9.4 labels Jun 29, 2022
@nahguam
Copy link
Contributor Author

nahguam commented Jun 30, 2022

/pulsarbot run-failure-checks

@nahguam
Copy link
Contributor Author

nahguam commented Jun 30, 2022

/pulsarbot run "Unit-BROKER_FLAKY Tests"

Copy link
Member

@shibd shibd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM

@RobertIndie RobertIndie merged commit 07c46fe into apache:master Jul 1, 2022
@nahguam nahguam deleted the topic-out-counters branch July 1, 2022 10:28
@mattisonchao
Copy link
Member

Hi, @nahguam

It looks like this pull request has many conflicts at branch-2.9 could you please help push a new PR to branch 2.9?

Nicklee007 pushed a commit to Nicklee007/pulsar that referenced this pull request Jul 5, 2022
…16214) (apache#16286)

### Motivation

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

### Modification

Provided a scoped implementation of the above methods for the single
counter required for each getter.
codelipenghui pushed a commit that referenced this pull request Jul 10, 2022
…#16286)

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

Provided a scoped implementation of the above methods for the single
counter required for each getter.

(cherry picked from commit 07c46fe)
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Jul 11, 2022
…16214) (apache#16286)

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

Provided a scoped implementation of the above methods for the single
counter required for each getter.

(cherry picked from commit 07c46fe)
(cherry picked from commit 63a3762)
wuxuanqicn pushed a commit to wuxuanqicn/pulsar that referenced this pull request Jul 14, 2022
…16214) (apache#16286)

### Motivation

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

### Modification

Provided a scoped implementation of the above methods for the single
counter required for each getter.
nahguam pushed a commit to nahguam/pulsar that referenced this pull request Jul 25, 2022
…16214) (apache#16286)

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

Provided a scoped implementation of the above methods for the single
counter required for each getter.

(cherry picked from commit 07c46fe)
nahguam pushed a commit to nahguam/pulsar that referenced this pull request Aug 4, 2022
…16214) (apache#16286)

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

Provided a scoped implementation of the above methods for the single
counter required for each getter.

(cherry picked from commit 07c46fe)
@mattisonchao
Copy link
Member

mattisonchao commented Aug 10, 2022

#16771 cherry-picked this improvement., I remove the release 2.9.4 label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/metrics cherry-picked/branch-2.10 doc-not-needed Your PR changes do not impact docs release/2.10.2 type/enhancement The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optimize AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
5 participants