Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter #16214

Closed
michaeljmarshall opened this issue Jun 24, 2022 · 2 comments · Fixed by #16286
Closed

Optimize AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter #16214

michaeljmarshall opened this issue Jun 24, 2022 · 2 comments · Fixed by #16286
Labels
good first issue Good for newcomers

Comments

@michaeljmarshall
Copy link
Member

Is your enhancement request related to a problem? Please describe.
In reading through the AbstractTopic code, I noticed that there are some inefficient stat generations. Specifically:

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java

Lines 1135 to 1141 in b236899

public long getMsgOutCounter() {
return getStats(false, false, false).msgOutCounter;
}
public long getBytesOutCounter() {
return getStats(false, false, false).bytesOutCounter;
}

Describe the solution you'd like
Based on quickly reading through the usages of these methods and the definition of getStats, I think we should find a more direct way to compute the values for these counters.

Additional context
I am happy to help review any PRs for this enhancement.
@michaeljmarshall michaeljmarshall added the good first issue Good for newcomers label Jun 24, 2022
@sijie sijie mentioned this issue Jun 24, 2022
Open
@nahguam
Copy link
Contributor

nahguam commented Jun 29, 2022

I can look at this one

nahguam pushed a commit to nahguam/pulsar that referenced this issue Jun 29, 2022
[improve][broker] Optimise msgOutCounter and bytesOutCounter (apache#…
0a1afed 
…16214)

### Motivation

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

### Modification

Provided a scoped implementation of the above methods for the single
counter required for each getter.
@nahguam nahguam mentioned this issue Jun 29, 2022
Merged
5 tasks
nahguam pushed a commit to nahguam/pulsar that referenced this issue Jun 29, 2022
[improve][broker] Optimise msgOutCounter and bytesOutCounter (apache#…
e237170 
…16214)

### Motivation

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

### Modification

Provided a scoped implementation of the above methods for the single
counter required for each getter.
RobertIndie pushed a commit that referenced this issue Jul 1, 2022
[improve][broker] Optimise msgOutCounter and bytesOutCounter (#16214) (…
07c46fe 
…#16286)

### Motivation

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

### Modification

Provided a scoped implementation of the above methods for the single
counter required for each getter.
mattisonchao added a commit that referenced this issue Jul 2, 2022
@mattisonchao
Revert "[improve][broker] Optimise msgOutCounter and bytesOutCounter (#…
2bedbab 
…16214) (#16286)"

This reverts commit 6f12f5e.
@michaeljmarshall
Copy link
Member Author

@nahguam - thanks for contributing the fix for this issue!

Nicklee007 pushed a commit to Nicklee007/pulsar that referenced this issue Jul 5, 2022
[improve][broker] Optimise msgOutCounter and bytesOutCounter (apache#…
57e1561 
…16214) (apache#16286)

### Motivation

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

### Modification

Provided a scoped implementation of the above methods for the single
counter required for each getter.
codelipenghui pushed a commit that referenced this issue Jul 10, 2022
@codelipenghui
[improve][broker] Optimise msgOutCounter and bytesOutCounter (#16214) (…
63a3762 
…#16286)

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

Provided a scoped implementation of the above methods for the single
counter required for each getter.

(cherry picked from commit 07c46fe)
nicoloboschi pushed a commit to datastax/pulsar that referenced this issue Jul 11, 2022
@nicoloboschi
[improve][broker] Optimise msgOutCounter and bytesOutCounter (apache#…
dcf80fa 
…16214) (apache#16286)

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

Provided a scoped implementation of the above methods for the single
counter required for each getter.

(cherry picked from commit 07c46fe)
(cherry picked from commit 63a3762)
wuxuanqicn pushed a commit to wuxuanqicn/pulsar that referenced this issue Jul 14, 2022
@weimob-wuxuanqi
[improve][broker] Optimise msgOutCounter and bytesOutCounter (apache#…
9c4304e 
…16214) (apache#16286)

### Motivation

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

### Modification

Provided a scoped implementation of the above methods for the single
counter required for each getter.
nahguam pushed a commit to nahguam/pulsar that referenced this issue Jul 25, 2022
[improve][broker] Optimise msgOutCounter and bytesOutCounter (apache#…
ef32401 
…16214) (apache#16286)

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

Provided a scoped implementation of the above methods for the single
counter required for each getter.

(cherry picked from commit 07c46fe)
nahguam pushed a commit to nahguam/pulsar that referenced this issue Aug 4, 2022
[improve][broker] Optimise msgOutCounter and bytesOutCounter (apache#…
ffc6561 
…16214) (apache#16286)

AbstractTopic#getBytesOutCounter and AbstractTopic#getMsgOutCounter
both generate full stats only to pick the single required counter for
the getters. These can be optimised to perform only the necessary work
to calculate the counters.

Provided a scoped implementation of the above methods for the single
counter required for each getter.

(cherry picked from commit 07c46fe)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[improve][broker] Optimise msgOutCounter and bytesOutCounter (#16214) nahguam/pulsar
2 participants
@nahguam @michaeljmarshall