Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Speed up hex string parsing #150

Merged
merged 1 commit into from
Apr 15, 2022
Merged

Speed up hex string parsing #150

merged 1 commit into from
Apr 15, 2022

Conversation

Xyene
Copy link
Member

@Xyene Xyene commented Apr 15, 2022

For a particular trace I had on hand, this brings down decode time from
7s to 6s.

Refs #93.

Signed-off-by: Tudor Brindus tbrindus@janestreet.com

@Xyene Xyene requested a review from cgaebel April 15, 2022 16:23
cgaebel
cgaebel requested changes Apr 15, 2022
View reviewed changes
src/perf_tool_backend.ml Outdated Show resolved Hide resolved
@Xyene
Speed up hex string parsing
69dbca0 
For a particular trace I had on hand, this brings down decode time from
7s to 6s.

Refs janestreet#93.

Signed-off-by: Tudor Brindus <tbrindus@janestreet.com>
@Xyene Xyene requested a review from cgaebel April 15, 2022 21:27
cgaebel
cgaebel approved these changes Apr 15, 2022
View reviewed changes
Copy link
Contributor

@cgaebel cgaebel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Branch free and lookup table free; nice!

@Xyene Xyene merged commit ce1a5dd into janestreet:master Apr 15, 2022
@lamoreauxaj lamoreauxaj mentioned this pull request Jun 23, 2022
Merged
lamoreauxaj added a commit to lamoreauxaj/magic-trace that referenced this pull request Jun 24, 2022
@lamoreauxaj
Improved hex parsing speed for offsets, instruction pointers, and per…
b4532b1 
…f maps with int64_of_hex_string.

This addresses issue janestreet#93. Although janestreet#150 implemented fast hex string parsing, when I was magic-tracing magic-trace, I noticed this in the trace:
![image](https://user-images.githubusercontent.com/15175891/175398604-f62a6da2-90f1-43ca-b8ae-e35349a7970c.png)

This section of the trace entirely occurs because of calling `Int.Hex.of_string offset` within `parse_symbol_and_offset`. And this tends to take around 200-250 ns. Since this occurs about once per line in the `perf.data` file, the trace I ran this on executed this around 1.8 million times which should save a few hundred ms (out of total time of a few seconds).

Here are the traces after and before this change (respectively) zoomed in on 3 decodes (which shows 3 executions of `parse_symbol_and_offset`) of a perf line containing an entry of the callstack sampled. The former is just below 2 us and the latter is around 2.6 us. However it is clear that most of the remaining time is now spent on evaluating regex in the former.

![image](https://user-images.githubusercontent.com/15175891/175397340-8a78c154-96c3-4e3c-a19b-34b08d448b40.png)
![image](https://user-images.githubusercontent.com/15175891/175398274-c3e15695-5c5d-4bd3-bf7e-297163545af4.png)

In regards to implementation, I extracted these functions out to a `Util` module and used them when possible else where in magic-trace (although only the offset calculation is expensive because it is called a lot, but it seems reasonable to use them if possible). I used first class modules to abstract around `Int` and `Int64`.

Signed-off-by: Aaron Lamoreaux <alamoreaux@janestreet.com>
lamoreauxaj added a commit to lamoreauxaj/magic-trace that referenced this pull request Jun 24, 2022
@lamoreauxaj
Improved hex parsing speed for offsets, instruction pointers, and per…
d578b37 
…f maps with

int64_of_hex_string.

This addresses issue janestreet#93. Although janestreet#150 implemented fast hex string parsing,
when I was magic-tracing magic-trace, I noticed this in the trace:

This section of the trace entirely occurs because of calling `Int.Hex.of_string
offset` within `parse_symbol_and_offset`. And this tends to take around 200-250
ns. Since this occurs about once per line in the `perf.data` file, the trace I
ran this on executed this around 1.8 million times which should save a few
hundred ms (out of total time of a few seconds).

Here are the traces after and before this change (respectively) zoomed in on 3
decodes (which shows 3 executions of `parse_symbol_and_offset`) of a perf line
containing an entry of the callstack sampled. The former is just below 2 us and
the latter is around 2.6 us. However it is clear that most of the remaining time
is now spent on evaluating regex in the former.

In regards to implementation, I extracted these functions out to a `Util` module
and used them when possible else where in magic-trace (although only the offset
calculation is expensive because it is called a lot, but it seems reasonable to
use them if possible). I used first class modules to abstract around `Int` and
`Int64`.

Signed-off-by: Aaron Lamoreaux <alamoreaux@janestreet.com>
lamoreauxaj added a commit to lamoreauxaj/magic-trace that referenced this pull request Jun 24, 2022
@lamoreauxaj
Improved hex parsing speed for offsets, instruction pointers, and per…
4dec2c3 
…f maps with int64_of_hex_string.

This addresses issue janestreet#93. Although janestreet#150 implemented fast hex string parsing,
when I was magic-tracing magic-trace, I noticed this in the trace:

This section of the trace entirely occurs because of calling `Int.Hex.of_string
offset` within `parse_symbol_and_offset`. And this tends to take around 200-250
ns. Since this occurs about once per line in the `perf.data` file, the trace I
ran this on executed this around 1.8 million times which should save a few
hundred ms (out of total time of a few seconds).

Here are the traces after and before this change (respectively) zoomed in on 3
decodes (which shows 3 executions of `parse_symbol_and_offset`) of a perf line
containing an entry of the callstack sampled. The former is just below 2 us and
the latter is around 2.6 us. However it is clear that most of the remaining time
is now spent on evaluating regex in the former.

In regards to implementation, I extracted these functions out to a `Util` module
and used them when possible else where in magic-trace (although only the offset
calculation is expensive because it is called a lot, but it seems reasonable to
use them if possible). I used first class modules to abstract around `Int` and
`Int64`.

Signed-off-by: Aaron Lamoreaux <alamoreaux@janestreet.com>
lamoreauxaj added a commit to lamoreauxaj/magic-trace that referenced this pull request Jun 24, 2022
@lamoreauxaj
Improved hex parsing speed for offsets, instruction pointers, and per…
df0029f 
…f maps with int64_of_hex_string.

This addresses issue janestreet#93. Although janestreet#150 implemented fast hex string parsing,
when I was magic-tracing magic-trace, I noticed unexpected lengthy calls to
`Scanf` and related functions.

This occurs because of calling `Int.Hex.of_string offset` within
`parse_symbol_and_offset`. And this tends to take around 200-250 ns. Since this
occurs about once per line in the `perf.data` file, the trace I ran this on
executed this around 1.8 million times which should save a few hundred ms (out
of total time of a few seconds).

On traces after and before this change (respectively) of 3 decodes (which
contain 3 executions of `parse_symbol_and_offset`) of a perf line containing an
entry of the callstack sampled. The former is just below 2 us and the latter is
around 2.6 us. However it is clear that most of the remaining time is now spent
on evaluating regex in the former.

In regards to implementation, I extracted these functions out to a `Util` module
and used them when possible else where in magic-trace (although only the offset
calculation is expensive because it is called a lot, but it seems reasonable to
use them if possible). I used first class modules to abstract around `Int` and
`Int64`.

Signed-off-by: Aaron Lamoreaux <alamoreaux@janestreet.com>
Xyene pushed a commit that referenced this pull request Jun 24, 2022
@lamoreauxaj @Xyene
Improved hex parsing speed for offsets, instruction pointers, and per…
3f3fee2 
…f maps with int64_of_hex_string.

This addresses issue #93. Although #150 implemented fast hex string parsing,
when I was magic-tracing magic-trace, I noticed unexpected lengthy calls to
`Scanf` and related functions.

This occurs because of calling `Int.Hex.of_string offset` within
`parse_symbol_and_offset`. And this tends to take around 200-250 ns. Since this
occurs about once per line in the `perf.data` file, the trace I ran this on
executed this around 1.8 million times which should save a few hundred ms (out
of total time of a few seconds).

On traces after and before this change (respectively) of 3 decodes (which
contain 3 executions of `parse_symbol_and_offset`) of a perf line containing an
entry of the callstack sampled. The former is just below 2 us and the latter is
around 2.6 us. However it is clear that most of the remaining time is now spent
on evaluating regex in the former.

In regards to implementation, I extracted these functions out to a `Util` module
and used them when possible else where in magic-trace (although only the offset
calculation is expensive because it is called a lot, but it seems reasonable to
use them if possible). I used first class modules to abstract around `Int` and
`Int64`.

Signed-off-by: Aaron Lamoreaux <alamoreaux@janestreet.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cgaebel cgaebel cgaebel approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Xyene @cgaebel