Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a small traces table for querying. #460

Merged
merged 2 commits into from
Jul 17, 2023
Merged

Use a small traces table for querying. #460

merged 2 commits into from
Jul 17, 2023

Conversation

spetrovic77
Copy link
Contributor

Suggested by @ghemawat.

The traces table now contains only the information requied for querying. The new format is more space efficient and leads to faster querying.

Also, insert all trace data into the database in a single transaction, as it is significantly more efficient:

Before:
cpu: Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz
BenchmarkStore
BenchmarkStore/1
BenchmarkStore/1-72 100 10763759 ns/op
BenchmarkStore/10
BenchmarkStore/10-72 10 108713097 ns/op
BenchmarkStore/100
BenchmarkStore/100-72 1 1055861276 ns/op

After:
BenchmarkStore
BenchmarkStore/1
BenchmarkStore/1-72 100 10609201 ns/op
BenchmarkStore/10
BenchmarkStore/10-72 100 10700368 ns/op
BenchmarkStore/100
BenchmarkStore/100-72 96 13052743 ns/op

@spetrovic77 spetrovic77 requested a review from ghemawat July 17, 2023 21:29
@spetrovic77 spetrovic77 self-assigned this Jul 17, 2023
Suggested by @ghemawat.

The traces table now contains only the information requied for querying. The
new format is more space efficient and leads to faster querying.

Also, insert all trace data into the database in a single transaction,
as it is significantly  more efficient:

Before:
cpu: Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz
BenchmarkStore
BenchmarkStore/1
BenchmarkStore/1-72  	     100	  10763759 ns/op
BenchmarkStore/10
BenchmarkStore/10-72 	      10	 108713097 ns/op
BenchmarkStore/100
BenchmarkStore/100-72         	       1	1055861276 ns/op

After:
BenchmarkStore
BenchmarkStore/1
BenchmarkStore/1-72  	     100	  10609201 ns/op
BenchmarkStore/10
BenchmarkStore/10-72 	     100	  10700368 ns/op
BenchmarkStore/100
BenchmarkStore/100-72         	      96	  13052743 ns/op
Copy link
Collaborator

@ghemawat ghemawat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great!

@spetrovic77 spetrovic77 merged commit ef99cdd into main Jul 17, 2023
@spetrovic77 spetrovic77 deleted the trace2 branch July 17, 2023 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants