Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial merge-base support #1557

Merged
merged 14 commits into from
Aug 26, 2024
Merged

initial merge-base support #1557

merged 14 commits into from
Aug 26, 2024

Conversation

Byron
Copy link
Member

@Byron Byron commented Aug 25, 2024

Translate tests from the Git test suite and assure that we produce the same results.

Tasks

  • Git baseline tests
  • implementation that can pass the baseline tests (needs handling of redundant commits)
  • borrowed cache for performance in multi-queries
  • more baseline tests
  • permutation tests to really hit it
  • Repository::merge_base()
  • gix merge-base for real-world scenarious
    • Test git merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e on the linux kernel

FollowUp

  • Optimize graph usage so that multiple merge-base runs can re-use each others graphs and thus save a lot of time for performance improvements.

Research

The commit-graph greatly accelerates the operation.

❯ hyperfine -w2 'git -c core.commitgraph=false merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e' 'git merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e'
Benchmark 1: git -c core.commitgraph=false merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):     953.6 ms ±  10.8 ms    [User: 903.0 ms, System: 45.9 ms]
  Range (min … max):   942.5 ms … 980.0 ms    10 runs

Benchmark 2: git merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):      21.8 ms ±   1.4 ms    [User: 7.3 ms, System: 12.3 ms]
  Range (min … max):    19.5 ms …  27.7 ms    119 runs

Summary
  git merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e ran
   43.68 ± 2.85 times faster than git -c core.commitgraph=false merge-base --independent 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e

Performance Comparison

gix without commit-graph is exactly as fast as Git. however, with commit-graph Git is 1.6x faster.

❯ hyperfine -w2 'git -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e' 'git merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e' 'gix -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e' 'gix merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e'
Benchmark 1: git -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):     641.8 ms ±   6.4 ms    [User: 600.6 ms, System: 36.9 ms]
  Range (min … max):   631.4 ms … 651.6 ms    10 runs

Benchmark 2: git merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):      62.4 ms ±   3.2 ms    [User: 45.2 ms, System: 14.7 ms]
  Range (min … max):    58.0 ms …  74.1 ms    49 runs

Benchmark 3: gix -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):     643.7 ms ±   6.1 ms    [User: 600.6 ms, System: 39.3 ms]
  Range (min … max):   636.8 ms … 657.7 ms    10 runs

Benchmark 4: gix merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
  Time (mean ± σ):      99.2 ms ±   1.4 ms    [User: 83.2 ms, System: 13.1 ms]
  Range (min … max):    96.0 ms … 101.9 ms    30 runs

Summary
  git merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e ran
    1.59 ± 0.08 times faster than gix merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
   10.28 ± 0.54 times faster than git -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e
   10.31 ± 0.54 times faster than gix -c core.commitgraph=false merge-base 2ecedd756908 d2360a398f0b 1253935ad801 160bab43419e 0e2209629fec 1d0e16ac1a9e

@Byron Byron force-pushed the merge-base branch 4 times, most recently from bdfc717 to 4fc6af8 Compare August 26, 2024 09:04
Translate first tests from the Git test suite and set up expectations.
…ta()`.

It's another way to insert parents, and tuned for the needs of merge-base.
`insert_data()` makes it easy to insert data for a single commit.
That way, operations that need a commit-graph (graph data structure)
can be run more efficiently as they won't consume the graph, but
can reuse it.
@Byron Byron force-pushed the merge-base branch 2 times, most recently from a6b4db3 to 5ae0985 Compare August 26, 2024 11:20
@Byron Byron marked this pull request as ready for review August 26, 2024 13:57
A simple method to obtain the merge-base between two commits.
For now it only supports the standard merge-base, but more derivatives
can be added easily on demand.
That way, the cost for creating intermediate commit objects are
cut in half.
@Byron Byron force-pushed the merge-base branch 4 times, most recently from df06b24 to 3d57f82 Compare August 26, 2024 16:00
Get an idea about he size of the graph.
In practice, what really matters is to reuse as much of the
previous computation as possible.

Re-using the graph (or its menory) doesn't do much, but reusing
the parsed commits would be a huge benefit when running many
queries after another.
@Byron Byron enabled auto-merge August 26, 2024 17:26
@Byron Byron merged commit 649f588 into main Aug 26, 2024
15 checks passed
@Byron Byron deleted the merge-base branch August 26, 2024 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant