Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More useful stats output by privPrecCompare #1642

Merged
merged 2 commits into from
Jan 17, 2025

Conversation

michael-schwarz
Copy link
Member

For my thesis, I was considering many different configurations, and the pairwise output got tedious:

protection equal to mine-W    (equal: 427, more precise: 0, less precise: 0, incomparable: 0, total: 427)
protection equal to lock    (equal: 427, more precise: 0, less precise: 0, incomparable: 0, total: 427)
protection equal to write    (equal: 427, more precise: 0, less precise: 0, incomparable: 0, total: 427)
protection equal to write+lock    (equal: 427, more precise: 0, less precise: 0, incomparable: 0, total: 427)
protection less precise than protection-interval    (equal: 51, more precise: 0, less precise: 376, incomparable: 0, total: 427)
protection less precise than mine-W-interval    (equal: 51, more precise: 0, less precise: 376, incomparable: 0, total: 427)
protection less precise than lock-interval    (equal: 51, more precise: 0, less precise: 376, incomparable: 0, total: 427)
protection less precise than write-interval    (equal: 51, more precise: 0, less precise: 376, incomparable: 0, total: 427)
protection less precise than write+lock-interval    (equal: 51, more precise: 0, less precise: 376, incomparable: 0, total: 427)
protection equal to protection-tid    (equal: 427, more precise: 0, less precise: 0, incomparable: 0, total: 427)
protection equal to lock-tid    (equal: 427, more precise: 0, less precise: 0, incomparable: 0, total: 427)
protection equal to write-tid    (equal: 427, more precise: 0, less precise: 0, incomparable: 0, total: 427)
protection equal to write+lock-tid    (equal: 427, more precise: 0, less precise: 0, incomparable: 0, total: 427)
protection less precise than protection-tid-interval    (equal: 51, more precise: 0, less precise: 376, incomparable: 0, total: 427)
protection less precise than lock-tid-interval    (equal: 51, more precise: 0, less precise: 376, incomparable: 0, total: 427)
protection less precise than write-tid-interval    (equal: 51, more precise: 0, less precise: 376, incomparable: 0, total: 427)
protection less precise than write+lock-tid-interval    (equal: 51, more precise: 0, less precise: 376, incomparable: 0, total: 427)
...

I had 18 configurations and thus 288 more pairwise comparisons like this (where many have equal precision).

With this PR, we still create the detailed output, but then at the end create buckets of approaches that have the same precision and then output one comparison per bucket.

For example, this output here (all approaches equally precise, TIDs do not matter, intervals pay off) is much easier to intuitively grasp than the 306 pairwise comparisons.

Bucket 0:
  protection-tid
  write
  write+lock-tid
  write+lock
  write-tid
  protection
  mine-W
  lock
  lock-tid
Bucket 1:
  write-interval
  lock-tid-interval
  write-tid-interval
  protection-interval
  mine-W-interval
  protection-tid-interval
  lock-interval
  write+lock-interval
  write+lock-tid-interval
Comparison between bucket 0 and 1: lock less precise than write+lock-tid-interval    (equal: 51, more precise: 0, less precise: 376, incomparable: 0, total: 427)

I merged this only into my dissertation branch back then (#1431 ). I opened this PR so we can consider if we want to have this or not.

Copy link
Member

@sim642 sim642 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think at one point when I looked at this I thought maybe there's some clever way to do this with DisjointDomain.PairwiseSet but it's not worth spending time to make this particularly elegant.

@michael-schwarz michael-schwarz merged commit ea7ee83 into master Jan 17, 2025
21 checks passed
@michael-schwarz michael-schwarz deleted the priv_prec_compare_stats branch January 17, 2025 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants