Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull in cop benchmarks into the repo itself #467

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Pull in cop benchmarks into the repo itself #467

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 2 additions & 6 deletions .rubocop.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,13 +105,9 @@ Metrics/ModuleLength:
Exclude:
- 'spec/**/*.rb'

Performance/Caller:
Performance:
Exclude:
- spec/rubocop/cop/performance/caller_spec.rb

Performance/CollectionLiteralInLoop:
Exclude:
- 'spec/**/*.rb'
- 'bench/**/*.rb'

RSpec:
Language:
Expand Down
2 changes: 2 additions & 0 deletions Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ git_source(:github) { |repo| "https://github.com/#{repo}.git" }

gemspec

gem 'benchmark-ips', require: false
gem 'benchmark-memory', require: false
gem 'bump', require: false
gem 'prism'
gem 'rake'
Expand Down
52 changes: 52 additions & 0 deletions bench/ancestors_include.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# frozen_string_literal: true

require_relative 'helper'

# https://github.com/rubocop/rubocop-performance/pull/123#issue-629164011

def fast
(Class <= Class) && # rubocop:disable Lint/BinaryOperatorWithIdenticalOperands
(Class <= Module) &&
(Class <= Object) &&
(Class <= Kernel) &&
(Class <= BasicObject)
end

def slow
Class.ancestors.include?(Class) &&
Class.ancestors.include?(Module) &&
Class.ancestors.include?(Object) &&
Class.ancestors.include?(Kernel) &&
Class.ancestors.include?(BasicObject)
end

bench_perf_and_mem(result: <<~RESULT) do |x|
********* IPS *********
ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]
Warming up --------------------------------------
less than or equal 718.264k i/100ms
ancestors.include? 103.830k i/100ms
Calculating -------------------------------------
less than or equal 7.186M (± 1.2%) i/s (139.17 ns/i) - 36.631M in 5.098690s
ancestors.include? 1.007M (± 2.7%) i/s (993.15 ns/i) - 5.088M in 5.056552s

Comparison:
less than or equal: 7185550.9 i/s
ancestors.include?: 1006894.0 i/s - 7.14x slower

********* MEMORY *********
Calculating -------------------------------------
less than or equal 0.000 memsize ( 0.000 retained)
0.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
ancestors.include? 1.000k memsize ( 0.000 retained)
5.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)

Comparison:
less than or equal: 0 allocated
ancestors.include?: 1000 allocated - Infx more
RESULT
x.report('less than or equal') { fast }
x.report('ancestors.include?') { slow }
end
89 changes: 89 additions & 0 deletions bench/array_semi_infinite_range_slice.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# frozen_string_literal: true

require_relative 'helper'

CHECKED = [3.3].freeze

# https://github.com/rubocop/rubocop-performance/pull/175#issuecomment-732061953

ARRAY = (1..100).to_a
range_take3 = (..2).freeze
range_drop90 = (90..).freeze

bench_perf_and_mem(result: <<~RESULT) do |x|
********* IPS *********
ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]
Warming up --------------------------------------
Array#take(3) 1.436M i/100ms
Array#[..2] 1.266M i/100ms
var Array#[..2] 1.253M i/100ms
Calculating -------------------------------------
Array#take(3) 14.001M (± 1.0%) i/s (71.42 ns/i) - 70.365M in 5.026331s
Array#[..2] 12.494M (± 1.3%) i/s (80.04 ns/i) - 63.316M in 5.068820s
var Array#[..2] 12.497M (± 1.2%) i/s (80.02 ns/i) - 62.650M in 5.013754s

Comparison:
Array#take(3): 14000869.7 i/s
var Array#[..2]: 12497303.5 i/s - 1.12x slower
Array#[..2]: 12493580.2 i/s - 1.12x slower

********* MEMORY *********
Calculating -------------------------------------
Array#take(3) 40.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
Array#[..2] 40.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
var Array#[..2] 40.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)

Comparison:
Array#take(3): 40 allocated
Array#[..2]: 40 allocated - same
var Array#[..2]: 40 allocated - same
RESULT
x.report('Array#take(3)') { ARRAY.take(3) }
x.report('Array#[..2]') { ARRAY[..2] }
x.report('var Array#[..2]') { ARRAY[range_take3] }
end

bench_perf_and_mem(result: <<~RESULT) do |x|
********* IPS *********
ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]
Warming up --------------------------------------
Array#drop(90) 1.394M i/100ms
Array#[90..] 1.255M i/100ms
var Array#[90..] 1.256M i/100ms
Calculating -------------------------------------
Array#drop(90) 13.911M (± 1.4%) i/s (71.88 ns/i) - 69.683M in 5.010116s
Array#[90..] 12.785M (± 0.5%) i/s (78.21 ns/i) - 63.983M in 5.004532s
var Array#[90..] 12.784M (± 0.6%) i/s (78.23 ns/i) - 64.062M in 5.011440s

Comparison:
Array#drop(90): 13911240.1 i/s
Array#[90..]: 12785455.6 i/s - 1.09x slower
var Array#[90..]: 12783591.2 i/s - 1.09x slower

********* MEMORY *********
Calculating -------------------------------------
Array#drop(90) 40.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
Array#[90..] 40.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
var Array#[90..] 40.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)

Comparison:
Array#drop(90): 40 allocated
Array#[90..]: 40 allocated - same
var Array#[90..]: 40 allocated - same
RESULT
x.report('Array#drop(90)') { ARRAY.drop(90) }
x.report('Array#[90..]') { ARRAY[90..] }
x.report('var Array#[90..]') { ARRAY[range_drop90] }
end
74 changes: 74 additions & 0 deletions bench/big_decimal_with_numeric_argument.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# frozen_string_literal: true

require 'bigdecimal'
require_relative 'helper'

# https://github.com/rubocop/rubocop-performance/issues/454

bench_perf_and_mem(result: <<~RESULT) do |x|
********* IPS *********
ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]
Warming up --------------------------------------
integer string new k 395.535k i/100ms
integer new k 809.005k i/100ms
integer string new kk
408.044k i/100ms
integer new kk 842.660k i/100ms
integer string new m 389.366k i/100ms
integer new m 813.443k i/100ms
Calculating -------------------------------------
integer string new k 4.302M (± 2.5%) i/s (232.47 ns/i) - 21.754M in 5.061086s
integer new k 8.795M (± 1.2%) i/s (113.70 ns/i) - 44.495M in 5.059773s
integer string new kk
4.075M (± 4.0%) i/s (245.43 ns/i) - 20.402M in 5.015977s
integer new kk 8.344M (± 7.0%) i/s (119.84 ns/i) - 42.133M in 5.077733s
integer string new m 3.754M (± 6.1%) i/s (266.40 ns/i) - 18.690M in 5.002906s
integer new m 8.590M (± 0.5%) i/s (116.42 ns/i) - 43.112M in 5.019137s

Comparison:
integer new k: 8795119.7 i/s
integer new m: 8589857.9 i/s - 1.02x slower
integer new kk: 8344151.7 i/s - same-ish: difference falls within error
integer string new k: 4301578.6 i/s - 2.04x slower
integer string new kk: 4074549.8 i/s - 2.16x slower
integer string new m: 3753692.1 i/s - 2.34x slower

********* MEMORY *********
Calculating -------------------------------------
integer string new k 88.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
integer new k 84.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
integer string new kk
88.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
integer new kk 84.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
integer string new m 92.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
integer new m 84.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)

Comparison:
integer new k: 84 allocated
integer new kk: 84 allocated - same
integer new m: 84 allocated - same
integer string new k: 88 allocated - 1.05x more
integer string new kk: 88 allocated - 1.05x more
integer string new m: 92 allocated - 1.10x more
RESULT
x.report('integer string new k') { BigDecimal('2000') }
x.report('integer new k') { BigDecimal(2000) }

x.report('integer string new kk') { BigDecimal('2000000') }
x.report('integer new kk') { BigDecimal(2_000_000) }

x.report('integer string new m') { BigDecimal('2000000000') }
x.report('integer new m') { BigDecimal(2_000_000_000) }
end
37 changes: 37 additions & 0 deletions bench/bind_call.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# frozen_string_literal: true

require_relative 'helper'

# https://github.com/rubocop/rubocop-performance/pull/92

umethod = String.instance_method(:start_with?)
bench_perf_and_mem(result: <<~RESULT) do |x|
********* IPS *********
ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]
Warming up --------------------------------------
bind.call 592.951k i/100ms
bind_call 1.151M i/100ms
Calculating -------------------------------------
bind.call 5.925M (± 0.5%) i/s (168.78 ns/i) - 29.648M in 5.003961s
bind_call 11.575M (± 1.4%) i/s (86.40 ns/i) - 58.722M in 5.074287s

Comparison:
bind_call: 11574705.6 i/s
bind.call: 5924959.7 i/s - 1.95x slower

********* MEMORY *********
Calculating -------------------------------------
bind.call 80.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
bind_call 0.000 memsize ( 0.000 retained)
0.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)

Comparison:
bind_call: 0 allocated
bind.call: 80 allocated - Infx more
RESULT
x.report('bind.call') { umethod.bind('hello, world').call('hello') }
x.report('bind_call') { umethod.bind_call('hello, world', 'hello') }
end
63 changes: 63 additions & 0 deletions bench/block_given_with_explicit_block.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# frozen_string_literal: true

require_relative 'helper'

# https://github.com/rubocop/rubocop-performance/issues/385

def if_block(&block)
!!block
end

def if_block_given
!!block_given?
end

bench_perf_and_mem(result: <<~RESULT) do |x|
********* IPS *********
ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]
Warming up --------------------------------------
block 1.688M i/100ms
block w/ block 579.312k i/100ms
block_given? 1.650M i/100ms
block_given? w/ block
1.627M i/100ms
Calculating -------------------------------------
block 17.046M (± 2.5%) i/s (58.66 ns/i) - 86.089M in 5.053948s
block w/ block 5.667M (± 0.5%) i/s (176.46 ns/i) - 28.386M in 5.009222s
block_given? 16.656M (± 0.8%) i/s (60.04 ns/i) - 84.158M in 5.053201s
block_given? w/ block
16.349M (± 1.9%) i/s (61.17 ns/i) - 82.981M in 5.077449s

Comparison:
block: 17046107.5 i/s
block_given?: 16655519.1 i/s - same-ish: difference falls within error
block_given? w/ block: 16349006.7 i/s - same-ish: difference falls within error
block w/ block: 5666956.8 i/s - 3.01x slower

********* MEMORY *********
Calculating -------------------------------------
block 0.000 memsize ( 0.000 retained)
0.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
block w/ block 80.000 memsize ( 0.000 retained)
1.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
block_given? 0.000 memsize ( 0.000 retained)
0.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
block_given? w/ block
0.000 memsize ( 0.000 retained)
0.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)

Comparison:
block: 0 allocated
block_given?: 0 allocated - same
block_given? w/ block: 0 allocated - same
block w/ block: 80 allocated - Infx more
RESULT
x.report('block') { if_block }
x.report('block w/ block') { if_block {} } # rubocop:disable Lint/EmptyBlock
x.report('block_given?') { if_block_given }
x.report('block_given? w/ block') { if_block_given {} } # rubocop:disable Lint/EmptyBlock
end
Loading
Loading