Skip to content

Commit

Permalink
Add rough benchmarks for comparing other gems
Browse files Browse the repository at this point in the history
Resolves #102
  • Loading branch information
kachick committed May 10, 2021
1 parent 527a683 commit e497170
Show file tree
Hide file tree
Showing 9 changed files with 148 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ build-iPhoneSimulator/
/vendor/bundle
/lib/bundler/man/

# For comapre with other gems, ref: https://github.com/kachick/ruby-ulid/issues/102
benchmark/compare_with_othergems/*/vendor/

# for a library or gem, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# Gemfile.lock
Expand Down
43 changes: 43 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -401,6 +401,49 @@ NOTE: It is still having precision issue similar as `ulid gem` in the both gener
1. [Fix to handle timestamp precision in parser](https://github.com/abachman/ulid-ruby/pull/5)
1. [Fix to handle timestamp precision in generator](https://github.com/abachman/ulid-ruby/pull/4)

### Generating benchmarks

This runs rough benchmarks

```console
$ rake benchmark_with_other_gems
(Do not use `bundle exec`!)
```

One of the result at 2021/05/10 on my machine

```plaintext
#### rafaelsales - ulid
cd ./benchmark/compare_with_othergems/rafaelsales && bundle install --quiet && bundle exec ruby -v ./generate.rb
ruby 3.0.1p64 (2021-04-05 revision 0fb782ee38) [x86_64-darwin20]
Warming up --------------------------------------
ULID.generate 10.118k i/100ms
Calculating -------------------------------------
ULID.generate 84.398k (±11.5%) i/s - 424.956k in 5.110762s
"`ulid gem - 1.3.0` generated products: 601789 - sample: [\"01F59WTR62H95XYW522GGH5CWJ\", \"01F59WTMKKF6CY2S6BSVSCZNRC\", \"01F59WTPBP7BQ6QR8AJAHNCE3Z\", \"01F59WTNS266NHSR1ABK65PN8W\", \"01F59WTN7NY9JQ4XRKSSJMXYKN\"]"
------------------------------------------------------------------------
#### abachman - ulid-ruby
cd ./benchmark/compare_with_othergems/abachman && bundle install --quiet && bundle exec ruby -v ./generate.rb
ruby 3.0.1p64 (2021-04-05 revision 0fb782ee38) [x86_64-darwin20]
Warming up --------------------------------------
ULID.generate 8.283k i/100ms
Calculating -------------------------------------
ULID.generate 68.643k (±10.9%) i/s - 339.603k in 5.010994s
"`ulid-ruby gem - 1.0.0` generated products: 496251 - sample: [\"01F59WV1VCYJ666ZYYY5MHA3ED\", \"01F59WTX6MXZGQEBAASZG851XR\", \"01F59WTWDZ0443FSYXAQGDC0F5\", \"01F59WV0A6EKNGZ49285WBTR3D\", \"01F59WTXD2WBM3QQ8Z34ZQJFDQ\"]"
------------------------------------------------------------------------
#### kachick - ruby-ulid(This one)
cd ./benchmark/compare_with_othergems/kachick && bundle install --quiet && bundle exec ruby -v ./generate.rb
ruby 3.0.1p64 (2021-04-05 revision 0fb782ee38) [x86_64-darwin20]
Warming up --------------------------------------
ULID.generate 7.197k i/100ms
Calculating -------------------------------------
ULID.generate 63.617k (± 8.5%) i/s - 316.668k in 5.018090s
"`ruby-ulid gem (this one) - 0.1.0` generated products: 454173 - sample: [\"01F59WV9DQJMGV29N4MJPWVP18\", \"01F59WV4QDH431FNMNP9YVEKVV\", \"01F59WV6PPR1MYYH4ZN822RCBG\", \"01F59WV8DFSXG4SJ69W9QWPAWE\", \"01F59WV7N9WG3ES8NB3PM6F4QF\"]"
```

I have an excuse, This gem does not aim `faster than other`.
So I think the results are acceptable.

## References

- [Repository](https://github.com/kachick/ruby-ulid)
Expand Down
12 changes: 12 additions & 0 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,18 @@ task :benchmark do
sh 'bundle exec ruby ./benchmark/sample.rb'
end

# This can't be used `bundle exec rake benchmark_with_other_gems`. Use `rake benchmark_with_other_gems` instead
task :benchmark_with_other_gems do
puts '#### rafaelsales - ulid'
sh 'cd ./benchmark/compare_with_othergems/rafaelsales && bundle install --quiet && bundle exec ruby -v ./generate.rb'
puts '-' * 72
puts '#### abachman - ulid-ruby'
sh 'cd ./benchmark/compare_with_othergems/abachman && bundle install --quiet && bundle exec ruby -v ./generate.rb'
puts '-' * 72
puts '#### kachick - ruby-ulid(This one)'
sh 'cd ./benchmark/compare_with_othergems/kachick && bundle install --quiet && bundle exec ruby -v ./generate.rb'
end

task :update_fixed_examples do
sh 'rm ./test/many_data/fixtures/dumped_fixed_examples_*.bin'
sh 'bundle exec ruby ./scripts/generate_many_examples.rb'
Expand Down
4 changes: 4 additions & 0 deletions benchmark/compare_with_othergems/abachman/Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
source 'https://rubygems.org'

gem 'ulid-ruby', '1.0.0'
gem 'benchmark-ips', '>= 2.8.4', '< 3'
26 changes: 26 additions & 0 deletions benchmark/compare_with_othergems/abachman/generate.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# coding: utf-8
# frozen_string_literal: true

require 'benchmark/ips'
require 'ulid'

raise "Bug to setup: #{ULID.methods(false)}" unless (ULID::VERSION == '1.0.0') && ULID::Identifier

products = []

Benchmark.ips do |x|
x.report('ULID.generate') do
products << ULID.generate
end
end

# Below sections ensuring basic behaviors

# This Regexp taken from my code. But case sensitive
STRICT_PATTERN_WITH_CROCKFORD_BASE32_SUBSET = /\A(?<timestamp>[0-7][0123456789ABCDEFGHJKMNPQRSTVWXYZ]{9})(?<randomness>[0123456789ABCDEFGHJKMNPQRSTVWXYZ]{16})\z/

unless (products.size > 42) && (products.uniq === products) && products.all?(STRICT_PATTERN_WITH_CROCKFORD_BASE32_SUBSET) && products.all?(String)
raise 'Some bugs in the gem or this benchmark exists!'
end

p "`ulid-ruby gem - #{ULID::VERSION}` generated products: #{products.size} - sample: #{products.sample(5)}"
4 changes: 4 additions & 0 deletions benchmark/compare_with_othergems/kachick/Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
source 'https://rubygems.org'

gem 'ruby-ulid', '0.1.0'
gem 'benchmark-ips', '>= 2.8.4', '< 3'
26 changes: 26 additions & 0 deletions benchmark/compare_with_othergems/kachick/generate.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# coding: utf-8
# frozen_string_literal: true

require 'benchmark/ips'
require 'ulid'

raise "Bug to setup: #{ULID.methods(false)}" unless (ULID::VERSION == '0.1.0') && ULID::MonotonicGenerator

products = []

Benchmark.ips do |x|
x.report('ULID.generate') do
products << ULID.generate.to_s
end
end

# Below sections ensuring basic behaviors

# This Regexp taken from my code. But case sensitive
STRICT_PATTERN_WITH_CROCKFORD_BASE32_SUBSET = /\A(?<timestamp>[0-7][0123456789ABCDEFGHJKMNPQRSTVWXYZ]{9})(?<randomness>[0123456789ABCDEFGHJKMNPQRSTVWXYZ]{16})\z/

unless (products.size > 42) && (products.uniq === products) && products.all?(STRICT_PATTERN_WITH_CROCKFORD_BASE32_SUBSET) && products.all?(String)
raise 'Some bugs in the gem or this benchmark exists!'
end

p "`ruby-ulid gem (this one) - #{ULID::VERSION}` generated products: #{products.size} - sample: #{products.sample(5)}"
4 changes: 4 additions & 0 deletions benchmark/compare_with_othergems/rafaelsales/Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
source 'https://rubygems.org'

gem 'ulid', '1.3.0'
gem 'benchmark-ips', '>= 2.8.4', '< 3'
26 changes: 26 additions & 0 deletions benchmark/compare_with_othergems/rafaelsales/generate.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# coding: utf-8
# frozen_string_literal: true

require 'benchmark/ips'
require 'ulid'

raise "Bug to setup: #{ULID.methods(false)}" unless (ULID::VERSION == '1.3.0') && ULID::Generator

products = []

Benchmark.ips do |x|
x.report('ULID.generate') do
products << ULID.generate
end
end

# Below sections ensuring basic behaviors

# This Regexp taken from my code. But case sensitive
STRICT_PATTERN_WITH_CROCKFORD_BASE32_SUBSET = /\A(?<timestamp>[0-7][0123456789ABCDEFGHJKMNPQRSTVWXYZ]{9})(?<randomness>[0123456789ABCDEFGHJKMNPQRSTVWXYZ]{16})\z/

unless (products.size > 42) && (products.uniq === products) && products.all?(STRICT_PATTERN_WITH_CROCKFORD_BASE32_SUBSET) && products.all?(String)
raise 'Some bugs in the gem or this benchmark exists!'
end

p "`ulid gem - #{ULID::VERSION}` generated products: #{products.size} - sample: #{products.sample(5)}"

0 comments on commit e497170

Please sign in to comment.