diff --git a/CHANGELOG.md b/CHANGELOG.md index 05b85fd8..904665fe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,7 @@ All notable changes to this project will be documented in this file. ## Unreleased +- Deprecated the term `slave` in favor of `replica` [#286](https://github.com/instacart/makara/pull/286) Matt Larraz - Drop support for Ruby < 2.5 and ActiveRecord < 5.2 [#281](https://github.com/instacart/makara/pull/281) Matt Larraz ## v0.5.0 - 2021-01-08 diff --git a/README.md b/README.md index 933b1851..2325eae5 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ [![Code Climate](https://codeclimate.com/repos/526886a7f3ea00679b00cae6/badges/7905f7a000492a1078f7/gpa.png)](https://codeclimate.com/repos/526886a7f3ea00679b00cae6/feed) -Makara is generic master/slave proxy. It handles the heavy lifting of managing, choosing, blacklisting, and cycling through connections. It comes with an ActiveRecord database adapter implementation. +Makara is generic master/replica proxy. It handles the heavy lifting of managing, choosing, blacklisting, and cycling through connections. It comes with an ActiveRecord database adapter implementation. ## Installation @@ -36,7 +36,7 @@ Next, you need to decide which methods are proxied and which methods should be s send_to_all :connect, :reconnect, :disconnect, :clear_cache ``` -Assuming you don't need to split requests between a master and a slave, you're done. If you do need to, implement the `needs_master?` method: +Assuming you don't need to split requests between a master and a replica, you're done. If you do need to, implement the `needs_master?` method: ```ruby # within MyAwesomeSqlProxy @@ -63,7 +63,7 @@ To handle persistence of context across requests in a Rack app, makara provides When `sticky:true`, once a query as been sent to master, all queries for the rest of the request will also be sent to master. In addition, the cookie described above will be set client side with an expiration defined by time at end of original request + `master_ttl`. As long as the cookie is valid, all requests will send queries to master. -When `sticky:false`, only queries that need to go to master will go there. Subsequent read queries in the same request will go to slaves. +When `sticky:false`, only queries that need to go to master will go there. Subsequent read queries in the same request will go to replicas. #### Releasing stuck connections (clearing context) @@ -116,13 +116,13 @@ Makara::Logging::Logger.logger = ::Logger.new(STDOUT) ## ActiveRecord Database Adapter -So you've found yourself with an ActiveRecord-based project which is starting to get some traffic and you realize 95% of you DB load is from reads. Well you've come to the right spot. Makara is a great solution to break up that load not only between master and slave but potentially multiple masters and/or multiple slaves. +So you've found yourself with an ActiveRecord-based project which is starting to get some traffic and you realize 95% of you DB load is from reads. Well you've come to the right spot. Makara is a great solution to break up that load not only between master and replica but potentially multiple masters and/or multiple replicas. By creating a makara database adapter which simply acts as a proxy we avoid any major complexity surrounding specific database implementations. The makara adapter doesn't care if the underlying connection is mysql, postgresql, etc it simply cares about the sql string being executed. ### What goes where? -In general: Any `SELECT` statements will execute against your slave(s), anything else will go to master. +In general: Any `SELECT` statements will execute against your replica(s), anything else will go to master. There are some edge cases: * `SET` operations will be sent to all connections @@ -132,7 +132,7 @@ There are some edge cases: ### Errors / blacklisting -Whenever a node fails an operation due to a connection issue, it is blacklisted for the amount of time specified in your database.yml. After that amount of time has passed, the connection will begin receiving queries again. If all slave nodes are blacklisted, the master connection will begin receiving read queries as if it were a slave. Once all nodes are blacklisted the error is raised to the application and all nodes are whitelisted. +Whenever a node fails an operation due to a connection issue, it is blacklisted for the amount of time specified in your database.yml. After that amount of time has passed, the connection will begin receiving queries again. If all replica nodes are blacklisted, the master connection will begin receiving read queries as if it were a replica. Once all nodes are blacklisted the error is raised to the application and all nodes are whitelisted. ### Database.yml @@ -156,14 +156,14 @@ production: sticky: true # list your connections with the override values (they're merged into the top-level config) - # be sure to provide the role if master, role is assumed to be a slave if not provided + # be sure to provide the role if master, role is assumed to be a replica if not provided connections: - role: master host: master.sql.host - - role: slave - host: slave1.sql.host - - role: slave - host: slave2.sql.host + - role: replica + host: replica1.sql.host + - role: replica + host: replica2.sql.host ``` Let's break this down a little bit. At the top level of your config you have the standard `adapter` choice. Currently the available adapters are listed in [lib/active_record/connection_adapters/](lib/active_record/connection_adapters/). They are in the form of `#{db_type}_makara` where db_type is mysql, postgresql, etc. @@ -177,7 +177,7 @@ The makara subconfig sets up the proxy with a few of its own options, then provi * sticky - if a node should be stuck to once it's used during a specific context * master_ttl - how long the master context is persisted. generally, this needs to be longer than any replication lag * master_strategy - use a different strategy for picking the "current" master node: `failover` will try to keep the same one until it is blacklisted. The default is `round_robin` which will cycle through available ones. -* slave_strategy - use a different strategy for picking the "current" slave node: `failover` will try to keep the same one until it is blacklisted. The default is `round_robin` which will cycle through available ones. +* replica_strategy - use a different strategy for picking the "current" replica node: `failover` will try to keep the same one until it is blacklisted. The default is `round_robin` which will cycle through available ones. * connection_error_matchers - array of custom error matchers you want to be handled gracefully by Makara (as in, errors matching these regexes will result in blacklisting the connection as opposed to raising directly). Connection definitions contain any extra node-specific configurations. If the node should behave as a master you must provide `role: master`. Any previous configurations can be overridden within a specific node's config. Nodes can also contain weights if you'd like to balance usage based on hardware specifications. Optionally, you can provide a name attribute which will be used in sql logging. @@ -188,16 +188,16 @@ connections: host: mymaster.sql.host blacklist_duration: 0 - # implicit role: slave - - host: mybigslave.sql.host + # implicit role: replica + - host: mybigreplica.sql.host weight: 8 - name: Big Slave - - host: mysmallslave.sql.host + name: Big Replica + - host: mysmallreplica.sql.host weight: 2 - name: Small Slave + name: Small Replica ``` -In the previous config the "Big Slave" would receive ~80% of traffic. +In the previous config the "Big Replica" would receive ~80% of traffic. #### DATABASE_URL @@ -217,8 +217,8 @@ connections: - role: master blacklist_duration: 0 url: <%= ENV['DATABASE_URL_MASTER'] %> - - role: slave - url: <%= ENV['DATABASE_URL_SLAVE'] %> + - role: replica + url: <%= ENV['DATABASE_URL_REPLICA'] %> ``` **Important**: *Do NOT use `ENV['DATABASE_URL']`*, as it inteferes with the the database configuration @@ -271,7 +271,7 @@ You can provide strings or regexes. In the case of strings, if they start with On occasion your app may deal with a situation where makara is not present during a write but a read should use master. In the generic proxy details above you are encouraged to use `stick_to_master!` to accomplish this. Here's an example: ```ruby -# some third party creates a resource in your db, slave replication may not have completed yet +# some third party creates a resource in your db, replication may not have completed yet # ... # then your app is told to read the resource. def handle_request_after_third_party_record_creation diff --git a/lib/active_record/connection_adapters/makara_abstract_adapter.rb b/lib/active_record/connection_adapters/makara_abstract_adapter.rb index 9d4c463c..9fe0d63b 100644 --- a/lib/active_record/connection_adapters/makara_abstract_adapter.rb +++ b/lib/active_record/connection_adapters/makara_abstract_adapter.rb @@ -114,31 +114,34 @@ def custom_error_message?(connection, message) SQL_MASTER_MATCHERS = [/\A\s*select.+for update\Z/i, /select.+lock in share mode\Z/i, /\A\s*select.+(nextval|currval|lastval|get_lock|release_lock|pg_advisory_lock|pg_advisory_unlock)\(/i].map(&:freeze).freeze - SQL_SLAVE_MATCHERS = [/\A\s*(select|with.+\)\s*select)\s/i].map(&:freeze).freeze + SQL_REPLICA_MATCHERS = [/\A\s*(select|with.+\)\s*select)\s/i].map(&:freeze).freeze SQL_ALL_MATCHERS = [/\A\s*set\s/i].map(&:freeze).freeze SQL_SKIP_STICKINESS_MATCHERS = [/\A\s*show\s([\w]+\s)?(field|table|database|schema|view|index)(es|s)?/i, /\A\s*(set|describe|explain|pragma)\s/i].map(&:freeze).freeze + SQL_SLAVE_MATCHERS = SQL_REPLICA_MATCHERS + deprecate_constant :SQL_SLAVE_MATCHERS def sql_master_matchers SQL_MASTER_MATCHERS end + def sql_replica_matchers + SQL_REPLICA_MATCHERS + end def sql_slave_matchers - SQL_SLAVE_MATCHERS + warn "sql_slave_matchers is deprecated. Use sql_replica_matchers" + sql_replica_matchers end - def sql_all_matchers SQL_ALL_MATCHERS end - def sql_skip_stickiness_matchers SQL_SKIP_STICKINESS_MATCHERS end - def initialize(config) @error_handler = ::ActiveRecord::ConnectionAdapters::MakaraAbstractAdapter::ErrorHandler.new @control = ActiveRecordPoolControl.new(self) @@ -154,8 +157,8 @@ def appropriate_connection(method_name, args, &block) handling_an_all_execution(method_name) do hijacked do - # slave pool must run first. - @slave_pool.send_to_all(nil, &block) # just yields to each con + # replica pool must run first. + @replica_pool.send_to_all(nil, &block) # just yields to each con @master_pool.send_to_all(nil, &block) # just yields to each con end end @@ -187,7 +190,7 @@ def needed_by_all?(method_name, args) def needs_master?(method_name, args) sql = coerce_query_to_sql_string(args.first) return true if sql_master_matchers.any?{|m| sql =~ m } - return false if sql_slave_matchers.any?{|m| sql =~ m } + return false if sql_replica_matchers.any?{|m| sql =~ m } true end diff --git a/lib/makara.rb b/lib/makara.rb index 11771952..749be871 100644 --- a/lib/makara.rb +++ b/lib/makara.rb @@ -2,7 +2,6 @@ require 'makara/version' require 'makara/railtie' if defined?(Rails) module Makara - autoload :Cache, 'makara/cache' autoload :ConfigParser, 'makara/config_parser' autoload :ConnectionWrapper, 'makara/connection_wrapper' diff --git a/lib/makara/config_parser.rb b/lib/makara/config_parser.rb index 2127296c..f458a37a 100644 --- a/lib/makara/config_parser.rb +++ b/lib/makara/config_parser.rb @@ -14,9 +14,9 @@ # blacklist_duration: 20 # connections: # - role: 'master' -# - role: 'slave' -# - role: 'slave' -# name: 'slave2' +# - role: 'slave' # Deprecated in favor of 'replica' +# - role: 'replica' +# name: 'replica2' module Makara class ConfigParser @@ -145,6 +145,9 @@ def initialize(config) @config = config.symbolize_keys @makara_config = DEFAULTS.merge(@config[:makara] || {}) @makara_config = @makara_config.symbolize_keys + + deprecate_keys(:slave_strategy, :slave_shard_aware, :slave_default_shard) + @id = sanitize_id(@makara_config[:id]) end @@ -164,12 +167,17 @@ def master_configs end - def slave_configs + def replica_configs all_configs .reject { |config| config[:role] == 'master' } .map { |config| config.except(:role) } end + def slave_configs + warn "#slave_configs is deprecated. Switch to #replica_configs" + replica_configs + end + protected @@ -208,5 +216,16 @@ def sanitize_id(id) end end end + + def deprecate_keys(*keys) + keys.each do |key| + next unless @makara_config[key] + + new_key = key.to_s.gsub("slave", "replica").to_sym + warn "Config key #{key} is deprecated, use #{new_key} instead" + + @makara_config[new_key] = @makara_config[key] + end + end end end diff --git a/lib/makara/logging/subscriber.rb b/lib/makara/logging/subscriber.rb index 3574cae7..4d01eba9 100644 --- a/lib/makara/logging/subscriber.rb +++ b/lib/makara/logging/subscriber.rb @@ -19,7 +19,7 @@ def sql(event) # uses the adapter's connection proxy to modify the name of the event # the name of the used connection will be prepended to the sql log ### - ### [Master|Slave] User Load (1.3ms) SELECT * FROM `users`; + ### [Master|Replica] User Load (1.3ms) SELECT * FROM `users`; ### def current_wrapper_name(event) connection_object_id = event.payload[:connection_id] diff --git a/lib/makara/proxy.rb b/lib/makara/proxy.rb index 1ea76915..4569602c 100644 --- a/lib/makara/proxy.rb +++ b/lib/makara/proxy.rb @@ -3,7 +3,7 @@ require 'active_support/core_ext/hash/keys' require 'active_support/core_ext/string/inflections' -# The entry point of Makara. It contains a master and slave pool which are chosen based on the invocation +# The entry point of Makara. It contains a master and replica pool which are chosen based on the invocation # being proxied. Makara::Proxy implementations should declare which methods they are hijacking via the # `hijack_method` class method. # While debugging this class use prepend debug calls with Kernel. (Kernel.byebug for example) @@ -159,16 +159,16 @@ def disconnect! def send_to_all(method_name, *args) - # slave pool must run first to allow for slave-->master failover without running operations on master twice. + # replica pool must run first to allow for replica --> master failover without running operations on master twice. handling_an_all_execution(method_name) do - @slave_pool.send_to_all method_name, *args + @replica_pool.send_to_all method_name, *args @master_pool.send_to_all method_name, *args end end def any_connection if @master_pool.disabled - @slave_pool.provide do |con| + @replica_pool.provide do |con| yield con end else @@ -179,7 +179,7 @@ def any_connection rescue ::Makara::Errors::AllConnectionsBlacklisted, ::Makara::Errors::NoConnectionsAvailable begin @master_pool.disabled = true - @slave_pool.provide do |con| + @replica_pool.provide do |con| yield con end ensure @@ -201,9 +201,8 @@ def appropriate_connection(method_name, args) end - # master or slave + # master or replica def appropriate_pool(method_name, args) - # for testing purposes pool = _appropriate_pool(method_name, args) yield pool @@ -211,7 +210,7 @@ def appropriate_pool(method_name, args) rescue ::Makara::Errors::AllConnectionsBlacklisted, ::Makara::Errors::NoConnectionsAvailable => e if pool == @master_pool @master_pool.connections.each(&:_makara_whitelist!) - @slave_pool.connections.each(&:_makara_whitelist!) + @replica_pool.connections.each(&:_makara_whitelist!) Kernel.raise e else @master_pool.blacklist_errors << e @@ -232,17 +231,17 @@ def _appropriate_pool(method_name, args) # stickiness is still valid @master_pool - # all slaves are down (or empty) - elsif @slave_pool.completely_blacklisted? + # all replicas are down (or empty) + elsif @replica_pool.completely_blacklisted? stick_to_master(method_name, args) @master_pool elsif in_transaction? @master_pool - # yay! use a slave + # yay! use a replica else - @slave_pool + @replica_pool end end @@ -290,7 +289,7 @@ def sticky? @sticky && !@skip_sticking end - # use the config parser to generate a master and slave pool + # use the config parser to generate a master and replica pool def instantiate_connections @master_pool = Makara::Pool.new('master', self) @config_parser.master_configs.each do |master_config| @@ -299,10 +298,10 @@ def instantiate_connections end end - @slave_pool = Makara::Pool.new('slave', self) - @config_parser.slave_configs.each do |slave_config| - @slave_pool.add slave_config do - graceful_connection_for(slave_config) + @replica_pool = Makara::Pool.new('replica', self) + @config_parser.replica_configs.each do |replica_config| + @replica_pool.add replica_config do + graceful_connection_for(replica_config) end end end @@ -311,13 +310,13 @@ def handling_an_all_execution(method_name) yield rescue ::Makara::Errors::NoConnectionsAvailable => e if e.role == 'master' - # this means slave connections are good. + # this means replica connections are good. return end - @slave_pool.disabled = true + @replica_pool.disabled = true yield ensure - @slave_pool.disabled = false + @replica_pool.disabled = false end diff --git a/spec/active_record/connection_adapters/makara_abstract_adapter_spec.rb b/spec/active_record/connection_adapters/makara_abstract_adapter_spec.rb index 8e923110..9a1e1efa 100644 --- a/spec/active_record/connection_adapters/makara_abstract_adapter_spec.rb +++ b/spec/active_record/connection_adapters/makara_abstract_adapter_spec.rb @@ -65,7 +65,7 @@ it "determines that \"#{sql}\" #{should_send_to_all_connections ? 'should' : 'should not'} be sent to all underlying connections" do proxy = klass.new(config(1,1)) proxy.master_pool.connections.each{|con| expect(con).to receive(:execute).with(sql).once} - proxy.slave_pool.connections.each do |con| + proxy.replica_pool.connections.each do |con| if should_send_to_all_connections expect(con).to receive(:execute).with(sql).once else diff --git a/spec/active_record/connection_adapters/makara_mysql2_adapter_spec.rb b/spec/active_record/connection_adapters/makara_mysql2_adapter_spec.rb index e02ea382..06d17a4a 100644 --- a/spec/active_record/connection_adapters/makara_mysql2_adapter_spec.rb +++ b/spec/active_record/connection_adapters/makara_mysql2_adapter_spec.rb @@ -22,11 +22,11 @@ expect(ActiveRecord::Base.connection).to be_instance_of(ActiveRecord::ConnectionAdapters::MakaraMysql2Adapter) end - it 'should execute a send_to_all against master even if no slaves are connected' do + it 'should execute a send_to_all against master even if no replicas are connected' do establish_connection(config) connection = ActiveRecord::Base.connection - connection.slave_pool.connections.each do |c| + connection.replica_pool.connections.each do |c| allow(c).to receive(:_makara_blacklisted?){ true } allow(c).to receive(:_makara_connected?){ false } expect(c).to receive(:execute).with('SET @t1 = 1').never @@ -45,7 +45,7 @@ establish_connection(config) connection = ActiveRecord::Base.connection - (connection.slave_pool.connections | connection.master_pool.connections).each do |c| + (connection.replica_pool.connections | connection.master_pool.connections).each do |c| allow(c).to receive(:_makara_blacklisted?){ true } allow(c).to receive(:_makara_connected?){ false } expect(c).to receive(:execute).with('SET @t1 = 1').never @@ -64,7 +64,7 @@ it 'should not blow up if a connection fails' do wrong_config = config.deep_dup - wrong_config['makara']['connections'].select{|h| h['role'] == 'slave' }.each{|h| h['username'] = 'other'} + wrong_config['makara']['connections'].select{|h| h['role'] == 'replica' }.each{|h| h['username'] = 'other'} original_method = ActiveRecord::Base.method(:mysql2_connection) @@ -86,8 +86,8 @@ original_method.call(config) end - ActiveRecord::Base.connection.slave_pool.connections.each(&:_makara_whitelist!) - ActiveRecord::Base.connection.slave_pool.provide do |con| + ActiveRecord::Base.connection.replica_pool.connections.each(&:_makara_whitelist!) + ActiveRecord::Base.connection.replica_pool.provide do |con| res = con.execute('SELECT count(*) FROM users') if defined?(JRUBY_VERSION) expect(res[0]).to eq('count(*)' => 0) @@ -111,9 +111,9 @@ end - it 'should have one master and two slaves' do + it 'should have one master and two replicas' do expect(connection.master_pool.connection_count).to eq(1) - expect(connection.slave_pool.connection_count).to eq(2) + expect(connection.replica_pool.connection_count).to eq(2) end it 'should allow real queries to work' do @@ -139,27 +139,27 @@ expect(con).to receive(:execute).with('SET @t1 = 1').once end - connection.slave_pool.connections.each do |con| + connection.replica_pool.connections.each do |con| expect(con).to receive(:execute).with('SET @t1 = 1').once end connection.execute("SET @t1 = 1") end - it 'should send reads to the slave' do + it 'should send reads to the replica' do # ensure the next connection will be the first one allow_any_instance_of(Makara::Strategies::RoundRobin).to receive(:single_one?){ true } - con = connection.slave_pool.connections.first + con = connection.replica_pool.connections.first expect(con).to receive(:execute).with('SELECT * FROM users').once connection.execute('SELECT * FROM users') end - it 'should send exists? to slave' do + it 'should send exists? to replica' do allow_any_instance_of(Makara::Strategies::RoundRobin).to receive(:single_one?){ true } Test::User.exists? # flush other (schema) things that need to happen - con = connection.slave_pool.connections.first + con = connection.replica_pool.connections.first expect(con).to receive(:exec_query).with(/SELECT\s+1\s*(AS one)?\s+FROM .?users.?\s+LIMIT\s+.?1/, any_args).once.and_call_original Test::User.exists? end @@ -176,7 +176,7 @@ end it 'should allow reconnecting when one of the nodes is blacklisted' do - con = connection.slave_pool.connections.first + con = connection.replica_pool.connections.first allow(con).to receive(:_makara_blacklisted?){ true } connection.reconnect! end @@ -199,11 +199,11 @@ load(File.dirname(__FILE__) + '/../../support/schema.rb') change_context - connection.slave_pool.connections.each do |slave| + connection.replica_pool.connections.each do |replica| # Using method missing to help with back trace, literally - # no query should be executed on slave once a transaction is opened - expect(slave).to receive(:method_missing).never - expect(slave).to receive(:execute).never + # no query should be executed on replica once a transaction is opened + expect(replica).to receive(:method_missing).never + expect(replica).to receive(:execute).never end end diff --git a/spec/active_record/connection_adapters/makara_postgis_adapter_spec.rb b/spec/active_record/connection_adapters/makara_postgis_adapter_spec.rb index 886716ef..85b80646 100644 --- a/spec/active_record/connection_adapters/makara_postgis_adapter_spec.rb +++ b/spec/active_record/connection_adapters/makara_postgis_adapter_spec.rb @@ -52,9 +52,9 @@ end end - it 'should have one master and two slaves' do + it 'should have one master and two replicas' do expect(connection.master_pool.connection_count).to eq(1) - expect(connection.slave_pool.connection_count).to eq(2) + expect(connection.replica_pool.connection_count).to eq(2) end it 'should allow real queries to work' do @@ -75,17 +75,17 @@ expect(con).to receive(:execute).with("SET TimeZone = 'UTC'").once end - connection.slave_pool.connections.each do |con| + connection.replica_pool.connections.each do |con| expect(con).to receive(:execute).with("SET TimeZone = 'UTC'").once end connection.execute("SET TimeZone = 'UTC'") end - it 'should send reads to the slave' do + it 'should send reads to the replica' do # ensure the next connection will be the first one allow_any_instance_of(Makara::Strategies::RoundRobin).to receive(:single_one?){ true } - con = connection.slave_pool.connections.first + con = connection.replica_pool.connections.first expect(con).to receive(:execute).with('SELECT * FROM users').once connection.execute('SELECT * FROM users') @@ -118,7 +118,7 @@ context 'with only master connection' do it 'should not raise errors on read and write' do custom_config = config.deep_dup - custom_config['makara']['connections'].select{|h| h['role'] == 'slave' }.each{|h| h['port'] = '1'} + custom_config['makara']['connections'].select{|h| h['role'] == 'replica' }.each{|h| h['port'] = '1'} ActiveRecord::Base.establish_connection(custom_config) load(File.dirname(__FILE__) + '/../../support/schema.rb') @@ -128,7 +128,7 @@ end end - context 'with only slave connection' do + context 'with only replica connection' do it 'should raise error only on write' do ActiveRecord::Base.establish_connection(config) load(File.dirname(__FILE__) + '/../../support/schema.rb') diff --git a/spec/active_record/connection_adapters/makara_postgresql_adapter_spec.rb b/spec/active_record/connection_adapters/makara_postgresql_adapter_spec.rb index 52ee91e6..f228443a 100644 --- a/spec/active_record/connection_adapters/makara_postgresql_adapter_spec.rb +++ b/spec/active_record/connection_adapters/makara_postgresql_adapter_spec.rb @@ -29,9 +29,9 @@ end - it 'should have one master and two slaves' do + it 'should have one master and two replicas' do expect(connection.master_pool.connection_count).to eq(1) - expect(connection.slave_pool.connection_count).to eq(2) + expect(connection.replica_pool.connection_count).to eq(2) end it 'should allow real queries to work' do @@ -52,27 +52,27 @@ expect(con).to receive(:execute).with("SET TimeZone = 'UTC'").once end - connection.slave_pool.connections.each do |con| + connection.replica_pool.connections.each do |con| expect(con).to receive(:execute).with("SET TimeZone = 'UTC'").once end connection.execute("SET TimeZone = 'UTC'") end - it 'should send reads to the slave' do + it 'should send reads to the replica' do # ensure the next connection will be the first one allow_any_instance_of(Makara::Strategies::RoundRobin).to receive(:single_one?){ true } - con = connection.slave_pool.connections.first + con = connection.replica_pool.connections.first expect(con).to receive(:execute).with('SELECT * FROM users').once connection.execute('SELECT * FROM users') end - it 'should send exists? to slave' do + it 'should send exists? to replica' do allow_any_instance_of(Makara::Strategies::RoundRobin).to receive(:single_one?){ true } Test::User.exists? # flush other (schema) things that need to happen - con = connection.slave_pool.connections.first + con = connection.replica_pool.connections.first expect(con).to receive(:exec_query).with(/SELECT\s+1\s*(AS one)?\s+FROM .?users.?\s+LIMIT\s+.?1/, any_args).once.and_call_original Test::User.exists? end @@ -97,7 +97,7 @@ context 'with only master connection' do it 'should not raise errors on read and write' do custom_config = config.deep_dup - custom_config['makara']['connections'].select{|h| h['role'] == 'slave' }.each{|h| h['port'] = '1'} + custom_config['makara']['connections'].select{|h| h['role'] == 'replica' }.each{|h| h['port'] = '1'} establish_connection(custom_config) load(File.dirname(__FILE__) + '/../../support/schema.rb') @@ -107,7 +107,7 @@ end end - context 'with only slave connection' do + context 'with only replica connection' do it 'should raise error only on write' do establish_connection(config) load(File.dirname(__FILE__) + '/../../support/schema.rb') @@ -130,13 +130,13 @@ load(File.dirname(__FILE__) + '/../../support/schema.rb') change_context - # Pre-loads the attributes so that schema queries don't hit slave + # Pre-loads the attributes so that schema queries don't hit replica # user = User.create(name: 'hello') - connection.slave_pool.connections.each do |slave| + connection.replica_pool.connections.each do |replica| # Using method missing to help with back trace, literally - # no query should be executed on slave once a transaction is opened - expect(slave).to receive(:method_missing).never - expect(slave).to receive(:execute).never + # no query should be executed on replica once a transaction is opened + expect(replica).to receive(:method_missing).never + expect(replica).to receive(:execute).never end end diff --git a/spec/config_parser_spec.rb b/spec/config_parser_spec.rb index 90497e10..427448f6 100644 --- a/spec/config_parser_spec.rb +++ b/spec/config_parser_spec.rb @@ -12,10 +12,10 @@ :name => 'themaster' }, { - :name => 'slave1' + :name => 'replica1' }, { - :name => 'slave2' + :name => 'replica2' } ] } @@ -108,8 +108,8 @@ expect(parser.id).to eq('myproxyidwithreservedcharacters') end - context 'master and slave configs' do - it 'should provide master and slave configs' do + context 'master and replica configs' do + it 'should provide master and replica configs' do parser = described_class.new(config) expect(parser.master_configs).to eq([ { @@ -120,16 +120,16 @@ :master_ttl => 5 } ]) - expect(parser.slave_configs).to eq([ + expect(parser.replica_configs).to eq([ { - :name => 'slave1', + :name => 'replica1', :top_level => 'value', :sticky => true, :blacklist_duration => 30, :master_ttl => 5 }, { - :name => 'slave2', + :name => 'replica2', :top_level => 'value', :sticky => true, :blacklist_duration => 30, @@ -141,7 +141,7 @@ it 'connection configuration should override makara config' do config[:makara][:blacklist_duration] = 123 config[:makara][:connections][0][:blacklist_duration] = 456 - config[:makara][:connections][1][:top_level] = 'slave value' + config[:makara][:connections][1][:top_level] = 'replica value' parser = described_class.new(config) expect(parser.master_configs).to eq([ @@ -153,16 +153,16 @@ :master_ttl => 5 } ]) - expect(parser.slave_configs).to eq([ + expect(parser.replica_configs).to eq([ { - :name => 'slave1', - :top_level => 'slave value', + :name => 'replica1', + :top_level => 'replica value', :sticky => true, :blacklist_duration => 123, :master_ttl => 5 }, { - :name => 'slave2', + :name => 'replica2', :top_level => 'value', :sticky => true, :blacklist_duration => 123, diff --git a/spec/connection_wrapper_spec.rb b/spec/connection_wrapper_spec.rb index 40060177..f7dc4a75 100644 --- a/spec/connection_wrapper_spec.rb +++ b/spec/connection_wrapper_spec.rb @@ -2,7 +2,7 @@ describe Makara::ConnectionWrapper do - let(:proxy){ FakeProxy.new({:makara => {:blacklist_duration => 5, :connections => [{:role => 'master'}, {:role => 'slave'}, {:role => 'slave'}]}}) } + let(:proxy){ FakeProxy.new({:makara => {:blacklist_duration => 5, :connections => [{:role => 'master'}, {:role => 'replica'}, {:role => 'replica'}]}}) } let(:connection){ subject._makara_connection } subject{ proxy.master_pool.connections.first } diff --git a/spec/middleware_spec.rb b/spec/middleware_spec.rb index f721d067..f8f601c8 100644 --- a/spec/middleware_spec.rb +++ b/spec/middleware_spec.rb @@ -32,7 +32,7 @@ _, headers, body = middleware.call(env) expect(headers).to eq({}) - expect(body).to eq('slave/1') + expect(body).to eq('replica/1') end it 'should use the cookie-provided context if present' do diff --git a/spec/proxy_spec.rb b/spec/proxy_spec.rb index e23fcf37..0690a6bd 100644 --- a/spec/proxy_spec.rb +++ b/spec/proxy_spec.rb @@ -5,22 +5,22 @@ let(:klass){ FakeProxy } - it 'sets up a master and slave pool no matter the number of connections' do + it 'sets up a master and replica pool no matter the number of connections' do proxy = klass.new(config(0, 0)) expect(proxy.master_pool).to be_a(Makara::Pool) - expect(proxy.slave_pool).to be_a(Makara::Pool) + expect(proxy.replica_pool).to be_a(Makara::Pool) proxy = klass.new(config(2, 0)) expect(proxy.master_pool).to be_a(Makara::Pool) - expect(proxy.slave_pool).to be_a(Makara::Pool) + expect(proxy.replica_pool).to be_a(Makara::Pool) proxy = klass.new(config(0, 2)) expect(proxy.master_pool).to be_a(Makara::Pool) - expect(proxy.slave_pool).to be_a(Makara::Pool) + expect(proxy.replica_pool).to be_a(Makara::Pool) proxy = klass.new(config(2, 2)) expect(proxy.master_pool).to be_a(Makara::Pool) - expect(proxy.slave_pool).to be_a(Makara::Pool) + expect(proxy.replica_pool).to be_a(Makara::Pool) end @@ -28,7 +28,7 @@ proxy = klass.new(config(1, 2)) expect(proxy.master_pool.connection_count).to eq(1) - expect(proxy.slave_pool.connection_count).to eq(2) + expect(proxy.replica_pool.connection_count).to eq(2) end it 'should delegate any unknown method to a connection in the master pool' do @@ -97,7 +97,7 @@ expect(proxy.sticky).to eq(true) end - it 'should provide the slave pool for a read' do + it 'should provide the replica pool for a read' do expect(proxy.master_for?('select * from users')).to eq(false) end @@ -161,20 +161,20 @@ expect(proxy.master_for?('select * from users')).to eq(false) end - it 'should use master if all slaves are blacklisted' do - allow(proxy.slave_pool).to receive(:completely_blacklisted?){ true } + it 'should use master if all replicas are blacklisted' do + allow(proxy.replica_pool).to receive(:completely_blacklisted?){ true } expect(proxy.master_for?('select * from users')).to eq(true) end - it 'should use master if all slaves become blacklisted as part of the invocation' do - allow(proxy.slave_pool).to receive(:next).and_return(nil) + it 'should use master if all replicas become blacklisted as part of the invocation' do + allow(proxy.replica_pool).to receive(:next).and_return(nil) test = double expect(test).to receive(:blacklisting).once expect(test).to receive(:using_master).once proxy.send(:appropriate_pool, :execute, ['select * from users']) do |pool| - if pool == proxy.slave_pool + if pool == proxy.replica_pool test.blacklisting pool.instance_variable_get('@blacklist_errors') << StandardError.new('some connection issue') pool.connections.each(&:_makara_blacklist!) @@ -189,22 +189,22 @@ proxy.ping # weird setup to allow for the correct - proxy.slave_pool.connections.each(&:_makara_blacklist!) - proxy.slave_pool.instance_variable_get('@blacklist_errors') << StandardError.new('some slave connection issue') + proxy.replica_pool.connections.each(&:_makara_blacklist!) + proxy.replica_pool.instance_variable_get('@blacklist_errors') << StandardError.new('some replica connection issue') proxy.master_pool.connections.each(&:_makara_blacklist!) proxy.master_pool.instance_variable_get('@blacklist_errors') << StandardError.new('some master connection issue') - allow(proxy).to receive(:_appropriate_pool).and_return(proxy.slave_pool, proxy.master_pool) + allow(proxy).to receive(:_appropriate_pool).and_return(proxy.replica_pool, proxy.master_pool) begin proxy.send(:appropriate_pool, :execute, ['select * from users']) do |pool| pool.provide{|c| c } end rescue Makara::Errors::AllConnectionsBlacklisted => e - expect(e.message).to eq('[Makara/master] All connections are blacklisted -> some master connection issue -> [Makara/slave] All connections are blacklisted -> some slave connection issue') + expect(e.message).to eq('[Makara/master] All connections are blacklisted -> some master connection issue -> [Makara/replica] All connections are blacklisted -> some replica connection issue') end - proxy.slave_pool.connections.each{|con| expect(con._makara_blacklisted?).to eq(false) } + proxy.replica_pool.connections.each{|con| expect(con._makara_blacklisted?).to eq(false) } proxy.master_pool.connections.each{|con| expect(con._makara_blacklisted?).to eq(false) } end end diff --git a/spec/support/helpers.rb b/spec/support/helpers.rb index 642ec973..7edfc3f6 100644 --- a/spec/support/helpers.rb +++ b/spec/support/helpers.rb @@ -3,16 +3,16 @@ def establish_connection(config) connection = ActiveRecord::Base.establish_connection(config) # make sure these are all reset to not be blacklisted - ActiveRecord::Base.connection.slave_pool.connections.each(&:_makara_whitelist!) + ActiveRecord::Base.connection.replica_pool.connections.each(&:_makara_whitelist!) ActiveRecord::Base.connection.master_pool.connections.each(&:_makara_whitelist!) ActiveRecord::Base.connection end - def config(masters = 1, slaves = 2) + def config(masters = 1, replicas = 2) connections = [] masters.times{ connections << {:role => 'master'} } - slaves.times{ connections << {:role => 'slave'} } + replicas.times{ connections << {:role => 'replica'} } { :makara => { # Defaults: diff --git a/spec/support/mysql2_database.yml b/spec/support/mysql2_database.yml index d980058e..1e0e42b2 100644 --- a/spec/support/mysql2_database.yml +++ b/spec/support/mysql2_database.yml @@ -14,5 +14,5 @@ test: master_ttl: 5 connections: - role: master - - role: slave - - role: slave + - role: replica + - role: replica diff --git a/spec/support/mysql2_database_with_custom_errors.yml b/spec/support/mysql2_database_with_custom_errors.yml index 7e226bcc..852242d3 100644 --- a/spec/support/mysql2_database_with_custom_errors.yml +++ b/spec/support/mysql2_database_with_custom_errors.yml @@ -14,8 +14,8 @@ test: master_ttl: 5 connections: - role: master - - role: slave - - role: slave + - role: replica + - role: replica connection_error_matchers: - !ruby/regexp '/^ActiveRecord::StatementInvalid: Mysql2::Error: Unknown command1:/' - "/^ActiveRecord::StatementInvalid: Mysql2::Error: Unknown command2:/i" diff --git a/spec/support/postgis_database.yml b/spec/support/postgis_database.yml index 4691074c..dd5ec495 100644 --- a/spec/support/postgis_database.yml +++ b/spec/support/postgis_database.yml @@ -11,5 +11,5 @@ test: master_ttl: 5 connections: - role: master - - role: slave - - role: slave + - role: replica + - role: replica diff --git a/spec/support/postgresql_database.yml b/spec/support/postgresql_database.yml index 490cb8b8..cf41dd7d 100644 --- a/spec/support/postgresql_database.yml +++ b/spec/support/postgresql_database.yml @@ -9,5 +9,5 @@ test: master_ttl: 5 connections: - role: master - - role: slave - - role: slave + - role: replica + - role: replica diff --git a/spec/support/proxy_extensions.rb b/spec/support/proxy_extensions.rb index b7b91abe..1c62869e 100644 --- a/spec/support/proxy_extensions.rb +++ b/spec/support/proxy_extensions.rb @@ -1,6 +1,6 @@ module ProxyExtensions - attr_reader :master_pool, :slave_pool, :id + attr_reader :master_pool, :replica_pool, :id def master_for?(sql) pool_for(sql) == master_pool