Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make pc #263

Closed
wants to merge 13 commits into from
Closed

Make pc #263

wants to merge 13 commits into from

Conversation

mmontagna
Copy link

@mmontagna mmontagna commented Aug 7, 2020

Tests pass

Will break pretty much anything that uses Makara's classes.

marcomontagna@Marco-makara % rake spec

Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Randomized with seed 29944

ActiveRecord::ConnectionAdapters::MakaraAbstractAdapter::ErrorHandler
  should properly evaluate connection messages like: PGError: server closed the connection unexpectedly This probably me
  should properly evaluate connection messages like: Mysql2::Error: Timeout waiting for a response from the last query
  should blacklist the connection
  should blacklist the connection
  should properly evaluate connection messages like: PG::UnableToSend: no connection to the server
  should properly evaluate connection messages like: Mysql2::Error (Cannot connect to MySQL server on '123.456.789.234' (111))
  should blacklist the connection
  should properly evaluate connection messages like: Can't connect to MySQL server on '123.456.789.235' (111)
  should raise the error
  should blacklist the connection
  should properly evaluate connection messages like: Mysql2::Error Can't connect to MySQL server on '123.456.789.235' (111)
  should properly evaluate connection messages like: Mysql2::Error: MySQL server has gone away: SELECT `users`.* FROM `users`
  should properly evaluate connection messages like: PG::ConnectionBad: FATAL: the database system is starting up
  should properly evaluate connection messages like: PG::ConnectionBad: PQconsumeInput() SSL connection has been closed unexpectedly: SELECT  1 AS one FROM "users"  WHERE "users"."registration_ip" = '10.0.2.2' LIMIT 1
  should blacklist the connection
  should blacklist the connection
  should properly evaluate connection messages like: Mysql2::Error (Can't connect to MySQL server on '123.456.789.234' (111))
  should properly evaluate connection messages like: PG::ConnectionBad: PQsocket() can't get socket descriptor:
  should properly evaluate errors like: Mysql::Error: : INSERT INTO `watchers` (`user_id`, `watchable_id`, `watchable_type`) VALUES
  should blacklist the connection
  should properly evaluate connection messages like: PG::ConnectionBad (could not connect to server: Connection refused
  should blacklist the connection
  should properly evaluate connection messages like: PG::ConnectionBad: timeout expired
  should blacklist the connection
  should properly evaluate connection messages like: PG::AdminShutdown: FATAL:  terminating connection due to administrator command FATAL:  terminating connection due to administrator command
  should properly evaluate connection messages like: Could not connect to server: Connection refused Is the server running on host
  should blacklist the connection
  should blacklist the connection
  should blacklist the connection
  should properly evaluate errors like: PGError: ERROR: column items.user_id does not exist LINE 1: SELECT "items".* FROM "items" WHERE ("items".user_id = 4) OR
  should blacklist the connection
  should properly evaluate connection messages like: Mysql2::Error: Lost connection to MySQL server during query: SELECT `geographies`.* FROM `geographies`
  should properly evaluate connection messages like: PG::ConnectionBad: could not translate host name "some.sample.com" to address: Name or service not known
  should blacklist the connection
  should blacklist the connection
  should blacklist the connection
  should blacklist the connection
  should blacklist the connection
  should blacklist the connection
  should blacklist the connection
  should raise the error
  should properly evaluate connection messages like: PG::ConnectionBad: FATAL: the database system is shutting down
  should properly evaluate connection messages like: Mysql2::Error: closed MySQL connection: SELECT `users`.* FROM `users`
  should properly evaluate connection messages like: org.postgresql.util.PSQLException: Connection to localhost:123 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
  custom errors
    blacklists the connection
    identifies custom errors

Makara::Context
  does not share stickiness state across threads
  release_all
    clears stickiness for all stuck proxies
    does nothing if there were no stuck proxies
  set_current
    sets stickiness information from given hash
  next
    doesn't store staged proxies with 0 stickiness duration
    sets expiration time with ttl based on the invokation time
    clears expired entries for proxies that are no longer stuck
    returns nil if there is nothing new to stick
    doesn't update previously stored proxies if the update will cause a sooner expiration
    returns hash with updated stickiness
  stick
    uses always the max ttl given
    doesn't overwrite previously stuck proxies with current-request-only stickiness
    sticks a proxy to master for the current request
    supports floats as ttl
  release
    does nothing if the proxy given was not stuck
    clears stickiness for the given proxy

Makara::ConfigParser
  should provide a default proxy id based on the recursively sorted config
  should use provided proxy id instead of default
  should replace reserved characters and show a warning for provided proxy ids
  ::merge_and_resolve_default_url_config
    parses the url parameter and merges it into the config
    does not use DATABASE_URL env variable
    does nothing to a config without a url parameter
  master and slave configs
    should provide master and slave configs
    connection configuration should override makara config

Makara::Pool
  provides the next connection and blacklists
  provides the same connection if the context has not changed and the proxy is sticky
  raises an error when all connections are blacklisted
  sends methods to all underlying objects if asked to
  does not provide the same connection if the proxy is not sticky
  should determine if its completely blacklisted
  skips blacklisted connections when choosing the next one
  should wrap connections with a ConnectionWrapper as theyre added to the pool
  only sends methods to underlying objects which are not blacklisted

Makara::Middleware
  should set the cookie if master is used
  should init the context and not be stuck by default
  should use the cookie-provided context if present

Makara::Proxy
  instantiates N connections within each pool
  should delegate any unknown method to a connection in the master pool
  sets up a master and slave pool no matter the number of connections
  #stick_to_master
    supports a float master_ttl for stickiness duration
    should use master if manually forced
    should persist stickiness by default
    optionally skips stickiness persistence, so it applies only to the current request
  #appropriate_pool
    should provide the master pool for a write
    should provide the slave pool for a read
    should raise the error and whitelist all connections if everything is blacklisted (start over)
    should not release master if it was stuck in the same request (no context changes yet)
    should not stick to master after without_sticking block if there is a write in it
    should not stick to master if stickiness is disabled
    should be sticky by default
    should release master if all stuck connections are released
    should use master if all slaves become blacklisted as part of the invocation
    should use master if all slaves are blacklisted
    should stick to master once used for a sticky operation
    should not stick to master if we are in a without_sticking block

Makara::ConnectionWrapper
  should have a default weight of 1
  should extend the connection with new functionality
  should invoke hijacked methods on the proxy when invoked directly
  #_makara_blacklisted?
    should store the blacklist status
    should handle frozen pre-epoch dates

Makara::Strategies::PriorityFailover
  should take the top weight
  should handle failover to next one
  should take given order if no weights
  should use the strategy

MakaraPostgreSQLAdapter
  should allow a connection to be established
  with only master connection
    should not raise errors on read and write
  transaction support
    when sticky is true
      behaves like a transaction supporter
        when querying through a polymorphic relation
          should respect the transaction
        when executing a query
          should respect the transaction
        when querying an aggregate
          should respect the transaction
        when querying for a specific record
          should respect the transaction
    when sticky is false
      behaves like a transaction supporter
        when querying through a polymorphic relation
          should respect the transaction
        when executing a query
          should respect the transaction
        when querying an aggregate
          should respect the transaction
        when querying for a specific record
          should respect the transaction
  with only slave connection
    should raise error only on write
  with the connection established and schema loaded
    should send writes to master
    should allow real queries to work
    should send SET operations to each connection
    should send exists? to slave
    should send reads to the slave
    should have one master and two slaves
  without live connections
    should raise errors on read or write

MakaraMysql2Adapter
  transaction support
    when sticky is true
      behaves like a transaction supporter
        when querying through a polymorphic relation
          should respect the transaction
        when querying for a specific record
          should respect the transaction
        when executing a query
          should respect the transaction
        when querying an aggregate
          should respect the transaction
    when sticky is false
      behaves like a transaction supporter
        when querying for a specific record
          should respect the transaction
        when executing a query
          should respect the transaction
        when querying an aggregate
          should respect the transaction
        when querying through a polymorphic relation
          should respect the transaction
  unconnected
    should allow a connection to be established
    should execute a send_to_all and raise a NoConnectionsAvailable error
    should execute a send_to_all against master even if no slaves are connected
    unconnect afterwards
      should not blow up if a connection fails
  with the connection established and schema loaded
    should blacklist on timeout
    should send writes to master
    should allow reconnecting
    should send reads to the slave
    should allow reconnecting when one of the nodes is blacklisted
    should send SET operations to each connection
    should have one master and two slaves
    should allow real queries to work
    should send exists? to slave

ActiveRecord::ConnectionAdapters::MakaraAbstractAdapter
  determines that "select get_lock('foo', 0)" requires master
  should not stick to master if handling sql like "show table"
  should stick to master if handling sql like "rollback transaction"
  determines that "
      UPDATE
        dogs
      SET
        max_treats = 10
      WHERE
        max_treats IS NULL
    " should not be sent to all underlying connections
  determines that "describe table" requires master
  should not stick to master if handling sql like "show indexes"
  should not stick to master if handling sql like "show full tables"
  should not stick to master if handling sql like "show tables"
  should stick to master if handling sql like "begin transaction"
  determines that "show fields" requires master
  determines that "select pg_advisory_lock(12345)" requires master
  determines that "INSERT INTO wisdom ('The truth will
set you free.')" should not be sent to all underlying connections
  determines that "insert into dogs..." requires master
  determines that "begin" requires master
  should not stick to master if handling sql like "show view"
  determines that "show index" requires master
  determines that "with fence as (select * from felines) insert to cats" requires master
  should not stick to master if handling sql like "show schema"
  determines that "SET @@things" requires master
  determines that "INSERT INTO wisdom ('The truth will set you free.')" should not be sent to all underlying connections
  should not stick to master if handling sql like "show index"
  determines that "set @@things" requires master
  determines that "insert into cats (select * from felines)" requires master
  determines that "show tables" requires master
  determines that "select * from users lock in share mode" requires master
  determines that "select * from users for update" requires master
  determines that "select release_lock('foo')" requires master
  should not stick to master if handling sql like "show database"
  should stick to master if handling sql like "commit transaction"
  determines that "select currval('users_id_seq')" requires master
  determines that "select * from users where name = "for update"" does not require master
  should not stick to master if handling sql like "show views"
  determines that "select pg_advisory_unlock(12345)" requires master
  determines that "select nextval('users_id_seq')" requires master
  determines that "with fence as (select * from users) select * from fence" does not require master
  should stick to master if handling sql like "
      UPDATE
        dogs
      SET
        max_treats = 10
      WHERE
        max_treats IS NULL
    "
  determines that "UPDATE dogs SET max_treats = 10 WHERE max_treats IS NULL" should not be sent to all underlying connections
  determines that "delete from people" requires master
  determines that "SET @@things" should be sent to all underlying connections
  determines that "select lastval()" requires master
  should stick to master if handling sql like "insert into"
  determines that "rollback" requires master
  determines that "select * from felines" does not require master
  determines that "select * from users where name = "lock in share mode"" does not require master
  should not stick to master if handling sql like "describe stuff"
  should not stick to master if handling sql like "explain things"
  should stick to master if handling sql like "begin deferred transaction"
  should not stick to master if handling sql like "show full table"
  determines that "    select * from users for update" requires master
  should not stick to master if handling sql like "set @@things"
  determines that "savepoint active_record_1" requires master
  should stick to master if handling sql like "update users"
  determines that "release savepoint" requires master
  should stick to master if handling sql like "delete from"
  determines that "    select * from felines" does not require master
  determines that "update users set" requires master
  determines that "commit" requires master

Makara::Strategies::RoundRobin
  should handle failover to next one
  should loop through with weights
  default config
    should default to the strategy
  bad config
    should raise name error
  given in config
    should use the strategy

Makara::Cookie
  fetch
    returns empty context data when the cookie contents are invalid
    parses stickiness context from cookie string
    returns empty context data when there is no cookie
  store
    sets the context cookie with updated stickiness and enough expiration time
    expires the cookie if the next context is empty
    allows custom cookie options to be provided
    does not set a cookie if there is no next context

Makara::Cache
  shows a warning

Finished in 15.03 seconds (files took 1.05 seconds to load)
219 examples, 0 failures

Randomized with seed 29944

@mlarraz
Copy link
Member

mlarraz commented Jan 19, 2021

Superseded by #286 and #290

@mlarraz mlarraz closed this Jan 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants