Feature/SMTP fail fast (#114)

* Added Truemail::Configuration#smtp_fail_fast, tests * Added Truemail::Validate::Smtp#smtp_fail_fast? * Added Truemail::Validate::Smtp#filtered_mail_servers_by_fail_fast_scenario * Updated Truemail::Validate::Smtp#establish_smtp_connection, tests * Updated readme, changelog * Updated linter configs * Updated gem version
truemail-rb · Dec 1, 2020 · a72374f · a72374f
1 parent 949af3c
commit a72374f
Show file tree

Hide file tree

Showing 9 changed files with 245 additions and 45 deletions.
diff --git a/.reek.yml b/.reek.yml
@@ -27,6 +27,7 @@ detectors:
     exclude:
       - Truemail::Configuration#whitelist_validation
       - Truemail::Configuration#not_rfc_mx_lookup_flow
+      - Truemail::Configuration#smtp_fail_fast
       - Truemail::Configuration#smtp_safe_check
       - Truemail::Wrapper#attempts
 

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,9 +2,35 @@
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.2.0] - 2020.12.01
+
+Ability to use fail fast behaviour for SMTP validation layer. When `smtp_fail_fast = true` it means that `truemail` ends smtp validation session after first attempt on the first mx server in any fail cases (network connection/timeout error, smtp validation error). This feature helps to reduce total time of SMTP validation session up to 1 second.
+
+### Added
+
+- Added `Truemail::Configuration#smtp_fail_fast`
+- Added `Truemail::Validate::Smtp#smtp_fail_fast?`
+- Added `Truemail::Validate::Smtp#filtered_mail_servers_by_fail_fast_scenario`
+
+### Changed
+
+- Updated `Truemail::Validate::Smtp#attempts`
+- Updated `Truemail::Validate::Smtp#establish_smtp_connection`
+- Updated gem documentation
+
+It's a configurable and not required option:
+
+```ruby
+Truemail.configure do |config|
+  config.smtp_fail_fast = true # by default it's equal to false
+end
+```
+
+Thanks to [@wikiti](https://github.com/wikiti) for timeout reports.
+
 ## [2.1.0] - 2020.11.21
 
-Collecting only unique ip-addresses for target mail servers. This update should reduce email validation time for case when remote server have closed connection via avoiding connection attempt to server with the same ip address.
+Collecting only unique ip-addresses for target mail servers. This update reduces email validation time for case when remote server have closed connection via avoiding connection attempt to server with the same ip address.
 
 ### Changed
 

diff --git a/Gemfile.lock b/Gemfile.lock
@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    truemail (2.1.0)
+    truemail (2.2.0)
       simpleidn (~> 0.1.1)
 
 GEM

diff --git a/README.md b/README.md
@@ -40,6 +40,7 @@ Configurable framework agnostic plain Ruby email validator. Verify email via Reg
       - [RFC MX lookup flow](#rfc-mx-lookup-flow)
       - [Not RFC MX lookup flow](#not-rfc-mx-lookup-flow)
     - [SMTP validation](#smtp-validation)
+      - [SMTP fail fast enabled](#smtp-fail-fast-enabled)
       - [SMTP safe check disabled](#smtp-safe-check-disabled)
       - [SMTP safe check enabled](#smtp-safe-check-enabled)
   - [Host audit features](#host-audit-features)
@@ -82,10 +83,12 @@ Also Truemail gem allows performing an audit of the host in which runs.
 - Minimal runtime dependencies
 - Supporting of internationalized emails ([EAI](https://en.wikipedia.org/wiki/Email_address#Internationalization))
 - Whitelist/blacklist validation layers
+- Ability to configure different MX/SMTP validation flows
 - Simple SMTP debugger
 - Event logger
 - Host auditor tools (helps to detect common host problems interfering to proper email verification)
 - JSON serializers
+- Ability to use the library as independent stateless microservice ([Truemail Server](https://truemail-rb.org/truemail-rack))
 
 ## Requirements
 
@@ -129,6 +132,8 @@ You can use global gem configuration or custom independent configuration. Availa
 - whitelisted domains
 - whitelist validation
 - blacklisted domains
+- RFC MX lookup flow
+- SMTP fail fast
 - SMTP safe check
 - event logger
 - JSON serializer
@@ -201,6 +206,13 @@ Truemail.configure do |config|
   # By default this option is disabled.
   config.not_rfc_mx_lookup_flow = true
 
+  # Optional parameter. This option will provide to use smtp fail fast behaviour. When
+  # smtp_fail_fast = true it means that truemail ends smtp validation session after first
+  # attempt on the first mx server in any fail cases (network connection/timeout error,
+  # smtp validation error). This feature helps to reduce total time of SMTP validation
+  # session up to 1 second. By default this option is disabled.
+  config.smtp_fail_fast = true
+
   # Optional parameter. This option will be parse bodies of SMTP errors. It will be helpful
   # if SMTP server does not return an exact answer that the email does not exist
   # By default this option is disabled, available for SMTP validation only.
@@ -233,6 +245,7 @@ Truemail.configuration
  @verifier_domain="somedomain.com",
  @verifier_email="verifier@example.com",
  @not_rfc_mx_lookup_flow=true,
+ @smtp_fail_fast=true,
  @smtp_safe_check=true,
  @logger=#<Truemail::Logger:0x0000557f837450b0
    @event=:all, @file="/home/app/log/truemail.log", @stdout=true>>
@@ -262,6 +275,7 @@ Truemail.configuration
  @verifier_domain="somedomain.com",
  @verifier_email="verifier@example.com",
  @not_rfc_mx_lookup_flow=true,
+ @smtp_fail_fast=true,
  @smtp_safe_check=true,
  @logger=#<Truemail::Logger:0x0000557f837450b0
    @event=:all, @file="/home/app/log/truemail.log", @stdout=true>>
@@ -294,7 +308,6 @@ Truemail.host_audit('email@example.com', custom_configuration: custom_configurat
 
 Please note, you should have global or custom configuration for use Truemail gem.
 
-
 ### Validation features
 
 #### Whitelist/Blacklist check
@@ -346,6 +359,7 @@ Truemail.validate('email@white-domain.com')
      @response_timeout=2,
      @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
      @not_rfc_mx_lookup_flow=false,
+     @smtp_fail_fast=false,
      @smtp_safe_check=false,
      @validation_type_by_domain={"somedomain.com"=>:mx},
      @verifier_domain="example.com",
@@ -392,6 +406,7 @@ Truemail.validate('email@white-domain.com', with: :regex)
      @response_timeout=2,
      @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
      @not_rfc_mx_lookup_flow=false,
+     @smtp_fail_fast=false,
      @smtp_safe_check=false,
      @validation_type_by_domain={},
      @verifier_domain="example.com",
@@ -424,6 +439,7 @@ Truemail.validate('email@domain.com', with: :regex)
      @response_timeout=2,
      @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
      @not_rfc_mx_lookup_flow=false,
+     @smtp_fail_fast=false,
      @smtp_safe_check=false,
      @validation_type_by_domain={},
      @verifier_domain="example.com",
@@ -458,6 +474,7 @@ Truemail.validate('email@black-domain.com')
      @response_timeout=2,
      @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
      @not_rfc_mx_lookup_flow=false,
+     @smtp_fail_fast=false,
      @smtp_safe_check=false,
      @validation_type_by_domain={},
      @verifier_domain="example.com",
@@ -492,6 +509,7 @@ Truemail.validate('email@somedomain.com')
      @response_timeout=2,
      @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
      @not_rfc_mx_lookup_flow=false,
+     @smtp_fail_fast=false,
      @smtp_safe_check=false,
      @validation_type_by_domain={},
      @verifier_domain="example.com",
@@ -542,6 +560,7 @@ Truemail.validate('email@example.com', with: :regex)
        @response_timeout=2,
        @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
        @not_rfc_mx_lookup_flow=false,
+       @smtp_fail_fast=false,
        @smtp_safe_check=false,
        @validation_type_by_domain={},
        @verifier_domain="example.com",
@@ -584,6 +603,7 @@ Truemail.validate('email@example.com', with: :regex)
        @response_timeout=2,
        @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
        @not_rfc_mx_lookup_flow=false,
+       @smtp_fail_fast=false,
        @smtp_safe_check=false,
        @validation_type_by_domain={},
        @verifier_domain="example.com",
@@ -637,6 +657,7 @@ Truemail.validate('email@example.com', with: :mx)
        @response_timeout=2,
        @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
        @not_rfc_mx_lookup_flow=false,
+       @smtp_fail_fast=false,
        @smtp_safe_check=false,
        @validation_type_by_domain={},
        @verifier_domain="example.com",
@@ -681,6 +702,7 @@ Truemail.validate('email@example.com', with: :mx)
        @response_timeout=2,
        @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
        @not_rfc_mx_lookup_flow=true,
+       @smtp_fail_fast=false,
        @smtp_safe_check=false,
        @validation_type_by_domain={},
        @verifier_domain="example.com",
@@ -702,6 +724,68 @@ If total count of MX servers is equal to one, `Truemail::Smtp` validator will us
 
 By default, you don't need pass with-parameter to use it. Example of usage is specified below:
 
+##### SMTP fail fast enabled
+
+Truemail can use fail fast behaviour for SMTP validation layer. When `smtp_fail_fast = true` it means that `truemail` ends smtp validation session after first attempt on the first mx server in any fail cases (network connection/timeout error, smtp validation error). This feature helps to reduce total time of SMTP validation session up to 1 second.
+
+```ruby
+require 'truemail'
+
+Truemail.configure do |config|
+  config.verifier_email = 'verifier@example.com'
+  config.smtp_fail_fast = true
+end
+
+Truemail.validate('email@example.com')
+
+# SMTP validation failed, smtp fail fast validation scenario
+=> #<Truemail::Validator:0x00007fdc4504f460
+    @result=
+      #<struct Truemail::Validator::Result
+        success=false,
+        email="email@example.com",
+        domain="example.com",
+        mail_servers=["127.0.1.1", "127.0.1.2", "127.0.1.3"], # there are 3 mail servers in a row
+        errors={:smtp=>"smtp error"},
+        smtp_debug=
+          [#<Truemail::Validate::Smtp::Request:0x00007fdc43150b90 # but iteration has been stopped after the first failure
+            @attempts=nil,
+            @configuration=
+              #<Truemail::Validate::Smtp::Request::Configuration:0x00007fdc43150b18
+                @connection_timeout=2,
+                @response_timeout=2,
+                @verifier_domain="example.com",
+                @verifier_email="verifier@example.com">,
+            @email="email@example.com",
+            @host="127.0.1.1",
+            @response=
+              #<struct Truemail::Validate::Smtp::Response
+                port_opened=false,
+                connection=nil,
+                helo=nil,
+                mailfrom=nil,
+                rcptto=nil,
+                errors={}>>],
+        configuration=
+          #<Truemail::Configuration:0x00007fdc4504f5c8
+            @blacklisted_domains=[],
+            @connection_attempts=2,
+            @connection_timeout=2,
+            @default_validation_type=:smtp,
+            @email_pattern=/(?=\A.{6,255}\z)(\A([\p{L}0-9]+[\w|\-.+]*)@((?i-mx:[\p{L}0-9]+([\-.]{1}[\p{L}0-9]+)*\.\p{L}{2,63}))\z)/,
+            @not_rfc_mx_lookup_flow=false,
+            @response_timeout=2,
+            @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
+            @smtp_fail_fast=true,
+            @smtp_safe_check=false,
+            @validation_type_by_domain={},
+            @verifier_domain="example.com",
+            @verifier_email="verifier@example.com",
+            @whitelist_validation=false,
+            @whitelisted_domains=[]>>,
+      @validation_type=:smtp>
+```
+
 ##### SMTP safe check disabled
 
 With `smtp_safe_check = false`
@@ -735,6 +819,7 @@ Truemail.validate('email@example.com')
        @response_timeout=2,
        @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
        @not_rfc_mx_lookup_flow=false,
+       @smtp_fail_fast=false,
        @smtp_safe_check=false,
        @validation_type_by_domain={},
        @verifier_domain="example.com",
@@ -784,6 +869,7 @@ Truemail.validate('email@example.com')
              @response_timeout=2,
              @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
              @not_rfc_mx_lookup_flow=false,
+             @smtp_fail_fast=false,
              @smtp_safe_check=false,
              @validation_type_by_domain={},
              @verifier_domain="example.com",
@@ -845,6 +931,7 @@ Truemail.validate('email@example.com')
              @response_timeout=2,
              @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
              @not_rfc_mx_lookup_flow=false,
+             @smtp_fail_fast=false,
              @smtp_safe_check=false,
              @validation_type_by_domain={},
              @verifier_domain="example.com",
@@ -891,6 +978,7 @@ Truemail.validate('email@example.com')
              @response_timeout=2,
              @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
              @not_rfc_mx_lookup_flow=false,
+             @smtp_fail_fast=false,
              @smtp_safe_check=false,
              @validation_type_by_domain={},
              @verifier_domain="example.com",
@@ -938,6 +1026,7 @@ Truemail.host_audit
          @response_timeout=2,
          @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
          @not_rfc_mx_lookup_flow=false,
+         @smtp_fail_fast=false,
          @smtp_safe_check=false,
          @validation_type_by_domain={},
          @verifier_domain="example.com",
@@ -964,6 +1053,7 @@ Truemail.host_audit
          @response_timeout=2,
          @smtp_error_body_pattern=/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i,
          @not_rfc_mx_lookup_flow=false,
+         @smtp_fail_fast=false,
          @smtp_safe_check=false,
          @validation_type_by_domain={},
          @verifier_domain="example.com",
@@ -1012,6 +1102,7 @@ Truemail::Log::Serializer::AuditorJson.call(Truemail.host_audit)
     "whitelisted_domains": null,
     "blacklisted_domains": null,
     "not_rfc_mx_lookup_flow": false,
+    "smtp_fail_fast": false,
     "smtp_safe_check": false,
     "email_pattern": "default gem value",
     "smtp_error_body_pattern": "default gem value"
@@ -1050,6 +1141,7 @@ Truemail::Log::Serializer::ValidatorJson.call(Truemail.validate('nonexistent_ema
     "whitelisted_domains": null,
     "blacklisted_domains": null,
     "not_rfc_mx_lookup_flow": false,
+    "smtp_fail_fast": false,
     "smtp_safe_check": false,
     "email_pattern": "default gem value",
     "smtp_error_body_pattern": "default gem value"
@@ -1090,6 +1182,7 @@ Truemail.host_audit.as_json
     "whitelisted_domains": null,
     "blacklisted_domains": null,
     "not_rfc_mx_lookup_flow": false,
+    "smtp_fail_fast": false,
     "smtp_safe_check": false,
     "email_pattern": "default gem value",
     "smtp_error_body_pattern": "default gem value"
@@ -1125,6 +1218,7 @@ Truemail.validate('nonexistent_email@bestweb.com.ua').as_json
     "whitelisted_domains": null,
     "blacklisted_domains": null,
     "not_rfc_mx_lookup_flow": false,
+    "smtp_fail_fast": false,
     "smtp_safe_check": false,
     "email_pattern": "default gem value",
     "smtp_error_body_pattern": "default gem value"

diff --git a/lib/truemail/configuration.rb b/lib/truemail/configuration.rb
@@ -21,7 +21,7 @@ class Configuration
                 :blacklisted_domains,
                 :logger
 
-    attr_accessor :whitelist_validation, :not_rfc_mx_lookup_flow, :smtp_safe_check
+    attr_accessor :whitelist_validation, :not_rfc_mx_lookup_flow, :smtp_fail_fast, :smtp_safe_check
 
     def initialize(&block)
       instance_initializer.each do |instace_variable, value|
@@ -102,6 +102,7 @@ def instance_initializer
         whitelist_validation: false,
         blacklisted_domains: [],
         not_rfc_mx_lookup_flow: false,
+        smtp_fail_fast: false,
         smtp_safe_check: false
       }
     end

diff --git a/lib/truemail/validate/smtp.rb b/lib/truemail/validate/smtp.rb
@@ -24,21 +24,31 @@ def run
 
       private
 
-      def request
-        smtp_results.last
+      def fail_fast?
+        configuration.smtp_fail_fast
+      end
+
+      def filtered_mail_servers_by_fail_fast_scenario
+        fail_fast? ? mail_servers.first(1) : mail_servers
       end
 
       def attempts
-        @attempts ||=
-          mail_servers.one? ? { attempts: configuration.connection_attempts } : {}
+        @attempts ||= begin
+          return {} if fail_fast? || !mail_servers.one?
+          { attempts: configuration.connection_attempts }
+        end
+      end
+
+      def request
+        smtp_results.last
       end
 
       def rcptto_error
         request.response.errors[:rcptto]
       end
 
       def establish_smtp_connection
-        mail_servers.each do |mail_server|
+        filtered_mail_servers_by_fail_fast_scenario.each do |mail_server|
           smtp_results << Truemail::Validate::Smtp::Request.new(
             configuration: configuration, host: mail_server, email: result.punycode_email, **attempts
           )