Received response with incorrect message id: ... warnings

[postmodern]

I'm noticing that sometimes I get these warnings when sending lots of DNS queries to 8.8.8.8:53 (UDP). Based on the code they appear to be coming from Async::DNS::Resolver#valid_response and cause the dispatch_request to not return a response object. Is there anyway to prevent these? Should I just retry the query if no response is returned?

[ioquatix]

I wonder if this is a race condition at the UDP packet level. Maybe the implementation is sending multiple requests and reading multiple responses and getting mixed up.

[postmodern]

Hmm, I can't seem to repro this locally. It might be happening due to the network (ex: AWS EC2) or other rate limiting.

#!/usr/bin/env ruby

require 'bundler/setup'
require 'async'
require 'async/queue'
require 'async/dns'
require 'open-uri'

WORKERS = 100

queue = Async::LimitedQueue.new(WORKERS)
subdomains = URI.open('https://raw.githubusercontent.com/rbsec/dnscan/master/subdomains-1000.txt').each_line(chomp: true)

resolver = Async::DNS::Resolver.new([[:udp, '8.8.8.8', 53]])

Async do |task|
  task.async do
    subdomains.each do |subdomain|
      queue.enqueue("#{subdomain}.example.com")
    end

    WORKERS.times do
      queue.enqueue(nil)
    end
  end

  workers = WORKERS.times.map do
    task.async do
      while (host_name = queue.dequeue)
        puts "Resolving #{host_name} ..."

        begin
          resolver.addresses_for(host_name)
        rescue Async::DNS::ResolutionFailure
        end
      end
    end
  end

  workers.each(&:wait)
end

[seandilda]

I ran into the same issue. I started troubleshooting it and found UDP source ports are being shared by different sockets which is causing the wrong socket to read the response. I've reported it as https://github.com/socketry/async-io/issues/83