Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEATURE: Allow asserting on requests in tests #1069

Open
wants to merge 16 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions lib/webmock/request_registry.rb
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,15 @@ def times_executed(request_pattern)
end.inject(0) { |sum, (_, times_executed)| sum + times_executed }
end

def requests_made
to_a
end

def to_a
requested_signatures.
array
end

def to_s
if requested_signatures.hash.empty?
"No requests were made."
Expand Down
4 changes: 4 additions & 0 deletions lib/webmock/request_signature.rb
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ def json_headers?
!!(headers&.fetch('Content-Type', nil)&.start_with?('application/json'))
end

def parsed_body
johngallagher marked this conversation as resolved.
Show resolved Hide resolved
JSON.parse(body, symbolize_names: true)
end

private

def assign_options(options)
Expand Down
16 changes: 15 additions & 1 deletion lib/webmock/util/hash_counter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,36 @@
module WebMock
module Util
class HashCounter
attr_accessor :hash
attr_accessor :hash, :array
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@johngallagher I believe we should separate the array of requests from the HashCounter. The HashCounter has a specific, single responsibility, and adding an array of requests expands its scope beyond its intended purpose.

Instead, I suggest moving the ordered list of requests up to the RequestRegistry object. This way, the RequestRegistry would manage both the HashCounter and the array of requests, maintaining a clearer separation of concerns.

What are your thoughts on this restructuring? It would allow the HashCounter to remain focused on its core functionality.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bblimke yup, that would work better. I do think that having all this cruft in the RequestRegistry might clutter it up, so it might be a RequestStore or something similar... although that's literally what RequestRegistry does I guess... so I'll look into that. Thanks! Great idea.

Copy link
Author

@johngallagher johngallagher Sep 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bblimke aha - I remember why I didn't put it into a separate object now. 💡

It's because of the design - HashCounter is exposed directly in lots of places like so:

https://github.com/johngallagher/webmock/blob/088aa2678f7efb34e910e8b430b54980283b9485/lib/webmock/http_lib_adapters/curb_adapter.rb/#L61

Personally, I'd rather encapsulate how the registry stores requests and hide that from clients.

That would mean we could have one store, two or more. But given how it's done, without a huge refactoring, I don't see how we'd get this in.

I'd be happy to take on that refactoring, to be clear - I love me some refactoring!

It'd be:

- WebMock::RequestRegistry.instance.requested_signatures.put(request_signature)
+ WebMock::RequestRegistry.instance.put(request_signature)

And the requested_signatures would be entirely private. We could have two different store classes that way and do something like (sketching it out roughly):

class WebMock::RequestRegistry
  def initialize
    @hash_counter_store = HashCounterStore.new,
    @array_store = ArrayStore.new
  end

  def put(request_signature)
    @hash_counter_store.put(request_signature)
    @array_store.put(request_signature)
  end

  def get(request_signature)
    @array_store.get(request_signature)
  end

  def to_a
    @array_store.to_a
  end

  def to_h
    @hash_counter_store.to_h
  end
  # ...
end

This would give us duck typing for stores, make their internals private and allow us to reach into the relevant store when we wanted to pluck out requests that were stored in a specific format.

To recap:

  1. Continue as I am munging it all into HashCounter
  2. Refactor before adding this functionality, then revisit this PR (defo a breaking change here!)

WDYT?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bblimke nudge :)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@johngallagher Thanks for the reminder. I agree that RequestRegistry internals should be private, with no other object having direct access to the hash counter or the array. I like your solution with two separate stores: hash_counter_store and array_store.

I'm not sure if this is a breaking change, since RequestRegistry isn't supposed to be used directly. It's not part of WebMock::API, which is the only versioned interface. However, there's a likelihood that some people have accessed RequestRegistry directly.

I don't mind releasing a new major version though, if we decide it's necessary.


def initialize
self.hash = Hash.new(0)
@order = {}
@max = 0
@lock = ::Mutex.new
self.array = []
@request_object_ids = {}
end

def put(key, num=1)
@lock.synchronize do
store_to_array(key:, num:)
hash[key] += num
@order[key] = @max += 1
end
end

def store_to_array(key:, num:)
request_object_id = @request_object_ids[key]
request_object_id = key.object_id if request_object_id.nil?
num.times do
array << ObjectSpace._id2ref(request_object_id)
rescue RangeError
johngallagher marked this conversation as resolved.
Show resolved Hide resolved
# Points to an invalid or recycled object so ignore
end
@request_object_ids[key] = key.object_id
end
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bblimke this is missing any comments to explain what's going on - maybe worth me adding some?

For others - doing this so that we are just referencing existing requests in the array, rather than creating new objects.

This is so that if we have massive numbers of requests that have the same signature, we're not bloating memory.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@johngallagher Thanks for your work on this. I've been looking at the store_to_array method.

I think it might be helpful to add a comment explaining the intention behind using object_id and _id2ref. This would really help to understand the reasoning behind this approach.

That said, I believe we might be able to simplify this code. I'm not sure whether we really need object_id and _id2ref at all. From my understanding, array is just storing references to the original objects, not clones.

I think the following achieves the same result:

def store_to_array(key, num)
  @request_objects ||= {}
  stored_object = @request_objects[key] ||= key
  num.times do
    array << stored_object
  end
end

This version uses a hash (@request_objects) to store the original request signature objects with same #hash result. It maintains the original object without the need for object_id.

It should be more efficient as it avoids the overhead of ObjectSpace._id2ref and potential RangeError rescues,
though I don't know how much faster that is.

Can you think of any scenarios where the original implementation behaves differently from this proposed one?

What are your thoughts on this?

I wonder if this can me optimised even further.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a much nicer solution than mine! I can't think of any scenarios just now but I'll think on it over the next few days...


def get(key)
@lock.synchronize do
hash[key]
Expand Down
68 changes: 68 additions & 0 deletions spec/unit/request_registry_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -92,4 +92,72 @@
end
end

describe "requests_made" do
it "returns the requests made" do
request_registry = WebMock::RequestRegistry.instance
request_registry.requested_signatures.put(WebMock::RequestSignature.new(:get, "www.example.com"))
request_registry.requested_signatures.put(WebMock::RequestSignature.new(:put, "www.example.org"))
expect(request_registry.requests_made.count).to eq(2)
expect(request_registry.requests_made[0].method).to eq(:get)
expect(request_registry.requests_made[0].uri).to eq(Addressable::URI.parse("http://www.example.com/"))
expect(request_registry.requests_made[1].method).to eq(:put)
expect(request_registry.requests_made[1].uri).to eq(Addressable::URI.parse("http://www.example.org/"))
end

it "returns the headers of the request" do
request_registry = WebMock::RequestRegistry.instance
request_registry.requested_signatures.put(WebMock::RequestSignature.new(:get, "www.example.com", headers: { 'Content-Type' => 'application/json' }))
expect(request_registry.requests_made.first.headers).to eq({ 'Content-Type' => 'application/json' })
end

it "only stores references to existing signatures" do
request_registry = WebMock::RequestRegistry.instance
signature = WebMock::RequestSignature.new(:get, "www.example.com")
duplicate_signature = WebMock::RequestSignature.new(:get, "www.example.com")
request_registry.requested_signatures.put(signature)
request_registry.requested_signatures.put(duplicate_signature)
expect(request_registry.requests_made[0].object_id).to eq(signature.object_id)
expect(request_registry.requests_made[1].object_id).to eq(signature.object_id)
end

context "when storing the same signature with a count" do
it "adds two references to the signature" do
request_registry = WebMock::RequestRegistry.instance
signature = WebMock::RequestSignature.new(:get, "www.example.com")
request_registry.requested_signatures.put(signature, 2)
expect(request_registry.requests_made[0].object_id).to eq(signature.object_id)
expect(request_registry.requests_made[1].object_id).to eq(signature.object_id)
end
end

context "with a JSON request" do
it "returns the parsed JSON body of the request" do
request_registry = WebMock::RequestRegistry.instance
request_registry.requested_signatures.put(json_request_with_body(a: 1))
expect(request_registry.requests_made.first.body).to eq({ a: 1 }.to_json)
expect(request_registry.requests_made.first.parsed_body).to eq(a: 1)
end

context "and invalid JSON in the body" do
it "crashes" do
request_registry = WebMock::RequestRegistry.instance
request_registry.requested_signatures.put(json_request_with_raw_body("{ invalid_json }"))
expect(request_registry.requests_made.first.body).to eq("{ invalid_json }")
expect do
request_registry.requests_made.first.parsed_body
end.to raise_error JSON::ParserError
end
end
end
end

private

def json_request_with_body(body)
WebMock::RequestSignature.new(:get, "www.example.com", body: JSON.generate(body))
end

def json_request_with_raw_body(body)
WebMock::RequestSignature.new(:get, "www.example.com", body: body)
end
end
Loading