Handling Webhooks

[fmaclen]

Hey there, I recently started using VCR and I'm trying to figure out how to properly handle webhooks.

My app has an endpoint POST /webhooks and during my tests I'm able to trigger the expected webhooks from an external API, the issue is that those incoming requests aren't recorded by VCR (my guess is because they originated outside of my testing environment).

I looked around VCR's docs, issues and previous discussions but couldn't find anything related to handling webhooks.
Is there some obvious configuration that I'm missing or is this simply not supported by VCR?

So far I'm able to circumvent this limitation by manually mocking those incoming webhooks but I'm running into an issue where if I mock an incoming webhook in this one test I also need to mock all of the other requests that are properly being recorded by VCR.

I'm wondering if there is a way I can check if VCR is in "record mode" in which case I can wait for the real webhook request and if it's in "playback mode" I mock them?

Your insight is much appreciated, thanks!

[nicolasiensen]

My app has an endpoint POST /webhooks and during my tests I'm able to trigger the expected webhooks from an external API

This seems odd. If you have to rely on an external service to call the system under test, you risk the determinism of your test suite.

Can't you make the request to POST /webhooks from the test itself?

the issue is that those incoming requests aren't recorded by VCR

It seems that you misunderstood the purpose of VCR. It doesn't record incoming requests to the system under test, but outgoing requests and their responses.

VCR was designed to improve the reliability of the test suite of systems that depend on third-party APIs. For example, suppose your system integrates with Twitter API. In that case, VCR will intercept all interactions your system has with this external API during tests and record them so your automated tests can run independently of Twitter's API.

[fmaclen]

If you have to rely on an external service to call the system under test...

I don't rely on it, it's just a request that my app receives but it's not originated as a response from a request I make from my tests.

The API I'm working with has a sandbox environment that allows me to trigger the webhook so I can test it, but when I call that function I don't get a response in the same request, it comes as a new incoming request a second or so later.

Even though it's a sandbox environment made for testing I was hoping VCR would capture it so I can avoid calling the API on every test run.

Can't you make the request to POST /webhooks from the test itself?

I can and that's how I'm handling this so far: manually mocking the webhook payload and sending the POST request to my /webhooks endpoint from my tests. This works fine in most cases.

The issue I have in some specific tests is that there are some time-delayed events that happen on the 3rd party API that my app needs to wait for, my app knows that those events are completed when I get a webhook from this API.

If I don't manually mock the webhook events when VCR is recording it all works well.
However, during playback those webhook events are never received so my app times out waiting for them since they were never recorded.

On the other hand, if I mock the webhook events during recording it all happens too quickly and when the test continues to the next assertion the 3rd party API is still processing my initial request and returns an error because it's not ready.

This is why I was wondering if there's a way to check if VCR is in record mode so I could use my skip my mocked webhook during recording and use it during playback.

It doesn't record incoming requests to the system under test, but outgoing requests and their responses.

I think this answers my question. I was hoping that VCR would record all of the requests my app sends and receives, not just the responses from the request originated from within the test.

That being said, here's a similar question from someone on Reddit.
The linked answer mentions that they can "record the payload received using VCR". Is that answer mistaken?

This seems odd.

It's possible that I wasn't explaining my situation correctly, hopefully it got clarified but let me know if it didn't.
Thanks again!

[nicolasiensen]

I never used VCR to record incoming requests. There might be a way to do it, but I'm not aware of it.

If I were in your place, I wouldn't try to record the incoming request but try to reproduce this request in my tests.

For example, if the external service makes a POST /webhooks to your API with a body { "foo": "bar" }, I would simulate this exact same request in the test and check that my system handled the webhook event as expected.

[fmaclen]

Understood, appreciate the help!

[krystof-k]

I think it would be great if VCR could expand in this direction and record/replay incoming requests – (Stripe) webhooks in my case as well.

I managed to find a workaround, but it would be nice to have it all in one package.

If anyone interested, here is my (simplified) approach:

RSpec.describe "Webhook test", :vcr do
  before(:each) do
    @real_request = false

    VCR.configure do |config|
      config.before_http_request(:real?) do |request|
        @real_request = true
      end
    end
  end

  before(:each) do
    allow_any_instance_of(ActionDispatch::Request).to receive(:body).and_wrap_original do |method, *args|
      payload = method.call.read
      parsed_payload = JSON.parse(payload)
      Dir.mkdir("spec/fixtures/webhooks") unless File.exist?("spec/fixtures/webhooks")
      File.write("spec/fixtures/webhooks/#{Time.now.to_i}.json", JSON.pretty_generate(parsed_payload))

      StringIO.new(payload)
    end
  end

  it "receives a webhook" do
    expect(SomeController)
      .to receive(:new)
      .once
      .and_call_original

    make_external_request

    unless @real_request
      payload = File.read("spec/fixtures/webhooks/1723632271.json")
      post "/webhook", params: payload
    end

    verify_webhook_was_handled
  end
end

[shunichi]

Hello everyone,

I'm excited to share a new Ruby gem I created that integrates with VCR to record and replay Stripe webhooks. This can be particularly useful for testing applications that rely on Stripe's webhook functionality without the need for actual network calls every time.

The gem is open-source and I would love for the community to try it out and contribute. You can find it here: vcr_stripe_webhook.

Feel free to provide feedback or contribute to the project!

Thank you!