Slow

[atomkirk]

What is causing it to be so slow? Is it exvcr itself or whatever vcr is using to intercept the network calls?

[parroty]

Network call happens only when recording the data (first time), but method mocking (stubbing methods) on each replaying involves certain overheads. There would be rooms for improvement, but haven't been able to identify good ways.

[jeffdeville]

@parroty Would you mind elaborating a bit on where the speed problems ultimately reside? Is it in the mocking in meck itself? I'm asking, because it shouldn't be possible for one spec to interfere w/ another, and if you don't set clear_mock, they can. If it's possible to fix the speed, and make that the default behaviour, I think it'd be worthwhile. (I can try to help, but if you could direct me on what you alrady know about the problem, I'd appreciate it)

[parroty]

I'm afraid that I haven't been able to investigate in detail. Profiling or something may be one approach.

[tompesman]

Hi,

I'm also analysing the performance of ExVCR, in my case a test with ExVCR is about 20x times slower than a test without, 230ms vs 20ms. The tests with the 3 adapters are equally slow so I think it's in the shared code. I've tried some tools to get some indicators where to look to fix the performance problem. With ExProf I got the following result on with this test: mix test test/adapter_hackney_test.exs:11. The two highest values in the uS / CALLS column are: :erlang.term_to_binary and :erlang.binary_to_term. It looks like it has something to do with the JSON Module.

Gist: https://gist.github.com/tompesman/7d15f60ccb3393cf929c608a5e081ea1

[tompesman]

After some more research it looks like the mock.ex file is causing the delay. The Task.await(:infinity) method is waiting for :meck.expect(module_name, function, callback) to return. Not sure why there is a delay here.

[jfrolich]

Also really interested, some of my tests with only 2 requests take 500ms which should be almost instant when no database connections are being made.

[StanBright]

Has anyway found a work-around?

[surik]

I had the same issue with mocking, so instead mock for every cassette I implemented mocking on startup, see #158

With this workaround, an application that heavily uses cassettes now needs 120 seconds less to run mix test

[dkuku]

I used mimic for mocking and I think it's way faster but it requires refactor of the app to allow for actual request
Meck takes as params only the function arguments and mimic requires mfa if you previously did mimic.stub call_original(Calculator, :add, [1, 2]) == 3 or just do nothining and it calls it. But currenly not everything is available currently in the context - for my testing I just hardcoded it but to make it flexible with all libraries it needs to be done properly. Also in mimic you call Mimic.copy(OriginalModule) only once - no need to do it every time you load a cassette (this may be the reason meck is so slow)