Pact-stub-server Multi-threaded

[Ajay2020813]

Hi Team

I was trying to run the pact stub server with multiple instances of the client but i am getting errors related to request timeouts.Looks like PactsubServer is not capable of serving the parallel requests.

I am using swift app with multiple simulators to see the behavior.

Is there any setting/param to be enable to make it support multi-thread?

Any support is appreciated. This is urgent

Thanks

[uglyog]

Looks like it was CPU bound (it was only running one thread, and that thread/CPU was being overwhelmed). I have pushed a fix for this.

[Ajay2020813]

Thank you so much for the quick fix.

…

On Mon, Jul 13, 2020 at 5:57 PM Ronald Holshausen ***@***.***> wrote: Looks like it was CPU bound (it was only running one thread, and that thread/CPU was being overwhelmed). I have pushed a fix for this. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQH3T65AKJEEP2YKFS4P7E3R3OUPXANCNFSM4OXKTV7A> .

[uglyog]

0.4.2 has been released with the fix

[Ajay2020813]

Hi Ronald I am still seeing the pact stubby server is not responding well as it was single client/app simulator. Definitely there is improvement, but still lot of time out comes. Pact stubby server responds with more errors if I increase more than 2 clients. Is there any way there could be more optimization on the server to handle multiple requests better or if require, Can we allocate more resources to this process on Mac machine? Appreciate your insight Thanks AJayGupta

…

On Mon, Jul 13, 2020 at 6:30 PM Ronald Holshausen ***@***.***> wrote: 0.4.2 has been released with the fix — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQH3T67QU7SSPJHGCDVI6N3R3OYJ3ANCNFSM4OXKTV7A> .

[uglyog]

How are you running the test, and what is the machine CPU and memory usage while it is running?

I ran a test with 5000 requests per second. This was on the loopback adaptor, so no network was involved.

$ echo 'GET http://127.0.0.1:8080/activities' | vegeta attack -duration=10s -rate=5000 | tee results.bin | vegeta report
Requests      [total, rate, throughput]         50000, 5000.11, 5000.00
Duration      [total, attack, wait]             10s, 10s, 226.018µs
Latencies     [min, mean, 50, 90, 95, 99, max]  119.339µs, 385.528µs, 218.218µs, 274.204µs, 311.223µs, 4.859ms, 39.723ms
Bytes In      [total, mean]                     8200000, 164.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:50000

By default, it will use 1 thread per CPU core plus one, so on my laptop it would use 13 threads. Your machine will be different.

[Ajay2020813]

Ronald Thanks for the response! I would like to show you live the behavior we are observing. When could you spare some time ? I will set up a meetup. It will be my pleasure to meet with you and discuss this further. Thanks AjayGupta

…

On Tue, Jul 14, 2020 at 9:31 PM Ronald Holshausen ***@***.***> wrote: How are you running the test, and what is the machine CPU and memory usage while it is running? I ran a test with 5000 requests per second. This was on the loopback adaptor, so no network was involved. $ echo 'GET http://127.0.0.1:8080/activities' | vegeta attack -duration=10s -rate=5000 | tee results.bin | vegeta report Requests [total, rate, throughput] 50000, 5000.11, 5000.00 Duration [total, attack, wait] 10s, 10s, 226.018µs Latencies [min, mean, 50, 90, 95, 99, max] 119.339µs, 385.528µs, 218.218µs, 274.204µs, 311.223µs, 4.859ms, 39.723ms Bytes In [total, mean] 8200000, 164.00 Bytes Out [total, mean] 0, 0.00 Success [ratio] 100.00% Status Codes [code:count] 200:50000 By default, it will use 1 thread per CPU core plus one, so on my laptop it would use 13 threads. Your machine will be different. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQH3T62JNNEK4RN5ADZWIU3R3UWLVANCNFSM4OXKTV7A> .

[Ajay2020813]

Hi Ronald One more thing to add is our pact-stub-server is loaded with pact files of around 10 MB, it’s basically around 1000 interactions. But we are also running on the same machine where client app is running, so no network involves. Thanks AjayGupta

…

On Tue, Jul 14, 2020 at 7:11 PM AJay GUpta ***@***.***> wrote: Hi Ronald I am still seeing the pact stubby server is not responding well as it was single client/app simulator. Definitely there is improvement, but still lot of time out comes. Pact stubby server responds with more errors if I increase more than 2 clients. Is there any way there could be more optimization on the server to handle multiple requests better or if require, Can we allocate more resources to this process on Mac machine? Appreciate your insight Thanks AJayGupta On Mon, Jul 13, 2020 at 6:30 PM Ronald Holshausen < ***@***.***> wrote: > 0.4.2 has been released with the fix > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#36 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AQH3T67QU7SSPJHGCDVI6N3R3OYJ3ANCNFSM4OXKTV7A> > . >

[Ajay2020813]

Hi Ronald I did quick 2 stress test and here are the results echo 'GET http://127.0.0.1:9992/orders/nonmerch/brand/10/market/US/store/04781' | vegeta attack -duration=1s -rate=100 | tee results.bin | vegeta report Requests [total, rate, throughput] 100, 101.02, 3.91 Duration [total, attack, wait] 25.597s, 989.892ms, 24.607s Latencies [min, mean, 50, 90, 95, 99, max] 888.178ms, 13.91s, 14.972s, 23.209s, 24.034s, 24.853s, 24.957s Bytes In [total, mean] 37900, 379.00 Bytes Out [total, mean] 0, 0.00 Success [ratio] 100.00% Status Codes [code:count] 200:100 Error Set: echo 'GET http://127.0.0.1:9992/orders/nonmerch/brand/10/market/US/store/04781' | vegeta attack -duration=1s -rate=500 | tee results.bin | vegeta report Requests [total, rate, throughput] 500, 501.27, 2.61 Duration [total, attack, wait] 30.992s, 997.47ms, 29.994s Latencies [min, mean, 50, 90, 95, 99, max] 58.383µs, 12.311s, 455.881ms, 30s, 30s, 30.001s, 30.003s Bytes In [total, mean] 30699, 61.40 Bytes Out [total, mean] 0, 0.00 Success [ratio] 16.20% Status Codes [code:count] 0:419 200:81 Error Set: Get "http://127.0.0.1:9992/orders/nonmerch/brand/10/market/US/store/04781": dial tcp 0.0.0.0:0->127.0.0.1:9992: socket: too many open files Get "http://127.0.0.1:9992/orders/nonmerch/brand/10/market/US/store/04781": context deadline exceeded (Client.Timeout exceeded while awaiting headers) *AFTER ulimit -Sn 20000* echo 'GET http://127.0.0.1:9992/orders/nonmerch/brand/10/market/US/store/04781' | vegeta attack *-duration=1s* -*rate=500* | tee results.bin | vegeta report Requests [total, rate, throughput] 500, 501.13, 2.71 Duration [total, attack, wait] 30.998s, 997.735ms, 30s Latencies [min, mean, 50, 90, 95, 99, max] 2.058s, 27.326s, 30s, 30s, 30s, 30.001s, 30.005s Bytes In [total, mean] 31836, 63.67 Bytes Out [total, mean] 0, 0.00 Success [ratio] 16.80% Status Codes [code:count] 0:416 200:84 Error Set: Get "http://127.0.0.1:9992/orders/nonmerch/brand/10/market/US/store/04781": EOF Get "http://127.0.0.1:9992/orders/nonmerch/brand/10/market/US/store/04781": context deadline exceeded (Client.Timeout exceeded while awaiting headers) *HERE IS MY LAPTOP CONFIGS* Ajays-MacBook-Pro:Sell ajay$ ulimit -a -S core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 20000 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 1418 virtual memory (kbytes, -v) unlimited Ajays-MacBook-Pro:Sell ajay$ ulimit -a -H core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) unlimited pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 65532 cpu time (seconds, -t) unlimited max user processes (-u) 2128 virtual memory (kbytes, -v) unlimited Please provide the guidance, we are blocked here. Any help is appreciated! Thanks Ajay

…

On Wed, Jul 15, 2020 at 4:00 AM AJay GUpta ***@***.***> wrote: Hi Ronald One more thing to add is our pact-stub-server is loaded with pact files of around 10 MB, it’s basically around 1000 interactions. But we are also running on the same machine where client app is running, so no network involves. Thanks AjayGupta On Tue, Jul 14, 2020 at 7:11 PM AJay GUpta ***@***.***> wrote: > Hi Ronald > > I am still seeing the pact stubby server is not responding well as it was > single client/app simulator. Definitely there is improvement, but still lot > of time out comes. > > Pact stubby server responds with more errors if I increase more than 2 > clients. > > Is there any way there could be more optimization on the server to handle > multiple requests better or if require, Can we allocate more resources to > this process on Mac machine? > > Appreciate your insight > > > Thanks > AJayGupta > > On Mon, Jul 13, 2020 at 6:30 PM Ronald Holshausen < > ***@***.***> wrote: > >> 0.4.2 has been released with the fix >> >> — >> You are receiving this because you authored the thread. >> Reply to this email directly, view it on GitHub >> <#36 (comment)>, >> or unsubscribe >> <https://github.com/notifications/unsubscribe-auth/AQH3T67QU7SSPJHGCDVI6N3R3OYJ3ANCNFSM4OXKTV7A> >> . >> >

[uglyog]

Unfortunately, I only get to work on this project in my spare time.

Can you explain what you are trying to achieve. The stub server was written to support prototyping new consumers, and not for performance testing. How many requests per second are you making to it, and what is the CPU and memory load of the machine while you are running the test?

[uglyog]

If you have a pact file with 1000 interactions, the stub server will have to match every incoming request against 1000 possible ones. That is a lot of work it has to do.

[uglyog]

dial tcp 0.0.0.0:0->127.0.0.1:9992: socket: too many open files probably means your terminal is not handling the amount of console output. You need to turn off the logging with the stub server if you want to run this type of load.

[uglyog]

I've done some performance enhancements. HTTP keep alive was disabled, so that might have been causing the timeouts. Try version 0.4.3 and see how it goes.

[Ajay2020813]

Thanks Ronald! I will try and let you know!

…

On Sat, Jul 25, 2020 at 7:53 PM Ronald Holshausen ***@***.***> wrote: I've done some performance enhancements. HTTP keep alive was disabled, so that might have been causing the timeouts. Try version 0.4.3 and see how it goes. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#36 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQH3T67H2DGH3GYSD76W4XDR5OLDDANCNFSM4OXKTV7A> .