test performance (speed + latency) of new services

[pbiggar]

Add simple benchmarks so that we can test out the new service:

• fizzbuzz, httpclient to itself, json, DB, math
• check that simple performance is comparable for each
• validate outputs and traces in each case
• do a load test for each
• when done, move these to an ops-performance canvas

[pbiggar]

Checklists:

• add benchmarks to the trydarkfsharp canvas
  • fizzbuzz
  • httpclient
  • json
  • DB
  • math
• do a simple performance test
  • fizzbuzz
  • httpclient
  • json
  • DB
  • math
• validate output and tests
  • fizzbuzz
  • httpclient
  • json
  • DB
  • math
• load test
  • fizzbuzz
  • httpclient
  • json
  • DB
  • math

[pbiggar]

I'm posting results with the goal to ensure that we can hit our current load and that there's no operational issues. Note that OCaml runs with 36 small but single-threaded machines and F# runs with 3 bigger multi-core machines that have not been tuned. My thinking is that the OCaml version is constrained by our single-threaded implementation while the F# version is constrained by networking somewhere along the way. None of the servers are really breaking a sweat here, and I haven't looked into the bottlenecks.

Fizzbuzz results:

F#

 $ wrk -t 50 -c 300 -d 5s https://trydarkfsharp.builtwithdark.com/fizzbuzz
Running 5s test @ https://trydarkfsharp.builtwithdark.com/fizzbuzz
  50 threads and 300 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.01s   384.82ms   2.00s    73.93%
    Req/Sec     8.41      6.00    40.00     74.02%
  1210 requests in 5.03s, 1.54MB read
  Socket errors: connect 0, read 0, write 0, timeout 17
Requests/sec:    240.75
Transfer/sec:    314.57KB

OCaml

 $ wrk -t 50 -c 300 -d 5s https://paul-darkocaml.builtwithdark.com/fizzbuzz
Running 5s test @ https://paul-darkocaml.builtwithdark.com/fizzbuzz
  50 threads and 300 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   399.45ms  229.06ms   1.91s    67.76%
    Req/Sec    14.11      8.26    50.00     75.81%
  3092 requests in 5.02s, 3.36MB read
  Socket errors: connect 0, read 0, write 0, timeout 19
Requests/sec:    615.87
Transfer/sec:    685.33KB

Verdict: perfectly capable of handling production loads.

[pbiggar]

After removing nginx from the mix, things look a bit better for F#:

 $ wrk -t 50 -c 300 -d 5s https://trydarkfsharp.builtwithdark.com/fizzbuzz
Running 5s test @ https://trydarkfsharp.builtwithdark.com/fizzbuzz
  50 threads and 300 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   702.72ms  464.06ms   1.81s    71.18%
    Req/Sec     9.06      6.10    40.00     79.90%
  1756 requests in 5.02s, 2.24MB read
Requests/sec:    349.75
Transfer/sec:    457.00KB

[StachuDotNet]

45% improvement in requests/s, awesome!

[pbiggar]

I think the number are quite bad honestly, but I think they're well within range of what we can ship with.

[StachuDotNet]

I'll take the delta positive

[pbiggar]

Things seem fine here and not planning on doing more, so closing.