Benchmark # of CPU cores for default config

[kentr]

Didn't find this in the issue queue...

There are reports that VMs with multiple cores are slower than VMs with 1 core, even with KVM:

• Allocating more than 1 core makes VM slower when on VirtualBox rdsubhas/vagrant-faster#5 (comment)
• https://ruin.io/2014/benchmarking-virtualbox-multiple-core-performance/
• http://www.mihaimatei.com/virtualbox-performance-issues-multiple-cpu-cores/

Have you tested this? Perhaps it's worthwhile to set vagrant_cpus: 1 in the default config.

[thom8]

This is something that should be optimized for the host machine and in my experience I've found the best performance is when the VM cpu's are the same as the host.

From Seth a maintainer of Vagrant -- https://sethvargo.com/faster-rails-development-with-vagrant/

see "Use all CPUs and half your RAM"

[kentr]

Yeah, but two of his references regarding the CPUs now show different results.

How to make Vagrant performance not suck has

Update: I used to recommend upping the number of CPU cores used by Vagrant, but it has been shown several times that adding more virtual cpu cores to a Virtualbox VM actually decreases performance. Bummer.

The vagrant-faster plugin (which automatically ups the VM CPUs) has a issue on it: rdsubhas/vagrant-faster#5.

The problem appears to be Virtualbox itself.

FWIW, on a 4-core Mac / Virtualbox 5.0.26 I benchmarked a client D7 project with 1 and 2 CPUs allocated, and 1 CPU consistently performed better for simple serial page request (like a developer would be performing for typical theme / module dev).

[geerlingguy]

I typically use VMware Fusion, which handles things a bit differently, and definitely feels faster with 2 CPUs than 1... but it would be good to have more hard data on whether 1 or 2 would be the best default. 2 seems at least safe since like 99.9% of all dev laptops have a 2+ core CPU nowadays.

[kentr]

@geerlingguy

2 seems at least safe since like 99.9% of all dev laptops have a 2+ core CPU nowadays.

Yeah, seems safe. Though, I gather from the reports that the primary factor is using the Virtualbox provider, not the number of host cores. Is it possible to set the default value based on the provider (like, if provider == virtualbox, vagrant_cpus = 1)?

Here are my simple benchmark results on a 2.3 GHz 4-core MacBook Pro:

ab -c1 -n100 http://example.com/
Ran 4 times, discarded 1st run

Avg requests / sec -- higher is better
1 CPU   3.49
2 CPU   2.87

Manual page reloads, devel page timer
Ran 10 times, caches warmed

Avg milliseconds / page load -- lower is better
1 CPU   406.77
2 CPU   420.88

Median milliseconds / page load -- lower is better  
1 CPU   399.16
2 CPU   425.79

wrk -t1 -c1 -d30s http://example.com/
Ran 4 times, discarded 1st run

Avg requests / sec -- higher is better
1 CPU   3.71
2 CPU   3.34

[geerlingguy]

Interesting results! I'll have to run them locally on VirtualBox to confirm/reproduce, and if it's the same, I'll definitely bump back to 1 vcpu and stick a note in the config that if using VirtualBox, consider leaving it unless you have a specific need for multi-core emulation (which could be slower).

[geerlingguy]

Testing on my i7 2013 MacBook Air (average of 3 runs, after discarding one):

ab -c 1 -n1000 http://drupalvm.dev/

VirtualBox:

• 1 CPU: 151 req/s
• 2 CPU: 139 req/s

Delta: ~8.2% performance increase when using 1 CPU on VirtualBox.

VMware:

• 1 CPU: 169 req/s
• 2 CPU: 171 req/s

Delta: ~1.1% performance increase when using 1 CPU on VMware.

So it seems 1 CPU is a sane default—no real regression on VMware, and makes it faster on VirtualBox. Might as well switch it!

[geerlingguy]

@kentr - Thanks again for all the benchmarking! I've attributed the commit to you :)

[kentr]

Awesome. I'm surprised it's also slightly faster on VMware.

[kentr]

Oh, I misread the VMware results. Now I see 2 CPUs is slightly faster.

[geerlingguy]

@kentr - Yeah, just barely... not worth defaulting to 2 IMO.