Hardware Recommendation and Choosing CPU Cluster or GPUs

[escorciav]

Hi all,

I'm new in this kind of deep learning algorithms and I would like to know other hardware specs beyond a powerful GPU? Specifically a powerful machine for training big models like the imagenet challenge and many others.
minimum RAM ?
minimum number of cores ?
minimum hdd ?

Thank you for sharing this valuable piece of code and create a active community!!!

[Yangqing]

Hardware requirements really depend on many factors - if you just want to do a hobby project, a machine with 4-8G memory and a few cores would be sufficient. Any hard disk would do, and given the current low price you probably want to buy a big disk anyway.

My imagenet model is trained with an i5 4570 CPU and 4G memory, if that helps.

[Cheng-Wang]

Dear all,

I am going to conduct some experiments based on caffe for training up to 5 million images. I have a chance to apply access permission of hardware resources from our Lab. I have two options:

(1) A 1000 core compute cluster comprising 25 nodes of 40 cores and 1 TB RAM each
(2) HP Server with 2 TB RAM and 64 cores and NVIDIA Tesla K20X

I can choose one of them, can you give me some suggestions regarding this problem from perspective of computing capability as well as the amount of configuration work? If I choose the first solution, how can I distribute my training work to different nodes ?

Thank you in advance !

[sguada]

Currently Caffe is not distributed, so getting to use multiple nodes will require some non-trivial extensions.
It will depend if you plan to train many different models in parallel or just one. For training multiple models you could use CPUs, but not sure how slow that would be. For training just one model GPU should be faster.

[Cheng-Wang]

Thank you for your suggestions @sguada , It's very helpful!