The recent version of Keras (>2.0) supports running your model in multiple GPUs. It is as simple as this:

from keras.utils import multi_gpu_model

model = multi_gpu_model(model, gpus=2) #in this case the number of GPus is 2

Hope this answers your question. Have a great day

I confirm that you use Keras (>2.0) for exploiting multiple GPUs. However, the three GPUs need to be from the same generation.

In your case, there is no problem for using the two GTX 1080 TI, but for the third one (Tesla K20), I'm not sure. It is a different card.

Please, if you test that, can you inform me if ti worked with the three.

Hope this helped.

Best regards,

Sidi,

Hi,

I have tried Keras with MXNET backend for that. It makes it super easy and it is comparably faster than Tensorflow backend.

The problem is that, as of now, MXNET backend is only available in Keras 1.2.2. They might also develop the backend for Keras 2 in future.

Here is a link that describes how to install and use Keras with MXNET backend for multiple gpus:

https://devblogs.nvidia.com/parallelforall/scaling-keras-training-multiple-gpus/

Here is another link to get a sense of how faster MXNET is compared to Tensorflow:

http://dlbench.comp.hkbu.edu.hk/mg2/?v=v7#tableContainer