Hello, everyone
I have two NVIDIA VGA Cards, whose model names are identical.
When coding CUDA C++ for CNN Deep Learning,
if batchsize = 4, I can do parallel processing
so that each process runs two batches
using cudaSetDevice
But, for fully connected layers,
some codes need to do intra batch summation
In this case, how should I code to do parallel processing?
Thank you in advance and have a nice day
Hello,
My first question is: What motivated your choice to code your own CNN directly in C++ with CUDA?
If it's an exercise in programming to increase your skills in C++/CUDA, that is a good reason. Otherwise, if your main objective is the research and results you will get once you have trained your CNN, you would be FAR more productive and have much less bugs by using a higher level framework/library like Keras, Pytorch or Tensorflow. Your research will also be much easier for others to reproduce and contribute. These software all take care of GPU computations and parallel training in a much better way than any single person ever could... and you would be wise to use them.
Please see this attachment:
https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html
visit this link
https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html
Thank all of you and have a nice day
I will refer to the pages you recommended
To Mr. Bruno Martin
I am studying and developing my own deep learning engine
including backpropagation for pure research purpose.
At this moment, I don't consider fast performance or parallel training.
I just want to make my engine work properly.
Thank you and have a nice day