It is clear that we need GPU ressources when training a Deep Learning. However, once the model is trained and the weights generated and exported, do we really need GPU capabilities to deploy the same trained model?
For example I have a model for predicting objects in an image. I finished the training task, loaded the weights and will do the prediction, do we need GPU for that or only CPU would be sufficient?
It's not always necessary even to train the neural network, the CPU can do this but it will be really slow. Moreover, some neural networks are not written as inherently parallel code (Instruction level parallelism approach) they always need a GPU to train and test. However, some very commonly used CNNs for classification, regression, detection, segmentation, tracking, etc. are quite old and can work on CPU.
As per my knowledge, the only thing which is a bottleneck in the CPU approach is memory i.e capacity and speed, same is true for GPUs.
As Khurram said, not necessary. Slow but doable on a single core. If a single core is too slow, try parallel programming (same physical memory) via a CPU's multiple cores and thread-spawning. Additionally, try multiple CPUs and their multiple cores within a Beowulf Cluster (https://en.wikipedia.org/wiki/Beowulf_cluster), a network of computers configured for distributed processing (different physical memories). Such a cluster network can be readily built from heterogeneous computers that happen to be laying around or have been tossed into a recycle pile.
Dear Peter Raeth and Khurram Hameed thank you very much for your answers.
Yes, I also went through some articles on the topic and I also tried some codeon Google Colab while deactivating the GPU execution mode, the weights were correctly uploaded by the predictor which means that no need for the GPU although the wrights file has a size of 1GB, but it works.
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training