CPU programming gives good results, especially when it requires a "fast response". But testing for programming, the GPU's vector calculus reports that significant improvements in the overall process time. No less important is the fact that you can do a lot of calculations spending a much lower power level.
It really depends on the application. While several tasks are easily parallelizable, some others might be more suited for sequential code (such as control structures with branching).
GPU programming might take more development time, but the performance benefits for certain tasks are really significant.
The "real benefit" of moving "everything possible" to GPU is close to none. In order to get benefit out of accelerators such as GPUs one has to find computationally expensive (= calculation dominated) parts of the programm which can run independently and seperate them into so called kernels. These kernels are then executed by the GPU.
This process is not always possible and will in no programm be "everything possible" ;) Some programmes have very little computation and a lot of copying memory around. These applications are bandwidth limited (= data dominated) and will not perform well on external accelerators.
So like in any development process: Find out what the properties of your problem are and find a better way to do it.
And finally: Don't use GPUs just because everybody seems to do that nowadays
And really finally: If you want to use GPUs, do so using OpenACC instead of CUDA!
Actually the GPU can outperform the GPU in memory bound algorithms if the data transfer can be done in parallel with the computations. See for example the computation of a histagram or a median filter. The computations are not dominant at all in a CPU implementation (it is the access to the memory that uses the most of the time in the serial computation) and it is very possible to be accelerated using a GPU.
The question is if your algorithm is parallelizable rather than if it is "computationally intensive". If so, most probably will be benefited from a GPU implementation.
About the interface to use: There are pros and cons in CUDA/OpenACC/OpenCL.
It really depends on the specific use that you are going to give to the implementation.
Overlapping computation and communication is an important point, agreed.
Choice of algorithm is important, agreed.
Choice on interface:
I always felt that CUDA is rather a library for internal use by nvidia rather thatn for the public. Of course, if you want and are able to optimize your code to the extreme, than CUDA is how you can do that. OpenACC and OpenCL are much more accessible. It's like choosing python over C.
Your last sentence: agreed. That was my point: Think about what you need and then make wise decisions.
Another different factor is how much effort are you willing to spend rewriting code? My suggestion is to take the directive based approach like OpenACC or OpenMP. Using this approach modification of the code more limited and the code can be used for conventional CPU:s as well. Computational paradigms come and goes, but comptuational codes usually have a longer timespan.
The real benefit of using GPUs are two:
(1) They provide up to an order of magnitude of speedup in FLOPS. The amount of floating point units in the chip is much higher than in the CPU.
(2) GPUs provide memory bandwidth that is potencially 7X~8X faster than CPU. However, this can only occur when the problem domain is perfectly mapped to the organization of the execution threads so that memory access can be optimal.
If your problem is "data-parallel" and large enough, then it is very probable that it can be solved much faster on the GPU.
On the other hand, if your problem is task-parallel, requires little numerical computation and has lots irregular and unpredictable memory access patterns, then the CPU implementation will probably be faster than the GPU.
It is not so clear where is the line that separates these two types of problems. Sometimes, you can make a perfect data-parallel mapping for a problem it was not meant to be for GPUs, this last aspect is research-level.
In the end, the CPU is very flexible (not so powerful) while the GPU is very powerful (not so flexible).
Please refer to my paper which exactly discusses about your question. From this paper, you may also find many other papers which further discuss about your question.
Article A survey of CPU-GPU heterogeneous computing techniques
The benefits you get depends on your application. GPU ussually has a higher mflops rate but moving data from RAM/CPU to GPU memory can be a bottleneck. You should carefully evaluate the memory transfer delay and the time taken by computation on CPU and GPU.
regards,
Bogdan
- Badges
- Science topic
- Similar topics
- Correction