I often use Mathematica software in my scientific work. Sometimes computations are very complex and spend a lot of time on standard PC (4-6 physical cores or 8-12 logical cores). I try to use parallel calculations but they accelerate a few times. Generally I solve numerically single or double integrals with nonlinear equations which can also contain integrals. One mentioned operation is calculated in non parallel mode but I need series of results which I can calculate parallel (for example ParalellTable[]).
I have question with CUDA in Mathematica. Theoretically very efficient graphic card should give significant acceleration. After reading Mathematica manual we can state that it is easy to calculate matrix operations with CUDA but integrals and nonlinear equations are rather difficult to obtain. I have question if somebody has good experience in solving said issues using GPU computing and could give me clear tips, literature, etc?
Dear Radoslaw,
i know that beginning with Mathematica 8 there is this possibility on using CUDA programming. But I've not experience. You can see the link below that contains some tutorials
http://reference.wolfram.com/language/CUDALink/tutorial/Overview.html
In the other hand you can try to post the same question on the stackexchange site in the section of Mathematica. There are a lot of experts and I found a lot of answers to my questions.
http://mathematica.stackexchange.com
Hope it is useful
mariano
Cuda processors are SIMD (single instruction multiple data) architectures. Cuda processors are extremely efficient at computation on data that is not dependent on one another. Depending on the algorithm several dependencies may arise that give way to a break in parallelism.This break in parallelism usually takes place through conditionals.
You would need to be specific on the algorithm to get a more specific answer, since integration could benefit from reduction operations in CUDA, but again it depends on the algorithm.
Functions like NIntegrate and FindRoot will not benefit from GPU; you'd have to code your own quadrature or rootfinder to go that route. ParallelTable can be used to compute several numeric integrals, say, but that would use CPU cores.
Dear Daniel,
I agree with you that I would have to code my numerical operations (quadrature or rootfinder). I think that I would spend more time writing code for CUDA than simply using parallel computing with CPU cores. But I have question for you: now we have Mathematica version 10.x but new (namely 11.x) will be more user-friendly for numerical computing with CUDA? How do you think?
Dear Mariano,
Thanks for interesting links. I have heard about stackexchange but do you know other valuable portals connected with Mathematica especially with parallel computing (or GPU computing). I am also interested in tips how to solve numerically standard problems (or write codes in Mathematica) which would be more efficient then typical solution.
I share the opinion that you are likely better off using coarse-grained parallelism on CPUs for this type of problem. I do not know whether there are plans for enhancing GPU possibilities in Mathematica. I'll send a link to this discussion to one or two people who might know but I'm not optimistic. As for CPU possibilities, depending on the specifics of your root-finding and numeric integrals, you might be able to get speed improvements by tweaking various options in NIntegrate and FindRoot away from their defaults. Method and Accuracy/PrecisionGoal come to mind for a start.
In addition to the StackExchange forum (which I was going to recommend until I saw there was already a response to that effect), there is Wolfram Community at community.wolfram.com. It also has some quite knowledgeable participants. Not surprisingly, there is considerable crossover amongst MSE participants.
I agree with Arturo;
Cuda on GPU is designed for the parallel SIMD algorihms. it is also possible to integrate MIMD parallel structure but it is not guaranteed to reach more interesting results.
All the new versions of the programming platforms integrate the parallel implementation tools. CUDA is one of the emerging tool.
The important question in this area is: How can the programmer manages his algorithms to became parallel running to reach the highest effectivness of the GPU device.