What Will Go to be Your Choice of parallel programming ..?

As per My verdict you can combine the powers of MATLAB and CUDA by

CUDAMAT or MATCUDA

CUDA

• Is marketed better.

• has developer-support in one package.

• Has more built-in functions and features.

• Only works on GPUs of NVIDIA.

OpenCL

• Has support for more types of processor architectures.

• Is a completely open standard.

• Only AMD’s and NVIDIA’s OpenCL-drivers are mature – Intel and IBM expected soon to mature their drivers.

• Is supplied by many vendors, not provided as one packet or centrally orchestrated. I hope you found that OpenCL is not a cute alternative to CUDA, but an equal standard which offers more potential. I also pointed out that OpenCL has to do some catch-up, but that will all happen this year.

Let's start with an official MATLAB documentation:

http://www.mathworks.com/help/pdf_doc/distcomp/distcomp.pdf

There is a lot of examples.

I have some experience with Matlab HPC (both parallel, distributive and GPU computation). Thus, I cannot judge if Matlab is better than Mathematica.

The first step is to look at how scalable is your application. For example, the very complex operations (usually matrix ones like eig, svd, inv, ...) can be easily implemented on CPU cluster (via Distributive computing server, DCS or/and Parallel computing toolbox, PCT). The number of continual threads is between two and some tens/hundreds.

On the other hand, if your applications have some separable parts (so-called kernels) that are computed many times use GPU card(s). The number of continual threads is between some hundreds and some thousands.

We use the CUDA cards by nVidia as a HW and the former Accelereyes' product Jacket as a SW. It is easy to implement, easy to understand and very effective. However, Accelereyes introduced new product, ArrayFire, which presently supports only C/C++ and Fortran. Matlab should hopefully be supported late in the 2013. Note, the PCT supports GPU as well.

I just finished watching this highly informative Webinar by Mathworks:

Speeding Up MATLAB Applications

https://www.mathworks.com/company/events/webinars/wbnr49643.html?s_iid=desc_rw_DW_cta2

It has good introduction on parallel and distributed computing on CPUs and GPUs with example code.