I am not too sure, so better wait for a "more" expert to answer, but my guess is that not much since GPUs are generally optimized to do floating-point operations.

If somebody has a good explanation of what is faster and how much, I am also interested.

from the Nvidia doc. (link below, page 5) on the Fermi architecture, it states the cores support ieee 754 (floating point) 64 bit precision as a feature. This, to me implies that Tesla GPUs probably do not support 64 bit floating point operations. Fixed point operations can always be programmatically enforced within a constant factor of processing penalty. In GPU processing in general, floating point operations often incur multi-cycle latency (i think up to 8 cycles). But with GPU core counts vastly exceeding CPUs (some times up to 1000:1), this may not be much of an issue.

http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_Fermi_Compute_Architecture_Whitepaper.pdf

I just do a couple of operations changing the float to int in matrix multiplication code. It shows the same performance in integers as a floating point calculations. Both variables have 4 bytes size. Since double and long both have 8 bytes size I can extrapolate it for them.

Think, that when we doing it on CPU we have some operations in registers like rA, rB, rC, rD, etc. And asm code is parallelized. That means that exponents adds up and mantissas are multiplying like integers. Don't know how it is on GPU, but I think it pretty much the same. So the answer is - yes, it takes more operations, but the resulting time is the same. I have Tesla C1060 videocard.

According to the documentation http://www.nvidia.co.uk/content/tesla/pdf/Tesla-KSeries-Overview-LR.pdf

The tesla K10 is optimised ("faster") for single precision and the tesla K20 is optimised ("faster") for double precision.

However, as to whether your application will be faster depends on the type of algorithm or problem you are solving.

For example, I have an application that uses single precision floating point but it's not faster on a K10 device even though the tesla K10 is optimised for this. This is because the K20 device gives more parrallelism for my algorithm and this outweighs the benefits of the K10 device being suited to single precision.

Tesla architecture absolutely supports 64 bit floating point. 64 bit fixed point is mainly used in CPU's because radix computation requires less hardware and fits the CISC processing structure. Tesla GPU's will computer faster and with better precision than CPU's assuming u purchase a GPGPU and not a gaming GPU. Precision is not the same in gaming graphics cards because it is not necessary to have a perfect value when rendering. The graphics will appear the same either way. For scientific application however a small difference makes all the difference in the world.

Robert, gaming GPUs have exactly the same precision as "non-gaming" GPUs. The difference is in using ECC memory (and more memory), and in the case of NVIDIA, artificially limiting the DP speed on the non-Tesla GPUs.

the hardware in gaming gpu's includes logic to enhance certain graphics calculations that general purpose GPU's do not have. This limits die space for DP units. This logic is slated to maximize gaming performance which mostly does not require DP calculations. General purpose GPU's will greatly out perform graphics cards in DP performance. Just because the internal chip may be built with 64 bit precision doesnt mean the card takes advantage of this.

Have you thought to use 32 bits floating point filtering? If you have not problem with accuracy you could get a good time improvement. Obviously you should save your data as float, then read and save again the results as float. In such case, you will have a higher thoughput in data reading and writing (2 times faster) and, if your algorithm is band limited, you will get at least this improvement. The algorithm speed (except in reding and writing) can improve or not depending on the gpu. Since you will have more shared memory and/or registers available (the data are smaller) it is probable you will be able to increase your parallelism. In case of accuracy problems you can use double only to store intermediate results, always keeping in mind to reduce to a minimum the amount of double precision calculation.

Gaetano, thanks for your suggestion, but indeed I need 64 bits precision. I don't really need floating point because the dynamics of the data is not large. That's why I'm thinking of moving to 64-bits fixed point, but I can't move to 32-bits floating point.

Is your algorithm bandwidth limited or is limited by processing speed? Is your reading of the data coaleshed?

Are you using double precision native operations and compiling for the nvidia card you have? Have you ever programmed using the cuda compiler instead of opencl?

As other researchers commented before, the operation on Nvidia are already optimized floating point to be executed in one clock cycle (or more depending on the operation), and using fixed point floating points will not save your time: on the contrary you could get worst performances.

Before to continue on the improvement of your code you need to answer to the questions about memory access: because optimization depends on this factor.

NVidia GPGPUs are 32-bit integer machines. Pointers can be set to 64 bits (-m64), also 64 bit integer arithmetic is possible (long long), but still integers ALUs are 32 bits at the HW level. Also memory bandwidth will be halved. For Kepler K20 peak performance in floating double precision is 1170 GFlops, with 208 GB/s of theoretical memory bandwidth (without ECC). Such a card uses 13 “SMX” for a total of 2496 “CUDA cores”. Double-precision performance is one third of single-precision performance.

http://en.wikipedia.org/wiki/Comparison_of_Nvidia_graphics_processing_units

All these numbers have a meaning only if the algorithm is not band limited. They are purely theoric. Tesla and Kepler also have dedicated floating point unit (not only integer ALU) that are also efficient in double precision.

Marco, when you write "memory bandwidth will be halved", what do you exactly mean? In my case, if I replace double floating point precision with 64-bits integer I don't see how memory bandwidth can be halved, but maybe I'm missing something.

Mario. I am realising my answer has been misunderstood. Let me retry. You can roughly assimilate them to 32-bit processors. Registers are 32 bits. Integer arithmetic is particularly efficient working on 32-bit words, as ALUs are 32 bit. You can use 64-bit arithmetic both for fixed and floating. On Fermi simple 64-bit ALU operations, such as addition and subtraction cost two operations, 64-bit integer multiplication (or a multiply-accumulate) requires four operations. I did not fully verified K20, but I believe the same holds as it seems they changed the SM micro architecture into SMX, not the core. The memory will be halved in terms of words transferred per sec since you need to work on 2x32-bit words (of course, not in term of GB/s). Bank conflicts are not affected, they can be managed at 64 bit boundary. Of course GFlops are peak numbers, thus upper bounds. In terms of application performance they are little or no predictive figures. This is true for all platforms, memory-wall is not a GPGPU-specific problem, it is "the problem" of parallel computing nowadays. However, due to reduced support for cache coherency, on GPGPU, the possible performance gain from a good memory access pattern (usage of block-shared memory, avoiding bank conflict, coalescing) is definitely larger with respect to multicores.

It is also true that high-end boards (tesla, k20, etc) are typically much more robust on double precision performance w.r.t. gaming devices (eg geforge). Transcendental operations are executed in specialised units (SFU).

http://www.realworldtech.com/fermi/7/

Mario, I am very glad to hear you raising this question. 64-bit precision is being blindly used as a panacea in so many applications, when the input arguments have only three or four decimals of accuracy and the only accuracy needed in the outputs is also three or four decimals. But numerically-ignorant programmers invoke 15 decimals of accuracy throughout a calculation, just to be safe. YES, there is massive savings possible over using 15-decimal "scratch" variables for the entire calculation, which is usually done because no one has the time or expertise to think about rounding error any more. You can increase speed and accuracy at the same time as you decrease memory footprint, pressure on bandwidth, energy usage, and power consumption, just by right-sizing the precision and dynamic range to what you need.

Any possibility of quantifying the error in your output as a function of errors in inputs? It's usually not easy to figure out, but if you can, you may be astonished at the economy of storage that the knowledge presents to you.

Double-precision IEEE floats provide over 600 orders of magnitude; I suspect you need less than 10 orders of magnitude in your calculation. That excess dynamic range consumes an 11-bit exponent when you probably only need about 5 bits of exponent. If you know the largest value and smallest value that your algorithm will ever require, by all means scale the problem to fit that range and used fixed-point math to get more digits of fraction, since you really don't need to spend precious bits on an oversized exponent. Nor do you likely need 52 digits of fraction, as provided by IEEE doubles. There is probably a way to get excellent results, better than what you're getting, using 32-bit fixed point. If you can figure out how, you'll nearly double your speed because almost all current architectures are bandwidth-limited, not FLOPS-limited.

Mario, you will see very little difference between 64bit GPU integers and 64bit GPU floating point. Your code sounds like it is memory bound. If you do two reads, plus some math, plus a write. You are doing 3 I/O ops and some math. The I/O on the GPU (or the CPU) is about 20x slower. So unless you have 60 math ops - the math cost is not the time killer. It takes exactly the same about of time to read 64 bits (of anything). I have a GPU GEMS chapter (on my web site) if you are interested.

Mario, I would like to make a few comments about details that are very important:

1) There are different versions of TESLA, each with different COMPUTE CAPABILITY (CC) versions ... CUDA Compute Capability 1.0 was in the initial introduction of CUDA. We now have CUDA CC 3.5. Each new version is NOT backwards compatible and requires the hardware to support a newer version of PTX ISA ({Parallel Thread Execution). So, why is this important ?

Each new version put more and more emphasis on double precision. CUDA CC 1.0 did not even support double's.

2) Also, GPU's are designed for float, not for integers ! Thinking that, integer-based operands will yield faster execution times is CPU-like thinking, and doesn't work for the GPUs.

3) You can use a CUDA Occupancy Calculator, available in the developer site to see if your program is IO-heavy, or core-heavy. If it is IO-heavy, like was suggested before, then, you might want to focus on memory access patterns.

4) Also, instead of trying to write your own routines, did you try ready-made libraries, such as, CUBLAS, which has a rich set of matrix-matrix, matrix-vector, and vector-vector arithmetic subprograms. There is a subprogram for EVERY type, including Complex ! So, you can use templates to keep changing it, to test your ideas, i.e., which TYPE is better ... Hope these answers helped a bit ... Tolga