because it also depends on the program you are running on the system. It is difficult to find a program that can be fully parallel. In scientific computing there are some example of program like computing the summation of a set of number. The number can be equally shared on the processors. Thus, each processor will produce a partial sum. However, still at the end only a reduced set of processors will be required to aggregate the partial results. 100% parallelism will also mean all the libraries used in a program are parallel. But as I said, not all tasks can be done in parallel. There are job that can be run only sequentially.

Thanks a lot!

Having worked on IBM BlueGene (Massively Parallel Processor) supercomputer, I can tell you that even if you have a parallel algorithm, it might not work beyond a certain level of parallelism.  I ported several programs that never anticipated > 512 possible concurrent cores/processors - and couldn't make use of the 1000s that BlueGene offered.

Guy Martin Tchamgoue is correct in that you cannot achieve 100% parallelism 100% of the time the problem runs.  At some point, the partial evaluations have to be aggregated - meaning that only a handful (usually only one) core/cpu will be doing that aggregation.

If you need further details you can find an overview of the dependencies which can be found in a program in the link below.

https://en.wikipedia.org/wiki/Dependence_analysis

What does 100% parallelism. In all cases you have context switching. Even in multi core. You have a large amount of synchronization - time shared variables and others. You even modeling treatment processes will always have an element of it that is a result of internal interactions, causing overhead. Besides the link that gave you Franchesco Argese, I recommend that you see articles from Rajkumar Buyya or:

https://www.researchgate.net/publication/2950435_Message-Passing_on_PCI-SCI_Network_Interfaces?ev=prf_pub

Article Message-Passing on PCI-SCI Network Interfaces

You cannot achieve full parallelism if there is still some sort of communication involved. But, lets take GPUs which, in some sense, offer 100% parallelism. Lets suppose that every GPU core gets the same number of data items to process. A bunch of cores will have to run in lock-step. This means that 32 or 64 cores (depending on the GPU) will all execute the same instruction at the same time. Even when in an if-then-else statement different data items will take different paths. Every core will execute the same instructions, but some will discard the results. So, in some sense you get 100% load, but you do not get the expected parallel speed-up.

With MPI, however, processors will never run in perfect sync. Also, it is too expensive to guarantee that every core has the same number of data items. Communication will synchronize your software from time to time which leaves some cores in idle states stealing from you 100% parallelism. Throw out the communication, which only works if you make the computations of each core independent, and you get perfect parallelism. Most problems will not allow for this optimization. This is why you cannot get 100% parallelism.

BTW, what makes you say you got an excellent programming model?

Simon, very well, that's a good question. I would answer so your question: Well, it's my programming model and it is excellent.

@ Sir Simon

I was questioned in an interview , where the interviewers were claiming even with an efficient and excellent programming model you can never achieve 100% parallelism, that's why i mentioned it.

I would say you can reach 100% parallelism, but only with not so optimal algorithms. You have parallelism when all cores are doing work all the time. You could achieve that by letting them do some extra work--just let your idle cores compute anything. The only problem with this is that it would slow down your algorithm. If you go for the fastest program, having 100% parallelism just to have 100% parallelism is the wrong route to go.

But, already with single core programming the algorithm with the highest performance is sometimes the algorithm that recomputes some things over and over again, just to reduce the accesses to RAM. Similarly, it could be faster to recompute the same thing on several compute nodes so that you do not have to communicate them. This would actually give you a higher degree of parallelism because sometimes all cores will compute exactly the same thing. The only problem with this approach is that parallel programming is already complicated enough. Taking this also into account complicates things even more.