In litterature i find lot of paper that discuss the problematic of parallel computing but i didn't find something relevant concerning how can we define the optimal number of compute nodes that will give us the maximum speedup.
Thanks for your help
#HPC #parallel computing #scientifique_simulation
#graph_partitionning
When you increase the number of nodes, speedup will begin to flatten or may even begin to drop off due to limited parallelism and increased overheads. The point where speedup reaches its maximum value can be treated as an optimal value for the number of nodes, if the objective is to maximize speedup.
Thanks Mr Ravikumar.
I totally understand this but the real question is : is there any equation that can be studied to find the max value of speedup using the caracteristics of the simulation.
Thank so much for your exaustive answer @M.Alvioli
I took a look on your exellent paper and it will help me so much.
Can we model our simulation as a graph and try to find the perfect combination between communication and computing load while doing graph partitionning?
I ll be very gratefull if you could take a look on this paper where we tried to see how the speed up behave on a multiple graph that represents simulations:
https://www.researchgate.net/publication/338363034_Impact_of_communication_volume_on_the_maximum_speedup_in_Parallel_computing_based_on_graph_partitioning
.... and give me your opinion on the relevance of this work it will help me a lot.
Thank you .
In the field of high-order finite element methods, we have a good idea of how to achieve 80% parallel efficiency on HPC systems. The rule of thumb we follow is 2000 points per MPI rank for CPU systems (e.g. Blugene Q, Sequoia, etc.). For GPU systems (e.g. Summit, Sierra, etc.) the number is 2 million points (since latency is very expensive). That is based on many years of development. You can read this paper which has many benchmarks and ideas you can follow to get an answer to your question Article Scalability of high-performance PDE solvers
I recommend that you read article
On topological scalability of computing systems
https://www.researchgate.net/publication/345359545_On_topological_scalability_of_computing_systems
- Badges
- Science topic
- Similar topics
- Psychology
- Mental Processes
- Learning