How to cope with available computing power as the machine size increases?
Amdahl's law gives you an upper bound about how mush an program can be parallelized. In other words, even if you have an infinite number of processors, your speedup is still limited by the sequential parts of your program. For instance, if 20% of your program is sequential, the speedup is limited to 1 / f = 5, no matter how many processors are available.
At a given point, increasing the number of processors only slightly decrease processing time, while still increasing the needed power. Therefore, there is a trade-off between execution time and needed power.
If you can estimate what is the sequential fraction of your program, then you can use Amdhal's law to estimate at which point increasing the number of processors only have a little impact on processing time.
I recommend you the book:
Kai Hwang. “Advanced Computer Architecture. Parallelism, Scalability, Programmability”, McGraw-Hill, Inc, 1993.
In Chapter 3 (Principles of Scalable Performance) of this book, you can find a good presentation of three speedup models: Amdahl’s law, Gustafson’s scaled speedup, and the memory-bounded by Sun and Ni.
Amdahl's law was seen as a pessimistic outlook by the parallel processing community. There is also a law called Gustafson's law which you may want to read about.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering