I teach the following subjects: 1) the history and architecture of computer systems. 2) Methods for creating multi-threaded applications. 3) Events and semaphores. 4) Classical problems in concurrent programming. 5) parallelization of recursive routines. 6) Pipes and pipeline parallelism.
Is this enough?
Does the course, in history and architecture (#1), cover MPP (Massively Parallel Processor) type architectures? which leads me into #2 and whether or not a discussion of MPI libraries, OpenMP compiler directives, and POSIX pthreads is done. Depending on the level of the course, deeper understandings of each of the three approaches becomes critical in thinking of large parallel applications.
Dear Ahmet Husainov I agree to Jerry it all depends the level of course. The the contents suggested by Kyoungson Jhang seem suitable for postgraduate research students. An Introduction to Parallel Programming by Peter Pacheco at http://www.cs.usfca.edu/~peter/ipp/
may be excellent resource in this regard.
Based on our experience with teaching parallel programming, I (along with a couple of colleagues) wrote a book on "introduction to parallel programming" which reflects my philosophy. It is a "prentice hall India" publication. It takes the view that 1) a good feel for various aspects is needed including architecture, dependency analysis, algorithms, languages, debugging, and so on. and 2) in depth practical exposure must be provided to write effective parallel programs in shared memory and distributed memory model. So, we devoted one chapter each to the various topics that belong to item 1. And a lot more coverage to process and thread model of shared memory programming, as well as PVM based distributed memory models.
In your list, 2 and 3 go together and in a related way. Some mutex and synch mechanisms are needed for shared memory programming (incl threads). Adding some framework for distributed memory programming provides a nice contrast and fills the big void in the coverage.
Some introduction to write kernel and asynchronous transfer between CPU and GPU or some examples in (CUDA , openCl).
I would also suggest memory consistency and coherency protocols, and the performance/cost trade-offs of using different kinds of memory synchronization primitives.
It obviously depends on what type of public you have in your course, and what is the goal of the course. Teaching OpenMP can be a very good choice if the course is not for computer science students. OpenMP does not need much knowledge of synchronization (as it is mainly implicit). However, OpenMP has bad performance in most cases compared to Pthreads programming. It is mainly because OpenMP adds unnecessary synchronizations (mutual exclusions) as in most cases it is not able to recognize the context. Thus, it uses a pesimistic synchronization policy.
I you want some computer science students to be able to make parallel programming with good performance, you should go for Pthread and MPI. For Pthreads I think having a good knowlegde of synchronization techniques is required. So, you will have to teach some classical synchronization problems in order to give them a good knowledge about synchronization, and especially mutual exclusion as well as the producers/consummers and readers/writer models.
One of the primary advantages of parallel programming is gain in performance. So in addition to the six broad topics mentioned, please consider adding performance analysis which includes topics like speedup, efficiency, scaling, Amdahl's law, Gustafson's law, time and space complexities, performance analysis & optimization of different algorithms.
As already pointed out, the major issue here is to define what is the intended public. I teach parallel programming for computer science undergrad and grad students. At undergrad level I'm more concerned with programming techniques using threads (shared memory) and MPI (distributed memory). In that course I have a fast introduction to HPC history, TOP500 data, and architectural evolution. After that I introduce some of the concurrency problems and solutions, parallel applications and some performance issues (speedup, Amdahl, ans so on).
For the graduate level course and separate the contents in two modules: one concerned with hardware (cpu, memory, networks) and one with software (parallel models, performance analysis, optimization techniques, programming). Surely there is some intersection between those courses, but this is not a problem....
- Badges
- Science method
- Similar topics
- Teaching