Hi! There is an elegant analysis in the book by Bertsekas and Tsitsiklis, "Parallel and Distributed Computation:Numerical Methods" regarding how asynchronous parallel computations affect the convergence speed. The book can be downloaded at
https://dspace.mit.edu/handle/1721.1/3719
I refer especially to Section 3.3 on parallell implementations of partially asynchronous computations. It shows how the maximal step length in the computations is proportional (or inversely proportional) to some fundamental measures such as how many components the decision vector has, how many processors are working, how fast communication is, and so on.