I would like to profile a web server running on EC2. However, the results have much more variability than I expected, which I eventually attributed to the dynamic nature of the CPU on the EC2 platform - if no other virtual machines are using my CPU, then my webserver can use these extra cycles. I would like to simulate minimum performance by limiting the total CPU cycles given to this process, but all solutions I have found to limit CPU work relatively (e.g. 10% of the available). Does anyone have advice on this?
Disclaimer: I've asked a similar question at http://unix.stackexchange.com/questions/104807/limit-cpu-cycles-inside-of-xen
Don't forget that you are not just sharing the CPU with another application: you are also sharing the caches (and TLB). If the other application always flushes you out of the cache(s) due to its high working set, then you'll see much worse performance.
Don't forget that you are not just sharing the CPU with another application: you are also sharing the caches (and TLB). If the other application always flushes you out of the cache(s) due to its high working set, then you'll see much worse performance.
James, great point! Can you recommend any publications showing this effect on a public cloud?
Not on a public cloud. I have seen variations of 50% in runtimes on a private cloud with multiple users sharing a node, which i GUESSED (no hard proof) were cache effects, Note that the users had exclusive access to their cores(and thereforeL1caches), so the effect was, I assumed, due to L2/3 caches. Note that an L2 cache hit can be 30 times faster than main memory..
I agree that caches are good secondary suspects - 30x may even be too conservative. In any case, the primary variability study that I've seen is by Schad et al 2010 (see http://dl.acm.org/citation.cfm?id=1920902). They only briefly mention cache, as it appears to be one of the minor performance contenders on the public cloud - a much larger part of your performance is the architecture of the machine you are actually assigned when choosing a specific amount of compute units, and I don't think they have done the analysis of performance effects under the condition of one architecture.
I can't read that (non-embeddded fonts - I've complained to ACM). But the graphs are bizarre. Note that the cache effect is definitely measurable: see Ristenpart,T., Tromer,E., Shacham,H. & Savage,S., Hey, you, get off of my cloud: exploring information leakage in third-party compute clouds.
Proc. 16th ACM conference on Computer and communications security - CCS '09, ACM, New York, 2009, pp. 199-212.
- Badges
- Science method