I am using this code snippet. But I am not sure this is the correct way. Is there any other methods or API to find the execution time of a MapReduce Job?
Date date; long start, end; // for recording starting and end time of job
date = new Date(); start = date.getTime(); // starting timer
job.waitForCompletion(true)
date = new Date(); endTime = date.getTime(); //end timer
log.info("Total Time (in milliseconds) = "+ (endTime-startTime));
log.info("Total Time (in seconds) = "+ (endTime-startTime)*0.001F);
Thanks V. Reddy. I have cheched that and also from web uri but there are differences between them. And also date.getTime does not give consistent run time for each run of the same job. I need complete execution time of a job including map, reduce, shuffle, sort etc. Can you please explain more.
I don't think that you can expect getting the same runtime constantly. As long as the standard deviation is not too big, I think you can claim that your data is credible!
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering