To be clear, you want to predict consistent high performance, across all subjects, not just consistent performance. Correct? Also, can we assume each subject is of comparable difficulty?

Edwin, actually we would like to give "consistency ranking" for each student based on how consistent the student is across all subjects and exams - solely on his own performance. So each student gets his own consistency ranking (not compared against other students / class high or low).

In other words, somebody who always fails no matter which exam or subject, may get 1st rank for being "consistent", though certainly he is not a high performer.

The subjects, they can be considered all as of equal difficult/easy.

The idea is, once we know the consistency rankings for all students, applying these rankings to their past performances should help us predict their performance in the next exam,and there by the top rankers.

Looking for a good method to calculate such consistency based on past.

Define high performance as upper quartile/ decile ranking for all subjects and all exams. Score student performance according to which quartile/decile (1 - 4/ 1 - 10) for each exam, for each subject, the students' exam performance reflected. Then with each students' total set of performance scores, take an average score, like a median score across all scores for an indicator of consistent quartile/decile performance. What do you think?

Wouldn't any measure of spread do? Edwin suggests using quantiles, but you could also just use within-subject variance. What is mind boggling about your problem is that most researchers are just used to see variance as a nuisance parameter, but nothing of interest by itself. (And I heartily disagree and therefore really like your question).

In case you want to do full inferential (not just descriptive) statistics with all bells and whistles, you'll probably run into problems, because the variance parameter doesn't obey the usual assumptions of asymptotic statistics (this is what most parametric tests ground upon). In this case, you'll probably want to resort to bootstrapping or MCMC fitting. Should be fairly easy to build such a model in JAGS or BUGS.

--Martin

Thank you Edwin and Martin - Quartiles and Measure of Spread are useful pointers.