I have set of questions of key performance indicators (KPIs) for academic performance of different colleges and program of my university. All the responses on each item are in number (File attached). My organization wants me to measures construct validity of the KPIs. I thought of factor analysis but ended up being perplexed and not sure if factor analysis will truly answer my questions. I did literature search and found most of the literature reported only content validity. Any help in this regard? How can I measure VALIDITY of the KPIs?
Review of the content of the questions by expert judges, is the validity of content, as part of the process of instrumental standardization also have to perform the internal validity through factor analysis and reliability of cronbach alpha.
Hello Mehwish,
Metrics for academic institutions abound; welcome the "new normal." Sometimes, they are chosen apparently in an attempt to compare programs / academic units. Other times, it may well appear as if they are implemented to punish specific units (based on "objective" data, of course!).
As Cristian's reply suggests, no set of metrics is likely to be successful if domain experts are not willing to endorse them as suitable for use as indicators of success, effectiveness, monetary viability, or any other chosen attribute of interest. So, content and so-called face validity evidence are likely a basic necessity here.
However...
The first question regarding mustering of evidence for validity of any measure is, how are the scores/results going to be used? Consider these possible answers:
1. Program improvement: I'd want empirical evidence that these characteristics are solidly associated with and distinguish programs that external reviewers would characterize as effective from programs that would be judged ineffective. That suggests a need for criterion-related validity evidence.
2. To compare programs: Here, the implication is similar, that these are the characteristics which reliably distinguish programs that are judged good vs. those judged poor based on some external criterion (which could be external reviewers' judgments). Again, criterion-related evidence.
3. To satisfy accreditation requirements. No need to investigate anything; if the accreditation in question is considered valuable, then one collects and reports the data. The key to changing the metrics is to work collaboratively with representatives of other institutions through the accrediting body.
4. To punish or terminate "under-performing" programs. In this case, the choice of metrics usually implies the features that are valued by whatever group established the metrics. Unfortunately, that often leads to program faculty and administration trying to figure out how to "game" the system so as to look good (or less miserable).
Good luck with your work; I have the feeling you'll need it!
https://essay.utwente.nl/60180/1/MA_thesis_X_Wang.pdf
Refer to this linke you will find research about: Performance measurement in universities --Managerial Perspective--
- Badges
- Science topic
- Similar topics
- Higher Education
- Universities