The test is reliable and valid. The second tells you that there is heterogeneity between the preferences of the population on the indicators measured. Which gives more merit to the test, is able to maintain high levels of validity and reliability even in a context of high heterogeneity. What is your opinion?
what is your evidence for reliability and validity? If the evidence is strong that should be enough to establish the merit of the scale.
Dear Loretta Secco:
On a proportional stratified random sample of school groups at a Mexican university (99 groups classes, 1627 students, 99 teachers), was obtained: Cronbach's alpha .971, agreed both the theoretical range of the scale as the empirical: 85 points . The main compenentes method yielded a single factor structure explained 68,376% of the variance.
The scale for one to one applied to teachers, obtained with the previous high significant correlation, and also good metric qualities. But none have a normal distribution.
¿What is your opinion?
the Cronbach coefficient tells you that there were a lot of questions asking about similar topics/concepts. The factor analysis told you there is construct validity.
Did you complete reliability analysis for the items to see if there were many items measuring the same question? Sometimes this provides a falsely high alpha coefficient (asking the same questions gives high inter-item correlations). An inter-item analysis with deletion of items that measure the same thing (>.8) might help determine the real internal consistency.
Also, did you revise the scale based on factor analysis and load strength? If items load poorly (.2 or less) you can delete/revise the questionnaire/measure.
I guess I am just trying to suggest there are further ways you can establish reliability and validity.
I am not sure if this helps at all.
Loretta
Dear Loretta Soto:
The factor loadings were between 0.779 and 0.882. In effect, we analyzed the absence of redundancies among items, covering concepts as diverse as almost spiritual aspects of the personality of the teacher related to your teaching vocation, through attitudinal issues and social skills to very specific issues of teaching methodology and evaluative. At 17 items.
In addition, student' responses allow a validity criterion for the responses of teachers, and the students' validity reciprocally.
The question is, does your mismatch to a graded response model of item response theory, speaks of defects on the scale or a characteristic of the population?
Reynaldo
I am wondering if the the high factor loadings (.779 - .882) for all 17 items suggests that there are repeated, or very similar, items. Items that are very similar will load strongly (high factor loads) and raise the internal consistency (high Cronbach alpha) because similar questions are being asked.
When the explained variance is 68%, I would expect some of the factor loads to be lower.. but I am not too sure.
I also wondered if all the items have the same Likert response categories.
I also wondered if you could run the graded response model analysis (which I really know nothing about) separately for each participant type? that way you might know whether the lack of model fit is due to the heterogeneity in the population and not the scale.
You are working beyond my knowledge as I have only used scaling/reliabilty and factor analysis to refine scales. I have not done graded response modelling -- sorry.
Loretta
What makes you say that there is poor adjustment to the GRM? Categorical confirmatory/restricted factor analysis is simply a reparameterization of item response models (including the GRM). If the factor analysis fit, so should the GRM.
A separate issue may be how the scale was administered. It appears that you have a hierarchical/nested data structure. The teachers should have the same latent trait score regardless of who rated them - thus there is a latent dependency in the data that is unmodeled by traditional factor and GRM analyses. Likewise if the same student rated more than 1 teacher, there is unmodeled latent dependency. To further complicate matters, students are also in classes. Two students in the same class should have more similar scores than students in different classes. A hierarchical model is necessary.
Dear Aaron Kaat:
Each student evaluated the teaching of only one professor. But, each teacher was evaluated for many student of one scholar class.
The problem with the graded response model was that almost all items presented a "infit" and "outfit" downer to -2.0 or upper to 2.0
What is your opinion?
Reynaldo
Dear Reynaldo?
Can the problems came because the extreme alternatives are not chose for enough people? In that case yopu do not have enough information to obtain consistent estimations of the parameters in those alternatives.
I do not, it is just an idea
Dear Luis:
En efecto, in student's evaluation teaching, is really difficult to write one item (a positive characteristic of one good teaching) that in student's perspective is not present in the teaching of teacher evaluated. That's is, it is dificult to write one item with a low affinity.
Dear Luis:
Of course, in student's evaluation teaching, is really difficult to write one item (a positive characteristic of one good teaching) that in student's perspective is not present in the teaching of teacher evaluated. That's is, it is dificult to write one item with a low affinity.
- Badges
- Science topic
- Similar topics
- Psychology
- Psychometrics