Dear colleagues,
I have some doubts about the correctness of the following analysis.
A class of 79 students underwent three different teaching methods that chronologically presented themselves in this sequence:
(a)Project based learning 1 (b)Traditional lecturing (c) Project-based learning 2
The first methodology (PBL1) and the last (PBL2) are the same although their implementations have been slightly changed in aspects external to the structure that scientifically makes them define as such. The idea was to verify that there was a compliance of data (performance at the examination and perception of the learning experience) despite the diversity of content involved, variations in the demands on the final product and different time location.
For the collection of data on the perceptive aspect, a 5-point Likert questionnaire was used (not at all, little, enough, very, very much) which investigated the following aspects:
PERSONAL PERCEPTION OF THE LEARNING EXPERIENCE
Items (a5 b5 c5; a8 b8 c8); (a6 b6 c6);(a9 b9 c9); (a12 b12)
PERCEIVED PRESENCE OF SPECIFIC DIDACTIC ELEMENTS DURING THE LESSON
Items (a1 b1 c1) ;(a3 b3 c3) ;(a4 b4 c4) ;(a5bis b5bis c5bis) ;(a6bis b6bis cbis6) ; (a7 b7 c7)
As you can see, for each methodology, students were asked the same types of questions (however, spaced out in the questionnaire). E.g. Sense of usefulness and meaningfulness conveyed to students (a4 b4 c4)
VALIDITY AND RELIABILITY
As far as Validity is concerned, it has been searched with data already acquired through the EXPLORATORY FACTORIAL ANALYSIS as follows:
The items, already selected on a theoretical basis after a first phase of questionnaire design, have been uploaded and refer to the PERSONAL PERCEPTION OF THE LEARNING EXPERIENCE.
It has been searched the number of factors with:
- EIGENVALUES above 1 (not parallel analysis, given the small number of the sample: 79 students).
- Rotation - oblique - oblimin
- Values above 0.3
RESULT: two factors emerge. Factor 1 items (a)+(c); Factor 2 items (b)
The items, already selected on a theoretical basis after a first phase of questionnaire design, have been uploaded and refer to THE PERCEIVED PRESENCE OF SPECIFIC DIDACTIC ELEMENTS during the lesson.
It has been searched for the number of factors with:
- EIGENVALUES above 1 (not parallel analysis, given the small number of the sample: 79 students).
- Rotation - oblique - oblimin
- Values above 0.3
- Items that do not load or load on more than one factor have been excluded
RESULT: two factors emerge. Factor 1 the items (a)+(c); Factor 2 the items (b)
As far as reliability is concerned, CRONBACH'S ALPHA has been used.
In the PERSONAL PERCEPTION OF LEARNING EXPERIENCE and in the PRESENCE PERCEPTION OF DETERMINED DIDACTICAL ELEMENTS DURING THE LESSON the items (a) and (c) were analyzed together as they refer to the same methodology (Project-based learning) while the items (b) on the traditional lesson apart:
a5, a6, a8, a9, a12, c5, c6, c8, c9 Cronbach's alpha 0.780
b5; b6; b8; b9; b12 Cronbach's alpha 0.740
In the PRESENCE PERCEPTION OF DETERMINED ELEMENTS DURING THE LESSON
a1, a3, a4, a5bis, a6bis, a7, c1, c3, c4, c5bis, c6bis, c7 Cronbach's alpha 0.780
b1, b3, b4, b5bis, b6bis, b7 Cronbach's alpha 0.703
I WOULD LIKE TO ASK YOU THE FOLLOWING QUESTIONS:
- Is the validation and reliability procedure correct?
- Was it right to separate for the factorial analysis and Cronbach's alpha DIDACTICAL ELEMENTS from LEARNING EXPERIENCE?
- Would it be more correct to analyze all items (a) together, then all items (b) together and all items (c) together (which I have already tried and they load each time only one factor and always reliable)?
Hello Claudia,
The estimation of score reliability after having "created" subscales via exploratory factor analysis very likely will result in inflated/overfitted values. There's a classic reference that addresses this practice: Cureton, E. E. (1950). Reliability, validity, and baloney. Educational and Psychological Measurement, 10, 94-96. doi: 10.1177/001316445001000107
Good luck with your work.
- Badges
- Science topic
- Similar topics
- Correction