Dear all,
I wish to have your precious help!
Data was collected from 340 respondents by using a structured questionnaire that has only 4 questions.
My question is:
What are the appropriate tests that measure reliability that can be done in such a case (4 questions)?
or
How can I defend that "not performing a reliability test in my manuscript text"?
N.B. I used a mixed-methods research methodology in three sequential phases (exploratory, descriptive, and explanatory). The mixed-methods research methodology consisted of two parts. 1) A quantitative cross-sectional study using a structured questionnaire, (my question about this part, a structured questionnaire has 4 questions). This was preceded and followed by 2) Two mutually-exclusive qualitative studies (exploratory and explanatory) using semi-structured questionnaires.
Thank you in advance
Yours,
Mohamed Bahlol
Because two items produce only a single correlation, the minimum number of items that you need for a reliability analysis with coefficient alpha is 3, so you are good to go.
Mohammed A Bahlol - You can use Cronbach's alpha approach. In most packages (eg SPSS) you can determine alpha for the full scale (ie all items included) and with items removed. Internal consistency is usually measured with Cronbach's alpha, a statistic calculated from the pairwise correlations between items. Internal consistency ranges between negative infinity and one. Coefficient alpha will be negative whenever there is greater within-subject variability than between-subject variability.
I wondered if there was anything helpful here; this is not a precise and learned response like the above from Dr Manzoor Hussain, but I realise the concern you have with a low number of questions :
http://www.npmj.org/article.asp?issn=1117-1936;year=2015;volume=22;issue=4;spage=195;epage=201;aulast=Bolarinwa
https://research-methodology.net/research-methodology/reliability-validity-and-repeatability/research-reliability/#:~:text=In%20simple%20terms%2C%20research%20reliability,times%20produces%20the%20same%20results.
https://select-statistics.co.uk/blog/assessing-questionnaire-reliability/
https://www.researchgate.net/post/How_can_I_evaluate_the_reliability_of_the_questionnaire
Good luck with this,
Mary
Hi Mohammed,
a defense for why not performing a reliability analysis difficult, -.... it is much easier to say that not every reliability test delivers valid results. You will finde arguments here and the references therein:
Maul, A. (2017). Rethinking traditional methods of survey validation. Measurement: Interdisciplinary Research and Perspectives, 15(2), 51-69.
One Quote:
" It is well established that the outcomes of covariance-based statistical procedures (including, but not limited to, estimates of reliability such as Cronbach’s alpha and latent variable models such as factor models) can be influenced by any causal factor that drives similarity in the manner in which individuals respond to items. This could certainly include psychological processes related to the actual content of the items, but could also include construct-irrelevant influences, such as factors that affect how individuals respond to survey items in general (e.g., response sets such as global (dis)agreeableness, extremity or edge aversion; Messick, 1991) or to a particular item type or format, and other such method effects (see Maul, 2013a). "
The other question, what would be an appropriate test for your design very much depends on your items. How many response levels? Ordinal nominal intervals scale? Are they theoretically derived to measure the same construct?
Hope this helps
Best
René
Mohammed A Bahlol - Since you have collected data from 340 respondents by using a structured questionnaire that has only 4 questions and you want to know the appropriateness of tests that measure reliability that can be done in such a case (4 questions). Number of questions is not the problem as long as the structure of questions in a questionnaire fulfills the objectives of the study. For the appropriateness of tests that measure reliability, you could use "Test - Retest Method". It is usually administered to a sample of subjects at two different occasions. Scores of tools administered at two different occasions is compared and calculated by using the following formulae of correlation coefficient. The correlation coefficient reveals the magnitude and direction of relationships between scores generated by research instruments at two separate occasions.
Interpretations of results - the results of coefficient correlation ranges between -1.00 through 0.00 and + 1.00 and the results are interpreted as follows:
1) + 1.00 score = perfect reliability;
2) + 0.00 score = no reliability;.
In order to measure the test-retest reliability, we have to give the same test to the same test respondents on two separate occasions. We can refer to the first time the test is given as T1 and the second time that the test is given as T2. The scores on the two occasions are then correlated.
Because two items produce only a single correlation, the minimum number of items that you need for a reliability analysis with coefficient alpha is 3, so you are good to go.
Dr David L Morgan's response has simplified it beautifully; so, as he wrote, 'you are good to go'. Good news!
I wanted to get it straight in my own head, so looked up for clarification and found that there were 2 questions in 2017 about this:
https://www.researchgate.net/post/what_is_the_minimum_number_of_items_necessary_to_calculate_Cronbachs_Alpha
https://www.researchgate.net/post/What_is_the_acceptable_range_for_Cronbach_alpha_test_of_reliability
and this in 2014 where Dr Morgan had the highest number of recommendations:
https://www.researchgate.net/post/What_is_the_minimum_number_of_required_items_for_an_attitude_scale
This is a link to an explanation about Cronbach's Alpha:
https://stats.idre.ucla.edu/spss/faq/what-does-cronbachs-alpha-mean/
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Probability
- Probability Theory
- Reliability