Hi Mohamed,

I think there is an easier way to detect affections on reliability and validity. One option could be to check atypical data with a Grubbs' test, like this you will see if you have outliers (so, atypical "response style"). For example, if someone answers 1; 5 (in a Likert scale) to all questions and scores the maximum (or the minimum), you will pop him out with the Grubbs' test. Then, it is generally suggested to delete these outliers but pay attention to this topic : https://www.researchgate.net/post/When_is_it_justifiable_to_exclude_outlier_data_points_from_statistical_analyses

Hope that helps, 

Julian

thank you Julian Nasello

What does a Grubbs' test ?

thank you Julian Nasello

What does a Grubbs' test ?

thank you Julian Nasello

What does a Grubbs' test ?

Hi Mohamed, 

You're welcome.

The Grubbs' test allows you to detect if your data contain extreme scores that affect the general distribution. We call extreme scores as outliers or atypical points/observations, because they are out of the general distribution (of your sample). They affect the distribution, so you can expect they affect also the validity and reliability. However, I am not an expert on that question and I hope that some RG colleagues will confirm or not this information. 

Here is the reference of Grubbs' paper : Grubbs, F.E. (1969). Procedures for Detecting Outlying Observations in Samples. Technometrics. http://www.jstor.org/stable/pdf/1266761.pdf?refreqid=excelsior%3Abeb2cdf05f84a73efbe6f7c5ee09455d

You have also other tests: Chauvenet's criterion, Dixon's Q test, and the Mahalanobis distance and leverage (but I am less familiar with these methods).

I think the Grubbs' test would help you, instead of producing a complex algorithm or equation.

Best regards,

Julian

i think there is not enough context given to satisfactorily answer your question. what exactly do you mean by "response style" (outliers, as suggested by another answer, or something completely different?), and what kind of "statistical equation" do you have in mind? have you got a specific idea in what way "response style" might affect the answers to your specific questionnaire, or did you just read a general warning somewhere about such a possibility and now you are trying to find a workaround in order not to have to face a potential problem?

of course, in general the "response style" must be taken in consideration when designing a questionnaire. for example, you might have to decide whether you prefer analogous scales or likert scales. in the latter case you probably need to decide if an odd or even number of levels makes more sense etc. all those choices will direct the "response style" in a certain direction and thus up to a certain degree you can control or at least influence the "response style". but what the role of some "statistical equation" would be in this i don't know...

or maybe you were asking about finding a "reponse pattern" after the data has been collected. but also in that case there is not enough information available to give you any sound piece of advice. of course there is a wealth of statistical methodology to this purpose but one would need to know about your data type, what kind of patterns you are looking for and what the consequences would be of finding a specific "response style".

What is the difference between EXTREME STUDENTIZED DEVIATE TEST and Grubb Test?

Extreme Studentized deviate (ESD) test is used to detect one or more outliers in a univariate data set that follows an approximately normal distribution.

The primary limitation of the Grubbs test and the Tietjen-Moore test is that the suspected number of outliers, k, must be specified exactly. If k is not specified correctly, this can distort the conclusions of these tests. On the other hand, the generalized ESD test only requires that an upper bound for the suspected number of outliers be specified.

Here is your answer and the equation you asked for

http://www.itl.nist.gov/div898/software/dataplot/refman1/auxillar/esd.htm