Katherine Blair :

First question: Are you talking about a single Likert-type item ? That is, a single question with the response 1 - 10 ?

Question 1b: Assuming that's a Yes: Would you want to treat your 1 to 10 responses as being equally spaced ? That is, do you suppose that respondents answer as if the distance between 1 and 2 is the same as the distance between 9 and 10 ?

Question 2: : Is your data such that you know Respondent A responded 5 before and Respondent A responded 6 after?

Salvatore S. Mangiafico

1a: yes, a single likert item (1-10). The Sam question is then asked again a few weeks later.

1b: yes, equally spaced

2: yes, my data is grouped to that I can see each respondent’s change in response.

Katherine Blair , do you have people not given the intervention?

And first thing to do would be a scatter plot, jittering the points a little in case some data points are at the same place.

Daniel Wright no, there is no control group needed

So scatter plot, include the y=x diagonal, and what you will likely want to be comparing is how much above or below the diagonal line those 20 points are. How depends on whether there is a relationship between these differences and the x variable, and the distribution of the these differences.

Katherine Blair , thank you for answering those questions. It helps a lot for us to be able to give some reasonable suggestions.

Daniel Wright gives you good advice. Start by plotting the data. I think the following is a case of what they have in mind: https://rcompanion.org/handbook/images/image062.png

It's important to ask yourself, in what way do I want to know if the responses are different ? That is, 1) is the mean of the differences in responses important ? Or, 2) whether the value of the after responses are generally greater or less than the before responses. Or 3) maybe you are interested in the medians? ... And there are many other possibilities.

A test that may be applicable for 2) is the Wilcoxon signed-rank test. A test that may be applicable for 1) is the paired t test, though there are additional assumptions for that test ---- though there may be versions, like permutation tests ---- that don't have those assumptions. A test that may be applicable for 3) may be the (paired, two-sample) sign test.

And to add to Salvatore S. Mangiafico 's list, a) is the association between the before and after scores interesting to look at, and b) is the value in testing the null hypothesis or would it be better to get an interval estimate of the effect (assume the effect is of a similar size for all values of the pre score, otherwise you may have some other things to do). Oh, and the scatter plot will help you to see if there are floor/ceiling effects, and this can affect your choice of analysis.

Hello Katherine!

Since you know pre-post data for each "patient", you'll want to use a paired-samples test.

Once you have conducted the experiment and have your data, you'll want to check for normality in the difference of the pre-post data (which way you subtract is not important, so long as you interpret the end result correctly). You can do this with the Kolmogorov-Smirnov test. If both the difference of data is normally distributed, you can run a paired-samples T-test. If either parameter is not-normally distributed, you'll have to use a non-parametric paired samples test, such as the Wilcoxon signed-rank test!

Best of luck with your endeavors!

EDIT: Edited based on input from Dr. Wright.

Anastasios Stylianos Tsarouchas , not sure why you highlighted or included the word separately. Can you expand? Obviously the paired ttest has the assumption that the difference between them is assumed to be normally distributed. Also, don't you think given the sample size the Kolmogorov-Smirnov test is under-powered? It cannot show the pre, post, or post-pre variables are normally distributed, only in extreme cases can it even reject this.

Hey Daniel Wright!

Many thanks for your input. Throughout my answer I assume little familiarity with statistics - our OP is a new user who could be still in the process of learning. Thus, I intended to stick to the point and provide a simple means for them to test whether a significant change is to be noted between pre-post samples.

In case this is not true, the OP has extensive knowledge on the matter and wishes to engage in deep discussion as to the optimal manner of tackling this problem, then I apologize for a rather simplistic, 'cook-book' style answer.

There are indeed better alternatives to the KS test for normality, but it's undeniably a household name in medical science. I agree that people trust it more than they should, but it's definitely a step above no testing at all.

Hi Anastasios Stylianos Tsarouchas , I have to head off to a meeting, so I'll just discuss two points. First, the separately, the paired t-test doesn't make any assumptions about the original pre or the original post variables. It subtracts them and then gets on with the procedure. Further, even if the pre and post (separately) are the same (normal or whatever), this does not imply the difference will have the same distribution.

Second, my point about normality testing wasn't specifically about other tests being more powerful, it was that with the sample size no test will be able to show the differences are close to normal.

Hey Daniel Wright you're absolutely correct! I amended my original response so as not to perpetuate the misconception I previously held on the subject!

Thank you Anastasios Stylianos Tsarouchas !